The Communication Success Screener: A Preliminary Investigation of Perceived Communicative Success Across Modalities, Environments, and Demands

Abstract

Purpose:

Behavioral intervention targeting speech, language, and communication concerns is an established therapeutic approach for patients with communication concerns across a range of acquired neurogenic disorders. The multidimensional factors that contribute to a person's self-identified communication challenges and strengths in their daily communication needs must be considered to provide functional and person-centered care. While assessments grounded in clinician observation or client self-report exist, there is a direct need for a screening tool that comprehensively evaluates the roles of modality (verbal, text, gesture) and environment (in-person, virtual) on self-reported success across communicative demands. In this study, we describe a preliminary approach to monitoring the progression of receptive and expressive communication skills in people with chronic poststroke aphasia in the context of communication practices of the 21st century, culminating in the development and exploratory implementation of a novel clinical instrument: the Communication Success Screener (COMSS).

Method:

Thirty-three participants with aphasia due to stroke were recruited to complete and evaluate the COMSS via an online survey. Quantitative responses and open-ended participant feedback were collected to validate and propose adaptations to the COMSS. Group-level analyses and case presentations were used to highlight COMSS features and outcomes.

Results:

Participant responses to the COMSS questionnaire suggest that this screening tool creates differentiated communicative success profiles based on self-report. Participant feedback also indicated that the COMSS appropriately evaluates self-reported success across modalities of verbal, text, and gesture in the context of in-person and virtual environments.

Conclusions:

The communication concerns experienced by people with aphasia are often heterogeneous and vary as a function of their daily activities of living, preferred language modalities, and environmental supports. We present preliminary feasibility analyses of the COMSS as a potential tool to support the monitoring of functional change by evaluating self-reported communicative success across form, modality, environment, and task demand.

Supplemental Material:

https://doi.org/10.23641/asha.24521044

Functional communication and its success can be operationalized as a person's ability to successfully interact in various communicative environments with multimodal communication (Clark, 1996; Crockford & Lesser, 1994; Holland, 1982). While easily defined, communicative success is complex to assess in a reliable, systematic manner that reflects real-world examples (Doedens & Meteyard, 2022) as well as client needs (Gallée & Volkmer, 2021; Volkmer et al., 2023). In response to this complexity, Gallée and Volkmer (2021) affirmed that all evaluation of speech and language abilities must consider “the use of language and how these aspects may impact functional communication” (p. 705). Identifying, monitoring, and acknowledging the manner in which specific impairments manifest in an individual are essential to conduct short and long-term care planning; moreover, these insights must be reflective of the person's multimodal and functional communication needs to best promote their everyday well-being (Gallée & Volkmer, 2021; Gallée et al., 2023). Rather than using an impairment-based framework to identify capacity in select linguistic functions, this functional-pragmatic approach focuses upon how effectively a person communicates and participates across situations (Doedens & Meteyard, 2022; Galletta & Barrett, 2014).

This philosophy is directly in line with the Life Participation Approach in Aphasia (LPAA), where enhancing a person's engagement and successful participation is the overarching aim of treatment outcomes (Chapey et al., 2000; Rogalski & Khayum, 2018; Ruggero et al. 2019; Volkmer et al., 2023), as well as the Living with Aphasia: Framework for Outcome Measurement (Kagan et al., 2008) and FOURC Model frameworks (Haley et al., 2019), where collaboration between the clinician and client is at the center for intervention in poststroke aphasia. Moreover, consistent with these frameworks, a person's self-evaluation of their communicative success must be considered in order to contextualize their strengths and weaknesses, as well as to best form and scaffold treatment goals (Galletta & Barrett, 2014). As such, the interplay of various factors—beyond pure preserved linguistic abilities—that can either positively or negatively impact a person's everyday communication experience must be examined directly. Here, we aim to create and provide a preliminary analysis of an assessment tool that captures self-reported success in receptive and expressive communication across modalities, environments, and communicative demands.

Motivation for a Novel Self-Report Measure

There is an increasing push to integrate stakeholder input and engagement to create evaluation protocols that accurately assess the functional impact of a condition on a person's life (Charalambous et al., 2020; Shippee et al., 2015). Person-centered assessment hinges upon client input (Gallée et al., 2023; Hersh & Boud, 2023). Patient-reported outcome measures (Irwin, 2012; Meadows, 2011) and self-report measures, in which clients self-evaluate their communication and participation capabilities, allow clinicians to tap into client-directed needs. This is essential to set the foundation for assessment and therapeutic processes that are completed in partnership with the client (Cruice et al., 2006; Swinburn et al., 2019). A requisite feature of the Communication Success Screener (COMSS) is that it is a direct measure of a person's self-perceived success. As such, a person's ratings can be used to characterize their insight or relative impact of their communication strengths and weaknesses. Mismatches can represent their weighting of perceived weaknesses; for example, a person may have a severe Western Aphasia Battery–Revised (WAB-R; Kertesz, 2022) Aphasia Quotient due to significant paucity of output and naming challenges but rate themselves as “almost always” successful in their given communicative demands due to limited verbal expression needs. Conversely, a person may rate their success across communicative demands within the not often or sometimes range but score highly on procedural discourse tasks and other assessments of discourse. These forms of mismatches may reveal priority areas for treatment that may not have been discovered otherwise.

Self-Report Measures for People With Aphasia

The LPAA is a service delivery model that necessitates the need to promote life participation in the face of communication challenges (Chapey et al., 2000). Therefore, it is imperative to assess the communicative satisfaction that people with aphasia (PWA) report to experience to best evaluate the role of their aphasia in their day-to-day life participation, personal identity, and overall quality of life (Irwin, 2012). PWA are the experts on their condition and should not be excluded from the assessment process (Jayes & Palmer, 2014). The value of PWA insight on the impact of their condition is increasingly recognized (Charalambous et al., 2020; Swinburn et al., 2019), driving the call for assessment practices to incorporate more and more client-directed feedback and goals (Hersh & Boud, 2023).

This has also led to the development and validation of a variety of quality of life measures and self-rating scales to assess life participation, well-being, and the functional impact of speech-language therapy for people with acquired neurogenic language disorders; examples of these include the Communication Participation Item Bank (Baylor et al., 2013, 2021), Assessment for Living with Aphasia (Simmons-Mackie et al., 2014), Communication Confidence Rating Scale for Aphasia (Babbitt et al., 2011; Cherney et al., 2011), Aphasia Needs Assessment (Garrett & Beukelman, 2006), Communicative Effectiveness Index (Lomas et al., 1989), Comprehensive Aphasia Test “The Aphasia Impact Questionnaire,” quality of life questionnaire for aphasics (Spaccavento et al., 2014), and Stroke and Aphasia Quality of Life Scale-39 (Hilari et al., 2003). The development of a new assessment tool is motivated by existing gaps relating to stakeholder involvement and the domains of communication addressed by these measurements.

Moreover, a recent review by Charalambous et al. (2020) revealed that only a small subset of self-report tools placed PWA at the center of the tool development. Creating assessment tools in partnership with PWA is essential to create ethical, meaningful, experientially driven, and empowering measures (Hersh et al., 2022). Therefore, in this preliminary investigation, stakeholder feedback is central in the development of an efficient yet comprehensive measure of self-reported communicative success for PWA. Further motivation for a novel self-report measure is the need to amplify the evaluation of gesture and virtual environments. Gesture has long been observed to facilitate and compensate for gaps communication in acquired neurogenic language disorders, especially poststroke aphasia (Rose, 2006, 2013; Sekine et al., 2013; Stark & Cofoid, 2022); however, we are unaware of any self-report assessment that measures a person's use or reliance upon gesture in communication. While clinician measures of gesture exist in scales such as the City Gesture Checklist (Caute et al., 2021) and components of the Nonverbal Semantics Test (Hogrefe et al., 2021), nonverbal communication and its place in everyday communication have yet to be investigated in self-report measures for people with communication concerns. Similarly, while a range of purposes are addressed by the evaluation tools, the evaluation of the role of technology in everyday communication has been limited. While certain assessment tools ask clients to evaluate their confidence or ability to use a phone or watch TV (see Table 1), the role of technology (relevant to today's technological tools, including but not limited to text-to-speech applications; word prediction software; and a myriad of platforms with text, audio, picture, and video messaging options; Dietz et al., 2011; Menger et al., 2020) has yet to be considered consistently and across communicative demands.

Table 1.

Existing self-report scales for people with aphasia.

Measure	Investigates	Total communicative demands	Evaluates gesture	Evaluates technology
The Communicative Participation Item Bank–General Short Form (Baylor et al., 2013)	Interference of communication concerns across situations on a typical day	10	No	No
Assessment for Living With Aphasia (Simmons-Mackie et al., 2014)	Quality of life as it relates to aphasia	37	No	No
The Aphasia Communication Outcome Measure (Hula et al., 2015)	Impact of communication disorder on everyday participation and quality of life	59	No	No
Communication Confidence Rating Scale for Aphasia: 7-item (Babbitt et al., 2011; Cherney et al., 2011)	Communication confidence across a variety of tasks	7	No	Limited to single item: “How confident are you about your ability to speak on the telephone?”
Communicative Effectiveness Index (CETI; Lomas et al., 1989)	Effectiveness of communication across situations	16	No	No
Quality of life questionnaire for aphasics (Spaccavento et al., 2014)	Perception of well-being across contexts	37	No	Limited to items: “To understand television programs” and “To use the phone”
Aphasia Needs Assessment (Garrett & Beukelman, 2006)	Communication concerns and priorities	83	No	Limited to single item: “talking on the phone”
Stroke and Aphasia Quality of Life Scale (SAQOL-39; Hilari et al., 2003)	Quality of life for people with chronic aphasia	39	No	No
The Simple Aphasia Stress Scale (Laures-Gores & Rice, 2019)	Assessment of experienced stress	1	No	No
Burden of Stroke Scale: communication and communication distress probes subsets (Doyle et al., 2003)	Functional participation and well-being	12	No	No

Environmental Contexts of Communication

There is ample incentive to assess the impact of environmental context on communicative success, with a particular lens toward virtual environments. As of April 2020, people of all ages across the globe have adjusted their daily digital habits, spending more time on social media, messaging applications, and videoconferencing than ever before (Kemp, 2020), with up to 79% of Internet users reporting increases in smartphone or mobile phone usage. The percentage of offline Americans, or Americans who report to not use the Internet, has decreased from approximately 50% in 2000 to less than 10% in 2021 (Pew Research Center, 2021). Furthermore, a significant increase in video-based calls was also observed (Dean, 2022; Koeze & Popper, 2020); as of October 2020, the number of annual meeting minutes spent on the popular video conferencing tool, Zoom, had increased by 3,300% from the same quarter in the previous year, with over 300 million individuals logging into meetings on a daily basis (Dean, 2022).

Studies of virtual communication in aphasia, such as texting, have emerged in the aphasia literature (Beeson et al., 2013; Fein et al., 2020; Kinsey et al., 2022; Lee & Cherney et al., 2022) alongside the growing work on the feasibility and benefits of speech-language intervention administered via service delivery models in the context of telepractice (Dial et al., 2019; Gallée et al., 2020; Hall et al., 2013; Kurland et al., 2018; Peñaloza et al., 2021; Schaffer et al., 2022; Steele et al., 2014; Weidner & Lowman, 2020), virtual participation in research (Dekhtyar et al., 2020; Doub et al., 2021; Marcotte et al., 2022), and technology use (Dietz et al., 2011; Menger et al., 2020; Sitren & Vallila-Rohter, 2019). Sitren and Vallila-Rohter (2019) determined that the success rate of tablet use in persons with aphasia is heterogeneous but is an appropriate goal for intervention use as their results showed PWA gaining independence in tablet use following a short course of technological intervention. Targeting technology use is, furthermore, supported by the growing push to create connections through support groups online, to not only eliminate geographic barriers but also create connections that may have otherwise not been possible (Brice & Hinckley, 2022; National Aphasia Association, 2022; Pitt et al., 2019). As such, evaluating PWA's communication via technology is necessary to comprehensively evaluate their overall self-perceived communicative success (Dietz et al., 2011). Of course, not every individual has access to or chooses to communicate via technological devices; however, this information must be established, rather than assumed, by the evaluating or treating clinician (Menger et al., 2016, 2020). Furthermore, an underlying aim of speech-language intervention is that it will generalize or enhance success across modalities; however, limited studies of such generalization currently exist in the literature (Webster et al., 2015).

Study Aims

In this study, we present the outcomes of a preliminary investigation of feasibility for a screening tool that reflects the vast variety of communicative contexts our speakers can currently operate in. We introduce the COMSS, a tool that directly assesses self-reported success across communication modalities, environments, and demands that are reflective of our world today. We partnered with individuals with chronic poststroke aphasia to use and reflect on the COMSS. The goal of this mixed-methods approach was to (a) characterize the success, or absence thereof, that people with chronic poststroke aphasia reported to experience across 18 communicative tasks; (b) examine how they felt regarding the COMSS questions' relevance and importance to their own life; and (c) explore the extent to which personal factors (e.g., time postonset, aphasia type) related to COMSS's perceived success by communication modality (verbal, text, gesture), form (expressive, receptive), and environment (in-person, virtual). Given a paucity of research in this area, we do not have an explicit directional hypothesis but instead propose to explore the impact of these factors across aphasia chronicity and other available aphasia characteristics, such as aphasia type.

Method

Participants

Thirty-three people living with stroke and aphasia participated in this study. One participant was excluded from the data set due to a diagnosis of dysarthria without aphasia. All participants were recruited from author B.C.S's aphasia participant bank at Indiana University-Bloomington (ethical approval via Institutional Review Board No. 16228). Participants were only contacted about possible participation in this study on the basis of their diagnosis of aphasia; each had already been recruited as part of several ongoing studies and consented to being recruited for further studies. All aphasia diagnoses were the result of a stroke or brain injury at least 6 months prior to recruitment. Prior to commencing the COMSS, participants were given the opportunity to self-describe their aphasia, with some choosing to describe a broad type of aphasia (e.g., expressive, receptive) and others the severity (e.g., moderate, global). Participants were provided the option of writing their aphasia type in an open-ended text box, embedded in the survey. For 14 of 32 participants, standardized assessment scores of speech, language, and communication were completed at the time of assessment (see Supplemental Material S1). Standardized assessment scores of speech and language were not available for all participants in B.C.S' aphasia participation database because some studies being conducted required only self-identification of aphasia and did not obtain standardized assessment scores identifying aphasia using some criterion.

Survey Development

The development of the COMSS was built upon the following principles:

• provides an immediate and structured opportunity for person-centered assessment and treatment targets;

• is built upon a functional-pragmatic, rather than impairment-based, model of communication (Doedens & Meteyard, 2022; Helm-Estabrooks et al., 2014);

• implementable across all adult populations with acquired neurogenic communication concerns;

• bridges the gap between performance on select isolated domains of speech and language and the effect of their interplay on a person's satisfaction in everyday communication (see Gallée & Volkmer, 2021); and

• is built upon self-evaluation and is therefore worded with aphasia in mind (e.g., by using clear language and pictographs).

The survey questions were developed by the first (J.G.) and second (V.E.T.B.) authors. The items in the COMSS were devised to establish a person's expressive and receptive participation in foundational language functions, such as responding to or understanding an open or closed question. Furthermore, communicative contexts with typical interlocutors, such as familiar or unfamiliar acquaintances and health care providers, were used to develop the scenarios represented in the COMSS. Questions were split approximately to address both expressive and receptive forms of language. Unique to the COMSS, five response options for participants to rate were created for each of the 18 questions, where options varied as a function of modality (verbal, text, and gesture) and environment (in-person and virtual) to comprehensively capture a person's perceived success in their functional participation across these features. Chronicity was included in analyses to account for this factor's potential role on feelings of success and adjustment to the effects of aphasia in communication over time.

Survey Structure and Design

The online survey was developed using Qualtrics software (2018) and hosted at Indiana University. Arial, a sans serif font, was used to display all survey instructions and items. The font size was set to size 16. Line spacing ratios were set to 1.5, and questions were separated by an additional empty space to clearly segment each question. The survey content was programmed to be compatible with mobile, tablet, and large screen devices to ensure comparable and easy-to-read presentations of the survey content. The survey scale (see Figure 1) was displayed with each question prompt to reduce working memory load and to support consistent responses.

Figure 1.

The COMSS rating scale. The rating scale of the COMSS includes pictographic supports to help clients evaluate and select their ratings for each statement. COMSS = Communication Success Screener.

Survey Items

Survey items asked participants to self-evaluate their sense of success when implementing verbal, text, or gesture-based communication across a series of communicative scenarios targeting either expressive or receptive language (e.g., participating in small talk with an acquaintance vs. understanding directions) in either in-person or virtual contexts. Survey questions were created with the aim of creating communicative contexts that (a) reflected both in-person and virtual environments; (b) equally evaluated expressive and receptive language performance; (c) probed verbal (speaking/listening), text (writing/reading), and gesture-based (facial expression and body language) modalities of communication; and (d) covered a variety of communicative demands that represent relevant yet generalizable scenarios that a person may encounter in a typical month. The complete set of questions can be found in Table 2 and the Appendix. To download the most current version of the COMSS (for print or virtual use), as well as further instructions, a preprint of this article with all corresponding materials can be accessed on the Open Science Framework (Foster & Deardorff, 2017; Gallée et al., 2022) using the following link: https://osf.io/c4maz/.

Table 2.

The 18 items of the Communication Success Screener (COMSS) and mean participant ratings.

Question	How successful do you feel in…	Domain	Mean rating	SD
Q1	Conversation with a person you are close to?	Speaking	3.61	1.08
		Writing	2.61	1.49
		Gesture	3.55	1.12
Q2	Small-talk with an acquaintance or colleague?	Speaking	3.34	1.38
		Writing	2.33	1.32
		Gesture	3.48	1.21
Q3	Communicating a transaction (e.g., ordering or paying for a meal)?	Speaking	3.00	1.39
		Writing	2.68	1.46
		Gesture	3.39	1.17
Q4	Asking a question with an open-ended answer?	Speaking	4.04	1.29
		Writing	3.17	1.59
		Gesture	3.54	1.48
Q5	Speaking to your doctor or other healthcare provider?	Speaking	3.50	1.21
		Writing	2.58	1.41
		Gesture	3.37	1.24
Q6	Providing instructions?	Speaking	3.61	1.09
		Writing	2.73	1.29
		Gesture	3.42	1.17
Q7	Communicating your opinion on a preferred topic?	Speaking	3.08	1.24
		Writing	2.84	1.25
		Gesture	3.44	1.19
Q8	Communicating your opinion on a novel topic?	Speaking	3.40	1.19
		Writing	2.67	1.09
		Gesture	3.39	1.20
Q9	Understanding conversation with a person you are close to?	Speaking	2.87	1.33
		Writing	2.22	0.96
		Gesture	3.00	1.26
Q10	Following small-talk with an acquaintance or colleague?	Listening	4.14	0.86
		Reading	3.43	1.28
		Gesture	3.71	1.18
Q11	Understanding a yes/no answer to a question you asked?	Listening	4.15	0.83
		Reading	3.42	1.30
		Gesture	3.63	1.47
Q12	Following a group conversation with two or more people?	Listening	3.55	1.31
		Reading	2.98	1.24
		Gesture	3.48	1.48
Q13	Understanding a non-yes/no answer to a question you asked?	Listening	3.94	0.93
		Reading	3.32	1.28
		Gesture	3.67	1.44
Q14	Understanding instructions?	Listening	3.53	1.12
		Reading	3.46	1.21
		Gesture	3.53	1.48
Q15	Understanding your doctor or other healthcare provider?	Listening	3.92	0.96
		Reading	3.38	1.27
		Gesture	3.42	1.41
Q16	Understanding other people's opinions?	Listening	3.83	1.30
		Reading	3.28	1.40
		Gesture	3.74	1.34
Q17	Understanding a movie or TV show?	Listening	3.389	1.34
		Gesture	3.46	1.79
Q18	Understanding what you hear on the radio or on the phone?	Listening	3.82	1.25

Survey Completion

An embedded mixed-methods approach was used (see Figure 2; Creswell & Plano Clark, 2018) to gather both quantitative and qualitative data on the COMSS, where the primary emphasis was placed on the quantitative participant responses. The qualitative data were meant to extrapolate on quantitative findings for the purposes of this preliminary study. Participants completed the online 18-question COMSS that asked them to self-evaluate their sense of success when implementing verbal, text, or gesture-based communication across a series of communicative scenarios targeting either expressive or receptive language (e.g., participating in small talk with an acquaintance vs. understanding directions). Participants received a $10 Amazon gift card for completing the survey. The survey was introduced to participants using the following prompt: “We will ask you questions about how successful you feel when speaking, writing, listening, reading, or using body language in different environments. We will ask about IN-PERSON (e.g., with another person at home or in a store) and virtual (e.g., on the phone, tablet, iPad, or computer) communication. We will also ask about BODY LANGUAGE (e.g., using your body and facial expressions to communicate).”

Figure 2.

Study design. An embedded design that incorporated both quantitative and qualitative assessment was used to evaluate stakeholder responses and feedback to evaluate the feasibility and usefulness of the COMSS. COMSS = Communication Success Screener. Visualization of methods described in from Creswell and Plano Clark (2018).

Participants were asked to rate each of the 18 questions using a Likert scale (0 = not applicable, 1 = never, 2 = almost never, 3 = sometimes, 4 = almost always, 5 = always) with associated visualizations for each score (see Figure 1). Responses could be selected by tapping the desired response or dragging the response scale to the intended target. At the end of the survey, participants were given the option to provide responses to both multiple-choice and open-ended feedback about the survey itself. Responses were not required. Open-ended feedback was elicited using question prompts with text boxes, in which respondents could type. The responses to the feedback were analyzed in the context of this study to validate and improve the screener using thematic analysis.

Survey Scoring

Composite scores for expressive and receptive language and verbal, text, gesture-based, in-person, and virtual communication were derived from the ratings provided for each prompt within the COMSS. Scores did not include scenarios that were rated as “not applicable,” as in these instances, rating scores of 0 were provided. Scores were calculated to reflect how successful a participant rated themselves to be in particular communicative contexts (i.e., 1 = never, 2 = not often, 3 = sometimes, 4 = almost always, and 5 = always). Exact scoring instructions are provided in Table 2.

Case Presentations

Three cases were chosen to illustrate three distinct examples of COMSS completion with varying aphasia diagnoses, each of which were validated by standardized assessment. These cases are presented below to demonstrate how the COMSS can help identify communicative success profiles across participants and diagnoses. Average and item-level self-report scores are reported for each to show the possible range of variation.

Analysis

Statistical Analyses of COMSS Outcomes

In the embedded mixed-methods design of this study, the quantitative outcomes of participant responses to the COMSS survey items were of primary interest. To address this aim, a linear regression was applied to statistically compare ratings of success as a function of modality (verbal, text, and gesture), form (expressive and receptive), environment (in-person and virtual) and chronicity (time since aphasia onset) while taking participant differences into account. For the modality of gesture, participants were asked to rate their success in using gesture across environment (e.g., collapsed across in-person and virtual) due to the relative consistency of facial expressions and body language across contexts and to mitigate respondent fatigue. As such, we evaluated the effect of environment in the absence of gesture success ratings. Here, another linear regression was used to examine the effect of modality (verbal and text), form (expressive and receptive), environment (in-person and virtual), and chronicity while also acknowledging participant differences as possible error contributors.

Thematic Analysis of Open-Ended Participant Feedback

Our second aim was to involve our participants as stakeholders in our research. We addressed this aim by collecting closed and open-ended feedback on the utility and relevance of the COMSS items. Qualitative methods supported our exploration of the perceived usefulness of the COMSS and to enhance our understanding of participant responses to the actual COMSS survey items. Six phases of thematic analysis (Braun & Clarke, 2021; Byrne, 2022) were applied to identify themes within the open-ended participant feedback. In this process, participant responses were collaboratively reviewed, annotated, and coded to derive their core meaning(s) by the first (J.G.) and second (V.E.T.B.) authors. Then, these codes were further evaluated to generate potential themes. The relative usefulness and weighting of the themes was then assessed to establish a coherent and cogent narrative of the data. Themes were finalized after this review by the first author (J.G.).

Results

Participant Demographics

The average reported age of aphasia onset of the 32 participants was 7.56 years (SD = 8.19 years). All participants reported to have aphasia due to stroke, where 20 out of the 32 participants provided additional descriptors of their aphasia type, such as “expressive,” “moderate,” “anomic,” or “global,” when prompted. Ten of the participants without standardized assessments reported to not know their specific diagnosis of aphasia (e.g., by stating “I don't know” or “unknown” or leaving this item blank). Four of the participants stated that they were still working at the time of completing the survey. Quick Aphasia Battery (QAB) scores (Wilson et al., 2018) were available for nine participants (M = 8.26, SD = 1.89), and WAB-R Aphasia Quotient scores were available for five participants (M = 81.6, SD = 15.4). Of the participants for whom standardized testing was obtained and who completed WAB-R testing, two were revealed to have a latent severity, eight have mild severity, and four have moderate severity.

COMSS Outcomes

Responses to the questions on the COMSS were optional. At least 25 responses were collected for each of the 18 COMSS items (M = 27.6, SD = 1.56, range: 25–31; see Table 2 and Figure 3 for the average success ratings by item and Table 3 for mean success ratings and ranges by modality, form, and environment). A two-way analysis of variance (y = Success ~ Modality × Form × Environment × Chronicity + Error(Participant)) was used to statistically compare success ratings (e.g., the ratings participants provided for each statement on a scale from 1 to 5 or identified as not applicable) as a function of modality (verbal, text, or gesture), form (expressive vs. receptive), and chronicity (time since aphasia onset) while taking participant differences into account. Results of this analysis revealed significance for modality, F(2) = 11.1, p < .001, and form, F(1) = 5.27, p = .025. No significant interactions of modality, form, environment, or chronicity were identified.

Figure 3.

Self-evaluated success by modality (verbal, text, or gesture) by question. The average success reported by all participants for each question across modalities. While gesture-based communicative success could be rated for all 18 items, verbal success was only relevant for Items 1–17 and text-based success for Items 1–16.

Table 3.

Mean self-ratings across the Communication Success Screener (COMSS) domain scores.

Measure	Mean rating	SD	Range
Expressive verbal	3.29	1.02	0.80–5.00
In-person Virtual	3.34 2.93	0.94 1.37
Receptive verbal	3.69	0.85	1.71–5.00
In-person Virtual	3.63 3.67	1.28 1.01
Expressive text	2.61	1.08	0.50–4.56
In-person Virtual	2.56 2.43	1.16 1.21
Receptive text	3.10	0.99	1.00–4.50
In-person Virtual	3.13 3.24	1.34 3.24
Expressive gesture	3.36	1.09	1.88–5.00
Receptive gesture	3.53	9.00	0.88–5.00

Post hoc analysis in the form of pairwise t tests (Holm method) demonstrated a significant difference between verbal, text, and gesture-based modalities, where communicative success ratings for text (M = 2.84, SD = 1.06) were significantly below gesture (M = 3.45, SD = 1.09, p = .004) and verbal (M = 3.49, SD = 0.953, p = .003). No significant difference between verbal and gesture modalities emerged (p = .834). Additionally, a significant difference between expressive (M = 3.11, SD = 1.09) and receptive (M = 3.50, SD = 1.02) language form was observed (p = .025), where participants rated themselves as more successful recipients than producers of communication.

Case 1

Our first case reported to have had a fluent aphasia for 10 years and to currently be working. Standardized assessment results revealed a QAB overall score of 9.55, with lowest scores of 7.5 for speech motor and 9.17 for sentence comprehension (see Supplemental Material S1). As such, performance was consistent with a classification of no or latent aphasia. This person reported to experience higher communicative success using gesture, reporting to experience it somewhere between almost always and always (M_{ExpressiveGesture} = 4.25, SD = 1.04; M_{ReceptiveGesture} = 4.70, SD = 0.483), than verbal (M_Speaking = 3.50, SD = 1.37; M_Listening = 4.11, SD = 0.963) or text-based (M_Writing = 3.40, SD = 1.37; M_Reading = 3.40, SD = 0.910; see Figure 4 and Table 4) modalities. In terms of communicative environment, this participant reported to experience communicative success between levels of sometimes and almost always across virtual (M_{ExpressiveVirtual} = 3.20, SD = 1.32; M_{ReceptiveVirtual} = 3.47, SD = 1.33) and in-person (M_{ExpressiveInPerson} = 3.69, SD = 0.946; M_{ReceptiveInPerson} = 4.05, SD = 1.22) situations. This participant completed all items of the COMSS.

Figure 4.

Three case studies of self-evaluated success ratings by form (expressive vs. receptive) and environment (in-person vs. virtual). Case 1 scored 9.55/10 summary score on the QAB (Wilson et al., 2018), Case 2 scored an aphasia quotient of 80.8, and Case 3 scored a 5.83/10 QAB summary score. Distinct profiles of success as a function of form and environment emerged. QAB = Quick Aphasia Battery.

Table 4.

Three case presentations of Communication Success Screener (COMSS) outcomes.

Participant		Verbal		Text		Gesture
Global aphasia	Question	In-person	Virtual	In-person	Virtual	Collapsed
	1	3	3	2	1	2
	2	4	3	1	1	3
	3	3	0	NA	NA	3
	4	4	NA	NA	NA	NA
	5	3	NA	NA	NA	NA
	6	3	NA	NA	NA	NA
	7	3	NA	NA	NA	NA
	8	4	NA	NA	NA	NA
	9	3	NA	NA	NA	NA
	10	4	NA	NA	NA	NA
	11	3	NA	NA	NA	NA
	12	1	NA	NA	NA	NA
	13	4	NA	NA	NA	NA
	14	3	NA	NA	NA	NA
	15	3	NA	NA	NA	NA
	16	3	NA	NA	NA	NA
	17	--	3	--	--	NA
	18	--	4	--	--	NA
Fluent aphasia
	1	4	3	4	4	4
	2	5	4	2	3	5
	3	4	2	2	3	5
	4	4	0	3		5
	5	5	3	4	3	2
	6	5	5	3	5	4
	7	3	3	4	5	4
	8	4	2	3	3	5
	9	2	3	4	3	4
	10	4	4	3	3	5
	11	4	5	5	5	5
	12	5	4	1	3	5
	13	5	3	4	3	5
	14	5	3	4	5	5
	15	5	5	5	5	5
	16	5	3	2	2	5
	17	--	5	--	--	4
	18	--	4	--	--	NA
Nonfluent aphasia
	1	3	5	0	3	5
	2	4	5	1	3	5
	3	1	4	1	1	5
	4	4	4	1	2	5
	5	2	4	1	2	5
	6	2	3	1	2	5
	7	2	3	2	3	5
	8	4	5	1	2	5
	9	1	3	1	1	5
	10	5	5	NA	3	4
	11	4	5	0	3	5
	12	1	1	1	1	1
	13	4	5	1	NA	5
	14	3	4	1	3	5
	15	2	2	0	3	3
	16	5	5	0	5	5
	17	--	5	--	--	5
	18	--	5	--	--	NA

Note. Dashes indicate absence of a participant response to a survey item. NA = not applicable.

Case 2

The second case reported to have nonfluent/expressive aphasia for 3 years and to be working at the time of completing the COMSS. WAB-R testing revealed an Aphasia Quotient score of 80.8 and a mild anomic aphasia classification (see Supplemental Material S1). This participant reported to experience communicative success between sometimes and almost always for the verbal modality (M_Speaking = 3.44, SD = 1.21; M_Listening = 3.61, SD = 1.58) and for writing (M_Writing = 3.40, SD = 1.63; M_Reading = 1.64, SD = 1.49), whereas they reported to always experience communicative success in using gesture expressively and to almost always experience success receptively (M_{ExpressiveGesture} = 5.00, SD = 0.000; M_{ReceptiveGesture} = 4.30, SD = 1.34; see Figure 3 and Table 4). While this participant reported to experience similar communicative success to Case 1 in their use of virtual communication (M_{ExpressiveVirtual} = 3.19, SD = 1.22; M_{ReceptiveVirtual} = 3.06, SD = 1.53), they rated in-person communication to be relatively lower at the level of not often or sometimes (M_{ExpressiveInPerson} = 1.88, SD = 1.26; M_{ReceptiveInPerson} = 2.17, SD = 1.98).

Case 3

The third and final case reported to have global aphasia for 22 years and to not be working at the time of this study. Standardized testing on the QAB revealed a summary score of 5.83, with the lowest scores being 2.5 for sentence comprehension and 2.36 for grammatical construction consistent with a moderate severity of aphasia (see Supplemental Material S1). This participant provided ratings that revealed higher self-rated communicative success for the verbal modality at a level of sometimes (M_Speaking = 3.00, SD: 1.09; M_Listening = 3.1, SD = 0.876) over the text (M_Writing = 1.25, SD = 0.500) and gesture-based (M_{ExpressiveGesture} = 2.67, SD = 0.577) modalities, for which no responses were provided for receptive communication. This participant, as is shown in Table 4, did not rate the majority of virtual or text-based communicative environments. For the communicative demands for which this participation rated their success in virtual communication, minimal success was reported (M_{ExpressiveVirtual} = 1.60, SD = 1.34) in contrast to in-person communication (M_{ExpressiveInPerson} = 2.73, SD = 0.942; M_{ReceptiveInPerson} = 3.10, SD = 0.876).

Participant Feedback

Consistent with the completion of the COMSS questions, at least 25 responses were provided per optional multiple choice feedback item (M = 26.7, SD = 1.03, range: 25–28). Results from the closed feedback questions revealed that 93% of respondents found the survey questions to be at least slightly useful (where 16 participants rated the questions as very to extremely useful), 81% found the questions relevant to their life, 80% reported that the questions at least slightly measured their communicative success, and 80% reported that the questions at least somewhat identified their communication strengths (see Figure 5).

Figure 5.

Participant feedback on the COMSS. Participant responses (“yes,” “no,” or “maybe”) on questions related to the relevance and utility of the COMSS to measure their self-perceived communicative success in their daily lives. COMSS = Communication Success Screener.

Eleven participants provided open-ended feedback. Beyond support for the COMSS, three central themes emerged from the thematic analysis of the open-ended feedback: (a) clarify terms, (b) enhance specificity of questions to address communication strengths and weaknesses, and (c) allow for reports of compensatory strategies. Examples of each theme are provided below.

Clarify Terms

Clarification of the terms used in the screener as they related to environment (e.g., in-person vs. virtual) was requested by two participants.

1. “Unclear about technology questions.”

2. “I got confused by the questions about in person reading. The optional questions were worded in the negative which confused me.”

Enhance Specificity of Communicative Demands

Two participants raised the point that the questions did not address within-day variability as it relates to changes in perceived communicative success as well as the specific events that may occur in a given day.

1. “I like the rating scale. However, not every day is same. In reality, I may use all the skills based on my level of neuro fatigue. Essentially going with what works on that given day day [sic]; Not everything always works as it should. The go-to compensation skill may not be available.” (Case 1)

2. “Even though I know that the survey is made to be short so that people will take it, I would like to see more questions about different aspects of one … for example, ordering food at a familiar restaurant versus a new one. In order to be able to assess my areas of strengths or weaknesses I would need to be able to see how I answered the different questions.”

Allow for Reports of Compensatory Strategies

Finally, one participant mentioned that the provision of a space to describe compensatory strategies to enhance communicative participation would be helpful.

1. “In-person researcher interviews would probably be more useful for this study AND finding out how patients like me developed work-arounds many years ago.”

Discussion

Here, we have presented the preliminary findings of a novel self-report tool that was developed to identify self-perceived communicative success across function, modality, and context. In particular, we propose the development of the COMSS to meet current needs for person-centered and stakeholder-informed evaluation measures. The COMSS is also designed to meet the need to measure the impact of technology across communicative demands, a feature that is currently not represented in existing measures. To accomplish this, the screener includes two broad domains of language abilities, namely, expressive and receptive language skills, that can be further broken down into verbal, text, and gesture-based communication and the environment in which these are used: in-person/face-to-face or virtually via technological means. Eight communicative scenarios taxing expressive language concerns and 10 drawing upon receptive language abilities were identified. Each of these scenarios is then rated to evaluate a person's subjective success in using verbal, text, and gesture-based communication strategies within each of these were created, with exception of Items 17 and 18 due to their limited verbal and/or text and gesture-based demands (see Table 2). Communication scenarios that are irrelevant to a given patient's everyday communication demands are excluded from scoring.

This preliminary study revealed that some PWA can self-report their communication successes and challenges; however, future work should examine potential barriers to completing this survey, such as reading comprehension challenges. The communication profile derived from the COMSS can serve to capture the challenges and successes an individual reports to face, providing critical information to determine appropriate restorative, maintenance, and compensatory approaches for personalized intervention targets and future care planning. To illustrate this point with an example, for an individual with chronic Broca's aphasia who predominantly communicates with colleagues via a virtual messaging application, compensatory goals related to work-related communication that tax verbal skills may not be as relevant as targets related to storing important words and phrases in their device to autocomplete their intended messages.

Ratings can additionally be collected from a close care partner or companion as well as the clinician to ascertain coherence and consistency across raters. Care partners have privileged insights in the day-to-day interactions a person with aphasia may have and can provide a unique perspective on their success across communicative demands (Hersh & Boud, 2023). Of course, divergences in care partner and client report will occur and are to be expected (Babbitt & Cherney, 2010; Gillespie et al., 2010) for a variety of reasons, ranging from personality differences to a power differential in the partner relationship (Gillespie et al., 2010). The outcomes from care partner ratings can therefore provide a good starting point to think about goals for communication partner training and counseling.

In this study, we solicited stakeholder feedback, namely, from our participants with aphasia, to address the face validity of this self-report measure. From our results, we believe to have received preliminary evidence that the COMSS measures what it sets out to: a person with aphasia's self-perceived communicative success across a variety of functional and relevant demands and communication environments, forms, and modalities. As will be touched upon in the Future Directions section, we hope to recruit a larger and more diversified sample of participants in terms of chronicity and type of diagnosis to gather more feedback. Ultimately, we hope to create a measure that captures communication concerns across the continuum of recovery and rehabilitation and, most importantly, increase the COMSS's face and content validities.

Clinical Applications of the COMSS

The COMSS is intended to be used in conjunction with standardized assessments that determine performance on isolated domains of speech and language, including but not limited to lexical retrieval, articulation, syntax, semantics, auditory comprehension, reading, and writing to contextualize their performance in these domains. Moreover, successful uses of verbal, text, and gesture-based communication are self-assessed, which enables clinicians to achieve what Hersh et al. (2012) define as “SMARTER goal setting”; in other words, establish goals in collaboration with a patient that are based on their direct life participation needs and abilities. For example, a client may desire to independently send their adult daughter a text every morning. For this patient, appropriate restorative goals may be related to the independent usage of their preferred texting application, whereas compensatory strategies may target the successful use of saved autocorrect and autocomplete functions. Person-centered care is at the heart of this screening tool as we believe that any subsequent intervention must be built in partnership with the clinician, client, and partners involved in care for it to have relevant, meaningful, and sustainable impact.

In our clinical work, we have found that a screening tool such as the COMSS can provide an immediate and structured opportunity for the patient to work in partnership with the clinician to determine the most appropriate and personalized functional communication targets in speech-language therapy across a variety of situations. Furthermore, the impact of such therapy goals on functional communication and subsequent life participation can be assessed by using and comparing COMSS ratings over time. From a research standpoint, the COMSS has the potential be used similarly, such as to evaluate the functional applications and contributions to quality of life of a novel treatment targeting naming. As patients specify their own strengths and challenges by communicative scenario, our priority is for the COMSS to be useful in guiding the clinician to identify the needs of their patients and how these may influence their intervention trajectory.

Case Discussion

All three cases presented in the results illustrate that gesture must be considered when it comes to identifying self-reported communicative success. Moreover, each of the three participants reported to experience communicative success in gesture that was comparable or higher than that of verbal expression. These findings provide further motivation for the use of gesture-based treatment programs, such as Visual Action Therapy (Helm-Estabrooks et al., 1982), or hybrid approaches, such as Promoting Aphasic Communicative Effectiveness (Pulvermüller & Roth, 1991), where gesture is paired with other modalities to functionally enhance both self-perceived and objective communicative success.

These three cases revealed the differentiated importance of the role of technology in typical communicative demands. For example, in contrast to Cases 1 and 2, Case 3 reported a strikingly low technology use in their average communication. Cases 1 and 3 reported to experience reduced communicative success in virtual in comparison to in-person environments, whereas for Case 2, the reverse was true. As such, treatment targets for the former two may incorporate technology, where virtual or technology-based communication could be paired with the stronger (Case 1) or weaker (Case 2) modality of in-person communication to make therapeutic gains. General recommendations for the administration and use of the COMSS are provided below.

Baseline Assessment

The COMSS is intended to create a baseline profile of self-perceived functional communication across a variety of communicative contexts and to efficiently and collaboratively determine a person's goals for speech-language intervention. While this was not achieved in the experimental setup of the present work, we recommend that the clinician initially familiarizes the patient with the concepts introduced in each section, provides definitions for expressive and receptive language, and offers both verbal and gestural explanations of the modalities listed in the screening tool. Furthermore, the communicative scenarios provided in the COMSS can be utilized to structure conversations as they relate to goal setting, building a relationship between the clinician and client, and to improve care partner support. A comprehensive set of administrative instructions can be found in the Appendix.

Completing the COMSS

As demonstrated in the three cases, participant responses create unique self-perceived communicative success profiles that resulted in personalized treatment recommendations. While only the client's evaluations within the COMSS were gathered in the context of this study, there are three possible reporters who can provide ratings of communicative success using the COMSS: the patient, a close care partner or companion, and the clinician. Firstly, the patient's self-report must be prioritized and performed in the presence of the clinician and possibly care partner to ensure full understanding of the communicative scenarios and rating scale. Then, the care partner and clinician can provide their evaluation of the patient's communicative success on a separate form. As a final step, the scores provided by these separate raters can then be compared to help determine interrater reliability. Discrepancies between scores can then be utilized as talking points in counseling or conversation with the patient and/or care partners, as well as to inform the need for further evaluation of self-awareness and monitoring. Of note, when the clinician is less familiar with the patient, filling out the form during the initial evaluation may be of interest as it can help the clinician monitor their own evaluation of the patient's abilities over time. Furthermore, this allows for an informal assessment of the patient's objective functional communication abilities with a stranger and reliability of their self-report. Finally, this exercise provides an opportunity for the clinician to provide feedback on the patient's performance, particularly if it helps the patient identify successes and strengths to build upon.

Future Directions

Here, we describe the preparatory steps of creating a feasible, valid, and reliable tool by reporting on our preliminary investigation of feasibility. We solicited stakeholder feedback in our overarching aim to create a PWA-validated tool and to achieve preliminary content validity. The majority of participant feedback indicated that the content of the survey was useful and relevant to evaluate success in everyday communication. Constructive feedback from participants revealed that certain specifications to the survey could enhance its utility. The COMSS will be adapted in response to the participant feedback. More specifically, the following adaptations will be implemented by the authors and made readily available online:

1. Create clear and accessible definitions of key terms to assist the interpretation of survey questions;

2. Enhance specificity of communicative environments while remaining general enough to be applicable to most, if not all, survey takers;

3. Clarify that responses should reflect an average week that encompasses day-by-day change; and

4. Allow for space to discuss compensatory strategies used in relation to the target modalities, environments, and communicative tasks.

The first two adaptations aim to address the precision of what the screener is capturing and validity of the information that it captures. In response to the third adaptation, while we believe that day-to-day variability is important to consider in assessment, treatment planning, and progress monitoring, the COMSS aims to capture typical or average self-rated communicative success. Finally, for the fourth aim, we will include a section for clients to share specific tools or strategies that they have incorporated into their typical communication to enhance their success. The adapted COMSS (see the Appendix) will be sent out for feedback to a different and larger sample size of participants consisting of both people with poststroke and speech-language pathologists. The currently proposed adaptations will provide the foundation for the next phase of this work: to build upon the current face validity of this measure and to also establish its content validity through a panel of clinical experts. A longer term goal of this project is to examine the sensitivity and stability of the COMSS over the course of a longitudinal study as well as to examine the relationship between client and clinician observations of communicative success. Despite the relatively high completion rate, a limitation of this work is that, by design, the COMSS did not require every question to be completed. However, the unique communicative success perception of the individual lies at the heart of this tool, which we believe to have illustrated through the inclusion of three distinct case studies.

Limitations

There are two primary limitations of this work. The first limitation is that actual completion of the COMSS survey occurred outside of a controlled laboratory setting. We did not require traditional speech and language assessment prior to participating in the COMSS in our efforts to work in partnership rather than undermine the individuals we were aiming to reach (Swinburn et al., 2019). Participants completed the survey at their own pace. As such, we cannot be entirely confident that participants completed the surveys entirely independently. Within the instructions, we asked participants to independently complete the survey and provide feedback. While it is possible that someone else could have completed the survey in place of the target participants, we do believe that the responses remained driven by the participants, even if a care partner assisted them in this process. In a future study, controls for this should be put into place to increase confidence in the independence of participant completion. A further limitation of this work is the process of recruitment via the aphasia database and contact via e-mail. While this process ensured that people eligible for participation in studies of aphasia were enrolled, this method also precluded individuals who may not utilize e-mail regularly or with ease. Future enrollment structures should therefore include in-person and multimodal options for contact. Due to this procedure, however, our confidence in validity of the participant response of an online survey is heightened, as this process has many parallels to the skills required to successfully respond to an e-mail.

Conclusions

The population of individuals with acquired neurogenic communication disorders, or other disorders that may secondarily impact speech, language, or communication, is underserved when it comes to systematically identifying and targeting communication needs that are personally relevant and aim to improve functional communication. Quantification of self-evaluated success is invaluable for planning person-centered speech-language therapy targets and evaluating the impact of behavioral intervention programs, or even possible pharmaceutical approaches that aim to mitigate disease impact. Here, we propose the development of the COMSS, a scoring system that results in a functional communication profile reflective of the patient's immediate communication concerns in terms of communicative form, modality, domain, and environment. Future work will be dedicated to the development of the reliability and validity of this screener, in strong partnership with PWA, to create a holistic and person-centered tool that evaluates representative communicative needs.

Data Availability Statement

The data sets generated during and/or analyzed during the current study are available in the Open Science Framework repository: https://osf.io/c4maz.

Appendix

The Communication Success Screener (COMSS)

Funding Statement

Brielle C. Stark is funded by the National Institute on Deafness and Other Communication Disorders (R01DC008524) and National Institute on Disability, Independent Living, and Rehabilitation Research (90SFGE0030). This work was made possible by the time and conversations that our clients and their families lent to share their experiences with the authors.