Effects of Written, Auditory, and Combined Modalities on Comprehension by People With Aphasia

Abstract

Background

People with aphasia experience reading challenges affecting participation in daily activities. Researchers have found combined auditory and written presentation modalities help people with aphasia comprehend contrived sentences and narratives, but less is known about the effects of combined modalities on functional, expository text comprehension.

Aims

This study's purpose was to examine comprehension accuracy, reviewing time, and modality preference of people with aphasia when presented with edited newspaper articles in written only, auditory only, and combined written and auditory modalities.

Method and Procedure

Twenty-eight adults with chronic aphasia read and/or listened to 36 passages. Following each passage, participants answered comprehension questions. Then, they ranked the modalities in accordance with preference and provided a rationale for their ranking.

Outcomes and Results

Comprehension accuracy was significantly better in the combined than auditory-only condition and in the written-only than auditory-only condition; the difference between combined and written-only conditions was not significant. Reviewing time differed significantly among conditions with the written-only condition taking longest and the auditory-only condition taking shortest. Most participants preferred the combined condition.

Conclusions

Access to combined modalities helps people with aphasia comprehend expository passages such as those found in newspapers better than auditory-only presentation. Furthermore, combined presentation decreases reviewing time from that needed for unsupported reading without compromising comprehension accuracy. Given that most participants preferred combined modality presentation, providing simultaneous auditory and written access to content through text-to-speech technology is a viable strategy when aphasia results in persistent reading challenges.

Reading is an expected activity of adults and is key to participating in social, leisure, and work activities (Kjellén, Laakso, & Henriksson, 2017; Parr, 1995; Smith, 2000; Webster, Morris, Howard, & Garraffa, 2018). Difficulty comprehending written texts can be a barrier to full societal participation and can prompt anger and frustration along with social exclusion, isolation, boredom, and depression (Chiou & Yu, 2018; Howe, Worrall, & Hickson, 2008; Parr, 2007). The result is a decrease in overall quality of life (Caute et al., 2016; Parr, 1996, 2007; Rose, Worrall, Hickson, & Hoffmann, 2011).

People with aphasia (PWA) frequently demonstrate reading deficits characterized by problems in both decoding words and comprehending written information (Knollman-Porter, Wallace, Hux, Brown, & Long, 2015; Leff & Behrmann, 2008). Materials easily accessed and understood prior to acquiring aphasia may now require considerable processing time and effort (Kjellén et al., 2017; Knollman-Porter et al., 2015). This change in ability leads to altered reading habits (Kjellén et al., 2017). Specifically, PWA tend to choose shorter texts and less complex materials, read less frequently, take longer to read passages, and have less independence than before acquiring aphasia (Knollman-Porter et al., 2015).

PWA rarely regain reading accuracy, speed, or pleasure commensurate with their premorbid level (DeDe, 2013; Holland, 2007; Parr, 1995; Pedersen, Vinter, & Olsen, 2003; Webb & Love, 1983). However, PWA still desire accessing written texts to gain information, stay updated about current events, assist with shopping, practice communication, connect with friends, and maintain control over their personal environment (K. Brown, Worrall, Davidson, & Howe, 2012; Knollman-Porter et al., 2015; Worrall et al., 2011). PWA also want to enjoy what they read, desire reading experiences that are personally relevant, and view literacy as an essential part of life (Elman & Bernstein-Ellis, 2006; Kjellén et al., 2017; Worrall et al., 2011). Therefore, finding ways for PWA to comprehend written material relatively quickly and efficiently is paramount for increased quality of life and community participation.

Functional Reading

Reading allows people to participate fully in society. One form of written material adults frequently encounter is expository text—that is, informational text that may include technical vocabulary, unfamiliar content, or facts (Roehling, Hebert, Nelson, & Bohaty, 2017). A common example is newspaper articles. Up to 96% of older adults read the newspaper often or on a regular basis (Champley, Scherz, Apel, & Burda, 2008), and newspapers and other periodicals are among the most commonly read materials by adults (Smith, 2000; White, Chen, & Forsyth, 2010).

Mixed findings exist about the relative difficulty of comprehending expository versus other types of text (e.g., narratives). Although researchers have not investigated this specifically with PWA, some have suggested that expository text is less difficult for unimpaired readers to comprehend than narratives (Champley et al., 2008; Zelinski & Gilewski, 1988); others contend the opposite because few causal connections and little predictability exists in expository passages (Britton, Glynn, & Smith, 1985; Budd, Whitney, & Turley, 1995; Hynd & Chase, 1991; Petros, Norgaard, Olson, & Tabor, 1989; Tun, 1989). Furthermore, expository texts rely heavily on working memory (Kintsch, Kozminsky, Streby, McKoon, & Keenan, 1975), which is a cognitive skill with which PWA may experience challenges (Caspari, Parkinson, LaPointe, & Katz, 1998; Downey et al., 2004; Friedmann & Gvion, 2003; Wright, Newhoff, Downey, & Austermann, 2003; Wright & Shisler, 2005). Regardless of its inherent difficulty, adults—including PWA—regularly encounter expository text and need to comprehend this type of written information for health, pleasure, and informational purposes.

Improvements in technology have prompted a steadily increasing number of older adults who access written materials via the Internet (Gell, Rosenberg, Demiris, LaCroix, & Patel, 2015; Zickuhr & Madden, 2012). With this, the amount of written text that PWA can access for communication and other needs has also increased. People use text messaging, the Internet, and e-mail to access and exchange information (Gell et al., 2015). Society's increasing reliance on technology-based written information will likely lead to an increased amount of information to which PWA have limited access unless they can use supports to improve their comprehension (Caute et al., 2016; Dietz, Ball, & Griffith, 2011).

Reading Supports

One way of promoting faster and more accurate comprehension of expository texts by PWA is through provision of reading supports. These can take the form of material modification or strategies to enhance decoding and comprehension directly.

Material Modification

PWA may choose to access modified written materials to enhance comprehension. Aphasia-friendly formatting is one form of material modification that can be helpful (Brennan, Worrall, & McKenna, 2005). This formatting includes abundant white space, large and clear fonts, and simple sentence structures and vocabulary. However, PWA often do not have immediate access to modified, aphasia-friendly texts because the high productivity demands placed on clinicians make modifying materials for single people unrealistic. Furthermore, given that many PWA want to read about personal friends, family, and community events (Knollman-Porter et al., 2015), texts that routinely appear in aphasia-friendly forms—such as shopping web pages or newspaper advertisements—may not match their reading interests. In short, PWA want to read the same materials as their peers (Elman & Bernstein-Ellis, 2006; Worrall et al., 2011). Finding a way for PWA to comprehend all written information easily and efficiently—particularly when aphasia-friendly formatting is unavailable—is ideal.

Support Strategies

PWA may use strategies to facilitate word decoding and reading comprehension. Some such strategies—such as reading aloud and rereading materials multiple times (Kjellén et al., 2017)—are independently implemented, but other strategies—such as having another person read materials aloud (Howe et al., 2008)—require the assistance of others. Partner assistance helps PWA comprehend when independent reading is too time-consuming or difficult (Knollman-Porter et al., 2015). The disadvantage is that such a strategy may promote feelings of being burdensome and force dependence on the availability and presence of others (Dalemans, de Witte, Wade, & van den Heuvel, 2010). PWA need support strategies that promote independent, efficient access to written materials.

One method of supporting independent processing of written information by PWA is through the simultaneous provision of written and auditory content via assistive technology devices. Because PWA have underlying cognitive deficits in attention, resource allocation, verbal memory, or processing speed that can cause or exacerbate linguistic processing impairments (Hula & McNeil, 2008; Majerus, Attout, Artielle, & Van der Kaa, 2015; McNeil, Odell, & Tseng, 1991; Neto & Santos, 2012), presenting information in two modalities simultaneously may lessen cognitive demands and improve overall reading performance (Hux, Weissling, & Wallace, 2008; Lasker, Hux, Garrett, Moncrief, & Eischeid, 1997).

J. Brown, Wallace, Knollman-Porter, and Hux (2019) found superior comprehension accuracy given combined auditory and written presentation when PWA chose pictures from selection sets of four to match written, spoken, or combined written and spoken sentences. Furthermore, the majority of participants preferred the combined written and auditory condition over either of the two single modality conditions. Another study by the same research group (Wallace, Knollman-Porter, Brown, & Hux, 2019) extended this work to examine the effects of single and combined modalities for narratives ranging from five to 10 sentences in length and Flesch–Kincaid (Flesch, 1948) grade equivalencies around 6.0. Participants performed with statistically greater accuracy when processing multisentence narratives given the combined modality than either of the single modalities and preferred having simultaneous presentation of auditory and written content. Although the parameters of the experimental stimuli Wallace and her colleagues used do not necessarily match those of expository texts PWA wish to read, initial research findings about the benefits of bimodal presentation for paragraph-length material are encouraging; implementing similar procedures with different text genres and texts of varying length may also be informative when trying to promote comprehension by PWA.

Text-to-Speech Conversion

One method of providing combined modality presentations to PWA involves text-to-speech (TTS) conversion available on electronic devices. Using a TTS conversion system allows for simultaneous access to written and auditory modalities but does not require manipulation of the reading material or reliance on other people to read the content; thus, PWA can use these systems independently via personal computers or electronic readers to access texts of interest to them. This may help PWA better comprehend information that would otherwise be difficult to understand in written form alone and to do so without having to rely on other people. Although PWA may have auditory comprehension deficits along with reading comprehension difficulties, researchers have established that they can comprehend computer-generated speech—which involves using a device to convert digital text to analog speech waveforms (Beukelman & Mirenda, 2013)—provided via popular operating systems (Hux, Knollman-Porter, Brown, & Wallace, 2017).

Many PWA who have participated in previous research studies have indicated regular use of computer-generated speech (Hux et al., 2017; Knollman-Porter et al., 2015; Wallace et al., 2019). However, the extent to which people with varying aphasia types and severities benefit from the TTS technology is not fully understood. What is known is that people with mild and moderate aphasia exhibit minimal performance differences across single and combined conditions when processing single sentences, but people with severe aphasia are significantly more accurate comprehending sentences given combined written and auditory presentation than given written-only presentation (J. Brown et al., 2019). Additional information about comprehension and processing efficiency benefits to people with different aphasia types and severities from single versus combined modality presentation of multisentence or multiparagraph expository texts is not yet available but is clinically important.

Personal Preferences of PWA

Considering the unique preferences and needs of a person with aphasia when recommending strategies and supports to help reading comprehension is important for facilitating functional and meaningful gains. Researchers have emphasized the importance of acknowledging the desires of PWA to define their own goals; collaborate with clinicians to resume relevant life activities; and close the gap between what they want, need, and currently can do (Byng & Duchan, 2005; Chapey et al., 2000; Duchan & Black, 2001; Helm-Estabrooks, Albert, & Nicholas, 2014). Choosing desirable strategies or supports increases the likelihood of subsequent use, which then leads to achieving functional gains (Dalemans, de Witte, Wade, & van den Heuvel, 2008; Wepman, 1953; Worrall et al., 2011). Thus, individual preferences should play a substantial role when choosing support strategies to help a person with aphasia comprehend written text.

This Study

Researchers have established that presenting content in combined rather than single presentation modalities can increase comprehension accuracy and reading efficiency by PWA (J. Brown et al., 2019; Wallace et al., 2019); PWA also prefer combined over single modality presentation (Wallace et al., 2019). However, materials used in extant research include only single sentences or relatively short narrative passages. Additional investigation is warranted to determine whether PWA retain these benefits and preferences when processing expository texts of comparable length and complexity to those encountered during everyday activities—such as content found in newspaper articles. Thus, the aim of this study was to evaluate the comprehension accuracy, reviewing time, and preferences of people with different types and severities of aphasia when processing expository passages derived from newspaper articles given written-only, auditory-only, and combined written and auditory modalities. All auditory presentation of stimuli used computer-generated speech from a TTS system to further validate the value of using this technology to promote independency by PWA. Specific research questions included the following:

1. How does the comprehension accuracy of PWA compare when processing edited newspaper articles in written-only, auditory-only, and combined written and auditory conditions?

2. How does the reviewing time of PWA compare when processing edited newspaper articles in written-only, auditory-only, and combined written and auditory conditions?

3. What are the presentation modality preferences of PWA when processing edited newspaper articles?

Method

Participants

Twenty-eight people with chronic aphasia participated in this study. The 17 male and 11 female participants ranged from 34 to 78 years of age (M = 59.43, SD = 10.98) and were between 8 and 253 months poststroke or encephalopathy (i.e., P2; M = 106.29, SD = 70.54). Years of education ranged from 12 to 18 (M = 14.89, SD = 2.11). Twenty-seven of the 28 participants were right-hand dominant prior to acquiring aphasia (exception: P24). All spoke American English as their primary language and, with the exception of two who wore audiologist-prescribed hearing aids, passed a hearing screening confirming perception in at least one ear of 1000-, 2000-, and 4000-Hz tones presented at 40 dB. The two participants using hearing aids demonstrated adequate hearing of conversational speech with the aids in place and had completed a hearing assessment by an audiologist within the past year. Participants also passed a vision acuity screening requiring 100% accurate identification of their first name each of 10 times it appeared in selection sets of five names; all names appeared in black, 24-point, Times New Roman font on a laptop computer. Each participant also completed a survey to provide information about technology use. Survey results revealed that 24 of 28 participants regularly used technology and 13 regularly used applications or programs with computer-generated voice output. Participant demographic information appears in Table 1.

Table 1.

Participant demographic information.

Participant	Gender	Age (years)	Education level (years)	TPO (months)	Currently receiving SLP services	Living status	Employment status	Regular technology use	Regular computer-generated speech use
P1	M	70	19	253	N	Independent	Retired	Y	N
P2	F	72	12	75	N	Spouse	Retired	N	N
P3	F	63	14	228	N	Spouse	Retired	Y	Y
P4	F	73	16	227	N	Spouse	Retired	Y	N
P5	M	68	18	22	N	Spouse	Retired	Y	N
P6	M	49	16	124	N	Spouse	Disability	Y	Y
P7	F	60	15	11	N	Spouse	Retired	N	N
P8	F	72	12	121	N	Partner	Retired	Y	Y
P9	F	34	18	40	N	Spouse	Disability	Y	Y
P10	F	59	12	73	N	Spouse	Retired	Y	N
P11	F	51	16	121	N	Children	Retired	Y	Y
P12	M	56	14	81	N	Parent	Disability	Y	Y
P13	M	47	12	108	N	Parent	Part time	Y	N
P14	F	53	14	17	N	Independent	Full time	Y	Y
P15	M	51	12	149	N	Independent	Retired	Y	N
P16	M	57	16	98	Y	Spouse	Retired	Y	Y
P17	M	46	16	183	Y	Independent	Part time	Y	Y
P18	M	48	16	91	Y	Parent	Retired	Y	Y
P19	F	59	18	128	Y	Spouse	Retired	Y	N
P20	M	70	16	194	N	Spouse	Retired	N	N
P21	M	54	12	154	Y	Family	Volunteer	Y	N
P22	M	59	12	181	Y	Independent	Part time	Y	N
P23	F	44	12	23	Y	Parent	Retired	Y	Y
P24	M	74	16	121	N	Independent	Retired	Y	N
P25	M	61	16	43	Y	Independent	Part time	Y	Y
P26	M	78	18	66	Y	Spouse	Retired	Y	N
P27	M	63	16	8	Y	Spouse	Retired	Y	N
P28	M	73	15	36	N	Spouse	Retired	Y	Y

Note. TPO = time postonset; SLP = speech-language pathologist; M = male; F = female; Y = yes; N = no.

We administered the Western Aphasia Battery–Revised (WAB-R; Kertesz, 2006); the Cognitive Linguistic Quick Test–Plus (Helm-Estabrooks, 2017); the Spoken Sentences, Spoken Paragraphs, and Written Sentences Comprehension subtests of the Comprehensive Aphasia Test (Swinburn, Porter, & Howard, 2004); and the Paragraph Factual subtest of the Reading Comprehension Battery for Aphasia–Second Edition (LaPointe & Horner, 1998) to gather information about participants' language and cognitive abilities. WAB-R Aphasia Quotient scores ranged from 15.6 to 97.0 (M = 66.77, SD = 21.75). Testing results for each participant appear in Table 2.

Table 2.

Participant standardized test scores.

Participant	Aphasia type as indicated by WAB-R scores	WAB-R	CLQT+ domains					CAT subtests			RCBA-2 subtest
		Aphasia Quotient (100)	Attention (215)	Memory (185)	Executive Function (40)	Language (37)	Visuospatial (105)	Spoken Sentences (32)	Spoken Paragraphs (4)	Written Sentences (32)	Paragraph Factual (10)
P1	Anomic	81.3	176	145	19	27	79	26	4	26	9
P2	Transcortical sensory	67.1	156	106	13	12	72	22	3	24	7
P3	Conduction	72.3	197	144	25	26	93	25	4	20	9
P4	Broca's	27	136	66	8	1	66	21	2	22	9
P5	Conduction	66.2	63	95	20	19	53	29	3	23	9
P6	Conduction	69.6	200	144	32	26	102	28	4	26	9
P7	Anomic	93.1	166	169	13	31	13	22	4	26	10
P8	Broca's	55.5	125	92	18	17	68	19	4	20	9
P9	Anomic	92.2	203	160	27	27	99	23	4	22	10
P10	Broca's	64.6	176	162	22	27	86	16	4	24	10
P11	Broca's	46.9	182	108	24	12	95	20	1	22	9
P12	Broca's	67.8	192	146	24	23	94	25	3	28	10
P13	Wernicke's	64.2	188	126	25	23	91	19	2	20	8
P14	Anomic	96.8	194	170	30	23	94	32	4	32	10
P15	Broca's	66.6	165	142	22	22	88	16	1	13	7
P16	Broca's	34.9	164	77	16	7	76	16	2	12	3
P17	Broca's	59.7	147	82	17	16	67	16	3	10	4
P18	Broca's	48.3	184	88	19	11	92	16	2	14	9
P19	Anomic	96.8	95	162	29	29	75	24	4	24	10
P20	Anomic	97.2	176	127	21	23	83	26	3	18	4
P21	Conduction	45	155	90	22	16	67	10	3	9	7
P22	Broca's	61.5	192	91	26	15	94	15	2	13	8
P23	Broca's	15.6	172	72	17	2	83	23	4	19	9
P24	Anomic	88.3	182	150	23	28	84	30	3	28	9
P25	Anomic	97	192	156	33	33	93	29	3	29	10
P26	Conduction	60.7	124	57	13	13.5	51	15	2	15	5
P27	Conduction	80.6	201	158	31	29.5	99	18	3	24	10
P28	Broca's	52.6	73	53	20	7	60	28	4	24	9

Note. WAB-R = Western Aphasia Battery–Revised; CLQT+ = Cognitive Linguistic Quick Test–Plus; CAT = Comprehensive Aphasia Test; RCBA-2 = Reading Comprehension Battery for Aphasia–Second Edition.

Materials

Study materials included visual and auditory presentations of edited passages selected from newspaper articles, researcher-generated comprehension questions relating to each passage, a laptop computer with E-Prime 3.0 software for stimulus presentation, and condition ranking materials.

Passages and Questions

We wanted experimental stimuli that reflected what PWA would encounter in real life but that were also of lengths and grade levels realistic for comprehension by people with language challenges. We chose newspaper articles that had between 180 and 220 words or that could be edited to this length so that our experimental stimuli were comparable to expository texts found in reading materials typically accessed by adults; the 180- to 220-word length was consistent with short articles we found in online newspapers. We determined a target grade equivalency range for our stimuli by evaluating the grade level of 69 randomly selected, unedited newspaper articles. These articles had grade equivalencies ranging from 6.5 to 13.0 (M = 9.93, SD = 1.69). Based on this finding and to maintain relative conformity across stimuli, we stipulated a grade equivalency between 9.0 and 11.0 for each passage.

We gathered and edited 39 newspaper passages to serve as stimuli. We used three as practice items and 36 as experimental items. We made no changes to the original articles other than shortening them by removing complete sentences and separating the remaining sentences into three logical paragraphs. For three stories, we changed sentence order to maintain flow and coherence. We made these editing changes purposefully to ensure the experimental materials were consistent with newspaper articles widely available to the general public yet still adhered to our desired length and grade-level criteria.

All stimulus passages came from U.S. newspapers available online and met the following criteria: (a) did not relay stories local to the region in which participants lived, (b) did not report national news likely to be familiar to some or all participants, (c) did not convey general knowledge information, (d) included nine or more sentences logically comprising three paragraphs, (e) only used acronyms assumed to be known by all (e.g., “U.S.”), (f) either contained between 180 and 220 words or could be logically shortened to that length, (g) included no more than two quotes, and (h) had a Flesch–Kincaid (Flesch, 1948) grade equivalency between 9.0 and 11.0. Following modification, the average number of words per passage was 204.6 (SD = 12.83), and the average grade equivalency was 10.11 (SD = 0.62). The passages ranged from nine to 15 sentences (M = 11.08, SD = 1.38).

We generated eight comprehension questions for each passage. All questions appeared as an incomplete sentence with the final word or phrase missing. Each statement was factual in nature and used words and phrases extracted from the passage. The target response and three foil responses appeared in a vertical list with the target response occurring in each of the four positions approximately the same number of times. The first question was always a gist question targeting the overall story meaning or theme. Remaining questions addressed explicit details stipulated in the story. An example passage not used in the study secondary to copyright issues but representative of the actual stimuli appears in the Appendix.

We performed a dependency analysis for each of the 36 stories and 288 questions comprising experimental stimuli using the passage dependency index—a computation yielding a value between −1.0 and 1.0 and for which higher values indicate greater need to rely on passage content to respond accurately to comprehension questions (Tuinman, 1974). The formula for computing the index is one minus the quantity of the proportion of correct responses without reading the passage divided by the proportion of correction responses having read the passage. The goal was to ensure that a substantial majority of adults without reading difficulties could correctly answer the comprehension questions after reading the passage but could not correctly guess the answers at a level substantially above chance prior to reading it. This meant that story content was not overly predictable and that one question did not inadvertently give away answers to subsequent questions. For this analysis, adults who self-reported via a questionnaire an absence of neurological impairments and reading difficulties answered the comprehension questions either before or after having read the associated passage. Validation criteria were that (a) at least 40% of adults selected an incorrect answer for each comprehension question when they had not first read the passage and (b) at least 85% of adults selected the correct answer for each question after reading the passage. Using these criteria ensured that the passage dependency index for each question was at least 0.53—a value consistent with those used by other researchers examining adult reading comprehension (Dietz, 2014; Dietz, Hux, McKelvey, Beukelman, & Weissling, 2009; Tian, 2006; Wallace, Dietz, Hux, & Weissling, 2012). Validation of the 36 passages and associated questions required four rounds of dependency analysis procedures involving 216 adults between the ages of 19 and 90 years (M = 49.04, SD = 13.05) and with 12–20 years of education (M = 16.54, SD = 2.05).

Stimulus Presentation

We used a Dell touch screen laptop computer to present the experimental stimuli to participants. We programmed E-Prime 3.0 to present the experimental stimuli in three conditions: written only, auditory only, and combined written and auditory. Using E-Prime allowed for controlled and systematic stimulus presentation along with data logging of response times and response accuracy as participants progressed through the experiment. We created nine E-Prime sets containing four passages in each of the three conditions, equaling 12 passages per set. Participants completed the experimental task across three sessions with a different set of 12 passages each time. The creation of nine E-Prime sets ensured that, across all participants, each story appeared an equal number of times in each condition. We used each grouping of three E-Prime sets (i.e., Sets 1, 2, and 3; Sets 4, 5, and 6; and Sets 7, 8, and 9) one time before repeating that grouping with a later participant. Figure 1 provides a visual representation of the distribution of stimulus passages across conditions and participants.

Figure 1.

Thirty-six validated experimental stimulus passages randomized across nine E-Prime sets.

The presentation of each condition in E-Prime began with a practice passage with three questions. After that, four passages with eight comprehension questions each appeared. We programmed E-Prime to first randomize each condition and then randomize each of the four passages within the condition to ensure that every participant received the stimuli in a unique order. Each validated passage appeared in black, 24-point, Times New Roman font centered on a white screen for presentation in the written-only and combined written and auditory conditions; a blank white screen appeared during the playback of passages during the auditory-only condition. A green icon labeled “Done” appeared in the bottom right-hand corner of each passage screen. Touching this icon progressed the E-Prime program to the screen, displaying the first comprehension question. An example screen display appears in the Appendix.

Auditory presentation used TTS generation of the David voice via a PC platform computer. Participants heard auditory stimuli via computer speakers and could adjust the volume to their desired level. We selected the David synthetic voice because Hux et al. (2017) found that PWA could understand it and preferred it to another commonly used computer-generated voice. We selected the default speed for auditory stimulus presentation in the auditory-only and combined written and auditory conditions. This resulted in a range of 127.42–159.47 words per minute (M = 145.37, SD = 10.00) across the 36 passages, with the range of passage audio length being 70–103 s (M = 84.81, SD = 7.13). The voice began speaking the passage following 1 s of silence.

Each comprehension question appeared on a white screen in black, 24-point, Times New Roman font following the presentation of a passage. Only one question appeared at a time.

Session Recording

We used a Canon HF R700 or R800 video camera to capture each session. We positioned the camera behind the participant to capture an over-the-shoulder shot of the laptop screen. This allowed for audio recording of participant comments throughout the experimental procedures. It also allowed identification of selected answers in two instances in which this aspect of E-Prime data collection failed.

Procedure

Institutional review boards at both universities at which data collection occurred approved all methods and procedures prior to recruitment. Participants performed the experimental task across three sessions after completing inclusionary/exclusionary screening procedures. Whenever available, we obtained diagnostic testing results for a participant from previous study records or clinic records. If we did not have assessment results available from these sources or the testing occurred more than 1 year prior to the start of data collection, we administered the WAB-R to determine aphasia type and the Aphasia Quotient during the first experimental session and supplemental testing, as needed, during the first and/or successive experimental sessions. For the experimental task, we read aloud the written instructions appearing on the laptop screen. A practice passage matching the subsequent condition provided an opportunity for a participant to adjust the volume of computer-generated speech output to a desired level, practice using the touch screen to respond to comprehension questions, and verify comprehension of the task instructions. After the practice questions, participants touched a “Continue” icon located near the bottom of the screen to advance to the first experimental passage. This allowed a participant to control the rest time between the practice and experimental passages. In the written-only and combined conditions, the text remained visible for whatever length of time desired by a participant, thus allowing for multiple readings of the text. Participants either read the text silently or aloud based on their preference. The auditory recordings occurred only once in the combined and auditory-only conditions. When participants finished reviewing a story, they pressed the “Done” icon to advance the program to the first comprehension question.

Participants could not refer back to a passage when answering questions. Use of a repeated-measures design in which each participant performed the experimental task in all conditions helped to control for potential variations in memory ability among participants. To minimize recall demands, comprehension questions appeared immediately following review of each associated passage. We read each question and response options aloud to participants unless they requested to read them independently. Participants responded by touching the circle to the left of their desired response. Participants could change their answer multiple times before progressing to the next question; however, they could not return to previous questions after having advanced to a subsequent one. The researcher advanced to the next question after the participant selected a final answer. Participants could request breaks between passages or conditions.

Following completion of four passages in each of the three conditions, we asked participants which of the three conditions they most and least preferred. Participants provided explanations for their preferences when possible given their speech and language challenges. We recorded responses for later analysis.

Data Analysis

Variables of interest included (a) comprehension accuracy, (b) passage reviewing time, (c) aphasia severity, (d) frequency of use of computer-generated voice technology, and (e) participant condition preference.

Comprehension Accuracy

We computed a repeated-measures analysis of variance (ANOVA) to identify significant accuracy differences across the written-only, auditory-only, and combined written and auditory conditions. We used the Huynh–Feldt correction because Mauchly's test of sphericity yielded a significant result, X²(2) = 9.310, p = .010, ε = .769. We then calculated post hoc analyses of pairwise comparisons using least significant difference (LSD) values to determine percentage accuracy differences across conditions.

Passage Reviewing Time

We defined reviewing time as the length of time in seconds participants spent with a passage. Depending on the condition, this meant either the time a participant spent reading the written text, listening to the audio, or both. Timing ended when the participant independently pressed the “Done” icon on the screen, thus advancing the program to the first comprehension question. As with percentage accuracy, we performed a repeated-measures ANOVA and post hoc analyses of pairwise comparisons using LSD values to determine reviewing time differences across conditions. We used the Greenhouse–Geisser correction because computation of Mauchly's test of sphericity yielded a significant result, X²(2) = 30.277, p < .001, ε = .604.

Relation Between Performance and Individual Factors

We performed a correlation analysis using Pearson correlation coefficient procedures to evaluate relations between the WAB-R Aphasia Quotient and comprehension accuracy and reviewing time across conditions. Additionally, we separated participants into two groups based on whether they did or did not self-report regular use of the TTS technology and computed independent-samples t tests to determine if familiarity with computer-generated speech influenced accuracy and reviewing time. Results of six separate Levene's tests of equality of variance confirmed the data did not violate homogeneity of variance assumptions either for accuracy (i.e., reading [F = 0.251, p = .621], auditory [F = 0.095, p = .760], combined [F = 0.251, p = .621]) or reviewing time (i.e., reading [F = 2.29, p = .142], auditory [F = 0.992, p = .328], combined [F = 0.13, p = .910]); thus, we used a parametric rather than nonparametric statistic for this computation.

Participant Condition Preference

We performed a descriptive analysis of condition preference by tallying the number of participants who selected each condition as their most and least preferred.

Results

Comprehension Accuracy

Depiction of the comprehension accuracy data appears in Figure 2. Participants achieved the highest accuracy in the combined written and auditory condition. The next highest accuracy occurred in the written-only condition, and the lowest accuracy occurred in the auditory-only condition. Computation of a repeated-measures ANOVA with Huynh–Feldt correction revealed a significant main effect across conditions, F(1.61, 43.52) = 20.607, p < .001, η_p ² = .433. Post hoc analyses using LSD procedures revealed significant differences between the written-only and auditory-only conditions, p < .001, and the auditory-only and combined conditions, p < .001. The written-only and combined conditions did not differ significantly, p = .495.

Figure 2.

Participants' average percent comprehension accuracy across conditions.

Passage Reviewing Time

Depiction of the reviewing time data appears in Figure 3. Participants had the longest reviewing time in the written-only condition. This was followed by the combined and auditory-only conditions, respectively. Computation of a repeated-measures ANOVA with Greenhouse–Geisser correction revealed a significant main effect across conditions, F(1.19, 33.99) = 13.94, p < .001, η_p ² = .341. Post hoc analyses using LSD procedures revealed significant differences between written-only and auditory-only conditions, p < .001; written-only and combined conditions, p = .006; and auditory-only and combined conditions, p < .001.

Figure 3.

Participants' average reviewing time across conditions.

Relation Between Performance and Individual Factors

Aphasia Quotient Scores

Results revealed significant correlations at the .01 level between WAB-R Aphasia Quotient scores and all accuracy conditions: written only (r = .563, p = .002), auditory only (r = .544, p = .003), and combined (r = .553, p = .002). In contrast, no correlation occurred between Aphasia Quotient scores and reviewing time: written only (r = .127, p = .518), auditory only (r = .211, p = −.282), and combined (r = .309, p = .109).

Computer-Generated Voice Exposure

When considering familiarity with computer-generated voice production, computation of independent-samples t tests revealed that accuracy (written only: t(26) = 0.306, p = .726; auditory only: t(26) = −0.107, p = .915; combined written and auditory: t(26) = 0.336, p = .739) and reviewing time (written only: t(26) = −0.095, p = .925; auditory only: t(26) = 1.69, p = .102; combined written and auditory: t(26) = 0.045, p = .965) did not differ significantly between the 13 participants who regularly used computer-generated voice and the 15 who did not.

Participant Condition Preference

We used the preference ranking provided after the final session to report participants' overall most preferred and least preferred conditions. We selected this ranking because it was the one generated when participants had the greatest amount of exposure to the three conditions. Data from one participant who ranked two of the conditions as equally preferred were not included in the preference numbers. Twenty-two of the 27 remaining participants (81.48%) ranked the combined condition as most preferred, four (14.81%) ranked the written-only condition as most preferred, and one (3.70%) ranked the auditory-only condition as most preferred. Seventeen participants (62.96%) ranked the auditory-only condition as their least preferred, seven (25.92%) ranked the written-only condition as their least preferred, and three (11.11%) ranked the combined condition as their least preferred (see Table 3). Rationales for preferring the combined condition referenced short reviewing time and ease of understanding (e.g., P14: Easier and more accurate…faster; P5: Having one was not enough). In contrast, participants reported greater difficulty when information appeared in a single modality, (P13: I didn't like that at all…The words…were so fast; P6: Terrible.)

Table 3.

Participant condition preference.

Modality	First choice	Second choice	Third choice
Written only	4	16	7
Auditory only	1	9	17
Combined written and auditory	22	2	3

Note. The data from one participant who ranked two of the conditions as equally preferred were not included in the preference numbers. Bolded numbers represent the most frequent condition selected first, second, and third by participants.

Discussion

PWA frequently experience persistent reading deficits that make decoding and/or comprehending written text difficult; as a result, PWA expend greater time and more effort than needed before acquiring aphasia to process written materials, and they may abandon reading activities that prove too challenging (Knollman-Porter et al., 2015). Presenting materials in combined modalities via the TTS technology may help PWA process information faster and with greater accuracy and independence than occurs given single modality presentation. Findings from previous studies suggest PWA perform better with combined modalities given sentence-level information or relatively long narrative passages (J. Brown et al., 2019; Wallace et al., 2019). The unique contribution of the study presented herein is determination of whether comparable benefits exist for expository materials similar to those frequently encountered when reading newspapers. Finding a way for PWA to use technology to support quick and accurate decoding and comprehension of written materials encountered during daily activities is important because this promotes independent participation in social, work, and leisure activities and may improve quality of life.

Comprehension Accuracy

Results from the current study support the findings of previous researchers who have reported that access to combined rather than single modality content presentation promotes comprehension by PWA (Hux et al., 2008; Lasker et al., 1997; Wallace et al., 2019). Once again, study participants were most accurate in comprehending materials presented in the combined written and auditory condition. More specifically, although the combined and written-only conditions did not yield significant accuracy differences, the combined versus auditory-only differences reached significance. This is consistent with what Wallace et al. (2019) found with relatively long narratives; J. Brown et al. (2019) reported a significant benefit for combined presentation of single sentences over both auditory-only and written-only presentation. This improvement, given the redundancy of combined modalities, may reflect a decrease in cognitive load facilitated by reducing the amount of information needed to be held in working memory and the freeing of cognitive resources for other processing needs (Wallace et al., 2012; Waller & Darley, 1978).

Results from the current study are consistent with those of Wallace et al. (2019) and suggest that combined modalities may support auditory comprehension more than they support reading comprehension when PWA attempt to process multisentence and multiparagraph materials. However, the lack of a significant finding between combined presentation and written-only presentation may be an artifact of the study procedures and thus requires cautious interpretation. Specifically, the presentation of auditory stimuli was limited to a single presentation both in the Wallace and current studies, but the written text remained available for multiple reviews if desired by the participant. Such procedures simulate the comprehension demands experienced by PWA during spoken conversations in which a speaker does not repeat himself/herself and no simultaneous written support is available. They are also consistent with the demands experienced when unlimited time and access allow repeated review of written materials. These procedures, however, limit direct comparison of the benefit afforded by combined modalities over auditory-only versus written-only presentation. Had we limited access to written passages in the written-only and combined conditions to a single review, differential comprehension accuracy may have resulted and distinguished the benefit provided by having simultaneous auditory and written content. Further investigation is warranted to ascertain the veracity of this notion. Regardless, the results confirm that assistive technology devices are viable means of presenting combined auditory and written content to facilitate comprehension and that combined modalities are superior than auditory presentation alone for PWA.

Passage Reviewing Time

Time spent reviewing passages is important because, when considered in combination with accuracy, it represents reading efficiency (Freeland, Skinner, Jackson, McDaniel, & Smith, 2000). PWA often take exponentially more time to decode and comprehend written text than they did prior to acquiring aphasia. Struggles decoding and comprehending can lead to fatigue, frustration, and task abandonment. In addition, PWA may choose to sacrifice reading accuracy to complete a reading task in what they perceive is a reasonable length of time (Knollman-Porter et al., 2015). Because of these issues, finding methods to help people maintain or improve comprehension while simultaneously decreasing the time required for task completion may be important for PWA.

The written-only and combined presentation conditions in the current study did not yield significant differences with respect to comprehension accuracy; however, they did yield significant results for the length of time PWA spent reviewing stimulus material. Specially, having simultaneous access to auditory and written content significantly shortened the length of time spent reviewing the content as compared to solely reading the passage, making performance more efficient in comparison to reading alone. Furthermore, the shortened reviewing time facilitated by having simultaneous auditory and written presentation did not result in compromised comprehension. This finding is consistent with those found by Harvey, Hux, and Snell (2013) when examining a single case of aphasia and cognitive impairment secondary to tumor resection and Wallace et al. (2019) when examining people with chronic aphasia. One reason for the apparent speed benefit of combined modalities may be that TTS output allows PWA to maintain momentum while reading, thus curbing the tendency to focus on decoding individual words not read automatically (Caute et al., 2016). This result suggests that using the TTS technology to provide combined written and auditory content presentation may facilitate greater comprehension efficiency when PWA must process lengthy texts relatively quickly.

A number of factors may have contributed to the differing lengths of passage review time noted across experimental conditions. One possibility already mentioned relates to the study procedures in that participants could review the text in the written-only and combined conditions multiple times and for whatever length of time they desired but could only listen to the auditory presentation one time. Future researchers may wish to eliminate this possibility by providing access to multiple repetitions of auditory content while continuing to allow unlimited time to review written content as would be feasible given some assistive technology applications. Another possibility is that participants may have opted to focus on only one of the two presentation modalities available in the combined condition. Thus, some people may have only listened during this condition while ignoring the presented text, some may have only read while choosing not to listen to the auditory presentation, and some may have done both simultaneously. A third and related possibility is that participants may have abandoned the attempt to process information in a given condition if they felt it was too difficult. Such behavior is not desirable in terms of experimental control and was not measured in the context of this study but is probably representative of strategies PWA routinely employ when confronted with language tasks they perceive as exceeding their capabilities. Future studies using technology to measure variables not considered in the present work (e.g., eye gaze) may reveal how PWA respond behaviorally when presented with different conditions.

Participant Condition Preference

Individual preference is an important factor when helping PWA choose support strategies. People are more likely to utilize strategies that they like and perceive as being helpful (Lasker & Bedrosian, 2000). Although the current participants as a group preferred the combined written and auditory modality, the selection of this option was not universal. Of the participants who selected one of the single modality conditions as most preferred, we observed no trends in preference based on aphasia type or severity. Also of note is the fact that not every person performed best in their most preferred condition. Variations across participants regarding preference, comprehension accuracy, and reviewing time highlight the need for clinicians to have individualized discussions with PWA about the benefits and drawbacks of various support strategies. As much as possible, clinicians should allow a person to choose for himself/herself the best method of maximizing reviewing time while simultaneously ensuring adequate comprehension. Clinicians need to consider, however, that some PWA have limited awareness of their deficits, and this may cause them to deny difficulties, underutilize resources, or request help inconsistently because they do not realize when comprehension problems occur (Kertesz, 2006; Kertesz & McCabe, 1977; Knollman-Porter, Dietz, & Dahlem, 2018; Lebrun, 1987). More specifically, PWA may not realize which of several strategies influence comprehension accuracy most positively and decrease reviewing time, so practitioners need to provide this information to support the decision-making process. The fact that we did not supply postexperimental accuracy or reviewing time information to study participants may have affected their selection of most and least preferred presentation conditions. Future researchers may wish to modify the procedures by providing participants with performance data prior to requesting a preference ranking.

Relation Between Performance and Individual Factors

Examining the relation both of Aphasia Quotient scores and frequency of technology use to comprehension accuracy and reviewing time is important because individual differences on these variables may help explain obtained findings. One method traditionally used to measure aphasia severity is the WAB-R Aphasia Quotient score (Kertesz, 2006). The WAB-R manual provides guidelines for classifying aphasia severity.

The correlation analysis performed with the current study data revealed a positive relation between WAB-R Aphasia Quotient scores and comprehension accuracy; thus, higher WAB-R scores corresponded with better comprehension accuracy on the experimental task regardless of condition. Although these results are not surprising, people with lower Aphasia Quotient scores—suggestive of more severe aphasia—may not fully realize the beneficial effects of particular supports on comprehension accuracy. Also important is the idea that differing aphasia severities may influence both the time needed to process information and the accuracy with which people can comprehend written materials regardless of presentation modality. For example, access to combined modalities may not lead to 100% accuracy by people with severe aphasia, but it may improve comprehension to a greater degree than content presentation in single modalities. Further exploration of the acceptance of improved but not perfect comprehension by PWA is warranted.

Technology use differences may also impact the extent to which people are familiar with and have had exposure to a variety of computer-generated voices. Previous researchers have documented that people with repeated exposure to computer-generated voices perform better on word recognition and comprehension tasks and display better long-term retention of presented content than people without such exposure (Rounsefell, Zucker, & Roberts, 1993; Schwab, Nusbaum, & Pisoni, 1985; Venkatagiri, 1994). However, this has not been evaluated for PWA nor does evidence exist to support the effect of repeated exposure using the many computer-generated speech options available today.

Performance of the analysis relating to computer-generated speech familiarity and comprehension accuracy revealed no systematic relation between these variables. This contrasts with findings from other researchers suggesting that repeated exposure to synthesized speech benefits comprehension by neurotypical adults (McNaughton, Fallon, Tod, Weiner, & Neisworth, 1994) and PWA (Huntress, Lee, Creaghead, Wheeler, & Braverman, 1990). The disparate finding may reflect the fact that we did not query participants about specific voices with which they were familiar when we asked about technology usage. We utilized the David voice via PC platform technology because our previous research established that many PWA could comprehend it with comparable accuracy as digitized natural speech (Hux et al., 2017), but multiple other computer-generated voices are available and in widespread use. Given substantial quality differences across the array of computer-generated voice options currently available, repeated exposure to one version of synthetic speech may not generalize to improved comprehension of all voices. Therefore, researchers need to examine further the influence of repeated exposure to various versions of computer-generated speech on comprehension by PWA, and they need to investigate the extent to which exposure to one voice generalizes to improved comprehension of other voices. The findings of such investigations will inform practitioners about whether repeated exposure facilitates comprehension by PWA who intend to use devices and applications with computer-generated speech output. The goal is to minimize the extent to which PWA must devote cognitive resources to decoding the acoustic–phonetic structure of synthetic speech, instead allowing a focus on the comprehension of conveyed content.

Limitations

We strove to examine the comprehension accuracy, reviewing time, and preferences of PWA when processing expository texts consistent with materials they might encounter in daily life. Still, we strategically chose and modified newspaper passages to ensure consistent grade levels and lengths and to minimize the presence of quotes and acronyms to maintain high levels of internal validity. These criteria may have resulted in our selection of materials that promoted better processing than PWA would experience with unmodified articles found online or in hard copies of newspapers. Hence, our stimulus selection presents a limitation to the generalizability or external validity of the research findings. Additional investigation is warranted to explore further the effects of combined presentation modalities on materials PWA routinely encounter that are not modified in any fashion and that represent personally relevant or preferred content.

Another limitation to the study is that we used the David voice available via the PC platform for the auditory output. Computer-generated speech, including that produced by David, may not consistently read words or symbols as humans would, especially regarding word timing and stress, abbreviations, and certain numbers (e.g., 1,000 read as one, zero, zero, zero). Although discrepancies between human and computer-generated readings of experimental passages may have negatively affected comprehension, we chose not to modify any TTS output errors or irregularities because they reflect what PWA would experience when using the technology in real-life situations. Computer-generated voices continue to improve on an on-going basis, and future researchers may wish to select different speech output when evaluating processing by PWA.

An additional limitation of the current study concerns the length of time participants spent reviewing texts. Because participants could reread experimental passages both in the written-only and the combined written and auditory conditions but could only listen one time in the auditory-only condition, reviewing times were not completely at the discretion of participants. This procedure may have resulted in shorter reviewing times in the auditory-only and/or combined conditions than would have occurred had participants been allowed to listen to passages repeatedly. We also did not allow participants to isolate specific words or phrases within a passage that they may have wished to hear multiple times as would typically be possible when using the TTS technology. These procedural limitations were purposeful on our part to maximize experimental control; however, they restricted the extent to which participants could take full advantage of the TTS technology features and, as such, limit the generalizability of the findings.

We noted previously that participants' unlimited access to the written text in the written-only and combined conditions may have contributed to comprehension accuracy differences across conditions. For the current study, participants could independently choose whether to reread a passage prior to answering comprehension questions. Some, but not all, participants chose to reread written passages; however, we did not systematically document by whom and how often rereading was utilized. For this reason, we are uncertain of the potential influence of multiple reviews on comprehension accuracy. Future researchers may wish to maintain consistency across conditions regarding participants' ability to review experimental stimuli multiple times or identify methods to measure rereading.

Finally, we could not complete a detailed analysis of the effects of varying aphasia types and severities on comprehension and reviewing time because of insufficient participant numbers. Studies with larger numbers of participants exhibiting specific aphasia types or severities may reveal differences in the extent to which people benefit from combined versus single modality presentations. Such findings may contribute to minimizing the time practitioners must spend administering assessments prior to introducing supportive comprehension strategies.

Conclusion

PWA can benefit from having access via the TTS technology to combined written and auditory modalities when processing multisentence expository texts similar to those found in many newspaper articles. Although a significant comprehension accuracy difference did not appear between written-only and combined written and auditory content presentations, the comprehension accuracy difference between auditory-only and combined conditions was significant. Furthermore, PWA took less time to review materials presented simultaneously via the TTS technology in two modalities rather than as written-only content, which suggests greater efficiency given combined modalities. The fact that utilizing the TTS technology can provide a means of achieving improved comprehension and efficiency without having to rely on other people to read content aloud is an additional benefit. Additionally, participants largely preferred the combined modality condition, thus lending further support to the value of this presentation option. Overall, the findings suggest that PWA can benefit from having simultaneous access to spoken and written renditions of functional expository text afforded by the TTS technology; however, clinicians need to consider accuracy, reviewing time, and individual preferences carefully when making suggestions about using the TTS technology to support comprehension.

Appendix

Example Passage

Funding Statement

Research reported in the article was supported by the National Institute on Deafness and Other Communication Disorders under Award 1R15DC015579 (awarded to all the authors). The content is solely the responsibility of the authors and does not necessarily represent the view of the National Institutes of Health.