A Comparison of Three Discourse Elicitation Methods in Aphasia and Age-Matched Adults: Implications for Language Assessment and Outcome

Abstract

Purpose

Discourse analysis is commonly used to assess language ability and to evaluate language change following intervention in aphasia. The purpose of this study was to identify differences in language produced during different discourse tasks in a large aphasia group and an age- and education-matched control group.

Method

Four structured discourse tasks across 3 discourse types (expositional, narrative, and procedural) were evaluated in a group of adults with aphasia (n = 90) and an age-matched control group (n = 84) drawn from AphasiaBank. CLAN software was used to extract primary linguistic variables (mean length of utterance, propositional density, type–token ratio, words per minute, open–closed class word ratio, noun–verb ratio, and tokens), which served as proxies for various language abilities. Using a series of repeated-measures analyses of covariance, with significantly correlated demographic and descriptive variables as covariates, main effects of discourse type were evaluated.

Results

Despite an impoverished output from the aphasia group (i.e., the control group produced significantly more overall output), there was a main effect of discourse type on most primary linguistic variables in both groups, suggesting that, in adults with and without language impairments, each discourse type taxed components of the spoken language system to varying extents. Post hoc tests fleshed out these results, demonstrating that, for example, narrative discourse produced speech highest in propositional density.

Conclusion

Each discourse type taxes the language system in different ways, verifying the importance of using several discourse tasks and selecting the most sensitive discourse tasks when evaluating specific language abilities and outcomes.

Although assessment of language abilities following aphasia often involves standardized testing batteries, linguistic discourse analysis provides a supplementary assessment that allows for the identification of impairments within a more naturalistic and ecologically valid domain (i.e., connected speech), allowing for the detection of additional difficulties (e.g., syntactic organization, word order, cohesion, and coherence) and compensatory strategies (e.g., circumlocution, self-cueing, retracing). Furthermore, unlike assessment of single-word retrieval, such as object naming, assessment of discourse allows for evaluation of the independence and interaction of different linguistic processes (e.g., phonology, morphosyntax, lexical semantics; Prins & Bastiaanse, 2004). Indeed, a recent review by Bryant, Ferguson, and Spencer (2016) noted that 86% of clinicians (of a 123-clinician sample drawn from five English-speaking countries) reported analyzing discourse within a clinical setting. Analyzing discourse at baseline and postintervention serves an important purpose: assessing efficacy and generalization of impairment-based treatments. In a systematic review of discourse analysis in aphasia including 165 treatment studies, spoken discourse analysis was used in 87 studies: as a primary outcome measure in 36, a secondary outcome measure in 19, and a measure of generalization in 37 (Bryant et al., 2016). However, the current state of research in spoken discourse analysis suffers from considerable flaws (Kintz & Wright, 2017). One flaw is the reliance on a single type of discourse elicitation method, and another concerns the consistency and appropriateness of the linguistic variables measured from the piece of discourse. The current project specifically addresses these concerns.

Spoken language can be elicited using a variety of discourse methods (for a review, see Bryant et al., 2016). The most common structured method is the expositional narrative, which provides subjects with a picture or picture sequences to describe. In the same systematic review of 165 studies, Bryant et al. (2016) noted that, across studies using expositional narrative, most studies used only one type of exponential narrative to acquire a language sample. The majority of studies analyzed a single picture description, such as the Cookie Theft Picture (Swinburn, Porter, & Howard, 2004). Another common method of discourse elicitation is narrative discourse, which involves the recounting of a personal story or the retelling of a well-known story, usually retold without visual aids. Narrative discourse, because of its reliance on memory and macrolinguistic structures such as story grammar and coherence, may elicit more complex language compared to expositional narrative (MacWhinney, Fromm, Holland, Forbes, & Wright, 2010). However, because a narrative discourse does not typically use visual aids, this type of discourse method may be inherently more difficult and therefore produced highly variable language for those who rely more on visual cues for lexical retrieval, as in severe aphasia. Finally, another common method of discourse elicitation is procedural discourse or the describing of a procedure/task (“tell me how to make a peanut butter and jelly sandwich”). Procedural discourse is thought to elicit more action words and gestural communication (Pritchard, Dipper, Morgan, & Cocks, 2015).

However, of the studies evaluating discourse in aphasia, few analyze more than one of these types of discourse, despite the suggestion from Brookshire and Nicholas (1994) that a combination of multiple structured language samples should be used because they generate a comprehensive language sample most resemblant of actual language use. In addition, given the variability commonly shown in aphasic performance within and across sessions, analyzing language from a single discourse method may not demonstrate the breadth of an individual's linguistic ability or be sensitive to linguistic changes postintervention. An additional issue concerns the consistency and appropriateness of the linguistic variable(s) being measured. When analyzing spoken language from a single type of discourse method, the extent to which different parts of the language system are taxed is likely highly dependent on the discourse method. For example, as stated above, narrative discourse may produce more syntactically complex language than procedural discourse. Therefore, there is a need to identify the extent to which each discourse method taxes parts of the spoken language system. Doing so has direct implications for research and clinical settings. Identifying discourse method(s) most sensitive to commonly measured aspects of the spoken language system will improve our ability to accurately and sensitively detect impairments, which is directly relevant to measuring treatment (and generalization) outcomes. Furthermore, we must also consider that the restraints on assessment and research time make it difficult to acquire and subsequently analyze many discourse samples. Indeed, although acquisition time is usually short, it is estimated that transcription and coding requires 6–12 min of time per minute of language sample (Boles, 1998). Therefore, an optimal solution is to identify which discourse method most sensitively taxes the property of spoken language one is most interested in measuring, thereby establishing which discourse method is most appropriate for sensitively evaluating abilities and impairments as well as intervention outcomes.

Here, we have selected primary linguistic variables that serve as proxies for different aspects of spoken language. We subsequently analyzed these primary linguistic variables across four different discourse tasks, comprising three different discourse types. We relied on data from a large database, AphasiaBank, to generate a sufficiently large sample size with which to compare language derived from different discourse elicitation methods across an age- and education-matched control group and an aphasia group (MacWhinney, Forbes, & Holland, 2011). The intent of the current project is to understand the extent to which each discourse task taxes components of the spoken language system.

Method

Participants

The AphasiaBank database was used to acquire spoken discourse data from adults with aphasia and controls. Data were extracted from persons with aphasia (< 93.8 Aphasia Quotient on the Western Aphasia Battery; Kertesz, 2007) who also had samples from all four discourse tasks (described in the next section). There was n = 90 included in this group (see Table 1 for demographics, including age, education, aphasia quotient, and duration of aphasia). The cause of aphasia was largely stroke. Anomic aphasia was present in 42 participants; Broca's, 18; conduction, 18; transcortical motor, five; transcortical sensory, one; and Wernicke's, six. Clinical impression of the presence or absence of apraxia of speech was evaluated in 80 of the 90 participants and was present in 30; dysarthria was evaluated in 79 participants and was present in 14.

Table 1.

Demographics of control and aphasia groups.

Group	Sex	Age (years) M (SD) Min–max	Education (years) M (SD) Min–max	AQ M (SD) Min–max	Duration of aphasia (years) M (SD) Min–max	Average tokens M (SD) Min–max	Average duration (s) M (SD) Min–max
Control n = 84	36 F 48 M	66.75 (15.51) 30–88	15.91 (2.31) 12–22			209.63 (117.63) 38–680	92.21 (48.41) 22.67–284.33
Aphasia n = 90	38 F 52 M	68.92 (15.58) 30–93	15.59 (2.84) 12–25	74.78 (14.86) 28.20–93.40	5.77 (4.23) 0.75–24.70	99.54 (69.24) 8.33–376.00	117.91 (90.48) 13.33–536.00

Note. AQ = Aphasia Quotient from the Western Aphasia Battery (Kertesz, 2007); F = female; M = male.

Spoken discourse data were extracted from an original N = 185 participants in the control group (99 women; M _age = 63.49 ± 17.41 years, range: 23–90 years). To closely match the control group to the aphasia group, case–control matching in SPSS 25 was employed, matching groups on the variables of sex, age, and education. Controls were matched to aphasia participants with a FUZZY interval of 1 SD of the aphasia group's mean age and education. Following this matching, the final control group consisted of 84 participants (see Table 1 for demographics).

Discourse Elicitation

Spoken language was analyzed during four discourse tasks. Tasks from three different types of discourse elicitation method were chosen: expositional, narrative, and procedural. All discourse samples were prompted using AphasiaBank protocol (MacWhinney et al., 2011; https://aphasia.talkbank.org/protocol/).

The procedural discourse involved participants telling the investigator “how to make a peanut butter and jelly sandwich” (which we have abbreviated here as PBJ). The narrative discourse was a story retelling, specifically the Cinderella story, which we call Cinderella. Finally, the expositional discourse involved two tasks. The first was a picture sequence description that involved describing a storyboard of four pictures in which a boy kicks a ball through a window, which we call here the Broken Window story or BW. The second was a picture description that involved participants describing a single picture of a cat being rescued from a tree by firemen, which we call here the Cat Rescue story or Cat.

Discourse Analysis

Spoken discourse production was audio-recorded (most included video) for each participant at the respective data collection site. Transcripts were coded by trained AphasiaBank raters using CHAT software (MacWhinney, 2000). Transcripts captured many aspects of spoken language production, including utterances, dysfluencies, and word- and utterance-level errors. As the CHAT transcriptions were coupled to the accompanying video, duration (in seconds) was automatically calculated.

CLAN (v23, downloaded on December 21, 2018), the accompanying analysis program to CHAT, was used to extract linguistic data from the transcripts. CLAN relies on the parsing of morphological and grammatical information by an automatic process called mor, which tags parts of speech within each utterance. Specifically, the following commands were used:

CLAN Command	Result
mor	Tag parts of speech automatically using mor script
eval +t*PAR +u	Evaluate transcripts to derive primary linguistic outcome variables • eval: evaluate microlinguistic information using the mor tier • +t*PAR: evaluate only the participant tier • +u: consolidate all files to single output

The goal of this project was to extract variables that served as proxies for different components of the spoken language production system. Therefore, variables that touched on language fluency, syntactic variation, and quality and diversity of output were selected. These variables are described in Table 2 and include mean length of utterance (MLU; measured in morphemes), verbs per utterance, type–token ratio (TTR), propositional density (Fromm et al., 2016), noun–verb ratio, and open–closed class word ratio. Total tokens were also extracted (i.e., total words, which did not include repetitions, retracings, or paraphasias but did include paraphasia targets [the word being replaced by the paraphasia]).

Table 2.

Primary linguistic outcome measurements.

Primary measure	Definition	Proxy of language component
Mean length of utterance	Average number of morphemes per utterance	Linguistic productivity
Propositional density	Number of verbs, adjectives, adverbs, prepositions, and conjunctions divided by the total number of words	Content richness
Words per minute	Total number of tokens divided by the duration (converted from seconds to minutes)	Fluency
Verbs per utterance	Average number of verbs per utterance; includes verbs, copulas, and auxiliaries followed by past or present participles and does not include modals	Syntactic complexity
Type–token ratio	Number of different words (types) divided by the number of words (tokens)	Lexical diversity
Open–closed	Ratio of open class words (all nouns, all verbs excluding auxiliaries and modals, all adjectives, all adverbs) divided by closed class words (all other classes)	Syntactic complexity
Noun–verb ratio	Ratio of nouns to verbs (excluding auxiliaries and modals)	Syntactic complexity
Tokens	Total number of words produced	Word retrieval, gross output

Note. Data derived from the CLAN software (MacWhinney et al., 2010).

Analysis

We had three lines of inquiry for this study.

Evaluation of Primary Linguistic Variables in Control Group

Goal: To determine the extent to which primary linguistic variables were different between discourse types in a group with no language impairment.

Demographic variables (age, education) and average tokens (across tasks) were correlated with primary linguistic variables of interest (also averaged across tasks; see Table 3a). Correlations were corrected for multiple comparisons using Benjamini–Hochberg correction (p < .05). Significantly correlated factors were then used as covariates in a sequence of repeated-measures analyses of covariance (ANCOVAs). For significant main effects, post hoc analyses adjusted for multiple comparisons using Bonferroni were conducted.

Table 3.

Correlations of demographic and descriptive variables with primary linguistic variables.

a. Correlations for the control group (n = 84)
Variables	Variables		Age	Education	Average tokens	Average MLU	Average verbs per utterance		Average density	Average TTR	Average WPM		Average noun–verb ratio	Average open–closed class word ratio
	Statistics		I	II	III	IV	V		VI	VII	VIII		IX	X
I	R		1	.039	–.063	–.101	–.189		–.034	.028	–.293		–.076	–.323
	p			.798	.798	.798	.258		.798	.798	.032		.798	.027
	n		84	84	84	84	84		84	84	84		84	84
III	R			1	.123	.095	.086		.022	.08	–.04		–.004	.095
	p				.846	.846	.846		.948	.846	.931		.972	.846
	n			84	84	84	84		84	84	84		84	84
III	R				1	.047	.069		.234	–.718	.157		–.223	.022
	p					.756	.732		.123	0	.344		.123	.842
	n				84	84	84		84	84	84		84	84
IV	R					1	.902		.172	–.186	.089		.206	.129
	p						0		.266	.266	.474		.266	.437
	n					84	84		84	84	84		84	84
V	R						1		.227	–.181	.115		–.066	.151
	p								.171	.225	.449		.552	.308
	n						84		84	84	84		84	84
VI	R								1	–.116	.194		–.184	−.001
	p									.438	.209		.209	.993
	n								84	84	84		84	84
VII	R									1	–.149		.213	.151
	p										.264		.225	.264
	n									84	84		84	84
VIII	R										1		–.033	–.08
	p												.763	.6
	n										84		84	84
IX	R												1	.326
	p													.018
	n												84	84
X	R													1
	p
	n													84
b. Correlations for the aphasia group (n = 90)
	Variables	Age	AQ	Education	Months poststroke	Presence of apraxia of speech	Presence of dysarthria	Average tokens	Average MLU	Average verbs per utterance	Average density	Average TTR	Average WPM	Average noun–verb ratio	Average open–closed class word ratio
Variables	Statistics	I	II	III	IV	V	VI	VII	VIII	IX	X	XI	XII	XIII	XIV
I	R	1	.541	.176	−.159	−.31	−.112	.281	.552	.492	.297	−.254	.261	−.095	−.039
	p		0	.155	.176	.013	.385	.015	0	0	.013	.026	.024	.406	.717
	n	90	90	85	90	80	79	90	90	90	90	90	90	90	90
II	R		1	−.026	−.008	−.165	−.034	.379	.663	.633	.401	−.244	.15	−.16	−.11
	p			.878	.942	.227	.878	0	0	0	0	.043	.227	.227	.395
	n		90	85	90	80	79	90	90	90	90	90	90	90	90
III	R			1	−.184	−.048	−.038	.252	.046	−.013	−.106	−.278	.012	.037	.038
	p				.348	.915	.915	.13	.915	.915	.866	.13	.915	.915	.915
	n			85	85	76	75	85	85	85	85	85	85	85	85
IV	R				1	.17	.12	.033	−.188	−.173	.039	.006	−.154	.099	.103
	p					.321	.507	.894	.321	.321	.894	.954	.321	.507	.507
	n				90	80	79	90	90	90	90	90	90	90	90
V	R					1	.277	−.2	−.415	−.447	−.159	.093	−.443	.383	.246
	p						.03	.124	0	0	.188	.444	0	0	.052
	n					80	78	80	80	80	80	80	80	80	80
VI	R						1	−.206	−.133	−.143	−.017	.085	−.056	.105	.001
	p							.442	.661	.661	.954	.744	.901	.661	.991
	n						79	79	79	79	79	79	79	79	79
VII	R							1	.491	.469	.291	−.715	.281	−.351	−.31
	p								0	0	.009	0	.01	.003	.007
	n							90	90	90	90	90	90	90	90
VIII	R								1	.931	.463	−.34	.449	−.436	−.392
	p									0	0	.001	0	0	0
	n								90	90	90	90	90	90	90
IX	R									1	.505	−.291	.502	−.54	−.424
	p										0	.007	0	0	0
	n									90	90	90	90	90	90
X	R										1	−.242	.352	−.444	−.373
	p											.039	.002	0	0
	n										90	90	90	90	90
XI	R											1	−.205	.367	.504
	p												.068	0	0
	n											90	90	90	90
XII	R												1	−.441	−.347
	p													0	.002
	n												90	90	90
XIII	R													1	.647
	p														0
	n													90	90
XIV	R														1
	p
	n														90

Note. Demographic and descriptive variables included age (years), AQ (aphasia severity, represented by the Aphasia Quotient from the Western Aphasia Battery; Kertesz, 2007), education (years), months poststroke, presence of apraxia of speech and of dysarthria of speech, and average tokens (across all discourse tasks). Primary linguistic variables have been averaged across all discourse tasks. In some cases, a data point for a discourse task was missing; total subjects evaluated for each correlation are shown in the n rows for every variable. p Values shown here have been corrected for multiple comparisons using Benjamini–Hochberg correction, and bolded p values are those which are significant at p < .05. MLU = mean length of utterance; TTR = type–token ratio; WPM = words per minute.

Evaluation of Primary Linguistic Variables in Aphasia Group

Goal: To determine the extent to which primary linguistic variables were different between discourse types in a group with language impairment.

Demographic variables (age, education, aphasia quotient, presence of apraxia of speech, presence of dysarthria, duration of aphasia) and average tokens (across tasks) were correlated with primary linguistic variables of interest (also averaged across tasks; see Table 3b). Correlations were corrected for multiple comparisons using Benjamini–Hochberg correction (p < .05). Significantly correlated factors were then used as covariates in a sequence of repeated-measures ANCOVAs. For significant main effects, post hoc analyses adjusted for multiple comparisons using Bonferroni were conducted.

Comparison of Primary Linguistic Variables Between Control and Aphasia Groups

Goal: To compare primary linguistic variables across discourse tasks between the aphasia and control groups.

Using appropriate statistical tests (parametric or nonparametric, based on Levene's Test for Homogeneity of Variances; Levene, 1960), primary linguistic variables were compared between the aphasia and control groups to identify significant differences in language ability between the groups. The extent to which discourse tasks differed in their elicitation of primary linguistic variables (i.e., as highlighted in the Evaluation of Primary Linguistic Variables in Control Group and Evaluation of Primary Linguistic Variables in Aphasia Group sections, described above) was compared between the aphasia and control groups.

Results

Differences in Demographics Across Groups

As expected, due to case–control matching, there was no significant difference in age, t(172) = 0.92, p = .36, or education, t(160.99) = 0.796, p = .42, between the control and aphasia groups. Pearson chi-square test showed no significant differences in the number of women and men between the control and aphasia groups, χ²(1, N = 174) = 0.007, p = .933.

Main Behavioral Results

The repeated-measures ANCOVAs were based on significant correlations between average primary linguistic variables (across discourse tasks) and demographic and descriptive variables; the significant results for each group are shown in Table 3.

Evaluation of Primary Linguistic Variables in Control Group

Evaluation of differences in primary linguistic variables in the control group identified how spoken language elicited by adults with no language impairment varied across discourse tasks (for full ANCOVA results, see Table 4; for post hoc and summary of results, see Table 5; for visualization of ANCOVA results, see Figure 1). The repeated-measures ANCOVAs demonstrated that there was a main effect of discourse type (expositional [BW, Cat], narrative [Cinderella], procedural [PBJ]) on most primary linguistic variables, with the exception of open–closed class word ratio and words per minute (WPM). Namely, there was a main effect of discourse type on MLUs, verbs per utterance, propositional density, and noun–verb ratio with no significant interactions (as no covariates significantly correlated with these variables; see Table 4). There was a significant interaction of tokens with TTR, such that, alongside a main effect of discourse type, production of more tokens correlated with a reduced TTR.

Table 4.

Analysis of the main effect of discourse type and interactions.

Primary linguistic variables	Significant correlations and covariates included in ANCOVA		Main effect of discourse type		Significant interactions
	Aphasia	Control	Aphasia	Control	Aphasia	Control
MLU	Age, p = .018 AQ, p < .001 AOS, p < .001 Tokens, p < .001	None	a F(1.62, 121.49) = 0.342, p = .665	a F(2.58, 195.88) = 16.18, p < .001**	Age, p = .049 Tokens, p < .001
Verbs per utterance	Age, p < .001 AQ, p < .001 AOS, p < .001 Tokens, p < .001	None	F(3, 225) = 3.924, p = .009**	a F(2.40, 182.33) = 38.11, p < .001**	Age, p = .012
Propositional density	Age, p = .018 AQ, p < .001 Tokens, p = .009	None	a F(2.61, 224.28) = 9.92, p < .001**	a F(2.55, 193.96) = 11.90, p < .001**	None
TTR	Age, p = .026 AQ, p = .043 Tokens, p < .001	Tokens, p < .001	a F(2.67, 229.85) = 3.66, p = .017*	a F(2.50, 187.18) = 0.002**	None	Tokens, p < .001
WPM	Age, p = .024 AOS, p < .001 Tokens, p = .01	Age, p = .032	a F(2.74, 235.32) = 6.77, p < .001**	a F(2.78, 236.14) = 2.04, p = .110	Tokens, p = .001 AQ, p < .001	Age, p = .022
Noun–verb ratio	AOS, p < .001 Tokens, p = .003	None	a F(2.45, 158.99) = 5.20, p = .004**	a F(1.64, 124.78) = 64.27, p < .001**	AOS, p = .012
Open–closed class word ratio	Tokens, p = .007	Age, p = .027	a F(2.35, 199.62) = 2.27, p = .097	a F(2.38, 201.90) = 2.48, p = .076	None	None
Tokens	Age, p = .015 AQ, p < .001	None	a F(1.15, 100.01) = 0.55, p = .484	a F(1.07, 81.26) = 115.53, p < .001	AQ, p = .01

Note. The first column highlights primary linguistic variables. The second column demonstrates the results of the repeated-measures analysis of covariance (ANCOVA) for the main effect of discourse type. The far right column includes any significant interactions with covariates (described in the first column). MLU = mean length per utterance; AQ = Aphasia Quotient; AOS = apraxia of speech; TTR = type–token ratio; WPM = words per minute.

^a Does not meet sphericity assumption; Greenhouse–Geisser reported.

^* p < .05.

^** p < .01.

Table 5.

Summary of the results of the repeated-measures analysis of covariance (ANCOVA) main effects and post hoc analyses.

a. Post hoc tests of significant main effects
Primary linguistic variables	Main effect of discourse type		Significant post hoc tests
	Aphasia	Control	Aphasia	Control
MLU	No	Yes		BW > PBJ, p < .001 Cat > PBJ, p < .001 Cind > PBJ, p < .001
Verbs per utterance	Yes	Yes	BW > PBJ, p < .001 Cat > PBJ, p < .001 Cind > PBJ, p < .001	BW > PBJ, p < .001 Cat > PBJ, p < .001 Cind > PBJ, p < .001
Propositional density	Yes	Yes	Cind > BW, p < .001 Cind > Cat, p < .001 PBJ > BW, p = .03	Cind > BW, p < .001 Cind > Cat, p < .001 Cind > PBJ, p < .001
TTR	Yes	Yes	BW > Cind, p < .001 BW > Cat, p = .011 Cat > Cind, p < .001 PBJ > Cind, p < .001	BW > Cat, p < .001 BW > Cind, p < .001 BW > PBJ, p < .001 Cat > Cind, p < .001 PBJ > Cind, p < .001
WPM	Yes	No	BW > Cind, p = .003 Cat > Cind, p = .046 PBJ > Cind, p = .002
Noun–verb ratio	Yes	Yes	PBJ > BW, p < .001 PBJ > Cat, p < .001 PBJ > Cind, p < .001	PBJ > BW, p < .001 PBJ > Cat, p < .001 PBJ > Cind, p < .001
Open–closed class word ratio	No	No
Tokens	No	Yes		Cind > BW, p < .001 Cind > Cat, p < .001 Cind > PBJ, p < .001 Cat > BW, p < .001 Cat > PBJ, p = .004
b. A layman's summary of repeated-measures ANCOVA results and comparison between groups
Primary linguistic variable	Language proxy	Aphasia	Control	Qualitative comparison of control with aphasia
MLU	Linguistic productivity	• No main effect of discourse type, but interaction with age and tokens ◦Regardless of discourse type: longer MLU correlates with greater age and more tokens produced	• Main effect of discourse type, no interactions • Procedural discourse (PBJ) had shortest MLUs of all types • No significant difference in MLU between expositional (BW, Cat) and narrative (Cind)	• MLU was more mediated by age and total tokens produced in the aphasia group but not in the control group • An effect of discourse type of MLU was only observed for the control group
Verbs per utterance	Syntactic complexity	• Main effect of discourse type, interaction with age ◦Regardless of discourse type: more verbs per utterance correlated with greater age • Procedural discourse (PBJ) had the fewest verbs per utterance of all types • No significant difference in verbs per utterance between expositional (BW, Cat) and narrative (Cind)	• Main effect of discourse type, no interactions • Procedural discourse (PBJ) had the fewest verbs per utterance of all types • No significant difference in verbs per utterance between expositional (BW, Cat) and narrative (Cind)	• Procedural discourse had the fewest verbs per utterance of all types • No interaction with age in the control group, but an interaction with age in the aphasia group
Propositional density	Content richness	• Main effect of discourse type, no interactions • Narrative discourse (Cind) was the densest type, though not significantly denser than procedural (PBJ) • Expositional (BW) was the least dense type	• Main effect of discourse type, no interactions • Narrative discourse (Cind) was the densest type • No significant differences in density between expositional (BW, Cat) and procedural (PBJ)	• Narrative discourse (Cind) was the densest type • For aphasia, expositional (BW) was the least dense; no significant difference for the control group between expositional and procedural
TTR	Vocabulary/lexical diversity	• Main effect of discourse type, no interactions • Narrative (Cind) was the least diverse • Expositional (BW) was the most diverse, but not significantly different from procedural (PBJ)	• Main effect of discourse type, interactions with tokens ◦Higher TTR correlated with fewer tokens produced • Expositional (BW) had the most diverse speech (highest TTR), thus the greatest different words to total words ratio • Narrative (Cind) was the least diverse	• Expositional (BW) tended to have the most diverse speech with few exceptions • Narrative (Cind) was the least diverse
WPM	Efficiency, fluency	• Main effect of discourse type, interaction with tokens and aphasia severity ◦More WPM correlated with more tokens and milder aphasia • Narrative (Cind) had the lowest WPM • No significant differences in WPM between expositional (BW, Cat) and procedural (PBJ)	• No main effect of discourse type, but interaction with age ◦Regardless of discourse type: more WPM correlates with greater age	• Main effect only in the aphasia group, and this main effect was mediated by aphasia severity and total tokens produced • No main effect of discourse type, but interaction with age in the control group
Noun–verb ratio	Syntactic complexity	• Main effect of discourse type, interaction with presence of AOS ◦This was a small relationship (R 2 = .15) and not linear; there was a tendency for a greater noun–verb ratio to correlate with presence of AOS • Procedural (PBJ) had the highest noun–verb ratio, suggesting it was the least complex discourse type • No significant differences in noun–verb ratio between expositional (BW, Cat) and narrative (Cind)	• Main effect of discourse type, no interactions • Procedural (PBJ) had the highest noun–verb ratio, suggesting it was the least complex discourse type • No significant differences in noun–verb ratio between expositional (BW, Cat) and narrative (Cind)	• Procedural (PBJ) had the highest noun–verb ratio, suggesting it was the least complex discourse type • No significant differences in noun–verb ratio between expositional (BW, Cat) and narrative (Cind)
Open–closed class word ratio	Syntactic complexity	• No main effect of discourse type, no interactions	• No main effect of discourse type, no interactions	• No main effect of discourse type, no interactions
Tokens	Word retrieval	• No main effect of discourse type, interaction with aphasia severity ◦Regardless of discourse type: milder aphasia correlates with more tokens produced	• Main effect of discourse type, no interactions • Narrative (Cind) produced the most tokens of all types • Expositional (Cat) produced more tokens than both expositional (BW) and procedural (PBJ) • No significant difference between expositional (BW) and procedural (PBJ)	• Main effect of discourse type only for the control group, where narrative produced the most tokens • Tokens was highly mediated by aphasia severity in the aphasia group, but no main effect was found

Note. (a) The first column summarizes if there was a main effect found (for full ANCOVA results, see Table 4). The significant post hoc tests are then described in the right column; p values have been corrected using Bonferroni correction for multiple comparisons. (b) This highlights main results, interactions, and a comparison of the aphasia and control groups in layman's terms. MLU = mean length of utterance; TTR = type–token ratio; WPM = words per minute; BW = Broken Window; PBJ = peanut butter and jelly; Cind = Cinderella.

Figure 1.

Primary linguistic variables compared within groups (aphasia, control) across discourse types. Raw means from the aphasia group (left column) and the control group (right column). Black lines indicate significant differences, as described in Tables 4 and 5. Error bars represent standard deviations. MLU = mean length of utterance; TTR = type–token ratio; WPM = words per minute, divided by 10; BW = Broken Window; PBJ = peanut butter and jelly.

Post hoc analyses showed that each discourse type (e.g., narrative) elicited language that differs across linguistic components (e.g., density, tokens). These post hoc analyses are more comprehensively explained in Table 5. Briefly, the control group results demonstrated that procedural discourse elicited the shortest MLU, the highest noun–verb ratio, and the fewest verbs per utterance, indicating that this type of discourse was the most syntactically impoverished. Furthermore, post hoc analyses suggested that narrative discourse elicited the densest language and most tokens and that expositional discourse (BW, Cat) tended to elicit the most diverse language (TTR).

Evaluation of Primary Linguistic Variables in Aphasia Group

Although evaluation of differences in primary linguistic variables in the control group identified how each discourse type was sensitive to producing certain primary linguistic components, we also quantified how linguistic components differed across discourse types in adults with aphasia (for full ANCOVA results, see Table 4; for post hoc and summary of results, see Table 5; for visualization of ANCOVA results, see Figure 1). In addition, this analysis also leant insight into significant interactions with covaried demographic and descriptive variables.

The repeated-measures ANCOVAs demonstrated that there was a main effect of discourse type (expositional [BW, Cat], narrative [Cinderella], procedural [PBJ]) on most primary linguistic variables with the exception of MLU, open–closed class word ratio, and tokens (see Table 4). There was a main effect of discourse type with no significant interactions for propositional density and TTR. Alongside a main effect of discourse type on verbs per utterance, there was also a significant interaction with age observed, such that greater age correlated with more verbs per utterance. Similarly, alongside a main effect of discourse type on WPM, there was a significant interaction with both tokens and age, where more WPM correlated with more tokens and greater age. The presence of apraxia of speech interacted with noun–verb ratio alongside a main effect of discourse type. Finally, despite no main effect of discourse type on number of tokens produced, there was a significant interaction of tokens with aphasia severity (Aphasia Quotient), where more tokens were produced in those with milder aphasia regardless of task.

These results complement the results of the control group, suggesting that each discourse type is sensitive to producing specific aspects of spoken language. In addition, the aphasia group results highlight mediating factors (e.g., aphasia severity, age) that interact with discourse type. Post hoc analyses in the aphasia group also complement evidence from the control group, in that procedural discourse (PBJ) elicited the fewest verbs per utterance and the highest noun–verb ratio (see Table 5). However, post hoc analyses in the aphasia group elaborated on this finding, suggesting that these main effects are often mediated by other variables, such as age and presence of apraxia of speech. Like the control group, narrative discourse (Cinderella) was the densest type (though not significantly denser than PBJ, which was a finding unique to the aphasia group), and expositional discourse (BW in particular) tended to have the highest TTR.

Comparison of Primary Linguistic Variables Between Control and Aphasia Groups

The control group elicited spoken language that was significantly different from the aphasia group across nearly all primary linguistic variables, except for TTR during expositional and narrative tasks and noun–verb ratio and open–closed class word ratio across all tasks (see Table 6). Therefore, the aphasia group had (expectedly) impoverished spoken output compared with the control group. One can best appreciate this comparison in Figure 2, indicating the difference between aphasia and control groups for all primary linguistic measures. The Evaluation of Primary Linguistic Variables in Control Group and Evaluation of Primary Linguistic Variables in Aphasia Group sections demonstrated that discourse types elicit consistent language properties in both the control and aphasia groups. This finding is particularly striking, especially as the control group produced significantly more output than the aphasia group.

Table 6.

Primary linguistic variables summarized for control and aphasia groups.

Primary linguistic variable	Discourse type	Aphasia M (SD) [n]	Control M (SD) [n]	Difference in means
MLU	Expositional (BW)	6.45 (2.85) [90]	11.34 (3.62) [84]	a t(172) = 9.92, p < .001*
	Expositional (Cat)	7.24 (3.01) [90]	11.43 (3.53) [78]	a t(166) = 8.30, p < .001*
	Narrative (Cind)	6.65 (2.45) [90]	11.26 (2.79) [84]	a t(172) = 11.61, p <.001*
	Procedural (PBJ)	5.77 (2.15) [90]	9.14 (2.70) [82]	a t(170) = 9.08, p <.001*
Verbs per utterance	Expositional (BW)	0.90 (0.52) [90]	1.62 (0.54) [84]	a t(172) = 9.02, p < .001*
	Expositional (Cat)	1.03 (0.56) [90]	1.74 (0.58) [78]	a t(166) = 8.12, p < .001*
	Narrative (Cind)	0.93 (0.48) [90]	1.61 (0.39) [84]	a t(172) = 10.23, p < .001*
	Procedural (PBJ)	0.69 (0.41) [90]	1.13 (0.26) [82]	b U = 6,192.00, p < .001*
TTR	Expositional (BW)	0.59 (0.13) [90]	0.61 (0.07) [84]	b U = 4,314.50, p = .11
	Expositional (Cat)	0.55 (0.13) [90]	0.53 (0.07) [78]	b U = 3,195.00, p = .32
	Narrative (Cind)	0.42 (0.14) [90]	0.39 (0.11) [84]	at(172) = 1.89, p = .06
	Procedural (PBJ)	0.60 (0.17) [90]	0.51 (0.11) [82]	b U = 2,487.00, p < .001*
WPM	Expositional (BW)	64.28 (38.33) [90]	140.75 (31.54) [83]	at(171) = 14.26, p < .001*
	Expositional (Cat)	60.30 (38.35) [90]	151.56 (32.20) [78]	at(166) = 16.56, p < .001*
	Narrative (Cind)	54.89 (30.75) [90]	130.92 (29.21) [84]	at(172) = 16.70, p < .001*
	Procedural (PBJ)	63.41 (33.73) [90]	152.64 (34.20) [82]	at(170) = 17.21, p < .001*
Propositional density	Expositional (BW)	0.38 (0.11) [90]	0.46 (0.05) [84]	b U = 5,934.00, p < .001*
	Expositional (Cat)	0.40 (0.10) [90]	0.45 (0.04) [78]	b U = 4,991.50, p < .001*
	Narrative (Cind)	0.44 (0.09) [90]	0.49 (0.03) [84]	b U = 5,546.50, p < .001*
	Procedural (PBJ)	0.41 (0.11) [90]	0.46 (0.05) [82]	b U = 4,793.00, p < .001*
Noun–verb ratio	Expositional (BW)	1.58 (1.49) [86]	1.21 (0.35) [84]	b U = 3,188.00, p = .19
	Expositional (Cat)	1.67 (1.72) [86]	1.20 (0.35) [78]	b U = 3,177.50, p = .56
	Narrative (Cind)	1.33 (1.03) [88]	1.12 (0.26) [84]	b U = 4,090.00, p = .23
	Procedural (PBJ)	2.63 (2.16) [83]	2.00 (0.78) [82]	b U = 3,238.00, p = .59
Open–closed class word ratio	Expositional (BW)	0.89 (0.60) [88]	0.78 (0.13) [84]	b U = 3,964.00, p = .41
	Expositional (Cat)	0.79 (0.53) [89]	0.74 (0.11) [78]	b U = 4,020.50, p = .08
	Narrative (Cind)	0.80 (0.85) [90]	0.69 (0.07) [84]	b U = 4,541.50, p = .02
	Procedural (PBJ)	0.82 (0.59) [89]	0.74 (0.16) [82]	b U = 4,032.00, p = .24
Tokens	Expositional (BW)	50.91 (33.18) [90]	83.86 (37.21) [84]	at(172) = 6.17, p < .001*
	Expositional (Cat)	73.62 (46.47) [90]	106.91 (43.48) [78]	a t(166) = 4.77, p < .001*
	Narrative (Cind)	204.89 (160.69) [90]	458.87 (307.97) [84]	b U = 5,973.50, p < .001*
	Procedural (PBJ)	42.81 (35.60) [90]	85.76 (50.63) [82]	b U = 6,000.50, p < .001*

Note. Statistically compared using, where appropriate, parametric independent-samples t test or nonparametric Mann–Whitney U test. Use of parametric versus nonparametric comparison of means was based on the significance of Levene's Test of Homogeneity of Variances based on mean (p > .05 indicates use of parametric test; p < .05 indicates use of nonparametric test). Raw means are shown in Figure 2. Original p values are listed, with an asterisk (*) indicating if this p value was statistically significant after Benjamini–Hochberg correction (p < .05). MLU = mean length per utterance; TTR = type–token ratio; WPM = words per minute; BW = Broken Window, expositional discourse; Cat = cat, expositional discourse; Cind = Cinderella, narrative discourse; PBJ = peanut butter and jelly, procedural discourse.

^a Independent-samples t test.

^b Mann–Whitney U test.

Figure 2.

Comparison of primary linguistic variables per discourse type for aphasia and control groups. Raw means from aphasia group (blue columns) and control group (orange columns), categorized by discourse type (BW, Cat, Cinderella, PBJ). Significant differences between control and aphasia groups are shown by a star. Error bars represent standard deviations. This figure is a complement to Table 6. MLU = mean length of utterance; TTR = type–token ratio; WPM = words per minute, divided by 10; BW = Broken Window; PBJ = peanut butter and jelly.

As was discussed in the introduction, most studies employing discourse as an outcome measure use single picture description to evaluate language production. In this study, we employed two expositional discourse tasks (BW and Cat) and were therefore able to directly compare primary linguistic data between tasks. Using a sequence of independent t tests corrected for multiple comparison correction (Benjamini–Hochberg), we established that the BW picture sequence and the Cat picture description elicited significantly different linguistic properties. In the control group, the Cat picture elicited longer MLU (p = .001, corrected p = .002), more tokens (p < .001), more verbs per utterance (p = .001, corrected p = .002), and more WPM (p < .001). However, the BW picture sequence elicited a higher TTR (p < .001), a greater density (p = .006, corrected p = .008), and a greater open–closed class word ratio (p = .037, corrected p = .042). There was not a significant difference in noun–verb ratio between the two expositional tasks (p = .07). When comparing linguistic data between BW and Cat in the aphasia group, the pattern was similar: The Cat picture elicited a longer MLU (p < .001), more tokens (p < .001), and more verbs per utterance (p < .001), whereas the BW picture sequence elicited a higher TTR (p < .001) and a higher open–closed class word ratio (p < .001). However, unlike in the control group, BW picture sequence elicited more WPM (p < .001), whereas the Cat picture elicited language that was denser (p < .001) and had a higher noun–verb ratio (p < .001). In summary, both expositional types of discourse may be useful for evaluating spoken language, but as prior results show, neither is the most ideal for evaluating density but may be useful when evaluating diversity (e.g., TTR) and perhaps syntactic complexity (verbs per utterance, noun–verb ratio, and open–closed class word ratio).

Summary

Overall, results from within each group (see the Evaluation of Primary Linguistic Variables in Control Group and Evaluation of Primary Linguistic Variables in Aphasia Group sections) and comparison across groups (see the Comparison of Primary Linguistic Variables Between Control and Aphasia Group section) demonstrate that spoken language elicited by each discourse task varies across linguistic properties in both the control and aphasia groups, even with an impoverished output (as in the aphasia group).

Discussion

In the current study, a large population of age- and education-matched adults and persons with aphasia was leveraged to evaluate the extent to which spoken language elicited by four different discourse tasks (spanning three discourse types) varied. As expected, the control group produced more output across discourse tasks, aligning with prior studies comparing proxies of language, such as noun–verb ratio and TTR, between age-matched control and aphasia groups (Fergadiotis & Wright, 2011; Thompson et al., 2012). Furthermore, the control group established that each discourse type elicited spoken language that differed in underlying linguistic properties. For example, narrative discourse was shown to elicit the densest speech, whereas procedural discourse, at least as measured here by the peanut butter and jelly story prompt, may exhibit the least syntactically complex spoken language. Strikingly, despite impoverished output across all discourse tasks, the aphasia group exhibited similar data, supporting the conjecture that narrative discourse elicits the densest language and procedural elicits the least syntactically complex language. Evaluating the aphasia group also demonstrated that significant interactions with variables such as aphasia severity, tokens, age, and presence of apraxia of speech are liable to have a mediating effect on primary linguistic variables measured from spoken language. This is, of course, not surprising given the large amount of research suggesting an interaction of aphasia severity and spoken language output (Bond, Ulatowska, Macaluso-Haynes, & May, 1983). However, considering the shared findings from the control and aphasia groups, this study's results present a compelling argument that the type of discourse used to elicit spoken language matters.

Indeed, each discourse type elicited spoken language that differed across linguistic properties. Lexical diversity, commonly measured not only by TTR but also by related entities such as moving-average TTR, has before been identified to differ across discourse types in people with aphasia (Fergadiotis & Wright, 2011; Fergadiotis, Wright, & Capilouto, 2011), and our results concur and expand on these findings. Put together, our findings and prior findings establish that, in both people with and without aphasia, each structured discourse method has its benefits for the production of certain aspects of spoken language. In summary, data suggested that propositional density was greatest in narrative discourse, WPM were reduced in narrative discourse, TTR was smallest for narrative discourse, and procedural discourse produced the least syntactically complex speech (highest noun–verb ratio, fewest verbs per utterance).

There are caveats to the current study. For example, a main effect of discourse elicitation method on propositional density for all groups was identified. Narrative discourse (Cinderella story) elicited the densest language, producing the most content-rich language regardless of aphasia severity or number of tokens. Narrative discourse may not prove to be the most sensitive to noun access or object naming (indeed, narrative discourse produced the smallest percentage of nouns across discourse types) but may prove to be the most sensitive elicitation method for evaluating depth of vocabulary and content richness and therefore may best evaluate impairments or changes in these language areas. Narrative discourse, unlike expositional discourse, does not rely on visual aids and relies more heavily on processes of memory (arguably, both long-term and working memory) and aspects of executive function, such as planning and organization. Therefore, in participants with concomitant impairments in these cognitive domains, narrative discourse may not produce the densest language. AphasiaBank does not collect cognitive scores, and we were therefore unable to explore the extent to which individuals did or did not demonstrate impairments in memory or executive function. However, given the large sample size of people with aphasia, who demonstrated that narrative discourse produced the densest language, this suggests that at least a large proportion of the greater aphasia population will likely also elicit the densest speech during this task. However, future work should evaluate the importance of measures of cognition on linguistic structures, like those explored here, as well as on macrolinguistic variables, such as story grammar (Nicholas & Brookshire, 1995; Roth & Spekman, 1986), coherence (Galetto, Kintz, West, Mrini, & Wright, 2013; Rogalski, Altmann, Plummer-D'Amato, Behrman, & Marsiske, 2010; Van Leer & Turkstra, 1999; Wright, Koutsoftas, Capilouto, & Fergadiotis, 2014), and production of main concepts (Richardson & Dalton, 2015). We did not evaluate the difference in macrolinguistic variables in the current study, but a complementary area of future research should identify the relative differences in macrolinguistic variables across discourse elicitation methods and the impact of aphasia severity on these differences. As a final caveat, the current aphasia sample consisted of 68 with fluent aphasia (conduction, transcortical sensory, Wernicke's, anomic) and 22 with nonfluent aphasia (Broca's, transcortical motor), and the present results may have been influenced by the larger proportion of people with fluent and/or milder aphasia, who have a tendency to produce more language than those with nonfluent and/or more severe aphasia.

Bryant et al. (2016) discuss how most studies utilizing discourse in aphasia employ the structured, expositional discourse elicitation method (overwhelmingly, a single picture description). This study directly evaluated linguistic variables from two types of expositional discourse tasks, a picture description (Cat) and a picture sequence description (BW). The control and aphasia groups both showed that the Cat picture description tended to elicit longer MLUs, more tokens, and more verbs per utterance and the BW picture sequence tended to elicit higher TTR and open–closed class word ratio. The aphasia group, unlike the control group, showed that the BW picture sequence elicited more WPM whereas the Cat picture elicited language that was denser and had a higher noun–verb ratio. The Cat and BW, though perhaps not eliciting the densest speech, may therefore be particularly helpful for those with more severe aphasia, who may rely more on visual cues for lexical retrieval (Doyles et al., 1998).

This study, which used a large data set of age- and education-matched control and aphasia data, specifically evaluates language produced during different structured discourse types. First, the results of the current study emphasize the importance of utilizing several types of structured discourse elicitation methods for a more comprehensive evaluation of language, demonstrating that the microlinguistic properties of spoken language elicited by each type of method is different. This finding lends support for the use of several types of discourse elicitation methods to acquire a comprehensive evaluation of language (Brookshire & Nicholas, 1994), which is important for the sensitivity of assessment and intervention. Word error (i.e., paraphasia) instability across discourse samples (particularly, short samples elicited using pictures, as in expositional tasks) also demands employing multiple discourse types to achieve a more reliable profile of paraphasias (Boyle, 2015). Second, we lay the foundation for more accurately selecting appropriate discourse elicitation methods to evaluate specific aspects of language. This is particularly relevant in clinical settings, when both acquisition and analysis time are a factor. The importance of choosing the most sensitive discourse type(s), according to your language variable(s) of interest, can be easily understood by an analogy to a common medical situation: measuring blood pressure. For example, if you want to measure blood pressure, you would acquire the most accurate and sensitive result from using the most appropriate measure (e.g., a well-fitted blood pressure cuff) than a less appropriate measure (e.g., a too big blood pressure cuff). This is the case in language research. If you want to examine a person's syntactic system in the most comprehensive fashion, one must use a discourse type that elicits spoken language that tends to most expansively tax this system. We show here, for example, that using a procedural discourse (alone) to measure syntactic complexity may not be the most sensitive discourse type for taxing syntactic processes. Furthermore, understanding the microlinguistic properties of spoken language produced by each type of discourse method is important for tailoring assessments to better appreciate the depth of specific language impairments and compensatory techniques, which may not be elicited to the same degree during standardized batteries or single-modality assessment (e.g., object naming). In addition, selecting the most appropriate discourse elicitation method may be especially important when identifying generalization of impairment-based training to more functional settings. For example, discourse-level improvement (or indeed, generalization of intervention) in aphasia following a syntactic impairment-based treatment may be best exemplified by assessing verbs per utterance, open–closed class word ratio, and/or noun–verb ratio elicited during narrative and expositional discourse. Meanwhile, improvements to spoken language density and productivity in aphasia following response elaboration training (Kearns, 1985) or semantic feature analysis/related therapies (Lowell, Beeson, & Holland, 1995; Ylvisaker & Szekeres, 1985) may be most sensitively evaluated using narrative discourse. Third, we suggest that evaluating tokens as the primary linguistic outcome from spoken discourse in aphasia is likely not a telling or sensitive measure of linguistic ability because of its relationship with overall severity of aphasia. This suggestion is of particular relevance for the field, as Bryant et al. (2016) showed that most structured discourse studies evaluated tokens as the linguistic variable of interest. We directly evaluated the difference in tokens across discourse types in aphasia and did not find a main effect of discourse task, but we did find a significant interaction of tokens with aphasia severity. This result suggests that analyzing tokens as the primary linguistic variable is highly dependent on the severity of aphasia and likely not sensitive to the type of discourse task. Furthermore, evaluating tokens alone does not provide information regarding other aspects of language that may be responding to treatment or that are relatively unimpaired despite aphasia severity (e.g., syntactic ability), which undermines the primary reason for using discourse in assessment or as an outcome measure.

Despite the ecological validity of using spoken discourse to assess language ability and outcomes in aphasia and indeed a well-used one, a variety of methodological issues have hampered its potential in research and clinical environments. To advance the field of discourse assessment, we provide compelling evidence for the use of multiple discourse task types and evidence for selection of the most appropriate task for analysis of specific aspects of spoken language, useful in both assessment and treatment outcome. We believe that these suggestions will improve upon some of the methodological issues in discourse assessment, particularly in improving the sensitivity of this method in identifying specific language abilities and outcomes.

Funding Statement

Funding for AphasiaBank is supported by National Institute on Deafness and Other Communication Disorders grant DC008524 (PI: Brian MacWhinney).