Automatic processing of wh- and NP-movement in agrammatic aphasia: Evidence from eyetracking

Abstract

Individuals with agrammatic Broca’s aphasia show deficits in comprehension of non-canonical wh-movement and NP-movement sentences. Previous work using eyetracking has found that agrammatic and unimpaired listeners show very similar patterns of automatic processing for wh-movement sentences. The current study attempts to replicate this finding for sentences with wh-movement (in object relatives in the current study) and to extend it to sentences with NP movement (passives). For wh-movement sentences, aphasic and control participants’ eye-movements differed most dramatically in late regions of the sentence and post-offset, with aphasic participants exhibiting lingering attention to a salient but grammatically impermissible competitor. The eye-movement differences between correct and incorrect trials for wh-movement sentences were similar, with incorrect trials also exhibiting competition from an impermissible interpretation late in the sentence. Furthermore, the two groups exhibited similar eye-movement patterns in response to passive NP-movement sentences, but showed little evidence of gap-filling for passives. The results suggest that aphasic and unimpaired individuals may generate similar representations during comprehension, but that aphasics are highly vulnerable to interference from alternative interpretations (Ferreira, 2003).

1. INTRODUCTION

Sentences with non-canonical movement are difficult for agrammatic (Broca’s) aphasic individuals to comprehend (Caramazza & Zurif 1976; Drai & Grodzinsky 2006; Grodzinksy 1990, 2000; Mauner, Fromkin & Cornell 1993; among others). This difficulty appears in a wide variety of tasks, including sentence-picture matching (Caramazza & Zurif 1976, Schwarz, Saffran & Marin 1980), makes-sense judgment tasks (Saffran, Schwartz & Linebarger 1998, Dickey & Thompson 2004), and even grammaticality judgment (Grodzinsky & Finkel 1998; though see Linebarger, Schwartz & Saffran 1983). For example, non-canonical NP-movement sentences like passives (1) and non-canonical wh-movement sentences like object relatives (2) often elicit chance performance in sentence-picture matching tasks:

1. The boy was kissed t by the girl.

2. I saw the boy who the girl kissed t.

The underlined constituents in (1–2) (who, the boy) represent the element which has been moved or displaced in these sentences, while the traces (t) represent the positions from which those elements have been moved. It is the task of the comprehender to associate the moved elements (or fillers) with the trace position (or gap) during comprehension, in order to arrive at a correct interpretation of the sentence (Frazier & Flores d’Arcais 1987). While there are important linguistic differences between wh- and NP movement (Chomsky 1986, 1995, Nevins and Anand 2003), and the two types of movement may be independently impaired and recovered in agrammatic aphasia (Friedmann 2006, Thompson & Shapiro 2005), they impose similar comprehension demands on readers or listeners. In both cases, a reader/hearer must associate a displaced element with a trace/gap in order to assign the element a semantic role in the sentence, while ignoring potentially competing information such as agent-first heuristics (Bever 1970, Ferreira 2003, Grodzinsky 1990, Townsend & Bever 2001) or lexical-semantic entailments (Piñango 2000). Some part of this process appears to be impaired for many agrammatic aphasic individuals crosslinguistically (see Drai & Grodzinsky 2006 for a survey and meta-analysis of crosslinguistic evidence).

Recently, evidence of this comprehension impairment has been provided using a novel experimental paradigm, eyetracking while listening (Dickey, Choy & Thompson 2007). Dickey et al. (2007) studied a group of aphasic and unimpaired listener’s eye movement patterns while processing wh-movement structures. Stories as in (3) were auditorily presented while subjects looked at images of characters and locations from the stories on a computer screen. The stories were followed by critical comprehension probes, and the participants’ eye-movements were monitored as they were processed these probes.

• (3)This is a story about a boy and a girl.

  One day, they were at school together.

  The girl was pretty, so the boy kissed the girl.

  They were both embarrassed after the kiss.

  1. Who did the boy kiss t that day at school?

  2. It was the girl who the boy kissed t that day at school.

  3. Did the boy kiss the girl that day at school?

The comprehension probes appeared in three forms: critical wh-movement structures, object wh-questions (3a) or object clefts (3b) and control yes/no questions (3c). Participants responded by answering the wh-question or by saying “yes” or “no” in response to the cleft structures, and yes/no questions.

Dickey et al. (2007) reported three main findings. First, the aphasic participants were reliably less accurate in their responses to both wh-questions and object clefts than they were to control yes-no questions without wh-movement. This finding replicated many previous studies, which have found comprehension impairments for these structures. Second, the eye-movement patterns were quite similar for the aphasic and control participants for both clefts or wh-questions. For example, for wh-questions, both groups exhibited a significant theme preference: upon hearing the verb “kiss,” which signaled the trace/gap, both groups shifted their visual attention to the girl (the theme of the kissing) even though the girl was not overtly mentioned at that point in the sentence. Dickey and colleagues interpreted this pattern as visual evidence of gap-filling, as it was parallel to previous eye-movement findings for wh-question comprehension among young unimpaired listeners (Sussman & Sedivy 2003). Third, the point at which eye-movement patterns differed for aphasic and control participants was during the final adverbial phrase (the locative “at school”) and after sentence’s end. Aphasic participants showed a much weaker (or even reversed) theme advantage, showing competition between theme and agent fixations. This difference appeared most strongly for trials which elicited incorrect answers. Parallel results have also been found for comprehension of pronouns and reflexives by agrammatic aphasic individuals: in a separate eyetracking study, Choy and Thompson (2005) reported no differences between aphasic and control participants’ eye-movements at the position of a pronoun (“her”) or reflexive (“herself”). Instead, they found competition between the target antecedent and an ungrammatical alternative downstream, later in the sentence.

Together, these findings provide suggestive evidence that at least some aphasic individuals are able to compute wh-movement dependencies in real time, and that they are able to associate fillers with gaps on a similar time-scale to unimpaired controls. However, the resulting representations are weak (viz. Avrutin 2006, den Ouden 2006), which leaves aphasic individuals particularly vulnerable to competition from other interpretations (viz. Ferreira 2003). An increased vulnerability to competition from non-syntactic sources of interpretation (such as agent-first heuristics, Bever 1970, or plausibility, Caramazza & Zurif 1976) is what differentiates aphasic individuals from unimpaired comprehenders under many accounts of aphasic comprehension deficits (e.g., Avrutin 2006, Grodzinsky 1990, Piñango 2000). It is also visible in the eye-movement record: it is what underlies the reduced theme advantage (and increased competition) found late in the sentence for aphasic individuals in the Dickey et al. results. This finding illustrates the methodological usefulness of eyetracking. Studies of aphasic sentence comprehension using other methodologies (for example, cross-modal lexical priming, e.g., Swinney & Zurif 1995) have not examined the availability or activation of competing interpretations/antecedents. These studies only examined activation of the displaced element, which should be reconstructed when comprehension is successful.

In spite of these clear patterns for object wh-questions, less clear cut results were found for object cleft structures. Although aphasic and unimpaired participants’ eye-movements did not differ statistically, methodological issues precluded a valid inspection of gap-filling for these structures due to a lack of appropriate control structures. Thus, further research is needed, with appropriate controls in place, in order to determine eye-movement patterns for other wh-movement structures. Also, Dickey et al. (2007) examined only wh-movement sentences. It would be useful to know whether similar patterns hold for non-canonical NP movement sentences such as passives, which pose similar problems for normal comprehenders and are also typically impaired in agrammatic aphasia.

The current study follows up on the results of Dickey et al. (2007). First, the current study examines comprehension of object relative clauses, another structure involving wh-movement (Chomsky 1977) for which comprehension is also typically impaired in agrammatic aphasia (Caramazza & Zurif 1976, among others). Second, it examines comprehension of non-canonical NP movement structures, specifically passive structures, for which comprehension is also typically impaired among agrammatic individuals (Schwartz, et al. 1980).

2. EXPERIMENT

This experiment used an eyetracking while listening paradigm to examine comprehension of object relative clauses (involving wh-movement) and passives (involving NP movement). There were three specific hypotheses tested. First, based on the results for wh-questions in Dickey et al. (2007), we hypothesized that both aphasic and control participants would exhibit visual evidence of gap-filling for wh-movement structures, in this case object relative clauses. That is, at the position of the verb signaling a trace, but not beforehand, participants should show a preference to gaze at a picture corresponding to the displaced element (the theme, in the case of object-extracted wh-movement).¹ Second, based on previous results from the literature on the processing of NP movement (Burkhardt, Piñango & Wong 2003), we hypothesized that both control and aphasic participants should exhibit visual evidence of gap-filling for passives as well. However, previous results from cross-modal lexical priming have suggested that gap-filling effects for NP movement may appear significantly downstream from the gap site (Burkhardt, et al. 2003), so these effects may be relatively delayed compared to the effects found for wh-movement in this experiment. (See Fodor 1989 for discussion.) Third, based on the Dickey et al. results, we hypothesized that the differences between aphasic and unimpaired participants should appear not early in the sentence but late, after the position at which automatic processing of wh- or NP movement takes place. More specifically, aphasic participants should exhibit competition between the correct interpretation (gazes at pictures corresponding to the theme, the extracted element) and a tempting but incorrect alternative one (gazes at pictures corresponding to the agent, and animate competitor). That is, parallel to the aphasic participants in the Dickey, et al. (2007) study, they should exhibit a decreased theme preference later in the sentence compared to controls.

The current experiment used a paradigm very much like that used by Dickey et al. (2007). Unlike that study, however, the current experiment embedded these structures inside prompts directing participants to point to one (of four) images on the screen: “Point to who the bride tickled in the mall” (object relative) and “Point to who was tickled by the bride in the mall” (passive). The reason for this choice was two-fold. First, previous work has found that listeners are more likely to gaze at an object during eyetracking if that object is directly relevant to an action they are about to perform, either overtly naming an object during spontaneous sentence generation (Griffin & Bock 2000, Griffin 2004) or pointing to or manipulating an object (Sussman, Campana, Tanenhaus & Carlson, 2003). Second, pointing tasks do not require a spoken response. This may reduce the task burden for non-fluent aphasic individuals, for whom overt speech production is demonstrably impaired.

2.1 Methods

2.1.1 Participants

Eight individuals diagnosed with agrammatic Broca’s aphasia (6 male) and fourteen unimpaired controls (4 male) participated in this experiment. The aphasic participants were mildly to moderately impaired based on their scores on the Western Aphasia Battery (WAB; Kertesz 1982), with WAB Aphasia Quotients ranging from 60.8 to 87.6. They ranged in age from 38 to 67 (mean 56.1 years) and were between 2 and 25 years post-onset (mean 9.3 years) at the time of testing. All were native monolingual speakers of American English, and reported no history of other speech-language, learning or neurological disorders. All aphasic participants but one (A3) were premorbidly right-handed. In addition, all had language profiles consistent with agrammatism. In spontaneous speech, they exhibited halting, effortful production with reduced syntactic complexity and little or no use of grammatical morphology. Furthermore, in their performance on the Northwestern Assessment of Verbs and Sentences (NAVS: Thompson, 2005), all aphasic participants but one (A1) demonstrated significant impairment in their production of complex sentences. All aphasic participants showed poorer performance in verb naming compared to verb comprehension on the NAVS. Demographic and language-testing data for the aphasic participants are presented in Table 1. NAVS scores were unavailable for one participant, A6.

Table 1.

Aphasic participants’ demographic and language testing data

Participant	Gender	Age	Years post- onset	Years of education	WAB AQ	WAB Fluency	WAB Comprehension	WAB Repetition	WAB Naming	NAVS Sentence production	NAVS Verb naming	NAVS Verb comprehension
A1	M	60	6	17	80.4	4	9.1	9.4	8.7	94%	88%	100%
A2	M	67	7	21	87.6	6	10	9.7	9.1	63%	63%	100%
A3	M	57	15	20	79.6	5	9.9	6.4	9.5	31%	91%	100%
A4	F	52	10	13	60.8	2	8	6.6	5.8	0%	62%	100%
A5	F	58	5	13	73.5	4	9.35	6.5	7.9	14%	82%	97%
A6	M	58	25	20	67.2	4	8.3	5.2	7.1	N/A	N/A	N/A
A7	M	59	2	17	72.4	5	9.25	7	7.9	32%	71%	98%
A8	M	38	4	19	82.4	5	9.2	8.8	9.2	40%	76%	100%
Mean		56.1	9.3	17.5	75.5	4.4	9.1	7.5	8.2	39%	76%	99%

Notes: N/A indicates that data were not available. NAVS scores represent percentage correct.

The control participants reported no prior history of speech-language, learning, or neurological disorders. All were right-handed, native monolingual speakers of American English.

2.1.2 Materials

Materials for this study consisted of forty-eight pairs of brief stories and panels depicting objects mentioned in the stories. The stories were presented monoaurally over a loudspeaker, while the panels were presented on a computer screen placed at a comfortable viewing distance for participants. Twenty-four of the forty-eight story-panel pairs served as experimental items, asking participants to point to the theme of the principal action of the story, while the remaining twenty-four served as fillers, asking them to point to either the agent of the action or to another animate distractor. More detail about the linguistic and visual stimuli is provided below.

2.1.2.1 Linguistic stimuli

The stories for all forty-eight stimulus pairs had the same structure. Each story was three sentences long and was followed by a comprehension probe. A sample story with comprehension probes is found in (4) below.

• (4)One day a bride and groom were walking in the mall.

  The bride was feeling playful, so the bride tickled the groom.

  A clerk was amused.

  1. Point to who the bride was tickling t in the mall.

  2. Point to who was tickled t by the bride in the mall.

Each story contained one transitive event, described in sentence two. Sentence one of each story introduced two animate NPs who were the agent and patient or theme of this event, as well as the location in which the transitive event took place. Sentence three served as a distractor sentence, introducing a third animate actor who responded in some way to the transitive event. The stories were kept deliberately simple to reduce working-memory burdens for aphasic participants.

The names of the actors, location, and distractor were matched for length in syllables for each story, and they did not overlap in their initial segments, so as not to create confusion or lexical competition during the stories or comprehension probes. Words with overlapping initial segments have been shown to compete with one another in tasks using an eye-tracking paradigm (Allopenna, Magnuson & Tanenhaus 1998).

Each story was followed by a comprehension probe. The comprehension probe asked participants to point to one of the pictures described in the story. For the twenty-four experimental items, the probe asked participants to point to the theme of the transitive event. Twelve of those items had probes with an object-extracted relative clause structure (4a), while the other twelve experimental items had subject relative clauses with passive structures for probes (4b). The twenty-four filler items were all followed by simple active comprehension probes. Twelve of those probes asked about the agent of the transitive event (“Point to who was tickling the groom in the mall”), while twelve of them asked about the distractor in the third sentence (“Point to who was amused at the scene”). Participants responded by moving a mouse and pointing to or clicking on one of the pictures on the screen.

All the linguistic stimuli were digitally recorded using SoundEdit 16 by a female native speaker of English, speaking at an average rate of 174.7 words per minute. This rate is within the preferred listening range of adults (Gade & Mills 1989; Wingfield & Ducharme 1999), and has been identified as the preferred rate for unimpaired adults (Cain & Lass 1974; Lass & Fultz 1976). It is slightly faster than the preferred rate of slightly over 150 words per minute for aphasic adults reported by Reinsche, Wohlert and Porch (1983). (See Love, et al. in press for evidence that faster speech rates cause comprehension difficulties for aphasic individuals listening to non-canonical wh-movement sentences.) A list of all experimental items is provided in the Appendix.

2.1.2.2 Visual stimuli

The visual stimuli consisted of forty-eight panels with four images arrayed around a central fixation cross. The panels were each divided (with gridlines) into nine separate squares, with the images of the three animate actors and the location centered in the top left, top right, bottom left, and bottom right squares. The panel of images for the sample item in (4) is in Figure 1.

Figure 1.

Sample visual display.

The images were obtained from commercial sources, either from Microsoft ClipArt or from ClipArt.com. They were assembled into panels using Adobe Photoshop 7.0. The position of the four elements was counterbalanced across the trials, so that the agent, theme, location and distractor occurred equally often in each of the four possible positions in the grid.

2.1.2.3 Presentation list

The story-panel pairs were pseudorandomized for presentation. The twenty-four filler items were interspersed among the experimental items so that no more than two experimental items appeared adjacent to one another. The experimental items were also distributed evenly within each list, so that each experimental condition appeared roughly equally frequently in each half of the experiment. The experiment also began with a filler item.

2.1.3 Procedures

After providing informed consent, participants were seated in a quiet, dimly-lit room in front of a computer. Pictures were displayed on a 15-inch color Macintosh monitor approximately 24 inches from participants’ eyes, and pre-recorded instructions as well as the stories and beeps indicating an upcoming comprehension probe were played over a loudspeaker beside the monitor. Eye movements were monitored and recorded by a remote camera for an Applied Science Laboratories (ASL) model 6000 remote eye tracker, which was linked to a Dell computer. The remote camera sampled the position and the direction of the participants’ gaze once every 16.6 milliseconds.

The eyetracking system was calibrated to each participant’s eyes at the beginning of the experiment-running session. For the calibration, the participants were first asked to click an image of a gopher, which popped out of different holes on the screen. They were then asked to look at one of nine numbers on the monitor as directed by an experimenter. Participants were told to move only their eyes, not their head, if possible. Additional calibrations took place following the practice items (described below) and after every twelve trials thereafter.

Following the initial calibration and a brief verbal explanation of the procedure, participants were presented with four practice trials. Practice trials used the same story structure used in the trials in the experiment, and the comprehension probes for these trials were similar to the comprehension probes in the experiment (two questioning the theme, one questioning the agent, and one questioning the location of a transitive event). After they had completed the practice trials, their calibration was then checked and the experiment continued.

An individual trial proceeded as follows. First, a blank white screen appeared on the computer monitor for 1500 milliseconds, replaced by a central fixation cross for 300 milliseconds. Then, the blank screen was replaced by a panel and an accompanying story began to play over the loudspeaker. Once the story was complete, there was a beep, followed after 300 milliseconds by a comprehension probe. Participants then responded to the comprehension probe by moving the mouse to click on one of the images on the screen. Once they had responded, the experiment-running program automatically advanced to the next trial.

The experiment terminated automatically after forty-eight trials with a message thanking the participant. The entire experiment, from obtaining consent to the last trial, lasted approximately forty-five minutes.

2.1.4 Data analysis and reliability

Responses to comprehension probes (involving clicks to pictured items on the screen) were recorded automatically by the computer. The responses were scored following the experiment, and the proportion of correct responses for the two experimental conditions (object relative and passive) was calculated. Clicking on the theme was counted as a correct response for both experimental conditions, since it was the theme (the person being tickled) that participants are asked to point to in both conditions. All other responses were counted as incorrect.

Eye movement patterns are reported as proportions of fixations to different elements in the panel during comprehension probes. For analysis purposes the comprehension probes were segmented into four regions corresponding to critical regions in the sentences, as shown in Table 2 below.

Table 2.

Sentence regions, comprehension probes

	Region 1	Region 2	Region 3	Region 4
Object relative	Point to who	the bride	was tickling	at the mall.
Passive	Point to who	was tickled	by the bride	at the mall.

The critical region for examining automatic processing of movement was the verb region. Visual evidence of automatically associating the moved element with the verb or trace in the wh-movement and NP movement sentences was expected to appear in this region, in the form of more fixations to the theme (the groom) compared to the agent (the bride). Intuitively, when participants heard “Point to who the bride tickled in the mall” or “Point to who was tickled by the bride in the mall,” they should start looking at the groom (the person tickled) upon hearing the verb “tickled.” The verb signals a trace (a trace of wh-movement in object relatives and a trace of NP movement in passives) and assigns a thematic role to the moved element “who.” In addition, if participants are paying attention to the unfolding structure of the sentence, they should also be expected to look at the agent when hearing agent-related linguistic material. In the case of object relatives (“Point to who the bride was tickling …”), when hearing the subject “the bride,” participants should gaze at the image of the bride (the agent). In the case of passives (“Point to who was tickled by the bride …”), they should gaze at the image of the bride when hearing the by-phrase.

In addition to the main sentence regions described above, there was a further post-offset analysis region. This region comprised all fixations made between the offset of the comprehension probe and the participant’s response, which signaled the trial’s end. Further, as is standard practice in studies of this kind, the temporal boundaries of each sentence region were shifted 200 milliseconds downstream for the purposes of analysis. This practice compensates for the time required to program and execute an eye-movement.

The duration of the sentence regions was measured for the comprehension probes for each of the twenty-four experimental sentences. These measurements were carried out independently by two trained native speakers using SoundEdit 16. Intercoder reliability for these measurements (to within 50 milliseconds) was 82%, with a mean difference of 9 milliseconds per measurement. For each measurement where there was a difference of greater than 50 milliseconds, the measurement was reviewed by a third coder (the first author) who reconciled the two measurements.

The primary dependent variable used for analysis of the eye-movement data was fixation proportions: the proportion of fixations to a given element on the screen out of all fixations made during a given region of the sentence. The criterion used to define a fixation was higher than that used in the Dickey et al. (2007) study: participants had to fixate on the same position for 100 milliseconds (six consecutive samples) for it to count as a fixation. (This limit is approximately three times as long as the sample used in other eyetracking while listening studies, e.g., Sussman & Sedivy 2003.) A derived theme advantage score was calculated based on the proportion of theme fixations versus the proportion of agent fixations, by subtracting the proportion of agent fixations from the proportion of theme fixations. The theme advantage score represents the mean preference to gaze at the theme (compared to the agent) during a given region of the sentence. A positive theme advantage score represents a preference to gaze at the theme during a sentence region, while a negative theme advantage score represents a preference to gaze at the agent during that region. The use of a derived target-advantage score like this one is common in the sentence-comprehension literature using eyetracking (Arnold, Eisenband, Brown-Schmidt & Trueswell 2000; Runner, Sussman & Tanenhaus 2003).

Fixation proportions and theme advantage scores were calculated for each participant individually. These numbers were then averaged across participants for the purposes of statistical analysis. All t tests reported below are two-tailed unless otherwise noted.

2.2 Results

Results for object relatives are presented first, followed by the results for passives. The object relative sentences, which involve wh-movement, are important for testing the first hypothesis above, that aphasic participants (and controls) will exhibit visual evidence of gap-filling when processing wh-movement structures, not only for wh-questions (as shown by Dickey et al. (2007) but also for relative clauses. Data derived from the passive sentences examines gap-filling in NP-movement.

2.2.1 Object relatives

2.2.1.1 Accuracy

Control participants were 100% accurate in their responses to object-relative comprehension probes. In contrast, aphasic participants had an accuracy of only 36.5% (range: 16.7–75%) in their responses to the same probes. The aphasic participants’ accuracy was significantly lower than the controls’ (t(20)= 11.89, p<.001). In addition, the aphasic participants’ accuracy did not differ significantly from chance (t(7)=.480, p>.05). (Chance performance in this study was 33%, since there are three animate images which could in principle be responses to a command starting with “Point to who …”) This pattern of performance indicates significantly impaired comprehension of object relative-clause structures among the aphasic participants in this study, and it is consistent with many previous results in the literature (Caramazza & Zurif 1976, among others).

2.2.1.2 Eye-movements

The overall pattern of gazes at the agent and theme across the comprehension probe are presented in Figure 2a–b.

Figure 2.

a–b: Gazes at agent and theme pictures over time at 16.6 millisecond intervals, object relative comprehension probes, for aphasic participants (a) and control participants (b).

The control participants’ eye-movements are plotted in Figure 2a, whereas the aphasic participants’ eye-movements are plotted in Figure 2b. The data plots represent the proportion of trials in which participants were gazing at the agent (grey line) and the theme (black line) every sixteen milliseconds, starting from the onset of the trial. The boundaries of the sentence regions are represented as vertical lines on the graph. These graphs thus trace participants’ changes in visual attention across the sentence, as they shift between the agent of the event (the bride) and the theme (the groom, the ultimate target).

As can be seen, the aphasic and control individuals demonstrated similar patterns. Neither control nor aphasic participants exhibited a strong preference for the agent or theme early in the sentence, during the first sentence region (“Point to who”) or the second (“the bride”). However, both groups shifted their attention to fixate the theme during the third sentence region (the verb “tickled,” which signaled the trace/gap). The control participants continued to gaze at the theme for the remainder of the sentence through the end of the trial. In contrast, the aphasic participants exhibited continuing visual attention to (fixations on) the agent.

These patterns were tested statistically by comparing the control participants’ theme advantage to the aphasic participants’ theme advantage in each sentence region, and by comparing each group’s theme advantage to chance. The theme advantage scores for both groups are shown in Figure 3.

Figure 3.

Theme-advantage scores for aphasic and control participants, object relatives, by sentence region

In Regions 1 (“Point to who”) and 2 (the subject region, e.g., “the bride”) neither group exhibited a reliable preference to gaze at the theme or the agent: aphasic participants’ theme advantage scores did not differ from chance (Region 1: t(7)=.804, p>.05; Region 2: t(7)=.617, p>.05), nor did control participants’ (Region 1: t(7)=1.228, p>.05; Region 2: t(7)=1.123, p>.05). However, in Region 3, the verb region (“was tickling”), Region 4, the prepositional phrase region (“at the mall”), and Region 5, the post-offset region, a theme preference emerged for both aphasic and control participants. The theme preference was significant for control participants in Region 3 (t(13)= 7.254, p<.001), Region 4 (t(13)=5.103, p<.001) and in the post-offset region (t(13)=7.252, p<.001). The aphasic individuals showed a significant theme preference in Region 4 (t(7)= 2.851, p<.05) and post-offset (t(7)=3.130, p<.05), but not in Region 3 (t(7)=1.221, p>.05). Comparing the two groups’ performance across sentence regions, the only region in which aphasic and control participants’ theme advantage differed significantly was at the post-offset region (t(20)= 2.382, p<.05). In this region, the aphasic participants exhibited a weaker theme preference than controls.

To examine the aphasic participants’ performance further, a separate analysis was conducted comparing the theme advantage scores for the aphasic participants’ correctly and incorrectly comprehended trials, as was done for correctly and incorrectly comprehended object wh-question trials in Dickey, et al. (2007). These data are shown in Figure 4.

Figure 4.

Theme-advantage scores for correct and incorrect object-relative trials, aphasic participants, by sentence region

As with the parallel analysis from the Dickey, et al. study, these results must be treated with caution, since they represent unequal numbers of observations in correct and incorrect trial sets and the number of correct trials is relatively small (36.5% of trials or 35 of 96 total trials across the aphasic participants). As the figure indicates, the two sets of trials exhibited somewhat different patterns of eye-movements. In correct trials, aphasic participants exhibited a significant theme preference in Region 4 and in the post-offset region (Region 4: t(7)=3.112, p<.05; Post-offset: t(7)=4.063, p<.05). This preference was not reliable at Region 3 (t(7)=1.004, p>.05). In incorrect trials, there was not a reliable theme preference in any of these regions (Region 3: t(7)=.492, p>.05; Region 4: t(7)=1.551, p>.05; post-offset: t(7)=.455, p>.05)

The post-offset region is the only region where correct and incorrect trials exhibited a significant difference, with correct trials showing a stronger theme preference (F[1,7]=9.91, p<.05). At this region, there was a numerical (though not reliable) preference to fixate the agent for incorrect trials, while there was a significant preference to fixate the theme in correct trials. The post-offset region was also the only one where aphasic and control participants’ theme advantage scores differed significantly in the analyses reported above. Taken together, these findings suggest that the incorrect trials were the source of the reliable difference between aphasic and control participants in the post-offset region: the aphasic participants showed a weaker theme preference during this region in incorrect trials.

The findings for object-relative structures in this study are thus similar to those reported for object wh-question structures by Dickey et al. (2007). Aphasic and control participants exhibited relatively similar eye-movement patterns for object-relative structures, especially early in the sentence. In particular, both groups exhibited a preference to fixate the theme after hearing the verb signaling the gap/trace. There was clear evidence of this preference at the verb itself for control participants, and somewhat less clear evidence of this pattern for aphasic participants. The aphasic participants did not exhibit a reliable theme preference at the verb, in contrast to the results reported by Dickey, et al., (2007), but only at the following regions of the sentence. However, there was not a reliable difference between control and aphasic participants’ theme preference at the verb. The clearest difference between the two groups (as well as between correct and incorrect trials for aphasic participants) appeared instead in the post-offset region, after the sentence was complete. At this position, aphasic participants exhibited competition (in the form of more gazes at the agent) in the incorrect trials in particular. This latter finding is directly parallel to the results for object wh-questions reported by Dickey, et al. (2007). In that study, aphasic participants also showed evidence of competition (again in the form of a weaker preference to gaze at the theme) late in the sentences, particularly for incorrect trials.

2.2.2 Passives

2.2.2.1 Accuracy

Control participants were 90% accurate in their responses to passive comprehension probes. In contrast, aphasic participants had an accuracy of only 20% (range: 0–67%) in their responses to the same probes. The aphasic participants’ accuracy was significantly lower than the controls’ (t(20)= 11.74, p<.001). In addition, the aphasic participants’ accuracy did not differ significantly from chance (t(7)= 1.79, p>.05). The aphasic participants in this study were thus significantly impaired in their comprehension of passive probes, in keeping with many previous results in the literature (Grodzinsky 1990, 2000; Saffran et al. 1980, among others).

2.2.2.2 Eye-movements

The overall pattern of gazes to the agent and theme across the comprehension probe are presented in Figure 5a–b.

Figure 5.

a–b Gazes at agent and theme pictures over time at 16.6 millisecond intervals, passive comprehension probes, for aphasic participants (a) and control participants (b)

The control and aphasic participants’ eye-movements are plotted in Figures 5a and 5b, respectively, which depict the mean proportion of looks to the agent (grey line) and theme (black line) every sixteen milliseconds across critical regions of the sentence. Here again, the boundaries of the sentence regions are represented as vertical lines on the graph.

As can be seen, both participant groups shifted their gaze in similar ways during the passive probes, much as they did in response to the object relative probes. Both groups showed relatively undifferentiated looks to the agent and theme in Region 1, and both groups shifted their attention to gaze at the agent during Region 3 (the by phrase, “by the bride”) and continued to gaze at the agent during Region 4 as well as after sentence end, during the post-offset region. Control participants but not the aphasic participants showed a slight increase in fixations to the theme during Region 2 (the verb “was tickled,” which signaled the trace).

These patterns were tested statistically by comparing control participants’ theme advantage to aphasic participants’ theme advantage in each sentence region, and by comparing each group’s theme advantage to chance.

As shown in Figure 6, the theme advantage for aphasic and control participants did not differ statistically during any region of the sentence (Region 1: t(20)= .676, p>0.05, Region 2: t(20)= 1.008, p>0.05, Region 3: t(20)= 1.025, p>0.05, Region 4: t(20)= 1.51, p>0.05) nor during the post-offset region (t(20)=.679, p>0.05). However, comparison of theme advantage to chance revealed that neither group showed strong evidence of gap-filling for the passive sentences. Neither group’s theme advantage differed from chance in Region 2, the gap region (control participants: t(13)= .982, p>.05; aphasic participants: t(7)= .384, p>.05). Looking at later regions of the sentence, control participants showed a significant agent preference during Region 4 (t(13)= 4.566, p<.05), following the by phrase (“by the bride”) and during the post-offset region (t(13)= 4.396, p<.05). In contrast, the aphasic participants did not show a reliable agent preference during these regions (Region 4: t(7)= 1.192, p>.05; post-offset region: t(7)= 1.971, p>.05).

Figure 6.

Theme-advantage scores for aphasic and control participants, passives, by sentence region

To examine the aphasic participants’ performance further, a separate analysis was conducted comparing the theme advantage scores for the aphasic participants’ correctly and incorrectly comprehended trials. These data are shown in Figure 7.

Figure 7.

Theme-advantage scores for correct and incorrect passive trials, aphasic participants, by sentence region

Once again, these comparisons must be treated with caution, as they are based on unequal numbers of observations and relatively few correct trials (20% of trials or 24 of 96 total trials across the aphasic participants). As indicated by the figure, correct and incorrect passive trials elicited different eye-movement patterns. Correct trials exhibited a non-significant theme preference at Region 4 (t(7)= 2.115, p=.068) and in the post-offset region (t(7)= 2.115, p=.072). Incorrect trials exhibited a significant agent preference was significant in the post-offset region (t(7)= 4.206, p<.05). The post-offset region was also the only region in which correct and incorrect trials differed significantly (F[1,7]= 17.93, p<.05), with correct trials showing a non-significant theme preference and incorrect trials showing an agent preference in this region. This stronger theme preference for correct passive trials may be evidence of successful comprehension, if not gap-filling per se, given its late appearance.

The eye-movement findings for passive probes were thus somewhat different from the findings for object relative clauses in this study. Neither control nor aphasic participants showed strong visual evidence of gap-filling for the passive sentences. Comparison of correct and incorrect trials for the aphasic participants did reveal some evidence of greater visual attention to the target (the theme) in correct trials: aphasic participants were more likely to gaze at the theme in correct trials later in the sentence, particularly post-offset. However, the two groups’ theme advantage scores did not differ reliably in any region of the sentence. Their overall patterns of eye-movements in response to passive sentences were thus fairly similar.

2.3 Discussion

Results from the object relative clause structures were similar to those shown by Dickey et al. (2007) for object wh-questions, but not for object clefts. In particular, the aphasic and control participants showed similar eye-movement patterns during the early parts of the object relative probes: both groups looked at the image corresponding to the moved element after hearing the verb. For control participants, this preference appeared at the verb itself, a pattern consistent with visual evidence of gap-filling (viz. Sussman & Sedivy 2003). For the aphasic participants, this theme preference was slightly delayed, becoming statistically significant at the sentence region following the verb. This was also later than the theme preference emerged for wh-question for aphasic participants in the Dickey, et al (2007) study. Possible reasons for this slight delay in the emergence of the aphasic participants’ theme preference are discussed below.

More importantly, in both studies, we found that the aphasic participants’ eye-movements differed from controls’ most dramatically late in the sentence: aphasic participants showed a weaker theme preference than the controls (reliably lower theme-advantage scores) after the offset of the sentence. This was the only position where a reliable between-group difference appeared. In addition, this position is the only one at which a reliable difference between correct and incorrect trials emerged. This finding is also parallel to what we found in the previous study.

As pointed out earlier, we (Dickey et al., 2007) did not find a theme preference following the verb for either our normal or aphasic participants when object cleft structures were tested. In that study, however, the object clefts were not presented along with appropriate subject-extracted cleft filler items to serve as distractors. In the current study, participants heard not only object-extracted relative clauses but equal numbers of (canonical-order) subject-extracted active fillers. This mix of sentence types was sufficiently varied to guard against the development of strategic looking patterns, which likely influenced participants’ eye-movement behaviors for object clefts in the first study. Indeed, with these controls in place, the patterns found for object relative clause structures mirrored those from object wh-questions found in our previous study. Together, these results suggest that both normal and aphasic listeners may compute wh-movement dependencies on-line, as sentences unfold.

The results from the passive structures present a more complicated picture. As with the early parts of the object relative probes, the aphasic and control participants exhibited very similar eye-movement behaviors in response to the passive probes, with no statistically significant differences in theme-advantage scores between the groups at any region of the sentence. However, neither group exhibited a reliable theme preference at the position of the verb/trace of NP movement (“Point to who was tickled _ by …”), nor did either group exhibit an increase in looks to the theme at the position of the verb or any regions thereafter. Neither control nor aphasic participants thus exhibited much evidence of gap-filling for the NP movement dependency in passive sentences. The aphasic participants were somewhat more likely to gaze at the theme in correct than in incorrect passive trials, but this theme preference was not fully significant at any region, and correct and incorrect trials were significantly different only in the post-offset region. While these differences may provide visual evidence that aphasic participants were successfully identifying the theme as the target (the picture to be pointed to) in correct passive trials, they do not seem to provide strong evidence of gap-filling.

Interestingly, control participants and to a lesser extent aphasic participants exhibited an increase in looks to the agent after hearing the by phrase (“by the bride”). (It is worth noting that this agent preference was not reliable for the aphasic participants, and that it was also weaker or missing for aphasic participants’ correct passive trials.) This result suggests that even though they showed little evidence of gap-filling for passives, both groups of participants were nonetheless sensitive to the unfolding structure of the sentence. After hearing the agent argument in a by phrase, participants shifted their gaze to the corresponding image, despite the fact that it was irrelevant to the ultimate response to the probe (which asked them to point to the theme, “Point to who was tickled …”). This finding is parallel to other findings in the sentence-comprehension literature, which show that listeners automatically (even anticipatorily) gaze at pictures of arguments occurring in prepositional phrases (Boland 2005).

This finding also suggests that the looking patterns seen in the current study were not simply driven by the overt response to the task. Even though participants were supposed to point to the theme (and often did, at least in the case of controls), they nonetheless gazed at the agent when the linguistic structure provided overt cues prompting them to do so. If their eye-movement patterns were controlled only by the response they were ultimately preparing to execute, they would not be expected to look at the agent after hearing the by phrase.

The absence of gap-filling looks for passives was unexpected and deserves further comment, especially when compared to the presence of on-line gap-filling for wh-movement structures in this study. One potential explanation of the missing gap-filling effects for passives in the current study is that such effects have long been difficult to obtain in psycholinguistic studies (viz. Fodor 1989, Townsend & Bever 2001). While there is ample evidence of gap-filling for wh-movement dependencies at the position of the verb/trace (Lee 2004, Traxler & Pickering 1996, Sussman & Sedivy 2003, among others), the majority of psycholinguistic evidence that a moved NP is associated with an NP trace during sentence comprehension comes from off-line studies. These studies report evidence of reactivation of the NP after the end of the sentence, rather than at the position of the NP trace in the sentence (Bever & McElree 1988; MacDonald 1989). There is some evidence of within-sentence gap filling for NP movement from cross-modal lexical priming (Burkhardt et al. 2003, e.g.), but this evidence comes from the comprehension of sentences with unaccusative verbs, not passive sentences. Some have argued that the relative lack of on-line gap-filling evidence for passives reflects deep differences in the grammatical representation or processing of NP movement and wh-movement (Fodor 1989). Regardless of whether this conclusion is correct, the failure to find evidence of gap-filling for passives in the current study is not without precedent.

Another potential explanation for the absence of gap-filling looks for passives in the current study may relate to task difficulty. The passive probes took a passive structure and embedded it in a headless relative-clause structure (“Point to who …”). While this choice was necessary to make passive sentence structures compatible with the pointing commands used in this study, it also added a level of syntactic complexity that was not required for object relatives. That is, understanding the passive probes required not only computing the NP-movement dependency but also computing a wh-movement dependency. The additional complexity associated with the passive probes may help explain the accuracy data in this study: the control participants correctly comprehended 89.9% of the passives compared to 100% of the object relatives, while the aphasic participants correctly comprehended only 19.8% of the passives, as compared to 36.5% of the object relatives. This performance pattern is not the typical one found in the literature: most studies find object relatives to be more impaired than passives among aphasic individuals (see Friedmann 2006). The particular form of the command may have not only reduced participants’ accuracy but adversely affected the strategies comprehenders used to solve these sentences.

If this explanation is correct, using a different behavioral task in conjunction with eye-tracking may be more appropriate. For example, aphasic participants could perform a sentence-picture matching task with passive sentences while their eye-movements are monitored. This technique has already been used successfully to examine the real-time comprehension of passives among typically-developing children (Stromswold, et al. 2002) and may be promising for testing sentence processing in aphasia as well.

The same sort of explanation may also shed light on the slightly delayed emergence of the aphasic participants’ theme preference for object relative clauses in the current study. This apparent delay must be treated with caution, since no significant group differences in theme advantage were found at the position of the verb, the site of gap-filling in the Dickey, et al. (2007) and Sussman and Sedivy (2003) studies. However, the aphasic participants’ accuracy for the object relative sentences tested in the current study was much lower than the aphasic participants’ accuracy for the object wh-questions tested in the Dickey, et al. (2007) study (36.5% versus 70%). This difference suggests that the aphasic participants had significantly more difficulty in understanding the more-complex object wh-movement sentences tested here. This complexity may have affected their automatic processing of these sentences as well. In particular, the added complexity of these sentences may have prompted some aphasic participants to engage in strategic or slowed processing. Perhaps for some highly complex object wh-movement sentences (such as the ones used in the current study), aphasic participants engage in particularly slowed automatic processing of the wh-movement dependency. Love, et al. (in press) report evidence suggesting that for complex object relative clauses, aphasic participants showed evidence of significantly delayed gap-filling in a cross-modal lexical priming paradigm. Slowed gap-filling correlated with these participants’ poor comprehension performance. In contrast, this pattern of delayed gap-filling did not appear for the simpler wh-question sentences tested in the Dickey, et al. (2007) study.

If this account of the differences across studies is correct, it predicts that the relative complexity of the object wh-movement structures may result in different eye-movement patterns, due to slowed processing for more complex sentences. In particular, more complex wh-movement stimuli should elicit not only more errors (as found in the current study) but delayed gap-filling patterns. This possibility could be tested by directly manipulating the complexity of the wh-movement sentence types within a single experiment. More complex object relative clauses would be expected to elicit slower eye-movements consistent with automatic gap-filling than less-complex object wh-questions. If this account of these differences bears out, it would provide another example of the effects of syntactic complexity on aphasic language performance. Structural complexity has also been found to affect not only the likelihood of comprehension and production errors (Friedmann 2006; Friedmann & Grodzinsky 1997) but also recovery and generalization patterns in response to treatment (Thompson, et al., 2003).²

3. CONCLUSION

The results of the current study provide a partial replication and extension of the eye-tracking results of Dickey et al. (2007). For a new class of wh-movement structures (object relatives), both aphasic and control participants showed visual evidence of gap-filling, computing the wh-dependency and associating the displaced element with a gap or trace in real time. As discussed above, the appearance of this evidence was delayed for aphasic participants compared to controls. Importantly however, aphasic participants differed from controls most clearly in later segments of the sentence and after the sentence’s end, as they prepared to choose which picture to point to. At this point, they showed lingering attention to a picture (the subject distractor) which represented a salient but grammatically impermissible competitor interpretation. This pattern of late looks to a competitor interpretation also characterized the difference between correct and incorrect trials for the wh-movement sentences. These findings are parallel to those reported for object wh-questions in Dickey, et al. (2007): the clearest difference between aphasic and control participants in that study (as well as between correct and incorrect trials) emerged later in the sentence, and also took the form of looks to a picture representing a tempting but grammatically impermissible interpretation. Together, these findings suggest that agrammatic aphasic adults’ impairments in understanding object wh-movement sentences lie at least in part in exaggerated competition from interpretations which are tempting but are syntactically unlicensed. They emerge late, as agrammatic individuals use the results of their syntactic computations to assign an interpretation to the sentence (and choose which picture to point to).

This result strengthens the case that aphasic individuals do successfully compute wh-movement dependencies in at least some cases (now based on evidence from two wh-movement structures, and two groups of aphasic participants). See also converging evidence in favor of this conclusion from cross-modal lexical priming (e.g., Love, et al. 2001, Love, et al. in press). It is worth noting that the evidence of successful resolution of the wh-movement dependency appears later in the complex object-relative sentences tested here, as it does for the object relatives tested in the Love, et al. (in press) study. Furthermore, the current results also suggest (again consistent with our earlier findings) that the difference between aphasic and unimpaired comprehension of these structures is not relative slowness of the aphasic participants’ computation of the relevant syntactic structure, but because of competition which the unimpaired controls do not experience (or at least not to the same degree, viz. Ferreira 2003, Townsend & Bever 2001). However, additional replications are needed to support these claims, particularly the claim that competition from tempting but ultimately irrelevant interpretations is what underlies aphasic individuals’ comprehension difficulties (viz. Avrutin 2006, den Ouden 2006; see also Grodzinsky 1990).³

The current results also provide novel (if limited) evidence regarding the on-line comprehension of passives by both aphasic and unimpaired comprehenders. As such, they are among the few studies which have provided evidence regarding the real-time comprehension of non-canonical NP movement structures. (See also Burkhardt, et al. 2003, and see Bever & McElree 1988 and MacDonald 1989 for off-line evidence regarding the computation of NP movement structures.) However, the particular findings reported here – showing little evidence of gap-filling for passive structures, for either aphasic or unimpaired participants – may be due in part to the fact that passive structures were embedded inside subject-extracted relative clauses in this study.

Finally, the current results provide another demonstration of the usefulness of eye-tracking for studying aphasic language phenomena (see also Hallowell in press). In the current study, examining eye-movements in response to object relatives allowed us to see just how similar aphasic and control participants’ real-time understanding of these sentences was, despite the large differences in their accuracy is responding to the probes. Methods like cross-modal lexical priming (Love, et al. 2001) and self-paced listening (Caplan & Waters 2003) have provided parallel evidence in this regard, showing often surprisingly similar patterns of on-line comprehension by aphasic and unimpaired individuals. This methodology is now also being extended to examine real-time processes during sentence production in agrammatic aphasia (Thompson, Dickey, Cho, Lee & Griffin 2007). Using eye-tracking can help provide a detailed picture of what goes right (as well as what goes wrong) during aphasic language processing.

APPENDIX: List of experimental materials

Object relative items

1. One day a bride and groom were walking in the mall.

The bride was feeling playful, so the bride tickled the groom.

A clerk was amused.

Point to who the bride was tickling in the mall.

2. One day, a duck and a shark were swimming in the bay.

The duck got scared so the duck pecked the shark.

A fish swam away quickly.

Point to who the duck was pecking in the bay.

3. One day, a bishop saw an angel in the train station.

The bishop almost missed the train, but the angel helped the bishop.

A conductor thought he was hallucinating.

Point to who the angel was helping near the station.

4. One morning a singer and an agent were sitting in a café.

The singer spilled coffee, so the agent scolded the singer.

A busboy came with a towel.

Point to who the agent was scolding in the café.

5. One day, a zebra and a giraffe were racing on the savanna.

All the way across the plain, the giraffe trailed the zebra.

A camel was amazed by their speed.

Point to who the giraffe was trailing on the savanna.

6. One afternoon, a girl and a boy were playing in an abandoned house.

The boy was clumsy and by accident, the boy tripped the girl.

A man outside heard them playing.

Point to who the boy was tripping in the house.

7. One day, a farmer and a schoolgirl were walking in a cornfield.

The schoolgirl almost fell in a hole, but the farmer saved the schoolgirl.

A driver stopped to see what the commotion was.

Point to who the farmer was saving in the cornfield.

8. One evening a model and an artist were in the bedroom.

The model was beautiful, so the artist sketched the model.

A dealer later bought the sketches.

Point to who the artist was sketching in a bedroom.

9. One day, a monkey and a lion were in a cage.

The lion was huge, and the lion scared the monkey.

The bear in the next cage became restless.

Point to who the lion was scaring in the cage.

10. One evening, a fireman ordered some food from a waitress at a diner.

The diner was very busy, but finally the waitress served the fireman.

The frycook was exhausted that night.

Point to who the waitress was serving in the diner.

11. One night, a squid saw a crab crawling under the docks.

The crab tried to escape but the squid captured the crab.

A shrimp swam away quickly.

Point to who the fisherman was capturing under the docks.

12. One night a squirrel and a raccoon were foraging in a garbage can.

They both found food, but the squirrel chased the raccoon away.

A mouse picked up some crumbs.

Point to who the squirrel was chasing from the garbage can.

Passive items

1. Once upon a time, a ghost and a witch were haunting a castle.

The ghost felt mischievous, so the ghost chased the witch.

A priest came to exorcise the castle.

Point to who was chased by the ghost in the castle.

2. One night, a tenor and an actress were performing in an opera.

On stage, the tenor threatened the actress.

A lawyer watched anxiously.

Point to who was threatened by the tenor at the opera.

3. One morning, a mother and her baby were visiting the city.

The baby became sad, so the mother tickled the baby.

An executive on the street was envious.

Point to who was tickled by the mother in the city.

4. One morning a dentist and a professional climber went rockclimbing in a ravine.

The dentist became scared, so the climber helped the dentist.

A camper watched them from below.

Point to who was helped by the climber on the cliff.

5. One day a parrot and a tiger were in the jungle.

The tiger became hungry, so the tiger trapped the parrot.

A leopard watched from a tree.

Point to who was trapped by the tiger in the jungle.

6. One day a puppy and a kitten were playing in a fountain.

The puppy was being playful, so the puppy drenched the kitten.

A pigeon flew around the fountain.

7. One morning a pelican and a seagull were flying near the shoreline.

The seagull got tired, so the pelican carried the seagull.

A heron was waiting when they landed.

Point to who was carried by the pelican near the shoreline.

8. One morning a queen and her servant were leaving the palace together.

The queen walked out and the servant escorted the queen.

A guard closed the gates behind them.

Point to who was escorted by the servant from the palace.

9. One day an athlete and a scholar were sailing on the ocean.

The scholar almost fell off the boat, but the athlete rescued the scholar.

A sailor on the boat was very relieved

Point to who was rescued by the athlete on the ocean.

10. One evening, a butler and a chauffeur were fighting in a mansion.

They both became furious, and finally the chauffeur chased the butler.

The cook was terrified.

Point to who was chased by the chauffeur in the mansion.

11. One day, a wolf and a deer were sleeping near a cave.

The wolf became crazed, and the wolf attacked the deer.

A hawk watched as the deer escaped.

Point to who was attacked by a wolf near a cave.

12. One day a coach and a reporter were talking in a bar.

They both drank too much, and the coach threatened the reporter.

A patron tried to intervene.

Point to who was threatened by the coach in the bar.