Neural basis for sentence comprehension: Grammatical and short‐term memory components

Abstract

We monitored regional cerebral activity with BOLD fMRI while subjects were presented written sentences differing in their grammatical structure (subject‐relative or object‐relative center‐embedded clauses) and their short‐term memory demands (short or long antecedent‐gap linkages). A core region of left posterior superior temporal cortex was recruited during all sentence conditions in comparison to a pseudofont baseline, suggesting that this area plays a central role in sustaining comprehension that is common to all sentences. Right posterior superior temporal cortex was recruited during sentences with long compared to short antecedent‐gap linkages regardless of grammatical structure, suggesting that this brain region supports passive short‐term memory during sentence comprehension. Recruitment of left inferior frontal cortex was most clearly associated with sentences that featured both an object‐relative clause and a long antecedent‐gap linkage, suggesting that this region supports the cognitive resources required to maintain long‐distance syntactic dependencies during the comprehension of grammatically complex sentences. Hum. Brain Mapping 15:80–94, 2001. © 2001 Wiley‐Liss, Inc.

INTRODUCTION

The present study focuses on patterns of brain activation attributable to grammatical and short‐term memory processes during sentence comprehension. We used functional magnetic resonance imaging (fMRI) to define regional activation patterns in healthy adults during the course of interpreting sentences that embody two independent contrasts: 1) syntactic complexity, and 2) the short‐term memory (STM) demands associated with sentence processing.

Syntactic Complexity

To achieve the syntactic complexity contrast we used sentences with relative clauses that were structured either canonically (subject‐relative sentences) or noncanonically (object‐relative sentences). A canonical structure is one in which the noun phrase (NP) preceding the verb is mapped as the agent of the action (e.g., “the boy” is the agent of “chased” in the sentence “The boy that chased the girl was lost”). A non‐canonical structure is one in which an NP preceding the verb is not the agent, but rather the entity acted upon or undergoing the action (e.g., “the boy” is not the agent of “chased” in the sentence “The boy that the girl chased was lost”).

Our capacity to account for a verb's arguments without the cue of word order is formally articulated in, among other places, Chomsky's theory of government and binding [Chomsky, 1981]. Non‐canonical sentence structures are hypothesized to involve constituent movement, and movement of this sort leaves an abstract, phonetically unrealized placeholder (e) in the vacated position or “gap.” If a thematic position contains a gap, then the gap is assigned the appropriate thematic role and the moved constituent (or antecedent as it is also called) gets its role only indirectly, by being co‐indexed to the gap (indicated by the subscript i). For example, “the boy” becomes the “chasee” by being linked to a gap after the verb in the center‐embedded clause, as in: [The boy]_i that the girl chased e _i was lost. Thus, one important feature of sentence comprehension is its reliance on the formation of syntactically‐licensed long‐distance dependency relations.

Whether canonical structures also undergo movement of precisely the same sort remains an unsettled issue. That is, it is not clear if our canonical example should be represented as: [The boy]_i that e_i chased the girl was lost.1 But, even if such movement is not theoretically warranted for subject relatives, syntactic dependency linkages must still be posited, albeit of a different sort. So in our example, “The boy that chased the girl was lost,” it is the pronoun “that” that is assigned the role of agent, the “chaser,” in the subordinate clause, and “the boy” becomes agent only via co‐indexation ( [The boy]_i that _i chased…). At any rate, noncanonical structures are more complex and more difficult to process than canonical structures, reflected by the relative order of the thematic roles brought about by syntactic movement. This is an empirical fact [Ferreira et al., 1996; Ford, 1983; Frazier and Rayne, 1982]. This fact possibly supports the position that canonical structures do not require subject extraction (constituent movement). By this reasoning, the link between two phonological entities, an antecedent and a pronoun, is easier to form than the link between an antecedent and a gap that is a phonetically unrealized placeholder [Grodzinsky et al., 1993]. But the canonical–noncanonical complexity difference can be explained as readily if one adopts the theory that a gap mediates subject extraction just as it does object extraction. Even from this perspective, extra processing is demanded for noncanonical structures whether the extra work has to do with aligning syntactic and semantic linking mechanisms [Pinãngo et al., 1997; Pinãngo and Zurif, 1998] or with different memory and integration costs as specified, for example, in Gibson's parsing model [Gibson, 1998].2

Short‐Term Memory Demand

Whatever the processing costs associated with the structural differences between canonical (subject‐relative) and noncanonical (object‐relative) sentences, there are also syntactically‐relevant STM demands that have a “span” or linear character. In this experiment we manipulate this linear demand on memory independently of the canonical‐noncanonical difference.

We capitalize on the fact that syntactic displacement, the mismatch between where a NP is encountered in a sentence and where it is interpreted, is not just a feature of linguistic theory, but also figures importantly in accounts of real‐time sentence processing. The link between a displaced constituent and the gap (or pronoun) is actually formed in real time, as comprehension unfolds. Specifically, priming patterns during sentence comprehension show that the meaning of a displaced constituent is activated when it is first encountered in a sentence and then reactivated at the gap site indexed by the trace. This operation is termed “gap‐filling” [Swinney and Fodor, 1989]. It is fast‐acting (reactivation occurs as soon as it is syntactically licensed) and, prima facie, dependent upon short‐term memory. On this last point, the NP “the boy” in “The boy that the girl chased was lost,” must be retained in short‐term memory until the gap is encountered later in the sentence. Only at the gap site does it receive its thematic role of “chasee.” Increasing the distance between the displaced constituent and the gap, simply by interpolating additional words, can be expected to increase memory demands. Moreover, this is to be expected whether the linkage has to be established between an antecedent and a gap as is surely the case for an object‐relative sentence, or even between an antecedent and a pronoun, as might be the case for a subject‐relative sentence.

Neural Basis of Sentence Processing

Information concerning the neuroanatomical distribution of these linguistic and memory operations has come in large measure from aphasia research. The focus of this work has been Broca and Wernicke aphasia, two syndromes contrasting both neuroanatomically and in terms of comprehension deficit [Alexander et al., 1990; Mohr, 1976; Naeser et al., 1987; Naeser and Hayward, 1978; Tonkonogy, 1986; Vignolo, 1988]. With respect to anatomy, Broca aphasia essentially signals prerolandic and inferior frontal involvement of the left hemisphere, even though there is variation in lesion site and extent for this syndrome that is certainly not limited to Brodmann areas (BA) 44 and 45. Some of the variability in the locus of the lesion causing Broca aphasia can be explained cytoarchitectonically, because the gross macroscopic features of brain structure used to define this crucial language region do not map consistently across subjects onto the borders of the underlying cytoarchitectonic region [Amunts et al., 1999; Roland and Zilles, 1998]. Still, the frontal focus of Broca aphasia differs from the lesion site associated with Wernicke aphasia, the latter involving auditory association cortex in the posterior‐superior portion of the superior temporal gyrus in the left hemisphere.

As for the difference in comprehension deficits, the problem in Broca aphasia seems more syntactically focused than in Wernicke aphasia. Patients with Broca aphasia show good comprehension of canonically structured sentences but poor comprehension of non‐canonically structured sentences [Beretta et al., 1996; Caplan and Futter, 1986; Caramazza and Zurif, 1976; Drai and Grodzinsky, 1999; Grodzinsky, 1989; Zurif and Pinãngo, 1999]. Wernicke aphasic patients do not show this crisp pattern, but instead have comprehension problems with both sentence forms that suggest the influence of semantic factors also [Caramazza and Zurif, 1976; Heilman and Scholes, 1976; Pinãngo and Zurif, 1998].

A number of investigators have sought to describe the canonical–noncanonical pattern in Broca aphasia at the level of syntactic representation [Grodzinsky, 1990; Hickok et al., 1993; Pinãngo, 1999]. Although the generalizations that have emerged do not all agree on the precise source of the deficit, they all agree that it involves the failure to establish long‐distance dependencies—a failure that may be circumvented by strategic means for canonical but not for noncanonical structures. Furthermore, there seems to be a fairly broad consensus that this grammatical limitation is not a matter of knowledge loss, but rather arises because of an inability to implement grammatical knowledge in real time, as comprehension unfolds [Kolk and van Grunsven, 1985; Swinney et al., 1996; Zurif et al., 1993].

In one such processing view, the failure to establish syntactic dependencies during comprehension is seen as the direct consequence of slower‐than‐normal lexical activation. This temporal alteration is tolerated when the lexical item to be activated is present in the acoustic stream (or visual record), that is, at the point when the word is physically encountered in the sentence. It is not tolerated, however, when there is no phonological presence to guide the reactivation of the word at the gap site. This view is supported by priming experiments that afford a real‐time perspective on the construction of links between displaced constituents and gap sites. These experiments have shown a stark contrast between Broca aphasic and Wernicke aphasic patients. Thus, Broca aphasics, who have been independently shown to have slowed activation, do not show gap‐filling, whereas Wernicke patients, who routinely exhibit normal initial activation patterns apart from sentence processing, do show gap‐filling—even for sentences that they have problems understanding [Prather et al., 1992; Swinney et al., 1996; Zurif et al., 1993]. In effect, slow processing and a related failure to construct syntactic dependencies, results only from left inferior frontal damage within the perisylvian language region.

There is, however, another possibility concerning the functional commitment of the left inferior frontal cortex associated with Broca aphasia. Several investigators have suggested that it accommodates the memory storage demands that arise during comprehension. It is well known that Broca aphasics have limited auditory‐verbal STM [Beeson et al., 1993; Butters et al., 1970; Cermak and Tarlon, 1978]. And as we have pointed out earlier, syntactic dependencies of the type described here require a memory buffer to effect the link between an antecedent and gap, that is, to allow the antecedent to be reactivated at the position at which it receives its thematic role.

It is possible also that the inferior frontal region implicated in Broca aphasia sustains multiple processing functions, including both speed of input activation and STM. Yet another possibility is that one processing requirement interacts with the other; that memory capacity is diminished in Broca aphasia only as a consequence of slower‐than‐normal processing.

The point to be taken is that this left anterior cortical region should show increased activation in neuroimaging studies of sentence comprehension in healthy adults as a function of increasing the cost of forming a syntactic dependency, a cost that can be measured by the amount of intervening material (number of words) between the two end‐points of the dependency relation. Moreover, the overall processing cost, and therefore the involvement of left inferior frontal cortex, is expected to be higher when the linking operation is carried out within a noncanonical structural context than when carried out within a canonical structural context [Gibson, 1998]. As already noted, quite apart from the length traversed by the dependency relation, processing complications are associated with noncanonical sentence structures that are not found in canonical structures [Pinãngo et al., 1997].

Clearly, left inferior frontal cortex will not be the only area activated during the processing of structurally complicated sentences. We fully expect activation in left posterior superior temporal cortex also, but not in a way that reflects sensitivity to the syntactic distinctions described here. As for other brain regions, any activation shown will have a participatory, as opposed to a necessary, status. This last distinction is dictated by the lesion data, by the fact that only left perisylvian cortex appears to be crucially involved in sustaining language.

In fact, some of our expectations have already been met by recent imaging analyses. Several studies with healthy adults have confirmed the importance of left inferior frontal cortex in syntactic processing, and with what seems to be far greater neuroanatomical precision than is available in the aphasia literature. The data from aphasia implicate not only the classical Broca area (BA 44 and BA 45), but also, as already mentioned, deeper and adjacent areas over a considerable extent of left inferior frontal cortex. By contrast, neuroimaging studies have tended to emphasize only BA 44 or BA 45 [Caplan et al., 1998a; Dapretto et al., 1998; Friederici et al., 2000; Stowe et al., 1998; Stromswold et al., 1996].

What is unclear, however, is whether claims for such specificity are warranted. Caplan and his colleagues have observed activation increases in BA 44 specifically for noncanonical sentences that contain object‐relative clauses [Caplan et al., 1998a; Stromswold et al., 1996].3 Not all manifestations of the canonical‐noncanonical difference point to BA 44, at least, not consistently so. In a theoretically relevant way the contrast between active and passive voice sentences manifests the canonical‐noncanonical difference no less than does the subject relative‐object relative difference. Yet, Caplan [2000] failed to find any reliable activation differences in a study comparing actives and passives. Moreover, this outcome is at odds with the results of a study carried out by Dapretto et al. [1998]. The latter did find residual activation in BA 44 for passives as compared to actives (although this comparison appeared among at least one other that did not involve canonicity) [Dapretto et al., 1998]. In addition, Ni et al. [2000] found left inferior and middle frontal (BA 44, 46, and 47) activation in an event‐related experiment of syntactic anomalies in sentences that avoided explicit judgments. This also suggests that areas adjacent to BA 44 and 45 are involved in syntactic processing.

Interpretation of these imaging data is made more difficult by the failure of much of this work to focus on minimal syntactic contrasts. Clearly this circumstance will change. Caplan and his colleagues have already started to address this issue in an imaging study using a subject‐object cleft construction contrast. They found that processing cleft object sentences, compared to cleft subject sentences, is associated with greater activation in BA 45 [Caplan et al., 1999]. Still, apart from this study, even the Caplan group conflates different syntactic features in their comparisons. Specifically, in each of their subject–object relative contrasts, they conflate place of embedding and site of extraction so that the noncanonical structures they used were also center‐embedded, whereas the canonical structures were configured in a less complex right‐branching format [Caplan et al., 1998a; Stromswold et al., 1996]. It is not clear, therefore, which complexity feature implicates recruitment of which portion of left inferior frontal cortex. Some investigators have examined regional recruitment patterns in sentences with canonical and noncanonical structures, where the former stimuli also featured shorter antecedent‐gap linkages than the latter stimuli [Just et al., 1996]. They found that the areas of activation increased in left superior temporal and inferior frontal regions (and to a lesser extent in homologous right hemisphere regions) as sentence complexity increased from active conjoined sentences to sentences containing subject‐relative clauses, and then to object relative clauses. Unfortunately, these studies have not controlled for antecedent‐gap distance, and as we suggested above, this factor too has likely shaped the activation patterns found for the canonical‐noncanonical contrasts.

The importance of this syntactic linkage distance factor is emphasized by the fact that left inferior frontal cortex is activated by verbal STM tasks, even apart from sentence comprehension [Awh et al., 1996; Cohen et al., 1994; Grasby et al., 1993; Rypma et al., 1999; Smith et al., 1996]. Relevant here also is the work of Stowe et al. [1998]. They assessed the role of STM in sentence processing more directly by charting activation for four levels of STM demand: 1) simple declarative sentences; 2) either complex sentences containing a center‐embedded clause or sentences with a list of adjectives modifying a noun; 3) ambiguous sentences where the initial words could be interpreted in two different ways; and 4) lists of unrelated words. They found activation of the left insula near Broca area in association with a greater verbal short‐term memory load (i.e., processing demands ordered as simple < complex = word list < ambiguous), but they found increased activity in the left posterior superior and middle temporal gyrus in association with a greater grammatical processing load (i.e., processing demands ordered as word list < simple < complex < ambiguous). Thus, the degree to which grammatical structure recruits left inferior frontal cortex may be modulated by the constraints of STM demand.

Building upon this body of work, the present experiment used BOLD fMRI to study the neural basis of sentence comprehension in healthy adults with sentence material that manipulated grammatical canonicity and varied short‐term memory load independently and in a linguistically relevant fashion. In particular, our fully penetrated design included four types of sentences that featured: subject‐relative or object‐relative center‐embedded clauses (i.e., canonical compared to non‐canonical sentence structure); and a three‐word or a seven‐word antecedent‐gap distance (i.e., short or long linkage between the gap or pronoun and its co‐indexed noun). We administered written sentences in a word‐by‐word fashion. Examples of the four types of sentences are presented in Table I. This design allowed us to specify in detail the neural recruitment patterns associated with canonically and noncanonically‐structured sentences that respectively have subject‐relative and object‐relative center‐embedded clauses, and with varying short‐term memory demands in sentences featuring short and long antecedent‐gap (or antecedent‐pronoun) distances. These semantically reversible sentences contained NPs with a male and a female, and subjects were asked to consider whether a male or a female performed the action described in the sentence. We used a statistical parametric mapping (SPM) approach for two types of analyses: to contrast the activation pattern for each type of sentence with a neutral baseline task that required the detection of one of two targets in strings of word‐like pseudofont characters; and to directly compare activation patterns associated with pairs of sentences.

Table I.

Mean (SD) accuracy and latency to respond to stimuli during sentence comprehension challenges

Sentence type	Stimulus example	Mean (SD) % correct	Mean (SD) msec latencya
Subject‐relative, short antecedent‐gap	[The strange man]i in black who e i adored * Sue was rather sinister in appearance	97.8 (0.019)	612 (365)
Subject‐relative, long antecedent‐gap	[The cowboy]i with the bright gold front tooth who e i rescued * Julia was adventurous	95.8 (0.041)	608 (280)
Object‐relative, short antecedent‐gap	[The flower girl]i who Andy * punched e i in the arm was five years old	96.4 (0.008)	977 (493)
Object‐relative, long antecedent‐gap	[The messy boy]i who Janet * the very popular hairdresser grabbed e i was extremely hairy	96.1 (0.048)	2504 (898)

^aLatency to respond was measured from the asterisk indicated in the sentence, since enough information was available only by this point in the sentence to respond to the probe.

METHODS

Subjects

We studied seven right‐handed native English‐speakers who were students at the University of Pennsylvania. These included five females and two males, with a mean (±SD) age of 20.7 years (±3.04 years), range 19–26 years. These subjects participated in an informed consent procedure approved by the IRB at the University of Pennsylvania.

Materials

Four types of sentences were presented to subjects, as summarized in Table I. This design included subject‐relative sentences with short and long antecedent‐gap distances as well as object‐relative sentences with short and long antecedent‐gap distances. Specifically, we presented sentences equal in overall word length that have subject‐relative or object‐relative center‐embedded clauses. In both types of sentences, we manipulated the distance between the gap and its co‐indexed noun by the strategic location of filler material that did not alter phrase structure, resulting in short antecedent‐gap distances of three words or long antecedent‐gap distances of seven words. Half of each type of sentence had a female as the agent, the remainder a male. The task for all sentences was to decide whether a female or male is the agent of the action in the sentence. Subjects indicated their response by pressing one of two buttons with the thumb of the left hand or the right hand.

We employed a word‐by‐word presentation technique to avoid eye movements associated with free‐field sentence reading, to control the rate of sentence presentation, and to monitor sentence processing time more carefully. To equate for duty cycle and the amount of time needed to process sentences of unequal difficulty, subjects responded to a sentence as soon as they felt that they had the correct response, and the next sentence was initiated by the subject's response. This also minimized grammatically irrelevant aspects of short‐term memory by not requiring subjects to retain their response until the end of sentence presentation. The latency to respond to a sentence was measured from the earliest point in each sentence at which the correct answer could be determined. For subject‐relative sentences this point was at the verb of the subordinate clause. For object‐relative sentences, this point was the subject of the subordinate clause. We avoided administering the stimuli aurally because the loud (∼80 dB) noise level associated with MRI magnet operation could interfere with the subjects' ability to hear the sentences clearly, we wished to avoid the interpretive confound of recruiting primary and secondary auditory cortices for the stimulus modality versus language‐sensitive cortices in a left peri‐Sylvian distribution, and we wanted to minimize phonologic task components.

In each run of stimulus presentation, subjects were acclimated to the MRI environment by viewing a blank screen for 20 sec and then an asterisk for 40 sec. Eight randomly‐ordered blocks of sentences (including two blocks of each type of sentence) were presented for 40 sec each and without a break between blocks of different sentence types. Subjects were not informed that blocks of different types of sentences were being administered. Four runs of sentence stimuli were presented in total, and the rate of word presentation alternated across runs at 750 msec/word and 500 msec/word. We found no effect for presentation rate, so we grouped data across levels of this factor in the analyses presented below. Two baseline blocks of stimuli designed to resemble the sensory‐motor properties of the sentence material were also presented in each run. The baseline task probed detection of one of two pseudofont targets in a sentence‐like string of pseudofont words, resembling the two‐choice probe of the sentences. These were presented analogously to the true sentences (each pseudofont word presented sequentially one at a time, each string containing 13 pseudofont words, the ability to respond as soon as the target was seen, a response triggered the next string). The targets were presented at the beginning of each of these baseline blocks for 1 sec. Pauses in performance were included between runs (every 8 min 20 sec).

An LCD projector (Epson 5000) compatible with high magnetic fields was used to back‐project visual stimuli onto a screen at the magnet bore. The subject viewed the screen through a system of mirrors available on the GE head coil. A portable computer (Macintosh 1400C) outside the magnet room used PsyScope presentation software [Cohen et al., 1993] to present stimuli and record response accuracy and latency.

Subjects were familiarized with the word‐by‐word presentation technique and the gender probe before entering the magnet bore, and the task was practiced by each subject. We monitored behavioral accuracy and latency while imaging data were being collected. One subject had been excluded from our analyses because her performance did not differ from random for several blocks of sentences. A technical malfunction prevented the collection of behavioral data in one subject, and a portion of the behavioral data (one run) in another subject.

Imaging Data Acquisition and Statistical Analysis

This experiment was carried out at 1.5 T on a GE Echospeed scanner. We used the standard clinical quadrature radiofrequency head coil. Foam padding was used to restrict head motion. Each imaging protocol began with a 10–15 min acquisition of 5 mm thick adjacent slices for determining regional anatomy, including sagittal localizer images (TR = 500 msec, TE = 10 msec, 192–256 matrix), T2‐weighted axial images (FSE, TR = 2000, TEeff = 85 msec), and T1‐weighted axial images of slices used for fMRI anatomic localization (TR = 600 msec, TE = 14 msec, 192 × 256 matrix). Gradient echo echoplanar images were acquired for detection of alterations of blood oxygenation accompanying increased mental activity. All images were acquired with fat saturation, a rectangular FOV of 20 × 15 cm, flip angle of 90°, 5 mm slice thickness, an effective TE of 50 msec, and a 64 × 40 matrix, resulting in a voxel size of 3.75 × 3.75 × 5 mm. The echoplanar acquisitions consisted of 24 contiguous slices covering the entire brain every 2 sec. A separate acquisition lasting 1–2 min was needed for phase maps to correct for distortion in echoplanar images. Raw data were stored by the MRI computer on DAT tape and then processed off‐line.

Initial data processing was carried out with Interactive Data Language (Research Systems, Boulder, CO) on a Sun Ultra 60 workstation. Raw image data were reconstructed using a 2D FFT with a distortion correction to reduce artifact due to magnetic field inhomogeneities. Individual subject data were then prepared for pseudosubject analysis and analyzed statistically using statistical parametric mapping (SPM96), operating on a MatLab platform, developed by the Wellcome Department of Cognitive Neurology [Frackowiak et al., 1997]. Briefly, the images in each subject's time series were registered to the initial image in the series. The images were then aligned to a standard coordinate system [Talairach and Tournoux, 1988]. The data were spatially smoothed with an 8 mm Gaussian kernel to account for small variations in the location of activation across subjects, and temporal smoothing was conducted with a 2.8 sec kernel to account for small variations in the hemodynamic response function. The data were pooled and analyzed parametrically using t‐test comparisons converted to z‐scores for each compared voxel. The initial set of statistical analyses we report compared each type of sentence to a pseudofont baseline, and these satisfied a rigorous statistical threshold of P < 0.05 following correction for multiple comparisons for both height and extent of activation. Subsequent statistical analyses comparing two different types of sentences were designed to test specific hypotheses, and we report differences that exceeded a statistically significant height criterion at least at the uncorrected P < 0.002 level, equivalent to a z‐score >3.00.

RESULTS

Behavioral Observations

Accuracy in understanding the four types of sentences during imaging is summarized in Table I. There were no significant differences in the subjects' accuracy for the different sentence types. The latency data are also summarized in Table 1. A repeated‐measures analysis of variance (ANOVA) with a 2 × 2 design (grammatical form [subject‐relative, object‐relative] × antecedent‐gap distance [short, long]) revealed significant main effects for grammatical form [F(1,5) = 48.92; P < 0.001] and antecedent‐gap distance [F(1,5) = 27.35; P < 0.003], and a significant grammatical x distance interaction effect [F(1,5) = 26.02; P < 0.004]. t‐Tests revealed that subjects responded significantly more slowly to object‐relative than subject‐relative sentences [t(5) = 6.99; P < 0.001], and significantly more slowly to long than short antecedent‐gap distance sentences [t(5) = 5.23; P < 0.003]. We also found that they responded significantly more slowly to object‐relative long antecedent‐gap than object‐relative short antecedent‐gap sentences [t(5) = 5.31; P < 0.003], and significantly more slowly to object‐relative short antecedent‐gap sentences than subject‐relative long antecedent‐gap sentences [t(5) = 3.32; P < 0.02] and subject‐relative short antecedent‐gap sentences [t(5) = 6.29; P < 0.001]. Subject‐relative long antecedent‐gap sentences did not differ statistically from subject‐relative short antecedent‐gap sentences. These findings confirm an order of difficulty indicating that object‐relative sentences with a long antecedent‐gap distance require the most time to understand, and that sentences featuring only a long antecedent‐gap distance (i.e., in the context of canonical subject‐relative sentences) are understood relatively rapidly.

Imaging Observations

Table II summarizes the coordinates of peak activation in the regions that were activated at least at the P < 0.05 level of significance in the four types of sentences, compared to the pseudofont baseline, following correction for multiple comparisons. The activation pattern associated with each type of sentence, contrasted with the pseudofont baseline, is illustrated in Figure 1. All of the sentences recruited the posterior superior temporal region of the left hemisphere, including portions of Brodmann areas (BA) 21, 22, and 39, in comparison to the pseudofont baseline. All of these contrasts also showed significant activation of the lingual and fusiform region that was somewhat more prominent in the left hemisphere than the right hemisphere, including portions of BA 18 and 19 in inferior temporal‐occipital cortex. We also observed significant recruitment of the right posterior superior temporal region for sentences with a long antecedent‐gap distance, including portions of BA 22 and 21. This included subject‐relative long antecedent‐gap sentences (Panel B) and object‐relative long antecedent‐gap sentences (Panel D). Finally, some areas of activation were associated only with a specific type of sentence. In particular, significant activation of left inferior frontal cortex (BA 47) was evident only in object‐relative long antecedent‐gap sentences (Panel D). Left hippocampal activation was associated only with object‐relative long antecedent‐gap sentences as well.

Table II.

Locus and extent of peak activation in brain regions during sentence comprehension in comparison to a pseudofont baseline

Scan	Activation locus (Brodmann area)	Co‐ordinates			Z‐value
		x	y	z
[Subject‐relative short]	Left posterior superior and middle temporal (22, 21, 39)	−68	−44	8	6.33
Bilateral lingual/fusiform (18, 19)	4	−84	−16	4.48
[Subject‐relative long]	Left posterior superior and middle temporal (22, 21, 39)a	−68	−44	12	3.89
Right posterior superior and middle temporal (22, 21)	60	−24	−4	4.34
Bilateral lingual (18, 19)	0	−88	−8	4.86
[Object‐relative short]	Left posterior superior and middle temporal (22, 21, 39)	−64	−56	8	6.09
[Object‐relative long]	Left posterior superior and middle temporal (22, 21, 39)	−64	−56	8	5.76
Right posterior superior and middle temporal (22, 21)	52	−68	16	4.56
Right posterior superior and middle temporal (22, 21)	60	−20	−8	4.44
Left inferior frontal (47)	−52	28	−8	5.01
Left hippocampus	−32	−20	−20	4.89
Bilateral lingual/fusiform (18, 19)	0	−92	−8	6.05

^aFollowing correction for multiple comparisons, this contrast was significant at P < 0.18.

Figure 1.

Regional activation patterns associated with contrasts of each type of sentence minus the pseudofont baseline. The recruited areas are displayed on right hemisphere (pictured on the left) and left hemisphere (pictured on the right) lateral views of a brain normalized to Talairach space. All displayed regions represent a significant difference between the contrasted conditions at P < 0.05 following correction for multiple comparisons, as summarized in Table II. (A) Subject‐relative short antecedent‐gap sentences − pseudofont baseline. (B) Subject‐relative long antecedent‐gap sentences − pseudofont baseline. (C) Object‐relative short antecedent‐gap sentences − pseudofont baseline. (D) Object‐relative long antecedent‐gap sentences − pseudofont baseline.

Additional analyses directly contrasted pairs of sentences to test a priori hypotheses about the role played by these brain regions in grammatical and short‐term memory aspects of sentence comprehension. For this reason, an uncorrected statistical threshold was established at P < 0.002, equivalent to a z‐score >3.00, unless otherwise stated. These contrasts are summarized in Table III and illustrated in Figure 2. We present first the contrasts highlighting structural features of a sentence, then the contrasts emphasizing short‐term memory, and finally the contrasts assessing the combined role of structural and short‐term memory features.

Table III.

Locus and extent of peak activation in brain regions during sentence comprehension in comparison to other types of sentences

Scan	Activation locus (Brodmann area)	Co‐ordinates			Z‐value
		x	y	z
[Object‐relative long] − [Subject‐relative short]	Left inferior frontal (47)	−52	28	−8	3.24
Right posterior superior temporal (22, 21, 39)	52	−68	16	3.06
Bilateral lingual (18)	0	−92	−8	3.74
[Object‐relative long] − [Object‐relative short]	Left inferior frontal (47)	−56	12	−4	3.11
Right posterior superior temporal (22, 39)	48	−64	16	3.07
[Subject‐relative long] − [Object‐relative short]	Right posterior superior temporal (22, 21)	56	−24	0	4.14
[Subject‐relative long] − [Subject‐relative short]a	Right posterior superior temporal (22, 21)	56	−24	0	2.98
[Object‐relative short] − [Subject‐relative short]	Left posterior temporal‐occipital (19)	−48	−68	−8	3.96
Bilateral lingual/fusiform (18, 19)	−4	−92	−8	4.30
Right lingual/fusiform (18, 19)	28	−68	−20	3.56
[Object‐relative long] − [Subject‐relative long]	Left posterior temporal‐occipital (19)	−40	−76	−4	4.42
Left ventral temporal/hippocampus	−32	−20	−20	3.49
Right fusiform (19, 37)	36	−40	−12	3.28
Right lingual (18)	16	−92	−12	3.28
[Object‐relative short] − [Subject‐relative long]	Left lingual/fusiform (19)	−4	−56	−12	3.28
Left posterior cingulate (30)	−8	−36	16	3.19

^aThis contrast was significant only at the P < 0.005 level.

Figure 2.

Regional activation during contrasts of specific types of sentences. The recruited areas are displayed on right hemisphere (pictured on the left) and left hemisphere (pictured on the right) lateral views of a brain normalized to Talairach space. All displayed regions represent a significant difference between the contrasted conditions at P < 0.002 (z > 3.00), as summarized in Table III. (A) Object‐relative short antecedent‐gap sentences − subject‐relative short antecedent‐gap sentences. (B) Object‐relative long antecedent‐gap sentences − object‐relative short antecedent‐gap sentences. (C) Subject‐relative long antecedent‐gap sentences − object‐relative short antecedent‐gap sentences. (D) Object‐relative long antecedent‐gap sentences − subject‐relative short antecedent‐gap sentences.

We contrasted object‐relative short antecedent‐gap sentences and subject‐relative short antecedent‐gap sentences (Panel A) to assess the role of a non‐canonical word order without the confound of a long antecedent‐gap distance. This contrast did not reveal activation in left inferior frontal cortex, although recruitment was evident in a temporal‐occipital distribution. Nor did we observe recruitment of left inferior frontal cortex in the contrast of object‐relative long antecedent‐gap and subject‐relative long antecedent‐gap types of sentences, or the contrast of object‐relative short antecedent‐gap sentences and subject‐relative long antecedent‐gap sentences.

To assess the role of short‐term memory without the confound of non‐canonical word order, we contrasted subject‐relative long and subject‐relative short types of sentences. This revealed significant recruitment only in right posterior superior temporal cortex (BA 22 and 21) at a P < 0.005 level of significance. We also contrasted object‐relative long antecedent‐gap sentences and object‐relative short antecedent‐gap sentences (Panel B). This also revealed recruitment of right posterior temporal cortex (BA 22 and 39) as well as activation of left inferior frontal cortex (BA 47). Antecedent‐gap distance was also assessed in a contrast of subject‐relative long antecedent‐gap sentences and object‐relative short antecedent‐gap sentences (Panel C), and this demonstrated only recruitment of right posterior superior temporal cortex (BA 22 and 21).

The combined role of structural and short‐term memory sentence features was evaluated with a comparison of object‐relative long antecedent‐gap sentences and subject‐relative short antecedent‐gap sentences (Panel D). This revealed significant activation of Broca area in the inferior frontal region of the left hemisphere (BA 47). We also observed greater recruitment of right superior temporal cortex (at the junction of BA 39, 22, and 21) during this contrast. Neither this contrast nor any of the other contrasts involving two different types of sentences revealed relative activation of left posterior superior temporal cortex, emphasizing the relatively equal demands made of this region across all types of sentences.

DISCUSSION

We observed a complex pattern of recruitment during sentence comprehension that varied depending on the structural and short‐term memory characteristics of the sentence. A core region in left posterior superior temporal cortex was activated under all sentence conditions. Ventral temporal‐occipital regions were recruited somewhat more robustly in the left hemisphere than the right hemisphere for all types of sentences as well. These two effects emerged when the sentences were contrasted with the pseudofont baseline, suggesting the roles of these brain regions in processes common to the comprehension of many different types of written sentences. Our findings also revealed distinct activation patterns associated with contrasts of particular types of sentences. Specifically, posterior superior temporal cortex in the right hemisphere was recruited during the comprehension of sentences with a long antecedent‐gap linkage compared to sentences with a short antecedent‐gap linkage. This area of activation was evident regardless of the grammatical complexity of the types of sentences being contrasted, emphasizing the role of this right temporal region in a short‐term memory component of sentence comprehension. By comparison, the effect of increasing STM demands on left inferior frontal cortex was more nuanced. Memory costs exerted some effect independent of structure. There was, after all, no residual activation in this area when long distance object‐relatives were contrasted with subject‐relatives that also had linkages spanning many words. Still, only the joint appearance of a long‐distance dependency link and a noncanonical structure reliably activated this area in the other contrasts.

Before elaborating upon each of these activation patterns, we again emphasize the distinction between necessary and participatory regions. As justified by aphasia research, the functional commitment of left inferior frontal cortex and that of left posterior superior temporal cortex are crucial to normal language processing; not so, the function of the other regions showing activation in this study. Viewed from the perspective of deficit analyses, these other regions seem to have participatory, yet non‐essential status.

Left Inferior Frontal Cortex

We observed an important interaction effect in our study, where left inferior frontal cortex was activated during object‐relative sentences that featured a long antecedent‐gap linkage. This region was not recruited during any of the other three sentence types minus baseline. This suggests that left inferior frontal cortex is not recruited solely under conditions of noncanonical word order, nor solely for the purpose of supporting verbal STM. This interaction is further supported by specific sentence contrasts. Left inferior frontal cortex was residually activated in two specific contrasts ([object‐relative long] − [subject‐relative short] and [object‐relative long] − [object‐relative short]), but not for two other contrasts ([object‐relative long ] − [subject‐relative long] and [object‐relative short] − [subject‐relative short]). The first contrast (i.e., [object‐relative long] − [subject‐relative short]) clearly indicates a canonicity‐length interaction: viz, inferior frontal activation is most apparent when constructing syntactic dependencies that span long distances within noncanonical structures. The second contrast (i.e., [object‐relative long] − [object‐relative short]) shows that this left anterior region is also engaged by memory demands associated with the formation of distant syntactic links in noncanonical sentences even when the grammatical structure of the sentence is controlled. The latter two observations suggest that pairs of sentences contrasting in grammatical structures do not necessarily recruit left inferior frontal cortex, particularly when the sentence pairs are matched in the length of their antecedent‐gap linkages. Moreover, the failure to recruit left inferior frontal cortex during the [subject‐relative long] − [subject‐relative short] contrast indicates that a lengthy antecedent‐gap linkage by itself is not sufficient to recruit left inferior frontal cortex.

This recruitment pattern can be aligned with data from aphasia. In both domains of inquiry, left inferior frontal cortex is seen to sustain elemental processing resources necessary for forming syntactic dependencies. The aphasia data suggest that this area underlies long‐distance syntactic linkages because of its commitment both to speed of information access as well as to STM capacity [Zurif, 1996]. The present fMRI data emphasize the commitment of this area to STM during the processing of long‐distance syntactic dependencies. This is not surprising given our use of carefully spaced, written word sequences. This form of sentence presentation contrasts with the rapid succession of words in speech. It de‐emphasizes temporal constraints and correspondingly increases memory demands over the longer course of successive word presentation.

As already forecast in our introduction section, there are several possible explanations for the finding that forming a distant syntactic dependency more clearly activates left inferior frontal cortex during the course of processing a noncanonical sentence than when processing a canonical sentence. One is that only the former type of structure undergoes syntactic movement. For a canonical structure, the syntactic link, however long, is not between an antecedent and a phonologically silent gap, but between two phonological shapes, an antecedent and a pronoun. This difference, we assume, makes the attachment easier to form.4

Even if we accept the possibility that gaps mediate subject, as well as object, extraction, it remains possible, however, to account for the greater involvement of left inferior frontal cortex when forming long distance dependencies in noncanonical structures. One possibility in this respect turns on the notion of a semantic linking mechanism that 1) acts roughly in parallel with syntactic linking, and 2) establishes correspondences between arguments and syntactic roles based on the purely semantic principle “agent first, undergoer second” [Feier and Gerstman, 1980; Pinãngo, 1999]. Such a mechanism is allowed to emerge when the syntactic algorithm that maps thematic roles to syntactic positions is somehow delayed. If in a non‐canonical sentence, syntactic information does not constrain the canonical thematic linking mechanism in time, as might be the case in a sentence with a long antecedent‐gap distance, then a mismatch arises that must be corrected. Such a mismatch occurs between semantic linking (that imposes a canonical order of arguments) and syntactic linking that follows a syntactic representation. We can, therefore, posit that it is this recovery procedure that is sustained by left inferior frontal cortex.5

There is another explanation suggested by Gibson's [1998] parsing model, an explanation that directly implicates STM. Gibson focuses on the fact that, compared to canonical structures, noncanonical forms contain extra discourse (event) referents that must be assigned thematic roles. These extra constituents add to the memory load independently incurred by the need to form a long‐distance dependency, a memory load that increases when the long‐distance dependency features a non‐canonical order. Whichever (if any) of these possibilities is eventually borne out, it is clear from the present data and the aphasia literature that left inferior frontal cortex plays a crucial role in processing noncanonical constructions with long antecedent‐gap links.

This finding adds extra detail to two recent imaging‐based characterizations of the role of left inferior frontal cortex. The first is Stowe et al.'s [1998] claim that the recruitment of Broca area during sentence comprehension is to be attributed exclusively to its support for “complex working memory.” The second is Caplan et al. [Caplan et al., 1998a; Stromswold et al., 1996] description of this region's syntactic commitment.

In partial support of Stowe et al. [1998], we find that left inferior frontal cortex is activated by the extra memory cost associated with an increase in the “length” of a syntactic dependency, and that this activation is at least partially independent of other features of syntactic complexity. Specifically, we find an activation contrast in left inferior frontal cortex when memory span was manipulated within the same structure (i.e., [object‐relative long] − [object‐relative short]). And we find no difference when, regardless of span length, the dependency involves sentence structures differing in canonicity, there is no residual activation in left inferior frontal cortex for long (complex) object‐relatives when compared to long (non‐complex) subject‐relatives. Therefore, memory span does count, as suggested by Stowe et al. [1998]. This does not, however, appear to be easily separable from specific linguistic constraints. Even though we have conventionally (and nonlinguistically) defined a memory span in terms of the number of words in a linear sequence, we found that this linear sequence is nonetheless rooted to the syntactic operation of gap‐filling (or antecedent‐pronoun linking). Although left inferior frontal cortex is involved in the formation of long‐distance syntactic dependencies in a general way, the fact remains that we observed engagement of this region most clearly when the operation is embedded in a noncanonical sentence structure. We found no left inferior frontal activation when the dependency spanned a relatively large number of words in a canonical structure (i.e., [subject‐relative long] − [subject‐relative short]). Although the differences in STM demand in terms of linear word span are identical in the [object‐relative long] − [object‐relative short] and [subject‐relative long] − [subject‐relative short] contrasts, the latter contrast cannot fully account for the STM resources used for gap filling.

We also extend the characterization put forth by Caplan and his coworkers [Caplan et al., 1998a; Stromswold et al., 1996]. This group has reported residual activation in left inferior frontal cortex for object‐relative minus subject‐relative sentences. They conflate this noncanonical‐canonical difference, however, with other syntactic differences. Their noncanonically ordered sentences are also center‐embedded and feature long antecedent‐gap distances, whereas their canonical sentences are constructed in simpler right‐branching formats with short‐distance syntactic dependencies. Using only center embedded constructions, we have detailed the basis for the difference that Caplan et al. [1998a] observed. Again, we isolated two jointly acting syntactic features, the length of the syntactic dependency and whether or not thematic roles are canonically ordered.

In our discussion so far, we have referred to the variably large left anterior region implicated in Broca aphasia. In fact, our object‐relative long sentences activated only one part of the frontal cortex associated with Broca aphasia, namely, Brodmann's area 47. Others have variously recruited BA 44, BA 45, and BA 47 [Caplan et al., 1998a; Dapretto et al., 1998; Friederici et al., 2000; Ni et al., 2000; Stowe et al., 1998; Stromswold et al., 1996]. It is unclear, however, whether we can attach any significance to the difference between BA 44, 45, and BA 47. BA 47 and 45 are adjacent, and all three of these anterior regions (44, 45, and 47) are implicated in Broca aphasia. Moreover, the gross macroscopic features that define BA 44, BA 45, and BA 47 are not entirely reliable landmarks of cytoarchitectonic borders [Amunts et al., 1999; Roland and Zilles, 1998].

The significance of the different Brodmann areas recruited in various studies is further obscured by task differences: Whereas we charted activation in a thematic‐role assignment task (subjects had to identify the agent of the action), some have used sentence plausibility judgments [Caplan et al., 1998a; Ni et al., 2000; Stromswold et al., 1996], and others have used sentence synonymy judgments [Dapretto et al., 1998]. In fact, even the difference in focusing on plausibility and synonymy seems to affect activation site for the canonical‐noncanonical contrast [Caplan, 2000; Dapretto et al., 1998]. Therefore, although left inferior frontal cortex is crucially involved in the processing of syntactic relations during comprehension, no single Brodmann area within this region has yet been convincingly assigned a privileged status. In the search for more precision, BA 47 must also be considered.6

Left Posterior Superior Temporal Cortex

Aphasia research shows that, like left inferior frontal cortex, left posterior superior temporal cortex must also be considered crucial for normal comprehension. Broadly stated, the aphasia data show that although left posterior superior temporal cortex is not involved in the reflexive identification and filling of gaps, it is involved in later, semantic forms of processing [Caramazza et al., 1976]. Our imaging data are consistent with this aphasia finding. Specifically, when compared to the baseline task, we observed activation in left posterior superior temporal cortex for all of our sentence types, an activation that might reasonably be hypothesized to signal a common level of semantic processing. Syntactic differences, whether having to do with the length of the dependency linkage or with canonicity, were irrelevant to this cortical area.

Others have also shown left posterior superior temporal activation during sentence processing in healthy adults [Bavelier et al., 1997; Carpenter et al., 1999; Just et al., 1996; Keller et al., 1998; Meyer et al., 1998; Ni et al., 2000]. More to the point, like the present work, previous studies have also observed statistically significant levels of recruitment of this brain region for multiple syntactic configurations [Stromswold et al., 1996]. Again, for left posterior superior temporal cortex, syntactic configurations seem irrelevant. What counts seems to be meaning. Obviously, however, many details remain to be filled in concerning the semantic commitment of this region during sentence processing.

Left Hippocampus

We observed recruitment of the left hippocampus during the contrast of object‐relative long antecedent‐gap sentences with the pseudofont baseline, and for the [object‐relative long antecedent‐gap] − [subject‐relative long antecedent‐gap] contrast. We have no explanation for these two observations. They do not connect to the few other reports in the literature with hippocampal recruitment that involves the processing of language materials. These have to do with single word processing during lexical encoding [Nyberg et al., 1996] and with lexical meaningfulness [Martin et al., 1997]. At present, then, in the absence of any coherent or consistent pattern across imaging studies, and in the absence of any demonstration of this region's significance with respect to aphasia, it seems unlikely that it has a linguistically‐specific role.

Right Posterior Superior Temporal Cortex

This area was consistently recruited for long antecedent‐gap (or pronoun) distances. That is, for each long‐short comparison, whether involving canonical or noncanonical structures, there was residual right posterior temporal activation. These data indicate that this area participates in the formation of dependency relations that span relatively many intervening words. Indeed, given our data, it is also reasonable to suppose that this commitment to a syntactically‐responsive memory buffer underlies previous imaging demonstrations of right posterior superior temporal activation during sentence processing [Just et al., 1996]. Possibly, the powerful bundle of callosal fibers linking homologous regions of the two hemispheres can support recruitment of the contralateral hemisphere as a short‐term memory buffer to retain a limited amount of information for a brief period of time during sentence comprehension.

Visual Association Cortex

As is to be expected, there was evidence in our study that visual sentence presentation activated brain regions responsible for written letter form recognition. Thus, we observed recruitment of visual association cortices during the presentation of these written sentences, including portions of the lingual and fusiform regions of both hemispheres. Consistent with previous work, activation was more robust in these inferior temporal‐occipital areas of the left hemisphere than the right hemisphere when contrasted with a baseline consisting of pseudofont characters that otherwise resembled letters in their visual characteristics [Petersen et al., 1990; Price et al., 1994, 1996; Puce et al., 1996; Pugh et al., 1996a].