Revisiting the role of Broca's area in sentence processing: Syntactic integration versus syntactic working memory

Abstract

Most previous neuroimaging studies of sentence processing have associated Broca's area with syntactic processing; however, the exact nature of the processes subserved by this brain region is yet not well understood. Although some authors suggest that Brodmann area (BA) 44 of the left inferior frontal gyrus (i.e., Broca's area) is relevant for syntactic integration processes, others claim that it is associated with working memory mechanisms relevant for language processing. To dissociate these two possible functions, the present study investigated hemodynamic responses elicited while participants processed German indirect wh‐questions. Activation increases were observed in left BA 44 together with superior temporal areas and right hemispheric homologues for sentences with noncanonical word order, in which a verb argument was dislocated from its canonical position over a relatively long distance. In these sentences, syntactic working memory load was assumed to be greatest. In contrast, no activation increase was elicited by object–initial as opposed to subject–initial sentences that did not differ with respect to working memory costs but with respect to syntactic integration costs. These data strongly suggest that Broca's area plays a critical role in syntactic working memory during online sentence comprehension. Hum. Brain Mapping 24:79–91, 2005. © 2004 Wiley‐Liss, Inc.

INTRODUCTION

Comprehending a sentence involves identifying the structural relations among its words and phrases. By assigning grammatical functions and thematic roles to different arguments of the verb, syntactic structure of a sentence indicates who is doing what to whom. The syntactic processing system reconstructs the structure of the sentence incrementally by assigning the perceived words to phrases as quickly as possible and by determining the hierarchical relations among the different phrases, a process often referred to as syntactic structure building.

In the past few decades, evidence has accumulated in favor of the assumption that Broca's area (encompassing the pars opercularis of the left inferior frontal gyrus/Brodmann area [BA] 44 and the posterior portion of pars triangularis/BA 45) [Uylings et al., 1999] is involved critically in syntactic processing during sentence comprehension. This assumption has been established based on neuropsychological studies of patients with Broca's aphasia and agrammatic comprehension [e.g., Caramazza and Zurif, 1976; Grodzinsky, 2000; Swinney et al., 1996; Zurif et al., 1972]. More recently, functional neuroimaging studies have been reported that support the notion of a localization of syntactic processes in left inferior frontal brain areas. Using positron emission tomography (PET), Stromswold et al. [ 1996] demonstrated that regional cerebral blood flow (rCBF) in left BA 44 was stronger for syntactically complex sentences (i.e., sentences with center‐embedded object–initial relative clauses) than for less complex sentences (i.e., sentences with right‐branching subject–initial relative clauses). This result has been replicated [e.g., Caplan et al., 1998, 2000] and the involvement of this brain area in syntactic processing is supported also by studies using functional magnetic resonance imaging (fMRI) [Ben‐Shachar et al., 2003; Cooke et al., 2001; Friederici et al., 2000; Just et al., 1996; Röder et al., 2002]. The results of these neuroimaging studies have been interpreted as supporting the notion that the pars opercularis/BA 44 of the left hemisphere is critical for processes that structure the incoming linguistic input [Caplan, 2001; Friederici, 2002; Kaan and Swaab, 2002]. Studies investigating lesion deficit correlations point to a more distributed representation of syntactic processes in the left perisylvian region [e.g., Berndt et al., 1996; Caplan et al., 1996; Willmes and Poeck, 1993; see also Dick et al., 2001]. In addition, several functional neuroimaging studies reported that temporal or parietal areas are modulated by syntactic complexity [e.g., Ben‐Shachar et al., 2003; Caplan et al., 2001; Just et al., 1996].

The exact nature of the processes subserved by Broca's area during syntactic processing is far from clear. Psycholinguistic models of syntactic processing often distinguish between two mechanisms involved in syntactic processing: (1) transient, computational processes relevant for integration of incoming input into the phrase structure representation under construction; and (2) more sustained working memory processes during online sentence comprehension. One prominent example of such a model is Gibson's syntactic prediction locality theory [Gibson, 1998], which distinguishes between syntactic integration costs and syntactic working memory costs (often associated with maintaining linguistic material active in working memory that has not yet been integrated). Both aspects of syntactic processing can contribute to the well‐established processing difficulty for object–initial versus subject–initial sentences [e.g., Ford, 1983; King and Just, 1991; King and Kutas, 1995]. Accordingly, both integration costs and syntactic memory costs might also contribute to the left inferior frontal complexity effects cited above.

To understand the cognitive processes responsible for activation increases seen in Broca's area for complex sentences, it is important to analyze the sentences used in previous studies as to which aspects of syntactic processing they tap. Consider, for example, the sentence types used by Stromswold et al. [ 1996] and Caplan et al. [ 1998; see (1a,b)] and the complex sentences used by Just et al. [ 1996; see (2a,b)]. All sentences consist of a main clause and of a relative clause that modifies an argument of the main clause (relative clauses are emphasized by italic font in the examples). Sentence (1a) contains a right branching relative clause whereas in all other sentences (1b, 2a, 2b), the relative clauses are center‐embedded and disrupt the main clause.

• 1aThe child spilled the juice that _i __i stained the rug.. (subject–initial)
• 1bThe juice that _i the child spilled __i stained the rug. (object‐initial)
• 2aThe reporter who _i __i attacked the senator admitted the error. (subject–initial)
• 2bThe reporter who _i the senator attacked __i admitted the error. (object‐initial)

As stated above, object–initial relative clause sentences are more difficult to process than are subject–initial relatives [e.g., Ford, 1983; King and Just, 1991; King and Kutas, 1995]. Two characteristics can lead to processing difficulty in these sentences. First, the processing difficulty might be a result of the interruption of the main clause. To process correctly a sentence like (1b) (“The juice […] stained the rug.”), the reader must be able to take up the first part of the main clause that is read before the embedded clause and integrate it with the verb and object at the end of the main clause. Consequently, center‐embedding is known to increase working memory load because the subject noun phrase of the main clause has to be held in working memory while the relative clause is processed [e.g., Miller and Isard, 1964; Wanner and Maratsos, 1978]. This increase in memory load might account for activation differences between sentences (1b) and (1a). However, an activation difference in Broca's area is also reported between (2b) and (2a), which both have center‐embedded relative clauses [Just et al., 1996; see also Caplan et al., 1999]. This finding indicates that the increased working memory load associated with center‐embedding cannot be the only cause of observed complexity effects in the brain.

Second, the relative clauses in (1b) and (2b) are more difficult to parse than are those in (1a) and (2a). This might be attributable to structural differences inherent to the embedded clauses, or might also be accounted for in terms of processing models. In all four sentence examples, an argument of the verb has been dislocated from its original position (indicated by “_” in the examples) to a clause initial position. In (1a) and (2a), the relative pronoun (that, who) represents the subject of the embedded clause, whereas in (1b) and (2b), the relative clause object was dislocated, yielding a clause with a noncanonical word order.1 Although object–initial relative clauses generally are said to be syntactically more complex than subject–initial relatives, it is not obvious that they differ in the complexity of their phrasal structures as the dislocated element resides in the same structural position in both sentence types. In addition, the number and types of phrasal nodes, a classic psycholinguistic measure of structural complexity [e.g., Frazier and Fodor; 1978; Gibson, 1998; Yngve, 1960], are identical in both sentence types. Accordingly, differences in brain activity between subject and object relatives cannot be due to differences in the complexity of the surface structures of these sentences.

With respect to processing models of syntactic complexity [e.g., Gibson, 1998; Yngve, 1960], the observed brain activation differences might be a result of differences in the syntactic processes triggered by subject as opposed to object–initial relative clauses. It was demonstrated recently that the application of syntactic transformation operations (argument dislocations like those in the sentence examples described above) leads to activation in the left inferior frontal region [Ben‐Shachar et al., 2003]. However, as both structures in question involve syntactic transformations this finding cannot explain fully the observed activation differences between subject–initial and object–initial relative clauses. Rather, differences in brain activation might occur because reconstruction of the underlying structure from a noncanonical sentence might be more costly for the cognitive system than processing a canonical subject‐first sentence, either because integrating words encountered in a noncanonical order evokes more transformational operations [e.g., Fodor et al., 1974; Gibson, 1998] or because of increased syntactic working memory demands [Caplan et al., 2000; Gibson, 1998].

It is not only transformational operations as such that are known to be costly. It was also demonstrated that the distance between a dislocated argument and another syntactic constituent (e.g., the subject or the verb or the distance between the dislocated element and the underlying structural position) influences processing difficulty [e.g., De Vincenzi, 1996; Gibson, 1998]. With respect to the sentence examples in question here, the greater distance between the clause‐initial object and the verb (or, alternatively, between the dislocated object and its original position) as compared to subject relatives is thus also a potential cause of the observed activation effects. Empirical evidence from psycholinguistics suggests that a dislocated element has to be maintained in working memory until it can be processed further [e.g., Frazier and Flores D'Arcais, 1989; King and Kutas, 1995; Kluender and Kutas, 1993]. From this, it follows that syntactic working memory processes are active for a longer duration in object‐first constructions than in subject‐first constructions. Caplan et al. [ 2000] convincingly demonstrated that rCBF increase in Broca's area for syntactically complex sentences is not due to phonologic rehearsal mechanisms that are part of the verbal working memory system. The authors concluded that the most likely source of observed rCBF effects in their syntactic processing studies was recruitment of working memory resources specialized for syntactic processing [Caplan et al., 2000; see also Caplan and Waters, 1999; Caplan, 2001]. This view is convergent with the conclusions drawn by several other authors [Friederici, 2002; Kaan and Swaab, 2002].

To summarize, it remains an open question whether the activation increases for complex sentences in Broca's area are due to differences in the transient, computational processes of integrating each new input word into the phrase structure representation, a process that might indeed be more demanding for sentences with a noncanonical word order, or whether these activation effects are due to increased working memory demands during processing of more complex sentences (or due to a combination of these two mechanisms). Both aspects of syntactic processing are involved in reconstructing sentence structure during online comprehension, they are interrelated closely, and both contribute to the processing difficulty of structurally complex sentences [Gibson, 1998]. It would be an important advancement for the neurocognitive study of syntactic processing to dissociate the brain regions involved in these two critical aspects of syntactic comprehension.

To achieve this goal, we investigated hemodynamic responses elicited during the processing of particular sentence types, from which we know that they involve an increased syntactic working memory demand over a well‐defined region of the sentence [Fiebach et al., 2002]. Rather than using relative clause sentences as was done in most previous neuroimaging work, we investigated the processing of indirect German constituent questions (i.e., wh‐questions) that are embedded within a matrix clause [e.g., (3)]. Just like relative clauses, wh‐questions are derived by dislocating an argument of the verb into the clause initial position. The main difference between the questions used here and the often‐used relative clause is that the dislocated element is represented by an interrogative pronoun rather than by a relative pronoun.

• 3aEr fragt sich, wer_i _ _i den Doktor gerufen hat. (subject–initial) he asks himself, who_nom the_acc doctor called has (word‐by‐word translation) He asks himself, who has called the doctor.
• 3bEr fragt sich, wen_i der Doktor _ _i gerufen hat. (object‐initial) he asks himself, who_acc the_nom doctor called has (word‐by‐word translation) He asks himself, who the doctor has called.

Similar to relative clause sentences, object wh‐questions are more difficult to process than are subject wh‐questions [De Vincenzi, 1996; Fanselow et al., 1999; Frazier and Flores D'Arcais, 1989]. A direct contrast between object wh‐questions (3b) and subject wh‐questions (3a) therefore replicates the comparison between object and subject relatives, and should identify brain regions specifically activated when syntactic integration costs are increased due to noncanonical word order. Importantly, both types of sentences are unmarked in the sense that a pronoun always has to precede the full noun phrase (NP) argument. The effect of noncanonical word order is thus investigated in a comparison of two unmarked sentence structures. The use of wh‐questions further allowed us to vary word order without having to embed the critical portion of the sentence within a disrupted main clause. In the present study, the comparison of object–initial and subject–initial sentences is therefore not contaminated by working memory load external to the critical region of the sentence.

The German sentences given in (3) demonstrate some typical properties of verb‐final languages with a relatively free argument order. As predicted by Greenberg [ 1966] for such languages, German is characterized by a relatively rich case morphology. Under the assumption of incremental sentence processing [e.g., Stabler, 1994], it was suggested that the presence of overt case marking in verb‐final languages allows for a direct relation of sentential arguments to each other without necessarily requiring a mediation by the verb [e.g., Hawkins, 2002]. This so‐called linguistic universal [Greenberg, 1966] receives empirical support from recent event‐related potential (ERP) work [Bornkessel et al., 2003; see also Fiebach et al., 2002; Frisch and Schlesewsky, 2001]. For example, when the preferred mapping from nominative case to an “Actor” interpretation and accusative/dative case to an “Undergoer” interpretation is disconfirmed by the interpretive properties of a clause final verb (e.g., the object‐experiencer verb gefallen [to be appealing to] with which the dative‐marked “Experiencer” outranks the nominative marked “Stimulus”), ERP measures show additional processing costs due to a reanalysis of thematic relations [Bornkessel et al., 2003]. This finding clearly indicates that an (at least initial) interpretative relation between two arguments was established on the basis of morphologic case information even before the clause‐final verb was encountered.

For sentences such as (3), it therefore has to be assumed that the two NP arguments marked for nominative and accusative, respectively, can be related to each other even before the verb is encountered. Unlike in English, this relation can be established independent of the structural positions in the sentence at which these constituents are encountered; the relevant cue in languages such as German is the case morphology of the arguments [Haider, 1993]. The case marking of the two arguments is also critical for the amount of working memory costs to be expected. If an NP marked for accusative is encountered in clause‐initial position, a subject and a verb have to be predicted to form a grammatical sentence. This process is assumed costly in terms of syntactic working memory [Gibson, 1998]. Upon encountering the subject, the subject and object arguments can be integrated as described above, and the associated working memory load is reduced. In the processing of subject–initial sentences, less complex syntactic predictions result, as only a verb has to be predicted to complete the sentence [Gibson, 1998].

In a previous ERP study [Fiebach et al., 2002], we took advantage of these characteristics of German by systematically varying the distance between the dislocated element (i.e., the wh‐pronoun) and the second NP. Doing this, we varied the distance during which a syntactically motivated working memory load had to be maintained. In half of the sentence stimuli, we inserted two long prepositional phrases between the wh‐pronoun and the second NP, whereas in the other half of the sentences, only one short prepositional phrase was introduced. We observed a sustained negativity over the left frontal scalp for long object wh‐questions when compared to long subject wh‐questions (see conditions Long‐object and Long‐subject in Table I). This sustained negativity was observable between the dislocated element and the subject noun phrase, i.e., between the two elements that are immediately related to each other, as described above. As the incremental interpretation of the clause‐initial object is dependent upon the subject NP, we argued that the sustained negativity reflects the temporary maintenance of the dislocated element until the processing of the subject licenses the integration of the dislocated object.2 Importantly, we demonstrated that the sustained negativity varied as a function of individual working memory capacity, a finding that provides strong evidence for the working memory interpretation of this effect [Fiebach et al., 2002].

Table I.

Example sentence stimuli

Condition	Example
Short‐subject	Thomas fragt sich, wer den Doktor am Dienstag nachmittag nach dem Unfall verständigt hat.
	Thomas asks himself, whonom the acc doctor on Tuesday afternoon after the accident called has
Short‐object	Thomas fragt sich, wen der Doktor am Dienstag nachmittag nach dem Unfall verständigt hat.
	Thomas asks himself, whoace thenom doctor on Tuesday afternoon after the accident called has
Long‐subject	Thomas fragt sich, wer am Dienstag nachmittag nach dem Unfall den Doktor verständigt hat.
	Thomas asks himself, whonom on Tuesday afternoon after the accident theace doctor called has
Long‐object	Thomas fragt sich, wen am Dienstag nachmittag nach dem Unfall der Doktor verständigt hat.
	Thomas asks himself, whoacc on Tuesday afternoon after the accident thenom doctor called has

The dislocated object and the subject noun phrase are indicated by italic font in object wh‐questions to demonstrate the distance of the hypothesized working memory demand. English translations are word‐by‐word translations.

In sentences with a short distance between the dislocated object and the subject, no comparable negativity was elicited. This latter observation strengthens the assumption that the presence of the working memory effect is dependent upon the length of the distance between the dislocated object and the subject in the sense of Gibson's syntactic prediction locality theory [ 1998]. Long and short subject–initial sentences do not differ with respect to their syntactic working memory demands in the way that object–initial sentences do. The reason for this is obvious: whereas the interpretation of the clause‐initial object is dependent upon the subject that is encountered later, the incremental interpretation of the clause‐initial subject is independent of the later appearance of an object argument. As discussed above, it is therefore not necessary to postulate additional syntactic memory demands due to the prediction of an object in subject–initial sentences.

To summarize, in our ERP study, we dissociated the canonicity of the word order from the distance over which an element is dislocated. In terms of processing, the former aspect is associated with the integrational costs elicited when processing a sentence, whereas the latter is related to the amount of syntactic working memory demands during sentence processing. We found evidence for a working memory involvement during processing of sentences in which the object argument was dislocated over a relatively long region of the sentence.

It is the aim of the present fMRI study to identify the neuroanatomical bases underlying the syntactic working memory effect described above. We set out to explore whether Broca's area is likely to be the neural generator of the syntactic working memory effects observed in the ERP studies, a likely assumption given the left anterior distribution of these effects [Fiebach et al., 2002]. To this end, we used the same long object and subject wh‐questions for the fMRI experiment as in the ERP study (Table I). It was necessary to modify the short sentence conditions because short sentences also differed from long sentences in terms of absolute length in the ERP study. This would not be acceptable for fMRI, as a direct comparison between short and long sentences would be confounded by length differences that might result in activation differences due to sensory processing. We constructed a new set of short wh‐questions in which the object was dislocated over a very short distance, but which had an overall length identical to that of the Long‐subject and Long‐object questions. This was done by inserting the same prepositional phrases as in the long sentences between the second NP and the verb, rather than between the dislocated element and the second NP (Table I). (The placement of prepositional phrases at different positions in the sentence is possible without rendering sentences ungrammatical due to the relatively free word order of German.)

Figure 1 shows across‐sentence ERPs for the long and for the new short wh‐questions that were reported by Fiebach [ 2001, Experiment 2]. It is observable that, as in our previous study [Fiebach et al., 2002], Long‐object questions induced a sustained negativity relative to Long‐subject questions. Short‐object questions elicited no comparable effect when compared to Short‐subject. (Statistical results are reported in the legend of Figure 1.) This observation is important, as it demonstrates that the same pattern of ERP effects as in the study by Fiebach et al. [ 2002] can be replicated using the present sentence material. This finding justifies the usage of the modified sentence material in the present fMRI study for identifying the neuroanatomical correlates of the syntactic working memory effect observed using ERPs.

Figure 1.

Event‐related brain potentials elicited by object wh‐questions (dashed line) as compared to subject wh‐questions (solid line) for questions with a short (left) or a long (right) distance between the dislocated element and the second noun phrase. Figure adapted from Fiebach [ 2001]. ERPs represent averaged data from 14 participants. Negative voltages are plotted upward. Experimental procedures, data acquisition, and data analysis were equivalent to procedures described in the ERP study of Fiebach et al. [ 2002]. The statistical analysis in the time window of 1,000–3,400 ms (in which the sustained negativity was observed for the Long‐object questions and that was also used in Fiebach et al. [ 2002]) indicates an interaction of word order and length (F[1,13] = 5.79; P < 0.05) that was resolved to show a significant negativity only for the Long‐object sentences (F[1,13] = 13.82; P < 0.005). ERPs to Short‐subject and Short‐object sentences did not differ in this time window (F < 0.5). Analysis of behavioral results indicates that responses to object wh‐questions took 46 ms longer than did responses to subject wh‐questions (F[1,13] = 6.39; P < 0.05), and that more errors were made for object than for subject questions (F[1,13] = 7.22; P < 0.05). Although there was a considerable difference in this effect between short questions (2.2% more errors in object than in subject questions) and long questions (5.2% more errors in object questions), the interaction of sentence type and length did not reach significance (F[1,13] = 2.27; P = 0.16).

The ERPs shown in Figure 1 indicate that a substantial working memory effect is present only for Long‐O sentences but not for Short‐O sentences. Consequently, when comparing long object wh‐questions with short object wh‐questions in the present fMRI experiment, activation differences should be observable that are related to increased syntactic working memory load but not to differences in the word order, as in both cases the object argument was dislocated. In this comparison, there should thus be no differences in syntactic integration costs but clear differences in syntactic working memory costs. To summarize the predictions of the present study, if transient syntactic integration costs associated with processing sentences in a noncanonical word order are sufficient to elicit increased activity in the area of Broca, greater activation should be seen for object than for subject wh‐questions. If, on the other hand, Broca's area is critical for temporarily maintaining unintegrated linguistic information in working memory, we would expect to see pronounced differences in neural activity in Broca's area when contrasting two conditions that clearly differ with respect to their working memory demands, i.e., when comparing long object wh‐questions to short object‐first questions.

SUBJECTS AND METHODS

Participants

Fourteen paid volunteers (six women; mean age 24.3 years; age range 19–31 years) participated in the fMRI experiment after being informed about the fMRI technique and after giving written consent. All participants were undergraduate students of the University of Leipzig, had normal or corrected to normal vision, and were without history of neurologic or psychiatric diseases. Furthermore, all participants were native speakers of German and consistently right‐handed, as indicated by a laterality quotient of 100 according to the Edinburgh Handedness Inventory [Oldfield, 1971].

Materials

Sentence stimuli used in this study consisted of four types of indirect German wh‐questions (see Table I for one sample set of stimuli). All questions contained a short matrix clause that consisted of a name and the verbal phrase “asks himself” (e.g., “Thomas fragt sich, …”). The matrix clause was followed by the embedded question. Wh‐questions with a short dislocation consisted of the interrogative pronouns, the second NP, two prepositional phrases (PPs), and a verb in participle tense followed by the corresponding auxiliary verb “hat” in sentence final position (i.e., “ … [(wer/wen) (NP) (PPs) V AUX ]”; Table I). Long wh‐questions were constructed from the same words. In these sentences, prepositional phrases were located between the dislocated argument and the second noun phrase (i.e., “ … [(wer/wen) (PPs) (NP) V AUX]”), resulting in a prolonged distance between the dislocated object and the subject in the object condition (Table I). For each sentence type, 48 stimuli were presented. All sentences used in the present study had the same length, i.e., 14 words; across conditions, word classes and word length in letters were matched. The complete set of sentence stimuli is available upon request.

The critical manipulation in the present experiment is the insertion of the two prepositional phrases either between the dislocated element and the second NP, or, in sentences with a short dislocation, between the second NP and the verb. This manipulation is possible in German, as German has a relatively free word order that allows one to place prepositional phrases at different positions within a sentence without rendering it ungrammatical. In our ERP study [Fiebach et al., 2002], ERPs did not provide evidence for an increased processing difficulty or even an ungrammaticality at the prepositional phrases when these were analyzed separately.

Experimental Procedure

Each trial consisted of a wh‐question and a subsequently presented probe assertion that had to be rated for correctness by the participants. Wh‐questions were presented visually in a sequence of 10 frames. Visual presentation was phrase‐by‐phrase, i.e., each frame consisted either of a word or a phrase (e.g., article and noun in an NP) to avoid possible ambiguities between articles and relative pronouns in German. Frames were presented either for 600 ms (when they contained only one word) or for 700 ms (when they contained a phrase). The probe assertion appeared on the screen 800 ms after offset of the last word. The comprehension task served to ensure that wh‐questions were processed correctly by the participants. For the comprehension probes, parts of the critical questions were rephrased in different ways (e.g., “The doctor was called after the accident.” or “The doctor called somebody on Tuesday.”). In half of the trials, certain aspects of the critical sentences were changed to form incorrect probe assertions. This was done in a way that ensured participants paid attention to all parts of the embedded wh‐questions. Behavioral responses were registered from the onset of the probe assertion.

fMRI Acquisition

Functional images were acquired in eight axial slices using a blood oxygenation level‐dependent (BOLD)‐sensitive gradient‐echo echoplanar imaging (EPI) sequence with an echo time (TE) of 30 ms, a flip angle of 90 degrees, a repetition time (TR) of 1 s, and an acquisition bandwidth of 100 kHz. The matrix acquired was 64 × 64 with a field of view (FOV) of 19.2 cm, resulting in an in‐plane resolution of 3 mm × 3 mm. Slice thickness was 5 mm with an interslice gap of 2 mm. The upper border of the second‐lowest slice was aligned to the anterior commisure–posterior commisure (AC–PC) line such that in all participants, the perisylvian region was covered. Each trial had a length of 16 s, thereby resulting in an effective intertrial interval (between the end of one wh‐question and the beginning of the next) of approximately 9 s.

The functional measurement was a single‐trial design carried out in three runs of about 15‐min length. Within each run, the different sentence types were presented in a pseudorandom order with equal frequencies and an even distribution across the runs. Before functional measurements, a T1‐weighted MDEFT scan [Ugurbil et al., 1993] was obtained (data matrix 256 × 256, TR = 1.3 s, TE = 10 ms) with a non slice‐selective inversion pulse followed by a single excitation of each slice [Norris, 2000]. These anatomical images were used for coregistration of the functional data sets with high resolution whole‐head 3‐D MDEFT [Lee et al., 1995; Ugurbil et al., 1993] brain data sets (128 sagittal slices, 1.5 mm thickness, FOV 25.0 × 25.0 × 19.2 cm, data matrix of 256 × 256 voxels), which were acquired in separate sessions.

fMRI Data Analysis

Functional data were submitted to a number of preprocessing steps including a slice‐time correction using sinc interpolation, a motion correction (rigid‐body realignment), and spatial smoothing using a Gaussian kernel with 4.9 mm full‐width half‐maximum (FWHM). All analyses were carried out with the LIPSIA software package [Lohmann et al., 2001]. Before the statistical analyses, functional data sets were coregistered with high‐resolution 3‐D structural images and normalized to stereotactic space [Talairach and Tournoux, 1988]. Normalized data sets were analyzed statistically in a fixed‐effects general linear model based on procedures from the SPM software package [Friston et al., 1995; Josephs et al., 1997]. Functional data were analyzed in an event‐related design from which were excluded the first 32 s obtained in each run and all incorrectly answered trials. The critical event was defined as the point in the sentences where the four sentence types started to differ, i.e., at the question word that introduced the embedded wh‐question. Observed data and the design matrix were convolved by a Gaussian kernel with 4‐s FWHM. High‐pass filtering was carried out with a cutoff frequency of 1/45 Hz. Continuous statistical Z‐maps were thresholded at Z = 2.81 (P = 0.0025; uncorrected) at the voxel level. To avoid accepting false positive activations, Z‐maps were also thresholded at P < 0.05 (corrected) at the cluster level [Worsley et al., 1996].

RESULTS

Behavioral Data

The main purpose of registering the behavioral performance of the participants was to ensure that critical sentences were processed correctly. Participants were instructed such that giving a correct answer was more important than giving a fast answer because based on behavioral performance, incorrectly answered trials were excluded from further analyses of fMRI data. For statistical analysis, reaction times and error rates were aggregated: (1) by participant and experimental condition (subject analysis); and (2) by item and condition (item analysis; see Clark, 1973), and then introduced into repeated‐measures analysis of variance (ANOVA) with the within‐subjects factors sentence type (i.e., subject vs. object wh‐question) and length (i.e., distance over which the dislocation took place; short vs. long). The mean reaction time across the different conditions was 1,735 ms (short‐subject, 1,740.19 ms [SE 101.57 ms); short‐object, 1,761.89 ms [SE 104.86 ms]; long‐subject, 1,692.56 ms [SE 87.27 ms]; long‐object, 1,746.96 ms [SE 99.62 ms]) from the onset of the probe assertion. There was no significant main effect of sentence type (F ₁ < 0.5; F ₂ = 1.74) or length (F ₁ < 0.5; F ₂ < 0.5) in the reaction times, and the interaction of sentence type and length was not significant (F ₁ < 0.5; F ₂ < 0.5).

The mean percentage of errors made was 10.9%. ANOVA of error rates revealed a reliable main effect of sentence type (F ₁[1,13] = 12.76; P < 0.005; F ₂[1,47] = 4.84; P < 0.05), because object wh‐questions elicited 4% more errors than did subject wh‐questions. The main effect of length reached significance only in the subject analysis (F ₁[1,13] = 6.48; P < 0.05; F ₂[1,47] = 2.96; P = 0.09). The interaction of sentence type and length was not reliable (F ₁ < 1; F ₂ < 0.5). Planned comparisons reveal that short object wh‐questions elicited 2.8% more errors than did short subject questions (13.84% [SE 1.81%] vs. 11.01% [SE 2.02%]; F ₁[1,13] = 3.25, P = 0.1; F ₂ = 1). In contrast, long object wh‐questions elicited 5.1% more errors than did long subject questions (11.9% [SE 1.91%] vs. 6.85% [SE 1.32%]; F ₁[1,13] = 6.65, P < 0.025; F ₂[1,47] = 5.34; P < 0.05).

fMRI Data

The functional imaging data speak a clear language with respect to the two factors complexity (i.e., object vs. subject questions) and working memory load (i.e., object questions with long vs. short dislocation). There were no brain areas that showed greater activation for object than for subject wh‐questions. This holds true for both short and long sentence conditions. The direct comparison between hemodynamic responses elicited by long object wh‐questions and short object questions resulted in reliable activation differences in inferior frontal and superior temporal areas of both hemispheres (Fig. 2; Table II). Stronger neural activity for long object wh‐questions was obtained bilaterally in the superior portion of the pars opercularis of the inferior frontal gyrus (BA 44), on the border to pars triangularis (BA 45). In addition, the inferior tip of pars opercularis, extending into the deep frontal operculum, was activated more strongly for long object questions. The latter activation difference was present only in the left hemisphere. In addition to the inferior frontal activations, we also observed increased activity at the junction of the left precentral and inferior frontal sulci. Pronounced activation differences were seen along the superior temporal sulci of both hemispheres (Fig. 2). A further activation increase was observed in the left thalamus; however, this effect did not pass the cluster‐size threshold as it had a volume of only 105 mm³.

Figure 2.

SPM{Z} for group statistics representing activity that was greater for long object wh‐questions than for short object wh‐questions. The statistical map was thresholded at Z > 2.81 (P < 0.0025 uncorrected) for visual display and rendered onto a high‐resolution structural MR scan of a representative individual brain. 3‐D renderings are presented in sagittal (A), axial (B), and coronal sections (C).

Table II.

Anatomic descriptions, Brodmann's areas, Talairach coordinates, and maximal Z‐values of reliably activated brain regions

Brain region	BA	Hemisphere	x	y	z	Zmax
Long > short object wh‐questions
Inf. frontal gyrus, pars opercularis (sup. portion), pars triangularis	44/45	Left	−44	21	11	3.44
	44/45	Right	45	21	10	4.06
Inf. frontal gyrus, pars opercularis (inf. tip)	44	Left	−46	17	4	3.46
Junction of precentral sulcus and inf. frontal sulcus	6/44	Left	−54	7	28	3.64
Sup. temporal sulcus (middle portion), middle temporal gyrus	21/22	Left	−54	−27	−1	5.52
Middle temporal gyrus (posterior portion)	22	Left	−52	−46	6	3.72
Sup. temporal sulcus, gyrus (middle portion)	21/22	Right	45	−18	−3	4.16
Thalamus		Left	−18	−18	12	3.09
Long > short subject wh‐questions
Parietooccipital sulcus (inf. portion)	17/30	Left	−20	−52	22	3.24
		Right	19	−59	20	3.58
Parietooccipital sulcus (sup. portion)	7	Left	−14	−65	43	3.43

BA, Brodmann's areas; inf., inferior; sup., superior.

To exclude that general factors, e.g., the different placement of the prepositional phrases in long vs. short object questions, might have caused the activation differences observed between long and short object wh‐questions in Broca's area, we also calculated the direct contrast between long and short subject–initial questions. No reliable activation differences were found in the areas showing a length effect for object questions, or in any other perisylvian language‐relevant brain areas when long and short subject wh‐questions were compared. An activation difference was seen in the parietooccipital sulcus (Table II). It was noted above that these two sentence conditions should not differ with respect to the induced working memory demands. This result is important as it demonstrates that activation effects found for object questions were not due merely to the different positions at which the two prepositional phrases were placed in the sentences, but rather were specific for the differences in the length of the region over which an assumed working memory load had to be maintained.

To examine whether the obtained activation difference in the left inferior frontal gyrus indeed consisted of two separable clusters with distinct local maxima (one more superiorly and one more inferiorly, as visual inspection of the data suggested; Fig. 2), we utilized multidimensional scaling (MDS) to identify independent clusters of neural activity. MDS maps high‐dimensional data into a 2‐D plane. The items contained in a similarity matrix are mapped into a low‐dimensional space such that the similarity values are transformed as nearly as possible into Euclidean distance values [for applications of MDS to investigation of cortical activations, see Friston et al., 1996; Tagaris et al., 1998; Young et al., 1995]. In our case, the similarity matrix contained correlation values obtained by comparing fMRI time series recorded at 38 voxels in a region of interest (ROI) that was located in Broca's area such that it covered the region exhibiting activation differences in the contrast of long and short object wh‐questions (Fig. 2). Relative distances between any two points determined by MDS represent similarities between the time series of the corresponding voxels as well as possible. We used the same ROI for each subject, so that we obtained 14 individual correlation matrices. These matrices were normalized using Fisher's Z‐transform and then averaged, resulting in a single average correlation matrix.

We then applied MDS to this matrix using the classic metric MDS method as described previously by Torgerson [ 1958] and Seber [ 1984]. The results of this analysis support the previous description of the left inferior frontal activation effect as consisting of two distinct clusters. In the MDS map (Fig. 3A), two separate clusters can be identified clearly. For better identification of the anatomic locations, the items in the MDS map (Fig. 3A) were labeled “1” (corresponding to the cluster in the more inferior part of BA 44 in Fig. 3B) and “2” (corresponding to the more superior cluster in Fig. 3B).

Figure 3.

A: Multidimensional scaling map displaying clusters of more inferior (1) and more superior (2) voxels activated in the left inferior frontal gyrus. Relative distances as determined by the MDS method are represented by the x‐ and y‐axes. Absolute distances indicated by the values on the axes are arbitrary and therefore cannot be interpreted in any way. B: Coronal, sagittal, and axial slices representing the results of the multidimensional scaling analysis. The border between the superior and inferior portion of BA 44 is clearly visible.

The described pattern of activation differences (i.e., no reliable main effect for object–initial vs. subject–initial sentences and a clear influence on Broca's area activation of the length of the dislocation in object but not in subject questions) could be captured by an interaction of sentence type and length in the general linear model. In this analysis, a reliable interaction of these two factors was observed at a slightly reduced threshold (P < 0.005) in the same brain areas that showed increased neural activity for long as opposed to short object wh‐questions. The results of this analysis can be obtained upon request. To ensure further that no subthreshold activation differences between object and subject sentences in Broca's area were overlooked due to the combined voxel‐ and cluster‐level threshold, we repeated the object‐subject comparisons at a less strict criterion (at P = 0.05; uncorrected). The only activation difference seen was a very small focus in the anterior left temporal lobe, with greater activity for long object than for long subject sentences.

DISCUSSION

In the current study, activation in Broca's area occurred together with activity in a bilateral network of inferior frontal and superior temporal brain regions. This activation was observed as a function of increased syntactic working memory costs due to the distance between a dislocated element and its original position. It was not seen in response to word order variations associated with object–subject differences. This result is partly in line with earlier studies, but clearly goes beyond these.

Concerning the finding of a distributed syntax‐related neural network involving bilateral inferior frontal and temporal brain regions, the present study replicates only in part previous PET studies of syntactic complexity [Stromswold et al., 1996; Caplan et al., 1998, 1999, 2000] in which activations were observed almost exclusively in left BA 44 and BA 45. Our data are consistent with other fMRI studies [e.g., Ben‐Shachar et al., 2003; Just et al., 1996; Röder et al., 2003] that also showed more distributed responses to syntactic manipulations. The difference between these fMRI results and the PET activation studies might be due either to an increased sensitivity of fMRI or to differences in task demands between the PET and the fMRI studies considered here [as was discussed by Caplan, 2001]. Whereas in the PET studies, plausibility judgments were required, tasks focusing more on syntactic properties of the sentences (i.e., either sentence comprehension or grammaticality judgments) were used in the cited fMRI studies and in the present experiment. As task demands were identical for all sentence types in the present study the involvement of superior temporal brain regions would have to be attributed to an interaction of task demands with the syntactic complexity manipulation under this assumption, rather than to task demands alone.

The present finding of a syntax effect distributed over frontal and posterior perisylvian regions is also in line with clinical studies that investigated the correlation between lesion sites and agrammatic performance [e.g., Basso et al., 1985; Caplan et al., 1996; Willmes and Poeck, 1993]. These studies indicate that syntactic functions might in fact be located in a neural network consisting of different areas within the perisylvian region [see also Berndt et al., 1996; Dick et al., 2001].

This discussion is focused mostly on activity in Broca's area, as the main goal of the present study was to specify this area's function during syntactic processing. Unlike earlier neuroimaging studies of syntactic complexity effects in the English language [e.g., Caplan et al., 1998, 1999, 2000; Just et al., 1996; Stromswold et al., 1996], we did not obtain the left inferior frontal gyrus activations when object–initial sentences were compared to subject–initial sentences. Interestingly, this result is consistent with several more recent studies comparing object extracted to subject extracted sentences [in English, Caplan et al., 2001; Cooke et al., 2001; and in German, Fiebach et al., in press]. The present study used two types of object–subject comparisons, one in which the dislocated element (in object–initial position) was far from its original position (long distance) and one in which the distance was short. Interestingly, neither of these object–subject comparisons reliably activated Broca's area. Activation in this area instead increased in a different comparison isolating syntactic working memory costs independent of the word order variation.

The posterior left inferior frontal gyrus, its right‐hemisphere homologue, and bilateral superior temporal brain regions were activated most strongly when object–initial sentences with a greater demand on working memory resources were contrasted with structurally very similar sentences that posed less demands on working memory, as is suggested by previous ERP research [Fiebach et al., 2002] (Fig. 1). Accordingly, our results support the assumption that Broca's area houses mechanisms that enable the sentence processor to keep syntactic information actively available over sustained periods of sentences while new linguistic information is being processed continuously [e.g., Caplan and Waters, 1999; Friederici, 2002].

This working memory‐based interpretation of Broca's area activation specific to syntactic aspects of sentence processing is supported by data from a recent fMRI study of English sentences with center‐embedded relative clauses. Cooke et al. [ 2001] compared subject and object relative clause sentences, and also compared sentences with long and short antecedent‐gap dependencies. The authors reached a similar conclusion, i.e., that left inferior frontal cortex was involved in maintaining long‐distance dependencies during sentence processing [Cooke et al., 2001]. However, the activation that Cooke et al. [ 2001] attribute to working memory aspects of sentence processing was located more anteriorly in the inferior frontal gyrus than were the present activation effects, namely in BA 47. This area is generally associated with semantic processing [e.g., Bookheimer, 2002; Dapretto and Bookheimer, 1999], and the authors interpret their results as indicating that dislocated constituents are maintained in working memory as semantic representations [Cooke et al., 2001, p. 92, Footnote 6]. Based on our data and in accordance with evidence from ERPs [e.g., Fiebach et al., 2002; King and Kutas, 1995], we strongly favor the alternative account that syntactic features of dislocated arguments are kept active until the dislocated element can be integrated. The difference in activation sites between the two studies might also be due to differences in the stimulus material used. Whereas the relative pronoun “who” in relative clauses, like those used by Cooke et al. [ 2001], has a head noun in the main clause to which it refers, the interrogative pronouns “wer” and “wen” used in the present study cannot be linked to a previously processed semantic entity and therefore are unlikely to trigger the maintenance of semantic information. The difference could also be attributable to the fact that the stimulus sentences used by Cooke et al. [ 2001] seem somewhat less balanced than the sentences used in the present study, with respect to syntactic structure as well as with respect to categories of words used across sentence conditions. In the Cooke et al. [ 2001] study, factors other than assumed differences in syntactic working memory demands thus might have influenced the critical contrasts.

Further evidence corroborating the assumption that Broca's area activation during the processing of complex sentences is associated with working memory processes comes from a recent study conducted in our laboratory [Fiebach et al., in press]. In this fMRI study, we investigated brain activation associated with the processing of temporarily ambiguous relative clauses in German. Syntactically ambiguous sentences are thought to tax working memory, as they require the maintenance of two alternative syntactic structures up to the point at which a new word disambiguates the sentence to one of the two alternative readings. Analogous to the present study, there were no activation differences between subject and object relatives in Broca's area. Ambiguous sentences that were disambiguated late in the sentence and, therefore, required the maintenance of two syntactic structures in working memory over a longer period, activated Broca's area more strongly than did sentences disambiguated early.

To summarize, the present findings in combination with some recent studies cited above suggest that inferior frontal (and, in some cases, superior temporal) differences in brain activation during syntactic processing are associated with increased demands on syntactic working memory during sentence processing. The present data do not exclude that computational processes of syntactic structure building are also supported by Broca's area. In fact, it has been demonstrated convincingly that the pars opercularis of the inferior frontal gyrus (BA 44) and the deep portion of the left frontal operculum are activated when syntactic information is in the focus of information processing [e.g., Dapretto and Bookheimer, 1999; Friederici et al., 2000]. On the other hand, our results provide a possible explanation for the fact that areas in the fronto‐opercular region are activated in some neuroimaging studies of language processing that experimentally put syntax into the focus, and not in others [e.g., Caplan et al., 2001; Kuperberg et al., 2000; Meyer et al., 2000]. We suggest that BA 44 is recruited mainly in cases when syntactic information has to be maintained temporarily in working memory. Parsing processes that are more computational in nature and temporally more circumscribed might be carried out partly in other brain regions.

Our results add to the discussion of whether in the context of syntactic processing, Broca's area functions as a unitary module or whether it is divided into different subregions contributing to different aspects of sentence processing. We observed one focus of activity in the superior portion of the left inferior frontal gyrus, present in both hemispheres, and we found that in the left hemisphere the activation difference extended also into the inferior tip of BA 44, where a further local maximum was found. The functional distinctness of the superior and the inferior cluster within Broca's area could be established using MDS. The inferior tip of BA 44 is associated generally with syntactic processes [e.g., Friederici, 2002]. The more superior area is located on the border between BA 44 and 45. BA 45 is associated by many authors with more semantic aspects of sentence processing, particularly strategic aspects of semantic processing like retrieval [e.g., Bookheimer, 2002; Gabrieli et al., 1998]. The activation seen in the present study is localized clearly in the posterior portion of BA 45, whereas semantic activations are often seen more anteriorly on the border to BA 47. A recent study by Gold and Buckner [ 2002] suggests that this posterior inferior frontal region might be involved in controlled processes independent of the precise domain of linguistic processing, as these authors found the posterior BA 45 region to be activated during both semantic and phonological tasks.

The functional subdivision of Broca's area into an inferior and a superior portion might also be related to the psycholinguistic differentiation of purely syntactic versus thematic aspects of sentence processing. Lending tentative support to this assumption, Newman et al. [ 2003] found increased activation in pars triangularis (BA 45) for sentences posing problems in the thematic domain, whereas sentences with syntactic violations (i.e., noun‐verb agreement violations) activated the pars opercularis more strongly. The consideration of thematic role assignment processes is also of relevance in the context of the present study. In general, thematic roles can be assigned based on several types of information, i.e., word order information in languages with strict argument order (e.g., English) and, in languages with relatively free argument order, subject verb agreement (e.g., Bulgarian) or case morphology (e.g., German) [see for example Frisch and Schlesewsky, 2001]. In the sentences investigated in the present study, case information was available overtly for the dislocated element (i.e., “wer” vs. “wen”) and on the determiner of the second NP. The observed involvement of BA 44/45 thus might indicate that part of the memory load present during sentence processing might have been due to the maintenance of thematic information in working memory. This would be in line with a psycholinguistic model suggesting that syntactic working memory costs are due to the temporary maintenance of thematic roles that could not be integrated [Gibson, 1991]. Working memory costs based on the maintenance of thematic information would also be greater for long than for short object questions.

The dissociation of inferior and superior portions of BA 44 and BA 44/45, which was demonstrated here by the application of MDS, suggests a functional differentiation between these areas. More detailed experimental work will be needed to test the different hypotheses discussed above.

Finally, the present data argue against a strict lateralization of language processes to the left hemisphere. Although this finding is inconsistent with classic lesion‐based models of language and the brain, it is in line with several recent brain imaging [e.g., Ben‐Shachar et al., 2003; Fiebach et al., in press; Just et al., 1996; Meyer et al., 2000] and lesion studies [e.g., Caplan et al., 1996; Schneidermann and Saddy; 1988]. It was suggested recently that right hemispheric structures might be drawn upon when additional, nonautomatic mechanisms of comprehension are required during language processing [e.g., Meyer et al., 2000]. This account might also hold for the present experiment, as the comprehension task carried out by the participants was relatively difficult.

CONCLUSIONS

We investigated the processing of German wh‐questions and manipulated two aspects of syntactic structure building, namely syntactic integration costs and syntactic working memory load. Increased syntactic memory costs, but not syntactic integration costs, elicited activation increases in inferior frontal and superior temporal brain areas bilaterally. These results strongly suggest that Broca's area is involved in syntactic working memory processes.