Syntactic processing is distributed across the language system

Abstract

Language comprehension recruits an extended set of regions in the human brain. Is syntactic processing localized to a particular region or regions within this system, or is it distributed across the entire ensemble of brain regions that support high-level linguistic processing? Evidence from aphasic patients is more consistent with the latter possibility: damage to many different language regions and to white-matter tracts connecting them has been shown to lead to similar syntactic comprehension deficits. However, brain imaging investigations of syntactic processing continue to focus on particular regions within the language system, often parts of Broca’s area and regions in the posterior temporal cortex. We hypothesized that, whereas the entire language system is in fact sensitive to syntactic complexity, the effects in some regions may be difficult to detect because of the overall lower response to language stimuli. Using an individual-subjects approach to localizing the language system, shown in prior work to be more sensitive than traditional group analyses, we indeed find responses to syntactic complexity throughout this system, consistent with the findings from the neuropsychological patient literature. We speculate that such distributed nature of syntactic processing could perhaps imply that syntax is inseparable from other aspects of language comprehension (e.g., lexico-semantic processing), in line with current linguistic and psycholinguistic theories and evidence. Neuroimaging investigations of syntactic processing thus need to expand their scope to include the entire system of high-level language processing regions in order to fully understand how syntax is instantiated in the human brain.

Graphical abstract

1. Introduction

Language processing is supported by an extended system of brain regions, primarily in the left frontal and temporal lobes (e.g., Binder et al., 1997; Fedorenko et al., 2010). Whereas evidence from both the patient and neuroimaging literatures strongly suggests that this system is selectively engaged in linguistic processes and not in other cognitive processes (e.g., Dronkers, Ludy & Redfern, 1998; Varley, Klessinger, Romanowski, & Siegal, 2005; Fedorenko, Behr, & Kanwisher, 2011; Willems, Benn, Hagoort, Toni, & Varley, 2011; Fedorenko, Duncan, & Kanwisher, 2012a; Monti, Parsons, & Osherson, 2012), the division of linguistic labor among its constituent regions is still heavily debated. A key question for understanding the internal structure of the language system is to what extent different aspects of language comprehension are localized to particular regions within the system versus distributed across the entire system. The answer to this question will reveal which functions are implemented in distinct neural circuits and which functions share neural resources. These organizational principles of neural architecture might, in turn, illuminate the cognitive architecture of the human language faculty (for similar inferences from neural to cognitive architectures in perception, see e.g., Kanwisher, 2010). In the current paper, we specifically focus on syntactic processing: is it localized or distributed across the language system?

Prior literature addressing this issue provides conflicting evidence, such that neuropsychological evidence – on the whole – supports a distributed view of syntactic processing whereas neuroimaging evidence appears to support a more localized view. On the one hand, investigations of patients with brain damage have revealed that lesions to many different parts of the language system can cause similar syntactic comprehension difficulties. Such regions include Broca’s region in the inferior frontal gyrus (e.g., Caramazza & Zurif, 1976; Schwartz, Saffran, & Marin, 1980; Caplan & Futter, 1986; Zurif, Swinney, Prather, Solomon, & Bushell, 1993; Grodzinsky, 2000), the arcuate fasciculus and/or the extreme capsule (e.g., Caramazza & Zurif, 1976; Papoutsi, Stamatakis, Griffiths, Marslen-Wilson, & Tyler, 2011; Rolheiser, Stamatakis, & Tyler, 2011; Tyler et al., 2011; Wilson et al., 2011), posterior temporal regions (e.g., Samuels & Benson, 1979; Selnes, Knopman, Niccum, Rubens, & Larson, 1983; Basso, Lecours, Moraschini, & Vanier, 1985; Tramo, Baynes, & Volpe, 1988; Caplan et al., 1996; Bastiaanse & Edwards, 2004; Wilson & Saygin, 2004; Amici et al., 2007; Tyler et al., 2011; Thothathiri, Kimberg, & Schwartz, 2012), and anterior temporal regions (e.g., Dronkers, Wilkins, Van Valin, Redfern, & Jaeger, 1994; Dronkers, Wilkins, Van Valin, Redfern, & Jaeger, 2004; Magnusdottir et al., 2013). For instance, lesions in all of these regions can impair the interpretation of semantically reversible sentences, such as The boy is chased by the girl, whose meanings (who did what to whom) depend on their syntactic form (i.e., word order, function words, and functional morphology). Consequently, some have argued that syntactic processing is supported by the language system as a whole (e.g., Caplan et al., 1996; Dick et al., 2001; Wilson & Saygin, 2004; Mesulam, Thompson, Weintraub & Rogalski, 2015).

On the other hand, many neuroimaging studies employing syntactic manipulations have found activations not across the entire language system but, instead, restricted to a subset of the system, most commonly in the inferior frontal and posterior temporal regions (e.g., Just, Carpenter, Keller, Eddy, & Thullborn, 1996; Stromswold, Caplan, Alpert, & Rauch, 1996; Cooke et al., 2001; Ben-Shachar, Hendler, Kahn, Ben-Bashat, & Grodzinsky, 2003; Wartenburger, Heekeren, Burchert, De Bleser, R., & Villringer, 2003; Constable et al., 2004; Bornkessel, Zysset, Friederici, von Cramon, & Schlesewsky, 2005; Fiebach, Schlesewsky, Lohmann, Von Cramon, & Friederici, 2005; Caplan, Stanczak, & Waters, 2008; Meltzer, McArdle, Schafer, & Braun, 2009; Peelle, Troiani, Wingfield, & Grossman, 2010; Christensen, Kizach, & Nyvad, 2012; see Friederici, 2011, for a recent meta-analysis). These studies suggest a localized view of syntactic processing, in line with many proposals that link syntax to Broca’s area (e.g., Bornkessel & Schlesewsky, 2006; Grodzinsky & Friederici, 2006; Grodzinsky & Santi, 2008; Friederici, 2009, 2011, 2012; Baggio & Hagoort, 2011; Tyler et al., 2011; Duffau, Moritz-Gasser, & Mandonnet, 2014; Ullman, 2012).

How can we reconcile these two sets of conflicting findings? One possibility is that the localized activation patterns in neuroimaging studies result from (i) the use of group analyses, which suffer from sensitivity loss due to inter-subject variability in the precise locations of activation peaks (e.g., Nieto-Castañon & Fedorenko, 2012); and (ii) differences across brain regions in the overall strength of response to language stimuli. In highly language-responsive regions one might expect relatively wide neighborhoods of strong activation, so that overlap across subjects could be evident despite individual variability in peak location. In regions that are language-selective but respond only weakly to language stimuli, however, one might expect smaller and shallower activation neighborhoods surrounding the (low) peaks, so that overlapping activations across subjects are less likely to emerge. Such reasoning suggests that neuroimaging methods that take into account inter-individual variability may be able to find evidence for distributed, rather than localized, syntactic processing. Therefore, here we use an individual-subjects approach (Fedorenko et al., 2010) that allows us to narrow in on the high-level language processing regions in each individual brain. We measure the effect of syntactic complexity on the response of these individually localized regions and show that, in fact, syntactic complexity modulates neural responses throughout the language system, consistent with the evidence from the patient literature.

2. Materials and methods

To test for sensitivity to syntactic demands, we chose a commonly used syntactic complexity manipulation: the contrast between subject- and object-extracted relative clauses, as in (1) (See also Figure 1).

• (1)
  1. Subject-extracted relative clause: the star that is greeting the circle
  2. Object-extracted relative clause: the circle that the star is greeting

In both (1a) and (1b), the verb phrase is greeting has two arguments (i.e., dependents): a subject who is doing the greeting (the star), and an object who is being greeted (the circle). However, the two sentences critically differ in the distance separating the verb phrase from its two dependents. Specifically, in the subject-extracted relative clause (1a), the dependencies are local: both the word that (which refers to the star) and the object the circle connect locally to the verb phrase is greeting. In contrast, the object-extracted relative clause (1b) has a more complex dependency structure: the verb phrase is greeting is separated from its object, the circle, by the subject the star. An appealing feature of this contrast is that a variety of factors that have been shown to affect sentence comprehension (e.g., Tanenhaus & Trueswell, 1995; Gibson & Pearlmutter, 1998) are matched across the two conditions, including lexical-level factors (the words are identical) and plausibility. So, only the dependency structure (i.e., syntax) varies.

Figure 1.

Schematic illustration of sample trials in the object-extracted condition. In these instances, the picture matching the sentence is on the left.

Across many languages, object-extracted relative clauses like (1b) have been shown to cause comprehension difficulty compared to subject-extracted relative clauses like (1a), as reflected in a variety of dependent measures including reading times and response accuracies to comprehension questions (e.g., English: Wanner & Maratsos, 1978; King & Just, 1991; Gibson, 1998; Grodner & Gibson, 2005; French: Holmes & O’Regan, 1981; Baudiffier, Caplan, Gaonac'h, & Chesnet, 2011; German: Mecklinger, Schriefers, Steinhauer, & Friederici, 1995; Schriefers, Friederici, & Kuhn, 1995; Dutch: Frazier, 1987; Mak, Vonk, & Schriefers, 2002; 2006; Japanese: Miyamoto & Nakamura 2003; Ishizuka, Nakatani, & Gibson, 2003; Ueno & Garnsey, 2008; Korean: O’Grady, Lee & Choo, 2003; Kwon, Polinsky, & Kluender, 2006; Kwon, Gordon, Kluender, & Polinsky, 2010; Russian: Levy, Fedorenko & Gibson, 2013). Therefore, the contrast between object- and subject-extracted relative clauses is considered by many to be a marker of syntactic processing, and has been used widely in both investigations of individuals with aphasia and brain imaging studies.

As mentioned above, in previous neuroimaging work, such contrasts between object- and subject-extractions as well as other, similar contrasts have produced activations largely restricted to Broca’s area, the surrounding regions in the inferior frontal gyrus and the posterior parts of the middle (and sometimes superior) temporal gyrus. Other regions in the language system – such as the orbital portions of the inferior frontal gyrus or the anterior temporal regions – did not show reliable responses. However, this data pattern does not necessarily imply that the former regions are significantly more sensitive to the syntactic manipulation than the latter. Before such a claim is put forward, two methodological issues warrant consideration.

The first issue concerns the sensitivity of fMRI analysis methods to syntactic complexity effects. The vast majority of previous studies have relied on traditional group analyses, where individual brains are transformed into a common space and their contrast maps are then averaged across participants, assuming a shared mapping of functional regions onto anatomy. Although such methods can be effective in detecting large regions of activation that align well across individuals, they suffer from sensitivity loss due to inter-subject anatomical and functional variability (e.g., Saxe, Brett, & Kanwisher, 2006; Nieto-Castañon & Fedorenko, 2012), which has been shown to be especially pronounced in the frontal and temporal cortices (e.g., Frost & Goebel, 2012; Tahmasebi et al., 2012). As a result, even when every subject shows a robust response to syntactic complexity manipulations individually, the effect may get “washed out” by group averaging (see e.g., Fedorenko & Kanwisher, 2011, for an example).

The second issue regards the validity of statistical tests. Namely, observing that some regions show a significant syntactic complexity effect, whereas others do not, cannot be taken as evidence that regions differ from one another in how important they are for syntactic processing. Such an inference would only be licensed by directly comparing contrast effects across regions, with some regions showing a stronger difference between the responses to object- and subject-extractions, compared to other regions. In other words, a region by extraction-type interaction is needed (see e.g., Nieuwenhuis, Forstmann, & Wagenmakers, 2011, for a recent discussion).

In summary, in order to argue that only a particular subset of the language system is engaged in syntactic processing (or, more generally, that different parts of the language system support different kinds of computations), it is important to use methods that take into account inter-subject variability in the exact location of syntactic effects and explicitly test hypotheses of interest. One way to take inter-subject variability into account in the second-level analyses is by using a functional “localizer” that narrows in on the relevant functional subset of each individual brain, in order to then examine the responses of those functionally defined regions to the critical conditions of interest. Thus, we here use a functional localizer for brain regions that support high-level linguistic processing (Fedorenko et al., 2010), which robustly activates the key language-responsive regions in the frontal, temporal and parietal cortices. We then employ a standard sentence-picture matching paradigm with object- and subject-extractions to examine whether syntactic complexity affects the response of these brain regions. In addition to testing the significance of this effect in each region, we also test for a region by extraction-type interaction, to assess whether some regions are more sensitive than others to syntactic complexity.

2.1. Participants

Thirteen participants (10 females) between the ages of 18 and 30 – students at MIT and members of the surrounding community – were paid for their participation. Participants were right-handed native speakers of English, naïve to the purposes of the study. All participants gave informed consent in accordance with the requirements of MIT’s Committee On the Use of Humans as Experimental Subjects (COUHES).

2.2. Design, materials and procedure

Each participant performed the language localizer task (Fedorenko et al., 2010) and the critical syntactic-processing task. Some participants also completed one or two additional tasks for unrelated studies. The entire scanning session lasted approximately 2 hours.

2.2.1. Language localizer task

Participants read sentences (e.g., A RUSTY LOCK WAS FOUND IN THE DRAWER) and lists of unconnected pronounceable nonwords (e.g., DAP DRELLO SMOP UL PLID KAV CRE REPLODE) in a blocked design. Each stimulus consisted of eight words/nonwords. For details of how the language materials were constructed, see Fedorenko et al. (2010). The materials are available at http://web.mit.edu/evelina9/www/funcloc/funcloc_localizers.html. Stimuli were presented in the center of the screen, one word/nonword at a time, at the rate of 350ms per word/nonword. Each stimulus was followed by a 300ms blank screen, a memory probe (presented for 1,350ms), and again a blank screen for 350ms, for a total trial duration of 4.8s. Participants were asked to decide whether the probe appeared in the preceding stimulus by pressing one of two buttons. (In previous work we established that similar activations obtain with a passive-reading task; see Fedorenko et al., 2010.) It is important to note that this localizer contrast (sentences > nonwords) does not specifically target syntactic processing: instead, it broadly targets high-level language processes, including processing of individual word meanings and combinatorial semantic and syntactic processing (see Fedorenko et al., 2010, 2012b, for further discussion) (see also section 4.2).

Condition order was counterbalanced across runs and participants. Experimental blocks lasted 24s (with 5 trials per block), and fixation blocks lasted 16s. Each run (consisting of 3 fixation blocks and 12 experimental blocks) lasted 336s. (Each run contained 4 blocks per condition: in addition to sentences and nonwords, the experiment included a third condition (lists of unconnected words), which was included due to its relevance to another study that was run in the same session.) Each participant completed 3 runs.

2.2.2. Critical task

Participants performed a sentence-picture matching task in an event-related design. On each trial, they saw two pictures – each including two characters interacting in some way – and heard a question prompting them to choose one of the pictures (see Figure 1), by pressing one of two buttons. Sentences contained either syntactically simpler subjectextracted relative clauses (e.g., Where is the star that is greeting the circle?) or syntactically more complex object-extracted relative clauses (e.g., Where is the circle that the star is greeting?).

For the pictures, we used 8 humanized simple shapes (a circle, a square, a triangle, a rectangle, an oval, a heart, a star and an arrow) and 7 easily depict-able actions (chasing, greeting, hugging, lifting, pulling, pushing and touching). Eight characters allowed for 28 unique character-pairs. These were distributed across the 7 actions with 4 character-pairs per action such that (i) each character was used once for each of the 7 actions, and (ii) each action was paired with each of the 8 characters. For each action/character-pair set, we created two versions of a picture (e.g., a circle greeting a star, and a star greeting a circle, as in Figure 1), for a total of 56 pictures. The position of the agent – on the left vs. on the right of the patient – was balanced across items.

For each pair of pictures (e.g., a circle greeting a star, and a star greeting a circle), four sentences were constructed (two per condition), as in (2):

• (2)
  1. Subject-extracted, version 1: Where is the circle that is greeting the star?
  2. Object-extracted, version 1: Where is the star that the circle is greeting?
  3. Subject-extracted, version 2: Where is the star that is greeting the circle?
  4. Object-extracted, version 2: Where is the circle that the star is greeting?

The sentences were recorded by a female native speaker of English, with a natural prosody, which was created to be as similar as possible across trials and conditions.

Each participant saw each pair of pictures four times over the course of the experiment, twice in the subject-extracted condition, and twice in the object-extracted condition. Pairs of pictures were distributed across four runs such that there was only one occurrence of each pair of pictures per run. So, across the experiment there were a total of 28 picture pairs (as in Figure 1) × 4 versions of a sentence = 112 trials (56 trials per condition). The position of the target picture (left, right) was randomized across trials. Trials were 6s long. Each trial began with a 200ms fixation, followed by the presentation of the pictures and the sentence. The pictures were presented for 4,000ms, followed by an extra 1,800ms of fixation. Sentence onset was simultaneous with picture onset and each sentence lasted between 4,510ms and 5,373ms (M=4,919ms). Participants could respond as soon as the sentence began and through the end of the trial. Each of the four runs lasted 252s, which included 28 6s trials and 84s of fixation (interleaved among the trials, such that the inter-trial interval varied between 0 and 8s). Four condition orders were created using the freely available optseq algorithm (Dale, 1999). These orders varied across runs.

2.3. fMRI data acquisition

Structural and functional data were collected on the whole-body 3 Tesla Siemens Trio scanner with a 32-channel head coil at the Athinoula A. Martinos Imaging Center at the McGovern Institute for Brain Research at MIT. T1-weighted structural images were collected in 128 axial slices with 1.33 mm isotropic voxels (TR=2000ms, TE=3.39ms). Functional, blood oxygenation level dependent (BOLD), data were acquired using an EPI sequence (with a 90° flip angle and using GRAPPA with an acceleration factor of 2), with the following acquisition parameters: thirty-one 4 mm thick near-axial slices acquired in the interleaved order (with 10% distance factor), 2.1mm × 2.1mm in-plane resolution, FoV in the phase encoding (A >> P) direction 200mm and matrix size 96 × 96, TR=2000ms and TE=30ms. The first 10s of each run were excluded to allow for steady state magnetization.

2.4. fMRI data preprocessing

MRI data were analyzed using SPM5 and custom Matlab scripts (available – in the form of an SPM toolbox – from http://www.nitrc.org/projects/spm_ss). Each subject’s data were motion corrected and then normalized into a common brain space (the Montreal Neurological Institute (MNI) template) and resampled into 2mm isotropic voxels. The data were then smoothed with a 4mm Gaussian filter and high-pass filtered (at 200s). For both the localizer task and the critical task, effects were estimated using a General Linear Model (GLM) in which each experimental condition was modeled with a boxcar function convolved with the canonical hemodynamic response function (HRF). The boxcar function for the localizer task modeled entire blocks; the function for the critical task modeled entire trials.

2.5. Traditional group analysis

Prior to conducting our key analyses, we aimed to replicate prior findings that used group analyses and reported activations for syntactic complexity manipulations mostly in the inferior frontal gyrus and posterior middle temporal gyrus. Therefore, we ran a group analysis of our critical task by: (i) creating a whole-brain, syntactic complexity contrast map for each participant, contrasting the GLM beta-weights for the object-extracted condition with the weights for the subject-extracted condition; and (ii) entering the contrast maps of all participants into a second-level GLM analysis (p<0.001, uncorrected).

2.6. Group-constrained, Subject-Specific analysis

Unlike the previous fMRI investigations of syntactic complexity that used traditional group analyses, our key analyses here were performed within regions of interest that were defined functionally in each individual participant. These regions of interest were defined using the sentences > nonwords contrast in the language localizer task. To do so, we used the Group-constrained Subject-Specific (GSS) analysis method developed in Fedorenko et al. (2010; Julian, Fedorenko, Webster, & Kanwisher, 2012). In particular, functional regions of interest (fROIs) were constrained to fall within a set of functional “masks” which indicated the expected gross locations of activations for this contrast and which were generated based on a group-level data representation from an independent group of participants (see Fig. 2a; Fedorenko et al., 2010). These masks were intersected with each individual subject’s activation map for the sentences > nonwords contrast, and the voxels falling within each mask were sorted based on their t-values for the localizer contrast, choosing the top 10% of voxels as that subject’s functional region of interest (see Fig. 2c for sample fROIs). This top n% approach ensures that the fROIs can be defined in every participant – thus enabling us to generalize the results to the entire population (Nieto-Castañon & Fedorenko, 2012) – and that fROI sizes are the same across participants. However, qualitatively similar results were obtained in an alternative analysis approach where the fROIs were defined as all the voxels that (i) fell within the relevant mask, and (ii) passed a fixed significance threshold (p<0.001, uncorrected) at the whole-brain level.

Figure 2.

Functional regions of interest (fROIs) in the language system. (a) The probabilistic overlap map for the contrast sentences > nonwords in a prior dataset of 25 subjects (Experiments 1 and 2 in Fedorenko et al., 2010). This map was used for generating group-based masks (outlined in gray) which were then used in the current experiment to constrain the selection of individual subjects’ fROIs. (b) The probabilistic overlap map of individual fROIs in the current experiment (shown in red), constrained to fall within the masks (outlined in gray) that were defined based on the prior data shown in (a). (c) Individual fROIs in six sample subjects in the current experiment.

Eight fROIs were defined in each subject. These included three fROIs on the lateral surface of the left frontal cortex in the inferior frontal gyrus (LIFG) and its orbital part (LIFGorb) as well as in the middle frontal gyrus (LMFG); and five fROIs on the lateral surface of the temporal and parietal cortex, in the anterior temporal cortex (LAntTemp), middle anterior temporal cortex (LMidAntTemp), middle posterior temporal cortex (LMidPostTemp), posterior temporal cortex (LPostTemp) and angular gyrus (LAngG). We here chose to focus on these “core” regions in the left hemisphere, which most robustly and consistently emerge in the investigations of the language system and which include the regions most frequently linked to syntactic processing (but see Appendix A for information on the responses to syntactic complexity of the right hemisphere homologues of these regions, and a couple of additional brain regions that consistently emerge for the localizer contrast).

To estimate the responses of fROIs to the conditions of the language localizer, we used an across-runs cross-validation procedure. In particular, each subject’s activation map was first computed for the sentences > nonwords contrast using all but one run of data, and the 10% of voxels with the highest t-values within a given mask were selected as that subject’s fROI. The response of each fROI to the same contrast was then estimated using the left-out run. This procedure was iterated across all possible partitions of the data, and the responses were finally averaged across the left-out runs to derive a single response magnitude for each condition in a given fROI/subject. This n-fold cross-validation procedure (where n is the number of functional runs) allows one to use all of the data for defining the ROIs and for estimating their responses (see Nieto-Castañon & Fedorenko, 2012, for discussion), while ensuring the independence of the data used for fROI definition and for response estimation (e.g., Kriegeskorte, Simmons, Bellgowan, & Baker, 2009). To estimate the responses of fROIs to the conditions of the critical experiment (i.e., to object-extracted and subject-extracted sentences), data from all runs of the language localizer experiment were used for defining the fROIs.

To summarize the logic of our approach: the language localizer allows us to identify a set of voxels / regions that respond robustly during language processing. We then focus specifically on these regions to test their responses to the critical contrast between syntactically more complex (object-extracted) and syntactically simpler (subject-extracted) relative clauses. If syntactic processing is distributed across the entire language system – rather than localized to particular regions – we should find (i) sensitivity to syntactic complexity in most or all of our fROIs, and (ii) no region-by-condition interactions, indicating that the different regions are similarly sensitive to syntactic complexity.

Statistical tests across subjects were performed on the beta weights extracted from the fROIs as defined above. Two contrasts were examined: (i) sentences > nonwords, and (ii) object-extracted > subject-extracted.

3. Results

3.1. Behavioral results

Due to a script error, behavioral responses for the sentence-picture matching task were not recorded for 6 of the 13 participants. However, 4 of these 6 were later brought in and re-tested behaviorally on exactly the same version of the experiment, so that altogether we obtained behavioral data from 11 of the 13 participants. In those 11 participants, we replicate the standard complexity difference, with slower RTs (4.57s vs. 4.40s; t₍₁₀₎=2.23, p<0.05) and lower accuracies (91% vs. 96%; t₍₁₀₎= −2.95, p<0.05) in the object-extracted condition. See also section 3.3.3 below.

3.2. Traditional fMRI group analysis

A whole-brain random-effects group analysis of the syntactic complexity contrast object-extraction > subject-extraction revealed several significant activation clusters, of which three appeared in regions of the left-hemisphere commonly associated with language processing. These clusters were located in (i) the posterior part of the middle temporal gyrus; (ii) the triangular part of the inferior frontal gyrus; and (iii) the mid-anterior part of the superior temporal gyrus (Table 1). The former two clusters are broadly consistent with the activation foci most commonly found in prior studies using similar contrasts. This can be seen in Figure 3, showing our group-level activation map along with marked locations of previously reported syntactic complexity effects.

Table 1.

Activation clusters for the syntactic complexity contrast (object-extracted > subject-extracted) identified with traditional group analysis

Regiona	Center coordinates (mm)b			Volume (mm3)	Peak t-value
	x	y	z
Left middle temporal gyrus (posterior)	−48	−58	5	4624	7.09*
Left inferior frontal gyrus, triangular	−40	28	0	88	5.36*
Left superior temporal gyrus (mid-anterior)	−57	−12	−5	280	7.26*

^aWe only report clusters that are located in left-hemispheric regions commonly associated with the language system.

^bCoordinates are reported in MNI space.

^*p<0.001, uncorrected for whole-brain multiple comparisons.

Figure 3.

Syntactic complexity effects in the left hemisphere identified with traditional group analysis. Both (a) and (b) show the activation map of our critical contrast, object-extraction > subject-extraction (p<0.001, uncorrected for whole-brain multiple comparisons) in hot colors. White circles show the locations of activations to similar syntactic complexity contrasts reported in prior studies as reviewed by Friederici (2011; referred to in that paper as “studies of movement”). Notice that the activations in the current study fall within the same general locations found previously, namely, the posterior middle temporal gyrus and the inferior frontal gyrus. (a) The effects are superimposed on sagittal slices of an anatomical scan from one of our participants. (b) The effects are projected onto an inflated cortical surface of an average brain in MNI space.

3.3. Group-constrained, Subject-Specific fMRI analysis

3.3.1. Are syntactic complexity effects localized to particular regions within the language system?

Replicating previous work, we find robust responses for the localizer contrast (sentences > nonwords) in each of the eight fROIs, using across-run cross-validation (for all regions, t₍₁₂₎>5, p<10⁻⁴; t-tests are across subjects). Critically, all of the regions defined by the localizer, except for the LAngG fROI, also showed a significant effect for our syntactic complexity manipulation. All effects remain significant after false-discovery rate (FDR) correction for the number of fROIs. These key results are summarized in Figure 4 and Table A.1.

Figure 4.

Responses of the language fROIs to the conditions of the language localizer and the critical experiment. Error bars represent standard errors of the mean by participants. The sentences > nonwords contrast is highly significant (p<10⁻⁴) in every region (this analysis was carried out using across-runs cross-validation, so that the data used to define the fROIs and estimate the responses are independent, as described in section 2.6). For the object-extracted > subject-extracted contrast: * significance at the p<0.05 level, and *** significance at the p<10⁻³ level or stronger. All effects remain significant after an FDR correction for the number of regions (n=8). (Note that it is difficult to directly compare the magnitudes of response to the sentences condition of the localizer task and the magnitudes of response to the two critical conditions, because of many differences in the design, materials and procedure across the two experiments.)

3.3.2. Do regions differ reliably with respect to how sensitive they are to syntactic complexity?

We found that every region within the language system (with the exception of the LAngG fROI) responds reliably more strongly during the syntactically more complex object-extracted condition than during the syntactically simpler subject-extracted condition. However, the difference between these two conditions is numerically larger in some regions than others. In particular, the largest syntactic complexity effects are observed in the brain regions that have been reported most consistently in previous studies (i.e., regions in and around Broca’s area and regions in the posterior temporal cortex; the LMFG fROI – also reported in a few prior studies (e.g., Meltzer et al., 2009) – also shows a large effect). One possibility is that these regions are in fact more strongly engaged in – and thus perhaps play a bigger role in – syntactic processing compared to the rest of the language system. Nevertheless, our data suggest that this is not the case.

In particular, the overall response to language (e.g., the response to the sentences condition of the language localizer relative to the fixation baseline) also varies across regions: it is numerically stronger in the more superior and dorsal frontal regions (the LIFG and LMFG fROIs) than in the inferior and ventral LIFGorb fROI, and it is stronger in the LMidPostTemp fROI than in the more anterior temporal regions (LAntTemp and LMidAntTemp fROIs) and the more posterior temporal/temporo-parietal regions (LPostTemp and LAngG fROIs). This pattern of different-strength BOLD responses across the language regions is consistent across participants and studies (see e.g., Fedorenko et al., 2010, 2011; Mahowald & Fedorenko, in revision.).

Given that effect sizes tend to scale with overall response strength, it is not surprising that the effects of syntactic complexity are more difficult to detect in brain regions where the response to language (or perhaps the BOLD signal more generally; see also Appendix B) is overall weaker. Indeed, we find that the overall language response (sentences > fixation effect) in a particular fROI is a significant predictor of that fROI’s response to syntactic complexity (object-extracted > subject-extracted effect), using a linear mixed-effects regression predicting the syntactic effect size from the overall response with random intercepts and slopes for fROI and participant (β=0.18, t=3.16, χ²₍₁₎ = 7.39, p<0.01). Note that this finding cannot be accounted for by differences in the number of voxels across fROIs, because we obtained similar results when equating the volumes of our fROIs (β=0.18, t=3.43, χ²₍₁₎ = 8.46, p<0.01).

In Figure 5, we show the relationship, across the eight ROIs, between the overall response to language (sentences > fixation, in the localizer experiment) and the size of the object-extracted > subject-extracted effect (effect sizes are averaged across participants). As can be seen, the LPostTemp and LIFG fROIs both show a larger syntactic effect than would be predicted by their overall language response (they fall above the trend line), whereas LAntTemp and LMidAntTemp fROIs show a smaller syntactic effect than would be predicted by their overall language response. However, none of these deviations are significant. Specifically, allowing the association between overall language response and syntactic complexity effect size to vary across fROIs, by including a random “overall language response” slope for each fROI, does not significantly improve the model (χ²₍₂₎ = 0.25, n.s.). The standard deviation of this random “overall language response” slope is very small (0.01) compared to the size of the corresponding fixed effect (β = 0.18), suggesting that our different fROIs contribute indistinguishable data to the model. Thus, although language regions may differ slightly in the relative strengths of the syntactic complexity effect, most of this variance appears to be accounted for by differences in the overall response to language stimuli across the language system. Beyond this explainable variance we find no evidence for a region-by-condition interaction and we cannot reject the hypothesis that our fROIs are all similarly sensitive to syntactic complexity manipulations.

Figure 5.

The syntactic complexity effect size co-varies with overall sensitivity to language. The mean size, across participants, of the syntactic complexity effect (object-extracted > subject-extracted) is plotted against the mean effect size of the overall response to language, as estimated in the localizer experiment (sentences > fixation). To control for inter-individual differences in the overall response strength, data for the eight fROIs were z-scored within each participant prior to averaging. Crosses show standard errors across participants for both effects. A dashed, black line depicts the linear regression line for predicting the syntactic complexity effect based on the overall language response, and was estimated for visualization purposes only (the linear mixed-effects regression reported in the Results section was carried out using individual data from all participants).

In the Appendix (sections B and C), we further explore the correlation between the size of the syntactic complexity effect and the magnitude of overall language response. First, we show that this correlation also holds across individual voxels (not just responses averaged across entire fROIs). Second, we show that this correlation is language-specific: the syntactic complexity effect size does not correlate with a nonlinguistic contrast based on a manipulation of working memory load. Because the correlation appears to be language-specific, we conclude that it is not explained by general physiological artifacts (i.e., differences across voxels in susceptibility to signal loss which might cause any two response measures to correlate across voxels). Third, we exclude the possibility that the magnitude of the overall language response (sentences > fixation) is simply another measure of syntactic processing, which would otherwise trivially explain why it correlates with the syntactic complexity effect size. Namely, we find that the size of the syntactic complexity effect is also correlated with the size of another effect: word lists > fixation, a lexical effect that does not include sentence-level syntax. This conclusion is also supported by a replication of our main result (a syntactic complexity effect distributed across the language system) in fROIs that were defined based on an alternative localizer contrast: word lists > nonword lists.

3.3.3. Does the size of the fMRI syntactic complexity effect predict any aspects of the behavioral performance?

Participants varied with respect to the size of the object-extracted > subject-extracted effect. Might these differences be functionally important? In an exploratory analysis, we examined the relationship between the size of the syntactic complexity effect and behavioral performance. Before doing so, we examined the reliability of the behavioral measures, because if the performance estimates are too noisy at the individual-participant level, there is no reason to expect them to correlate with the effect size in fMRI data. We considered four measures: difference in the accuracy / RT between the object-extracted and subject-extracted conditions, and overall (averaged across the two conditions) accuracy / RT. For each measure, we split the data into odd and even-numbered trials and looked for correlations across subjects. Although both of the RT measures were highly reliable (r=0.91 for the overall RT, and r=0.51 for the object-extracted vs. subject-extracted difference), neither showed a reliable relationship with the size of the fMRI syntactic complexity effect. As for accuracies, the difference measure was not correlated between odd- and even-numbered trials (r=−0.14), but the overall accuracy was highly reliable across the two data halves (r=0.70).

When we correlated the overall behavioral accuracies with the size of the fMRI syntactic complexity effect, we found that participants with larger syntactic complexity effects in fMRI performed significantly worse in the task. As we see in Figure 6, all 8 fROIs show this trend, again highlighting the similarity among these regions with respect to their engagement in syntactic processing. The effect is significant in a linear, mixed-effects model predicting the size of the fMRI syntactic complexity effect from the logit-transformed accuracies with random intercepts for participant and random intercepts and slopes for fROI (β=−0.21, t=−3.48, χ²₍₁₎=9.29, p<0.01). (This relationship remains significant after a Bonferroni correction for the number of behavioral measures examined, i.e., 4.)

Figure 6.

The relationship between task accuracy and the size of the syntactic complexity effect (object-extracted > subject-extracted) in fMRI. Data is shown for each of the 8 fROIs, which all show a downward trend. Blue lines are based on a simple linear regression for each region, with smoothed 95% confidence intervals shaded in gray. Most of the points fall above 0, which shows the main effect of increased fMRI response to the object-extracted condition relative to the subject-extracted condition.

One way to interpret this relationship is in terms of comprehension efficiency. For example, some participants may have greater exposure to syntactically complex object-extracted structures and/or have greater working memory capacity (see Discussion), and consequently may not need to activate their language regions more strongly to process the more syntactically complex structures. Such participants are also likely to be overall better in their language comprehension ability, thus answering comprehension questions more accurately.

3.4. Why do the traditional group-analysis and the group-constrained, subject-specific analysis produce different results?

Traditional group analyses, by design, identify regions of activation that are overlapping across many participants. Regions in which activations show less inter-individual overlap will therefore be missed by such analyses (see e.g., Nieto-Castañon & Fedorenko, 2012, for the underlying math and simulation data). Our alternative analysis method, in contrast, allows for some variability in the locations of activations across people due to its use of individually defined functional regions of interest. Given our results above, we reasoned that activation maps for the syntactic complexity effect would show relatively higher inter-individual overlap in the LIFG and LPostMidTemp compared to other fROIs, as these regions were identified by the traditional group analysis.

To visualize these potential differences across regions, we identified regions showing syntactic complexity effects in each participant and evaluated their inter-individual overlap. To this end, we first identified activation peaks in individual maps of the syntactic complexity contrast (object-extracted > subject-extracted) using a watershed algorithm (to prevent the algorithm from over-parcellating the contrast maps, they were smoothed with a 8mm FWHM Gaussian kernel). An “activation neighborhood” around each peak was then defined (in the original, non-smoothed activation maps) as the largest contiguous set of surrounding voxels having numerically positive contrast estimates. Finally, for each voxel in common MNI space, we counted the number of participants for whom that voxel belonged to an activation neighborhood.

Figure 7 shows the overlap measures we obtained. As can be seen, activations in LIFG and LMidPostTemp show the highest overlap across participants, along with LMFG. These three regions are also the ones where the syntactic complexity effects are numerically the strongest, and these two observations are plausibly linked. Given that regions of the language system show relatively high inter-individual variability in their functional-to-anatomical mapping (e.g., Amunts et al., 1999; Fischl et al., 2008; Frost & Goebel, 2012; Tahmasebi et al., 2012), overlap in activation maps across participants is mainly expected in the most responsive regions that have high peaks and, thus, larger activation neighborhoods.

Figure 7.

Overlap across participants in the anatomical location of the syntactic complexity effect. Heat maps depict voxels in which more than 40% of participants have an “activation neighborhood” for the syntactic complexity contrast (object-extracted > subject-extracted). Neighborhoods were defined as maximal sets of contiguous voxels that surrounded an activation peak and had contrast estimates numerically greater than zero. Black contours depict our group-based masks (from Fedorenko et al., 2010) used to define fROIs. Numbers correspond to the order of fROIs in Figure 4: 1, LIFGorb; 2, LIFG; 3, LMFG; 4, LAntTemp; 5, LMidAntTemp; 6, LMidPostTemp; 7, LPostTemp; 8, LAngG. Data are superimposed on horizontal slices of Freesurfer’s average T1 scan in common MNI space. Slices were chosen to maximize visibility of the greatest overlap in each mask. Note the especially high overlap (shown in dark red) in the LIFG and LMidPostTemp.

4. Discussion

Our results demonstrate that syntactic complexity effects – greater responses to more syntactically complex sentences – are not localized to particular regions within the language system, but are instead found throughout the entire system. Although our results are consistent with prior studies that had observed these effects in the inferior frontal and posterior temporal brain regions, we also show that these effects obtain in the rest of the language regions (with the sole exception of the LAngG fROI), including the language-responsive regions in the orbital LIFG, and in the anterior portions of the lateral temporal cortex.

As discussed in the Introduction, an architecture where syntactic resources are distributed across the language system fits well with the findings from the patient literature: deficits in syntactic comprehension have been reported following damage to many different components of the language system (e.g., Caplan et al., 1996; Dick et al., 2001; Wilson & Saygin, 2004; Mesulam et al., 2015; see Mesulam, 1990, for an early discussion of distributed language processing).

Some previous neuroimaging results further support the idea of distributed syntactic processing, although they do not isolate syntactic processing from other aspects of language comprehension, as the current manipulation does. For example, a contrast between sentences (in which words combine to form syntactic structures) and lists of unconnected words (devoid of such structures) produces activation across the language system (e.g., Snijders et al., 2009; Fedorenko et al., 2010; Pallier, Devauchelle, & Dehaene, 2010; Bedny, Pascual-Leone, Dodell-Feder, Fedorenko, & Saxe, 2011; Brennan & Pylkkanen, 2012; see also earlier studies – Mazoyer et al., 1993; Schlosser, Aoyagi, Fulbright, Gore, & McCarthy, 1998; Vandenberghe, Nobre, & Price, 2002; Humphries, Love, Swinney, & Hickok, 2005; Humphries, Binder, Medler, & Liebenthal, 2006 – although those typically found activations only in parts of the language system). Admittedly, the sentences > word lists contrast is not a “pure” syntactic manipulation, because sentences differ from word lists in additional ways: they also engage compositional semantic processes and possibly, at least for auditory presentation, prosodic processes. A somewhat syntactically purer contrast, between Jabberwocky sentences (which preserve the word order, function words and functional morphology of real sentences but use nonwords) and lists of unconnected nonwords, also produces a response throughout the language system (e.g., Fedorenko et al., 2010; Bedny et al., 2011) (see also section 4.1.3 and Appendix D).

Nonetheless, a vast majority of prior neuroimaging studies of syntactic complexity have instead supported a localized, rather than distributed, view of syntactic processing. We have argued that these prior investigations may not have observed effects in some parts of the language system because of poor sensitivity of traditional group-based analyses (e.g., Nieto-Castañon & Fedorenko, 2012) and because those regions have an overall weaker response to language and thus smaller, harder to detect, effects, especially for subtle manipulations. Our current findings support this claim: first, we directly contrasted a traditional group-based analysis that found evidence for a few localized foci of syntactic complexity effects, with an analysis based on individual localization of language-responsive fROIs that found these effects robustly present throughout the language system. Second, the regions that the group-based analysis failed to identify were shown to have higher inter-individual variability (i.e., less overlap) of activations. This poorer overlap appeared to coincide with lower responsiveness to language in those regions, compared to the regions that the group-based analysis did successfully identify.

Similar reasoning applies to other studies that have targeted syntactic processing and reported effects only in the inferior frontal and posterior temporal regions (e.g., syntactic violation manipulations: Embick, Marantz, Miyashita, O’Neil, & Sakai, 2000; Cooke et al., 2006; Friederici, Kotz, Scott, & Obleser, 2010; Herrmann, Obleser, Kalberlah, Haynes, & Friederici, 2012; or syntactic priming: Santi & Grodzinsky, 2010; Menenti, Segaert, & Hagoort, 2012; Segaert, Menenti, Weber, Petersson, & Hagoort, 2012). We hypothesize that those effects (and possibly other, non-syntactic, effects; e.g., Devlin et al., 2000), like the syntactic complexity effects studied here, are actually present throughout the language system.

It is worth noting that, contra proposals about Broca’s area and parts of the posterior temporal cortex being the core syntactic centers of the brain, a number of researchers have argued that parts of the anterior lateral temporal cortex are instead critically engaged in combinatorial syntactic (and/or semantic) processing (e.g., Humphries, Willard, Buchsbaum, & Hickok, 2001; Vandenberghe et al., 2002; Humphries et al., 2005; Rogalsky & Hickok, 2009; Baron & Osherson, 2011; Bemis & Pylkkanen, 2011; Brennan, Nir, Hasson, Malach, Heeger, & Pylkkänen, 2012; Zhang & Pylkkänen, 2015) (for a further discussion of the anterior temporal lobe, see Appendix C). We suspect that, as with the above studies, the observed effects are present across the language system, although it is not at present clear why these studies differ from the studies above in observing the effects in the anterior temporal as opposed to inferior frontal and posterior temporal regions.

4.1. What do syntactic complexity effects reflect? Interpretations and limitations

4.1.1 Causal involvement in syntactic processing

The finding that a distributed set of language regions are all sensitive to syntactic complexity manipulations should not be interpreted as demonstrating that all of these regions play an equal role in syntactic processing. For example, we do not suggest that every region that shows a stronger response to syntactically complex sentences than to syntactically simpler sentences is causally involved in syntactic processing. A possible alternative is that only a subset of our language fROIs are critical for processing syntax, but their output rapidly travels to the rest of the language system and is therefore reflected in the temporally slow BOLD signal. Furthermore, whereas syntactic complexity primarily modulates syntactic processing, it may additionally modulate other comprehension processes (like those related to the processing of information structure; e.g., Jackendoff, 1972), thus leading in some fROIs to linguistic but non-syntactic “secondary effects” masking as syntactic effects (but see section 4.2).

Like all fMRI studies, our current study is not designed to (and could not) distinguish regions that are causally involved in syntactic processing from those that are more epiphenomenally recruited. Neuropsychological studies are also limited in their ability to identify such distinctions among regions, because naturally occurring brain damage typically encompasses multiple functional areas, as well as extending to white matter tracts connecting regions that may themselves be unaffected by the lesion (e.g., Mesulam et al., 2015). Identifying regions that are critical for syntactic processing ultimately requires causal measurements with both high temporal and spatial resolution, such as invasive stimulation studies using subdural electrodes (inserted pre-surgically for medical reasons; e.g., Ojemann, Ojemann, Lettich & Berger, 1989). Nonetheless, we emphasize that our current contribution is the demonstration that signals reflecting the modulation of neural activity by syntactic complexity (whatever such activity reflects at the mechanistic and cognitive levels) are present throughout the language system, contrary to many previous suggestions.

4.1.2. Experience-based vs. working-memory-based accounts of syntactic complexity

For brain regions that are causally linked to syntactic processing – whatever subset of the language system these may turn out to correspond to – another question arises: which of the factors underlying the complexity difference between object-extracted and subject-extracted structures modulate the activity of these regions? Two classes of proposals have been advanced to account for such complexity differences: experience-based theories (e.g., Hale, 2001; Levy, 2008; Gennari & MacDonald, 2008, 2009; Wells, Christiansen, Race, Acheson, & MacDonald, 2009) and working-memory-based theories (see O’Grady, 2011; Gibson, Tily, & Fedorenko, 2013, for overviews). According to the former, object-extractions are more difficult to understand because they are less frequent in the input. According to the latter, processing object-extractions places greater demands on working memory because one of the dependents of the verb has to be retrieved from memory when the verb is encountered (e.g., Gibson, 1998; Gordon, Hendrick & Johnson, 2001, 2004; McElree, Foraker & Dyer, 2003; Grodner & Gibson, 2005; Fedorenko, Gibson & Rohde, 2006; Lewis, Vasishth & Van Dyke, 2006; Fedorenko, Woodbury & Gibson, 2013). Neither class of proposals can fully explain the rich empirical picture that has emerged from dozens of sentence processing studies, and most researchers now agree that a complete account of language comprehension requires both a probabilistic grammar component and a (plausibly domain-general) working memory resource (e.g., Boston, Hale, Vasishth, & Kliegl, 2011; Demberg & Keller, 2008; Lewis et al., 2006; Gibson et al., 2013; Levy et al., 2013).

Does the syntactic complexity effect we observed throughout the language system reflect the differences in frequency between object-extracted and subject-extracted constructions, or the different demands they place on working memory? Fedorenko et al. (2011) showed that regions of the language system do not respond to general working memory demands, although in the left frontal cortex they lie adjacent to other, distinct regions that are strongly modulated by working memory demands (Fedorenko et al., 2012a; Fedorenko, Duncan & Kanwisher, 2013). The effects reported here in the language regions are thus unlikely to reflect differences in working memory (cf. Fiebach, Schlesewsky, & Friederici, 2001; Kaan & Swaab, 2002; de Vries, Monaghan, Knecht, & Zwitserlood, 2008; Rogalsky & Hickok, 2011). We therefore conjecture that these effects reflect differences in the relative frequencies of the two constructions, although we note that the design of current experiment cannot provide evidence favoring either account over the other.

This interpretation of our results does not contradict the contribution of general working memory resources to the syntactic complexity effects. Namely, whereas we have here focused on the regions of the language system, syntactic complexity manipulations also produce responses in the regions of the domain-general fronto-parietal “multiple demand (MD)” system (e.g., Barde, Yeatman, Lee, Glover, & Feldman, 2012), and damage to some MD regions can lead to difficulties with syntactically complex structures (e.g., Amici et al., 2007). More generally, MD regions respond to diverse executive tasks (e.g., Duncan & Owen, 2001; Corbetta & Shulman, 2002; Duncan, 2010; Fedorenko et al., 2013) across many domains, including language (e.g., Rodd, Davis, & Johnsrude, 2005; Novais-Santos et al., 2007; January, Trueswell, & Thompson-Schill, 2009; McMillan, Clark, Gunawardena, Ryant, & Grossman, 2012; McMillan et al., 2013; Nieuwland, Martin, & Carreiras, 2012; Wild et al., 2012). An important goal for future work is thus to understand the division of labor between language and MD regions during syntactic processing (see also Fedorenko, 2014, for discussion). For example, which regions exhibit sensitivity to syntactic complexity earlier? Does activity in each system relate to distinct aspects of behavior? Is MD activity causally important for language comprehension (e.g., Amici et al., 2007)?

4.1.3. Interactions between stimulus and task

The discussion above assumes that the syntactic complexity effects we observed reflect, in some way or another, an inherent difference between object-extracted and subject-extracted clauses – a “pure” difference between construction types that would replicate whenever such sentences are processed. Is it possible that these effects instead result from an interaction between construction and our particular sentence-picture matching task? Perhaps some extra-linguistic aspects of this task are more difficult when hearing object-extracted sentences compared to subject-extracted sentences, accounting for our results.

For instance, it has been previously argued that both sentence types tend to be initially parsed by assigning the active role of an agent to the first noun encountered (a “subject-first” assumption; e.g., Frazier, 1987; Frazier & Flores d’Arcais, 1989; Schriefers et al., 1995; Schlesewsky, Fanselow, Kliegl & Krems, 2000; Traxler et al., 2002); this assignment is correct only in subject-extracted sentences (the circle that is greeting the star), and require reanalysis in the case of object-extracted sentences, where the first noun is the patient of an action (the circle that the star it greeting). Perhaps then, upon hearing the first noun in our sentence stimuli, participants searched for a picture in which that noun was depicted as the agent rather than the patient. Such a strategy would correctly solve the task for the subject-extracted sentences, but would force participants to switch pictures upon reanalysis of the object-extracted sentences. Some cognitive process involved in this picture switching might underlie the stronger activations in language regions observed for the latter sentences compared to the former.

Interpreting our results as reflecting stimulus-task interactions appears to require that the extra-linguistic differences in task performance for the two sentence types involve executive functions (guiding behavioral strategies), response inhibition, working memory or other, similar, domain-general cognitive resources. However, previous data show that such mental processes do not recruit the language system (as discussed in section 4.1.2). Specifically, language regions respond at or below a low-level baseline to tasks that have general demands similar to those of the sentence-picture matching task (see Fedorenko et al., 2011, 2012a).

Furthermore, effects of syntactic complexity like the one studied here are among the most robust sentence-level linguistic phenomena and have been shown to hold across a wide range of paradigms in the prior literature (see section 2; e.g., reading with comprehension questions or plausibility judgments, listening with comprehension questions, listening with a concurrent lexical-decision task or nonword detection task, sentence repetition, etc.). It is generally assumed that the mental processes underlying syntactic complexity effects across all these diverse paradigms are the same.

Importantly, syntactic complexity effects also replicate in naturalistic materials under passive reading conditions, where no interaction with an externally imposed task is expected (e.g., Demberg & Keller, 2008). To further support this claim, Appendix D includes an analysis of a syntactic contrast under passive reading conditions from a previously reported dataset (Fedorenko et al., 2010). Consistent with our main results, this supplementary analysis demonstrates that all language regions (except for the left AngG) show a stronger response as syntactic processing demands increase. We therefore conclude that extra-linguistic processes caused by an interaction between sentence type and the sentence-picture matching task are not likely to affect the observed responses in the language system.¹

4.2. Is syntactic processing cognitively inseparable from other aspects of language comprehension?

Perhaps the most important consequence of the finding that syntactic processing is not localized to a subset of the language system is the suggestion of strong (and probably complete) overlap between regions that support syntactic processing and those that process word-level meanings (e.g., Fedorenko et al., 2012b; see Bates & Goodman, 1999, for an earlier extensive review and discussion; cf. Marin, Saffran, & Schwartz, 1976; Caramazza & Berndt, 1978, for earlier opposing views). Indeed, lexico-semantic processing appears to be similarly distributed across the language system. For example, contrasts between single words and various baselines (fixation, false fonts, pseudowords, etc.) elicit responses in all the language regions considered here (e.g., Humphries, Binder, Medler, & Liebenthal, 2007; Diaz & McCarthy, 2009; Fedorenko et al., 2010; Bedny et al., 2011).

Of course, it is not straightforward to compare roughly similar distributions of syntactic and lexico-semantic effects across separate studies, especially given the high inter-individual variability in the precise anatomical locations of language regions. It is possible that, within the same individual, each language region consists of several sub-regions, some more heavily recruited during syntactic processing and other more heavily recruited during lexico-semantic processing. Sub-regions of the latter kind might have been missed by our language localizer contrast (sentences > nonwords) if this contrast was somehow biased, such that syntactic differences across its two conditions were stronger than lexico-semantic differences.

However, even when we change our localizer contrast to a “purely” lexical comparison between word lists and nonword lists, the identified language regions show the critical syntactic complexity effect (Appendix C). More generally, other studies have directly contrasted lexical and syntactic manipulations and found overlapping activations. For example, Röder, Neville, Bien, & Rösler (2002; see also Keller, Carpenter, & Just, 2001) examined syntactically complex vs. simpler sentences that were made up of real words vs. pseudowords. Inferior frontal and posterior temporal regions showed sensitivity to both manipulations: sentences composed of real words produced stronger responses than pseudoword sentences, and syntactically complex sentences produced stronger responses than syntactically simpler sentences. Thus, at least at the spatial scale of voxels measured with fMRI, syntactic and lexico-semantic processes appear to recruit the same set of regions distributed across the entire language system.

What are the theoretical implications of an overlap between syntactic processes and lexico-semantic processes at the level of their neural implementation? Specifically, does such overlap indicate that these processes are cognitively inseparable? This conjecture is in line with most current linguistic frameworks and the wealth of available psycholinguistic evidence. Specifically, when we know a language, we possess (i) a large but limited inventory of linguistic knowledge representations (e.g., words); and (ii) an ability to combine these stored knowledge representations to form a potentially infinite number of new meanings, i.e., a compositional capacity (e.g., Frege, 1914). Early proposals (e.g., Chomsky, 1965) linked lexico-semantic processing to the storage component of language (i.e., our lexicon), and syntactic processing – to its combinatorial component. However, over the last several decades, the nature of stored linguistic representations has evolved to allow for greater complexity, including information about how morphemes and words can combine with one another (e.g., Joshi, Levy, & Takahashi, 1975; Bresnan, 1982; Schabes, Abeille, & Joshi, 1988; Pollard & Sag, 1994; Bybee, 1998, 2010; Goldberg, 1995; Chomsky, 1995; Jackendoff, 2002, 2007; Culicover & Jackendoff, 2005). Consequently, many current proposals construe language knowledge as a continuum from the sounds of the language, to morphemes and words, to more complex units like words stored with the syntactic/semantic contexts in which they frequently occur (the degree of abstractness of these contexts varies depending on the details of the particular proposal). This view is supported by much experimental work showing that comprehenders appear to keep track of co-occurrences at different grain sizes, crossing the boundaries between words and combinatorial rules (e.g., Clifton, Frazier, & Connine, 1984; MacDonald, Pearlmutter, & Seidenberg, 1994; Trueswell, Tanenhaus, & Garnsey, 1994; Garnsey, Pearlmutter, Myers, & Lotocky, 1997; Traxler, Morris, & Seely, 2002; Reali & Christiansen, 2007; Gennari & MacDonald, 2008), or between sounds and words (Farmer, Christiansen, & Monaghan, 2006; Schmidtke, Conrad, & Jacobs, 2014). A similar picture obtains in the domain of language production (see e.g., Vigliocco & Hartsuiker, 2002, for a review).

Strong neuro-scientific support for the cognitive inseparability of syntactic and lexico-semantic processes cannot, however, rely on spatial overlap alone. It also requires (i) evidence for temporal overlap between the different processes recruiting a given language region; and (ii) causal evidence that the region in question is necessary for the different processes. Unfortunately, joint temporal, spatial and causal evidence cannot be obtained with fMRI. As discussed earlier, it requires methods such as electrocortical stimulation (Ojemann, Ojemann, Lettich & Berger, 1989). Still, the spatial overlap between the responses to individual word meanings and to syntactic complexity throughout the language system allows us to at least entertain the hypothesis that the very same brain regions (i) store our language knowledge, and (ii) support the combination of those knowledge representations to form new meanings (see Hasson, Chen & Honey, 2015, for a recent discussion of this idea as applied to neural computation in general; cf. proposals like that of Baggio & Hagoort, 2001, according to which different brain regions of the language system support storage vs. combinatorial processing).

4.3. Dissociations within the language system?

As we argued in the Introduction, uncovering the division of linguistic labor among the regions of the fronto-temporal language system is key to understanding the cognitive architecture of the language faculty. However, the most fundamental aspects of the language system’s architecture remain to be discovered. For example, how to divide the language system into constituent regions is still under debate: the division into eight regions based on the average topography of language activations adopted here (from Fedorenko et al., 2010) is only a suggestion (see also Mahowald & Fedorenko, in revision). In fact, it is not even clear whether division of the language system into regions is warranted. On the one hand, the different regions of the language system show broadly similar functional profiles as measured with fMRI: they all respond more to meaningful and structured language stimuli like phrases and sentences than to “degraded” stimuli like lists of words, Jabberwocky sentences or lists of nonwords (e.g., Fedorenko et al., 2010; Pallier et al., 2010; Bedny et al., 2011). As shown here, they also all show sensitivity to finer-grain syntactic manipulations. In addition, language regions exhibit synchronized low-frequency oscillations during rest (e.g., Cordes et al., 2000; Hampson, Peterson, Skudlarski, Gatenby, & Gore, 2002; Turken and Dronkers, 2011; Newman, Kenny, Saint-Aubin, & Klein, 2013; Yue, Zhang, Xu, Shu, & Li, 2013; Blank, Kanwisher & Fedorenko, 2014) and language comprehension (Blank et al., 2014). Finally, various functional properties of the language regions – such as, how large or lateralized they are – are strongly correlated across regions (Mahowald & Fedorenko, in revision). All these results suggest that language regions form a functionally integrated system and should be considered as such when thinking about the architecture of language processing (e.g., Fedorenko & Thompson-Schill, 2014).

On the other hand, this is not to say that no functional dissociations exist within the language system. Indeed, a number of prior studies have reported differences among some of the language regions (e.g., Thompson-Schill, D’Esposito, & Kan, 1999; Bedny, Caramazza, Grossman, Pascual-Leone, & Saxe, 2008; Snijders et al., 2008; Mesulam et al., 2015). As discussed above, we should also keep in mind the low temporal resolution of fMRI: it is possible that dissociations would be more apparent when examining the language system through a finer temporal lens. Nevertheless, if one is to argue that some region or regions of the language system are functionally distinct from the rest of it, region by condition interactions are critical, and differences in overall responsiveness to language may further need to be taken into account.

4.4. Conclusion

Our study provides evidence that sensitivity to syntactic complexity is widespread across the language system, contrary to many previous neuroimaging studies that reported only a few, localized foci of syntactic complexity effects. Investigations of syntactic processing therefore need to expand their scope to include the entire system of high-level language processing regions in order to fully understand how syntax is instantiated in the human brain. More generally, we recommend that neuroimaging studies of the language system follow two methodological considerations. First, analysis methods should allow for inter-individual variability in the exact anatomical location of functional regions. In this regard, functional localization of language regions individually in each participant is one promising method, showing increased sensitivity compared to traditional group analyses. Second, any hypothesized functional differences across regions of the language system should be tested by directly comparing effect sizes across regions (i.e., explicitly testing for a region-by-condition interaction), while taking into account more general differences in overall sensitivity to language. These considerations should guide us as we continue to accumulate evidence about the functional profiles of the regions of the language system; they will enable us to advance and evaluate specific hypotheses about the kinds of representations that such regions are likely to store and the computations that they are likely to perform.

Appendix

A. Sensitivity to syntactic complexity in the “core” language system and the extended language system

Table A.1.

Effects^a,^b for the localizer contrast^c (sentences > nonwords) and the critical contrast (object-extracted > subject-extracted) in the “core” language system

fROI	Localizer effect		Syntactic complexity effect
LIFGorb	t=5.79;	p<10−4	t=3.41;	p<0.01
LIFG	t=8.31;	p<10−4	t=4.66;	p<10−3
LMFG	t=6.72;	p<10−4	t=4.63;	p<10−3
LAntTemp	t=6.28;	p<10−4	t=2.34;	p<0.05
LMidAntTemp	t=7.52;	p<10−4	t=4.19;	p<10−3
LMidPostTemp	t=10.44;	p<10−4	t=5.43;	p<10−3
LPostTemp	t=9.17;	p<10−4	t=5.43;	p<10−3
LAngG	t=8.16;	p<10−4	t<1;	n.s.

^aWe report uncorrected p values (df=12), but all effects remain significant after an FDR correction for the number of regions (n=8).

^bSee also Figure 4.

^cEstimated in data not used for defining the fROIs, using across-runs cross-validation.

Table A.2.

Effects^a for the localizer contrast (sentences > nonwords)^b and for the critical contrast (object-extracted > subject-extracted) in the extended language system

fROI	Localizer effect	Syntactic complexity effect
Right hemisphere homologues of “core” language regions
RIFGorb	t=3.71; p<0.005	t<1; n.s.
RIFG	t=3.89; p<0.005	t=2.19; p<0.05
RMFG	t=2.05; p<0.05	t=1.23; n.s.
RAntTemp	t=5.37; p<10−4	t<1; n.s.
RMidAntTemp	t=3.75; p<0.005	t<1; n.s.
RMidPostTemp	t=5.67; p<10−4	t=2.39; p<0.05
RPostTemp	t=4.44; p<10−3	t<1; n.s.
RAngG	t=2.59; p<0.05	t<1; n.s.
Medial frontal cortex region
LSFG	t=5.51; p<10−4	t<1; n.s.
Cerebellar regions
RCereb	t=4.63; p<10−3	t=1.88; p<0.05
LCereb	t=6.28; p<10−4	t<1; n.s.

^aWe report uncorrected p values (df=12).

^bEstimated in data not used for defining the fROIs, using across-runs cross-validation.

B. The spatial pattern of syntactic complexity effects is better explained by language-specific responsiveness than by general, non-specific proneness to signal loss

In section 3.3.2 we report that the size of our syntactic complexity effect in a given language fROI is strongly predicted by the response magnitude to language in that fROI. Analyses of the relationship between these two effects was performed on contrast estimates that were averaged across voxels in each fROI. Yet this averaging might have obscured potential heterogeneity within these regions. It is therefore possible that, on a finer-grain spatial scale, one would not find an association between syntactic complexity effect sizes and overall language response. To test this possibility, we here explore the relationship between the two effects across individual voxels.

A correlation between the syntactic complexity effect size and overall language response across voxels would be compatible with two interpretations. One possibility is that the association is not language-specific: a strong correlation across voxels would be expected for any two effects, linguistic or non-linguistic, due to physiological artifacts. In particular, inter-regional differences in vascularization (e.g., Harrison, Harel, Panesar, & Mount, 2002; Ances et al., 2008; Ekstrom, 2010; Wilson, 2014) or proneness to signal loss (e.g., Jezzard & Clare, 1999; Menon & Kim, 1999) might explain why different contrasts co-vary across voxels (e.g., regardless of the particular contrast, effect sizes across voxels might scale with the voxels’ distance from air-tissue interfaces). An alternative interpretation, however, is that the association between the two effects is language-specific and would not generalize to non-linguistic effects.

To distinguish these possibilities, we ran a model predicting the size of the syntactic complexity effect (object-extracted > subject-extracted) across individual voxels using two predictors: a non-linguistic effect and a language-specific response (sentences > fixation in the localizer task). Our non-linguistic effect contrasted two versions of a spatial working memory task differing in difficulty (hard > easy). In this task, which our participants performed in the scanner for another study, participants have to keep track of four vs. eight locations within a 3 × 4 grid. This task has previously been shown to have reliable variability across cortical voxels (allowing, in particular, the functional localization of frontal and parietal regions of the “cognitive control” or “multiple demand” system; e.g., Fedorenko et al., 2013). According to the first interpretation above, predicting the size of the syntactic complexity effect from the size of the non-linguistic effect would not benefit from adding the language-specific response magnitude as a predictor (given that all contrasts should show strong correlations). However, according to the second interpretation, the size of the syntactic complexity effect would be predicted by the size of the language-specific response magnitude above and beyond the non-linguistic effect size.

A linear, mixed-effects regression model with random intercepts and slopes for both participant and fROI supported the second interpretation: the contribution of the language-specific response magnitude to the model was significant (β=0.16, t=5.22, χ²₍₁₎=15.7, p<10⁻⁴). In fact, when the language-specific response magnitude was included in the model, the non-linguistic effect size was not a significant predictor of the syntactic complexity effect size (β=−0.03, t=−0.31, χ²₍₁₎=0.09, n.s.).² Moreover, this lack of association between syntactic complexity and non-linguistic effects is not due to the restricted range of contrast values in our fROIs (where non-linguistic effects are very weak), as our results extend beyond those regions. Specifically, similar results were obtained when we ran the model on all individual voxels falling within the group-based masks instead of only including voxels falling within participant-specific fROIs (contribution of language-specific response magnitude: β=0.15, t=5.21, χ²₍₁₎=17, p<10⁻⁴; contribution of non-linguistic effect size: β=−0.01, t=−0.11, χ²₍₁₎=0.01, n.s.). Therefore, the correlation between the size of the syntactic complexity effect and the response magnitude to language is functionally specific, and does not generalize to non-linguistic contrasts. It is therefore unlikely that physiological artifacts, such as differences across voxels in proneness to signal loss, are the main factor underlying this correlation.

C. Language fROIs are sensitive not only to sentence-level syntax, but also to lexical information

We would like to stress that the results reported in Appendix B should not be taken as indication that our localizer contrast (sentences > nonword lists) was in fact just a localizer for syntactic processing. Our localizer targets regions involved in various aspects of high-level linguistic processing, including both semantic and syntactic processing at both the lexical and sentence levels, as previous work from our lab has shown (Fedorenko et al., 2010, 2012b). To more directly ensure that our localizer did not exclusively target sentence-level syntactic processing, we took advantage of our localizer design which included, besides sentences and nonword lists, a third condition – word lists that did not form sentences (and that were included here for the purposes of another study). Below, we briefly report three analyses targeting the contrasts between word-lists and control conditions (either nonword lists or fixation) as a measure of lexical processing unrelated to sentence-level syntax.

First, we measured the size of the words > nonwords effect in our fROIs (localized with the sentences > nonwords contrast). Replicating our prior work (Fedorenko et al., 2010) and using across-runs cross-validation as in the other analyses, we observed reliable responses in all fROIs except for LMFG (ts>1.92, ps<0.05; all regions except for LIFG remained significant after FDR correction for the number of regions). Second, we found that across individual voxels in all eight fROIs, the size of the sentences > fixation effect was predicted by the size of the words > fixation effect, above and beyond the prediction provided by the non-linguistic working memory effect (linear, mixed-effects regression with random intercepts and slopes for both participant and fROI: β=1.18, t=16.23, χ²₍₁₎=36.21, p<10⁻⁸). These findings demonstrate that our fROIs are sensitive not only to sentence-level syntactic information, but also to lexical information.

Third, we repeated our main analysis of the syntactic complexity effect (reported in section 3.3.1), but now defined fROIs using the words > nonwords contrast (instead of using the sentences > nonwords contrast). Despite the fact that this alternative localizer tends to produce weaker contrast effects compared to the localizer reported in the paper, we found a significant syntactic complexity effect in all regions except for LAntTemp and LAngG (ts > 2.83, ps < 0.03; all regions remained significant after FDR correction for the number of regions) (see Figure C.1).

The apparent lack of a syntactic complexity effect in the LAntTemp (cf. a significant effect reported in the main text, when this fROI was localized with the sentences > nonwords contrast) should be interpreted with care. First, the effect sizes of the words > nonwords localizer contrast are much weaker than those of the sentences > nonwords localizer contrast, throughout the entire language system. Thus, the words > nonwords localizer contrast has inferior localization capacities, and this weakness might account for the lack of a syntactic complexity effect in the LAntTemp. Second, we emphasize that a functional difference between the LAntTemp and the other language regions requires a region-by-condition interaction.

Alternatively, the lack of a syntactic complexity effect in a fROI localized with a lexical contrast might imply that the anterior temporal lobe contains a sub-region that is recruited for processing word-level information but not sentence-level syntactic information. Such a sub-region is perhaps separate from the sub-region that did show a syntactic complexity effect in the main text (for a similar suggestion, see: Pascual et al., 2015). This conjecture might explain why the role of anterior temporal lobe in language processing remains debated: on the one hand, it has been reported to engage in syntactic and semantic combinatorial processes above the word level, and only for linguistic stimuli but not for other meaningful stimuli (e.g., Humphries et al., 2001; Vandenberghe et al., 2002; Humphries et al., 2005; Rogalsky & Hickok, 2009; Baron & Osherson, 2011; Bemis & Pylkkanän, 2011; Brennan et al., 2012; Zhang & Pylkkänen, 2015); on the other hand, it has been identified as an amodal (and non-linguistic) semantic hub for simple concepts (Lambon-Ralph, Sage, Jones, & Mayberry, 2010; Mesulam et al., 2015; for reviews, see: Patterson, Nestor, & Rogers, 2007; Wong & Gallate, 2012; Jefferies, 2013).

Figure C.1. Replication of the main result using an alternative language localizer contrast (words > nonwords). Responses of the language fROIs are shown to the conditions of the alternative language localizer and the critical experiment (object-extracted > subject-extracted), using the same conventions as described in Figure 4. The results using the original localizer are reported in section 3.3.1.

D. Language fROIs are sensitive to syntactic manipulations during a passive reading task

In section 4.1.3, we discuss the possibility that the syntactic complexity effects we observed do not simply reflect the difference between object-extracted and subject-extracted sentences, but instead result from an interaction between sentence type and the sentence-picture matching task. Although we find this account of our results unlikely, we wanted to demonstrate that the same language regions exhibit sensitivity to syntactic processing during a passive reading task. For this purpose, we analyzed data reported in Fedorenko et al. (2010), from 12 participants (experiment 1) who passively read linguistic stimuli, including critically, Jabberwocky sentences (which preserve the word order, function words and functional morphology of real sentences but use nonwords) and lists of unconnected nonwords. Because only the stimuli in the former condition contain identifiable syntactic structure (e.g., due to the presence of function words), we interpret the Jabberwocky > nonwords contrast as a syntactic contrast. These data were analyzed with the same procedures described in the Methods section, and are presented in Figure D.1. Consistent with our main results, all language fROIs show a stronger response to Jabberwocky sentences than to nonwords (including the AngG, which shows the smallest effect size and a negative beta weight for nonwords).

We note that our critical contrast reported in the main text (object-extracted > subject-extracted) still provides stronger evidence for sensitivity to syntactic processing compared with the Jabberwocky > nonwords contrast, because the former (i) is “tighter” and based on a minimal pair of sentences that contain identical words and only differ in their word order (syntax); and (ii) uses sentences with real words, as opposed to the less natural nonwords and, therefore, has stronger ecological validity. Nevertheless, the Jabberwocky > nonwords contrast reported here contributes converging evidence in support of our main result. We believe that these data, measured during a passive-reading task, alleviate the concerns about our results reflecting an interaction between sentence-type and the sentence-picture matching task.

Figure D.1. Responses of the language fROIs to real sentences, Jabberwocky sentences and lists of nonwords. The sentences > nonwords contrast was used to localize the fROIs. The effect sizes for all three conditions, as well as the jabberwocky > nonwords contrast, were then evaluated in independent data, using across-runs cross-validation (see section 2.6). Asterisks denote the significance of the Jabberwocky > nonwords contrast. The same conventions described in Figure 4 are used.