Off‐line sentence processing: What is involved in answering a comprehension probe?

Abstract

The aim of this study was to better characterize the influence of the comprehension probe on syntax‐related activation patterns observed in fMRI studies of sentence comprehension. In this study, sentence comprehension was assessed by presenting a true/false statement after each sentence. To disassociate the sentence reading from the comprehension probe activation, a 6‐s delay was placed between these processing phases. Two factors were manipulated, one affected the sentence and the other affected the probe. The sentences were manipulated by varying their syntactic complexity; conjoined‐active and object‐relative sentences were examined. The comprehension probes asked whether one of the first two mentioned nouns in the preceding sentence performed the action of one of the two verbs. The probes were manipulated by varying the distance (number of intervening words) between the noun and verb within the sentence; there were three probe types: short distance, long distance, and false statements. The results, which focused on the processing taking place during the probe, showed that the distance manipulation resulted in significant differences in both behavioral and brain activation measures. This was particularly true of BA 44, which revealed an interaction between complexity and distance such that the complexity effect was all but eliminated for the long‐distance condition. Additionally, we replicated our previous finding of syntactic complexity effects during the probe phase. Finally, post hoc analysis revealed that participants used two distinct strategies during sentence reading; significant effects of strategy use on both behavioral and brain activation data were observed. Hum Brain Mapp 2009. © 2009 Wiley‐Liss, Inc.

INTRODUCTION

There have been a number of recent studies investigating the neural basis of sentence processing and syntactic analysis in particular [Ben‐Shachar et al., 2003; Bookheimer, 2002; Caplan, 2006; Caplan et al., 1998, 1999, 2001; Constable et al., 2004; Cooke et al., 2002; Fiebach et al., 2001, 2005; Friederici et al., 2000, 2003, 2006; Homae et al., 2002; Jobard et al., 2007; Just et al., 1996; Keller et al., 2001; Kuperberg et al., 2001, 2003; Meyer et al., 2000; Newman et al., 2003; Ni et al., 2000; Stowe et al., 1994, 1998]. However, few have focused on off‐line processing. On‐line sentence processing is viewed as the interpretive processes of “recognizing words and appreciating their meanings and syntactic features; constructing syntactic and prosodic representations; and assigning thematic roles, and other aspects of propositional and discourse‐level semantics” [Caplan and Waters, 1999, p. 78]. On the other hand, off‐line processing is thought of as the postinterpretive processes associated with the post hoc usage of extracted meanings to accomplish other tasks [e.g., responding to a comprehension probe; Caplan and Waters, 1999]. Although, typically, it is the sentence processing and not the probe phase that is of interest, the processing taking place during the probe phase plays a nontrivial role in the brain activation as well as the behavioral pattern of results. Given that previous studies have shown that factors such as differences in output modality (e.g., responding with a button press vs. subvocally) affect brain activation when the cognitive task itself is the same [Becker et al., 1999; Jennings et al., 1997], characterizing the effect of off‐line processes on brain activation is critical.

Although there are a number of studies focusing on syntactic analysis, these studies do not always agree. In fact, there are a number of discrepancies found across studies of syntactic processing. One possible explanation is differences in task design. For example, Cooke et al. [2001] examined the effect of syntactic complexity in which the task was to respond as to whether the agent was male or female. There they found an effect of complexity in the inferior frontal gyrus, but not in temporal cortex. Keller et al. [ 2001] examined syntactic complexity and lexical frequency using a sentence comprehension task in which the comprehension probe followed the sentence and found no syntactic effects for sentences containing high‐frequency words in either the inferior frontal gyrus or temporal cortex. In another study, using a paradigm similar to the Keller study, in which sentences were presented visually and auditorily, syntactic complexity effects were observed in both the inferior frontal gyrus and temporal cortex [Michael et al., 2001]. These methodological differences across studies make it difficult to synthesize the results into a comprehensive model of syntactic analysis.

In a recent study designed to determine the effects of ancillary cognitive processes, those related to task demands and not to sentence processing itself, on syntactic analysis researchers examined three separate tasks that were performed by the same group of participants [Caplan et al., 2008]. There, sentence verification, plausibility judgment, and nonword detection tasks were used with sentences that varied in their syntactic complexity, object‐extracted versus subject‐extracted sentences. Caplan et al. found a larger set of regions revealed syntactic complexity effects when the task was more demanding. For example, the verification and plausibility tasks elicited syntactic complexity effects in many more cortical regions than did the nonword detection task. This widespread syntactic effect for more demanding tasks does not necessarily reflect sentence or syntactic level processing per se but may reflect ancillary processes. These ancillary processes, therefore, may mask the true sentence processing cortical activity.

The primary aim of this study was to determine how these off‐line, ancillary processes interact with syntactic, fMRI measured brain activation patterns. To accomplish this aim, we attempted to examine the processing taking place during the off‐line, probe phase separate from that during the on‐line, sentence phase by using an experimental paradigm employed by Lee et al. [ 2008]. In previous fMRI studies, it has not been possible to examine these two phases separately because of the overlap in their hemodynamic responses. Here, syntactic complexity was manipulated by comparing conjoined‐active and object‐relative embedded sentences. A comprehension paradigm was employed in which each sentence was followed by a true or false comprehension probe. Finally, the distance between the sentence verb and its agent was manipulated in the probe creating a short‐ and long‐distance manipulation. To examine off‐line processing separately from on‐line processing, a 6‐s delay was interposed between the sentence and probe. The 6‐s delay period allowed for the partial separation of the two hemodynamic responses (see Methods).

METHODS

Participants

Twenty‐two participants took part in the experiment. They were all Indiana University students without any history of neurological disorders. Data from 18 participants (eight male, 10 female, age = 22.4 ± 0.44) were used for this data analysis; data from four participants were discarded because of either excess motion (>3 mm) or excess errors (>30% in any condition). All participants gave signed informed consent, which was approved by the Indiana University Institutional Review Board.

EXPERIMENTAL PROCEDURES

The study was composed of three sessions, training, imaging, and debriefing sessions. During the training session, all participants were administered the Edinburgh handedness inventory and the Daneman and Carpenter [ 1980] Reading Span Test to obtain a measure of working memory capacity. Participants were all right‐handed and their reading span scores ranged from 2 to 5 (Mean = 3.4 ± 0.22). During the training session, participants were also introduced to the sentence comprehension experiment and completed 16 practice trials to familiarize them with the experimental procedure. Immediately after the fMRI scan, participants completed a debriefing questionnaire. The questionnaire was designed to determine how the participant performed the task as well as how difficult they felt the task was.

This fMRI experiment used a single trial event‐related design in which each trial was treated as an event block [Kruggel and von Cramon, 1999; Zarahn, 2000; Zarahn et al., 1997; Fig. 1]. A trial could be divided into two phases; a sentence reading phase and a responding to a comprehension probe phase. Participants were instructed to read each sentence thoroughly and respond as quickly and accurately as possible to probes that were presented 6s later. Participants were told to place a greater weight on accuracy than speed of responding. Sentence materials were taken from Keller et al. [ 2001], which were derived originally from Just et al. [ 1996]. A 2 × 2 design was employed with syntactic complexity (conjoined‐active and object‐relative sentences) and distance (short vs. long) as within‐participant variables. The object‐relative sentences are syntactically more complex [Caplan and Waters, 1999; Just et al., 1996; Keller et al., 2001; Prat et al., 2007]. Stimuli were equated across conditions for frequency, word length, sentence length, and animacy.

Figure 1.

fMRI protocol.

Probes were constructed by asking if one of the nouns performed the act denoted by one of the two verbs. Thirty‐three percent of the probes were false and used as fillers. The distance manipulation was at the comprehension probe. Here, distance was defined by the number of intervening words between the noun (subject of verb) and verb within the sentence. Example, stimuli include:

Conjoined active: The pilot scared the escort and broke the mirror on the closet.

short: The pilot scared the escort. (1a)

long: The pilot broke the mirror. (1b)

Object‐relative: The pilot that the escort scared broke the mirror on the closet.

short: The escort scared the pilot. (2a)

long: The pilot broke the mirror. (2b)

As demonstrated, for the conjoined‐active stimuli for the short distance probe pilot and scared have no intervening words, whereas pilot and broke have four. The same metric is used for the object‐relative stimuli. However, the short‐distance condition is more computationally demanding than the long because of the relative clause.

The duration of each trial was 16 s. A trial began with a sentence being presented in the middle of the screen for 5 s (see Fig. 1). After 5 s, a delay to allow the hemodynamic response to approach baseline was presented for 6 s. By inserting the 6‐s delay, the on‐line sentence reading phase and off‐line comprehension phase could be distinguished. During the delay, an X was presented on the screen and the participants were instructed to fixate on it. Finally, a comprehension probe was presented for 5 s with a cue (i.e. F|T). The cue indicated the appropriate response (a right index finger for true and the left index finger for false). It is our policy to use both hands to respond to ensure that no laterality differences as a function of motor response will be observed. After each trial, a 12‐s ITI was added to let the hemodynamic responses go back to baseline. In the beginning of the fMRI session, two practice trials were included to remind participants how to perform the task.

There were 14 trials per condition, which were evenly and randomly presented across 4, 8‐min functional runs. In addition, each run contained 3, 28‐s fixation periods located at the beginning, middle, and end of each run. The baseline hemodynamic response was measured by averaging signals during the 28‐s fixation periods (fixation to a star sign, *). Stimuli were presented on the screen located behind the scanner and viewed by participants via a mirror attached on the head coil. Fiber optic button boxes in each hand were used to record behavioral responses. Two participants responded worse than the criteria of 30% of overall error rates, so, their data were totally excluded from the data analysis. To eliminate possible contamination from careless responses, incorrect responses from each subject were taken out for the response time analysis and were excluded from the fMRI data analysis.

fMRI Acquisition and Analysis

Functional MRI was conducted on a 3T Siemens TRIO scanner with an eight‐channel radio frequency head coil located in the Imaging Research Facility at Indiana University. Functional images were obtained in 18 oblique axial slices with 5 mm thickness and a 1 mm gap (TR = 1000 ms, TE = 25 ms, flip angle = 60°, matrix size = 64 × 64, FOV = 240 × 240 mm²) by a gradient echo planar imaging (EPI) sequence. Before the statistical analysis, for all the functional images, conventional preprocessing procedures such as slice timing correction, head motion correction by realignment and spatial normalization were conducted by using the SPM2 software (Wellcome Department of Imaging Neuroscience; http://www.fil.ion.ucl.ac.uk/spm). In the spatial normalization step, all functional images were warped directly to the Montreal Neurological Institute (MNI) EPI template and resampled to the 2 × 2 × 2 voxel dimensions, which were supported by the SPM package. A conventional statistical inference was performed on the normalized functional images from each individual by using the general linear model and Gaussian random field theory [Friston et al., 1995]. A canonical hemodynamic response function (HRF) with onset and duration for each phase was setup to generate a statistical parametric map (SPM). For example, a HRF function with 5‐s duration was constructed at the onset of sentence presentation for the on‐line sentence reading phase. An isolated HRF function with the same shape was built at the onset of the probe presentation for the off‐line comprehension phase.

The primary analysis in this article is the ROI time course analysis. A cortical language network in healthy subjects has been suggested by recent neuroimaging research [Gitelman et al., 2005; Papathanassiou et al., 2000]. Seven functional ROIs were specified to construct a language network based on previous data [Lee et al., 2008, see Fig. 3]. Three frontal ROIs in the left inferior frontal gyrus (namely, BA 45/47, BA 44, and BA 13–the anterior insula) and two temporal ROIs in the left anterior and posterior middle temporal gyrus (BA 21, BA 22). In addition, the left inferior parietal lobule (BA 40) and the supplementary motor area (SMA) were also examined. (see Figure 3 and Table I for a summary of the ROI information).

Figure 3.

Activation maps. The top figure depicts the probe minus sentence contrast for conjoined‐active sentences (left) and object‐relative sentences (right). The bottom figure depicts the resulting activation from the long minus short contrast for conjoined‐active sentences. There were no regions showing differential activation as a function of distance for the object‐relative sentences (for either the long minus short or the short minus long contrast).

Table I.

Description of functional ROIs

Anatomical reference	BA	MNI coordinate (x, y, z)
L. inferior frontal gyrus	45/47	−34, 28, −4
L. inferior frontal gyrus	44	−48, 16, 30
L. insula	13	−40, 16, −4
Supplementary motor area (SMA)	6	−6, 10, 58
L. ant. middle temporal gyrus	21	−58, −14, −10
L. post. middle temporal gyrus	22	−56, −44, 4
L. inferior parietal lobule	40	−40, −56, 54

The ROIs were obtained from a study conducted by Lee et al. [ 2008]. In that study, the same sentence stimuli (conjoined‐active and object‐relative sentences) were presented using the same experimental paradigm (sentences presented for 5 s, followed by a 6‐s delay and a probe for 5 s). The ROIs were determined by the sentence minus fixation and the probe minus fixation contrasts with FWE correction (P < 0.05).

The functional ROIs were defined as a cube with a size of 10 × 10 × 10 mm³ in x, y, and z directions with the center being defined by the previous data. Using the Marsbar toolbox [Brett et al., 2002], averaged time course data of all the voxels within a cubic ROI were extracted for each individual normalized imaging dataset and sorted by experimental conditions (e.g., conjoined‐active condition). The averaged time course signals across all trials were converted into percentage signal change (PSC) using the formula (signal–baseline/baseline) × 100 for each time point, where the baseline constant was the mean signal of the fixation periods. Then, the PSC time courses were baseline corrected to 0. For each individual subject, baseline corrected PSCs for the 5‐s on‐line phase (from 6 to 11 s, the first 6 s were taken out for the delayed hemodynamic response) and the 5‐s off‐line phase (from 18 to 22 s) were averaged to obtain a mean signal change for each processing phase. The averaged PSC value for each phase was considered as a representative activation level of each ROI for each subject. With these values, two‐way repeated measures ANOVA (syntactic complexity × distance × phase) was conducted to test the main effects and interactions between three factors for each ROI.

RESULTS

Behavioral Results

Behavioral measurements were taken into account for overall task performance. The syntactic complexity effect (conjoined‐active vs. object‐relative) was tested on the response time and error rate data using an ANOVA with complexity and distance as within‐participant variables. Response time showed a significant main effect of syntactic complexity and distance as well as a significant interaction [complexity: F(17) = 44.99, P < 0.0001; distance: F(17) = 6.56, P < 0.05; interaction: F(17) = 5.55, P < 0.05] (see Figure 2). Error rates also showed a significant effect of syntactic complexity [F(17) = 10.88, P < 0.005]; however, the effect of distance and the interaction failed to reach significance [distance: F < 1; interaction: F(17) = 3.4, P > 0.08]. Post hoc tests revealed that the conjoined‐active sentences revealed an effect of distance [error: F(17) = 3.05, P < 0.1; RT: F(17) = 13.42, P < 0.005], but the object‐relative sentences did not [error: F < 1; RT: F < 1].

Figure 2.

Behavioral data.

fMRI Results

Overall, the imaging results presented here, like the behavioral results, demonstrate the effect of distance was primarily observed for conjoined‐active sentences (see Tables II and III and Fig. 4). This effect was observed in a region of the inferior frontal gyrus, BA 44, in the inferior parietal cortex, and in the posterior temporal cortex. Additionally, there were a number of regions that appeared to be significantly more involved during the probe compared with the sentence phase; those regions included the inferior parietal cortex, the insula, middle frontal gyrus, and the anterior temporal cortex.

Table II.

Brain activation clusters

Anatomical region		BA	Cluster size	z‐score	Coordinates
					x	y	z
Probe minus sentence—conjoined active
Left	Superior temporal gyrus	22	645	6.07	−44	−2	−4
Left	Insula	13		4.5	−40	12	4
Right	Superior temporal gyrus	22	261	5.68	46	−2	0
Right	Insula	13		4.77	46	10	−4
Left	Inferior parietal lobule	40	733	5.26	−52	−32	56
Left	Postcentral gyrus	2		5.02	−50	−30	34
Left	Inferior parietal lobule	40		4.95	−58	−30	32
Right	Inferior parietal lobule	40	174	4.93	56	−26	30
Left	Precentral gyrus	6	31	4.88	−20	−18	62
Probe minus sentence—Object‐relative
Left	Insula	13	19	7.42	−44	14	4
Left	Inferior frontal gyrus	47	33	7.65	−36	20	−4
Long minus short—conjoined active
Left	Superior temporal gyrus	22	5419	6.97	−32	−52	18
Left	Lingual gyrus	19			−32	−54	4
Left	Inferior frontal gyrus	47	499	5.93	−34	24	−10
Left	Inferior frontal gyrus	45			−38	24	4
Left	Insula	13			−44	6	14
Left	Middle temporal gyrus	21	33	5.56	−56	−8	−10
Left	Postcentral gyrus	2	38	5.42	−42	−26	28
Left	Cuneus	18	30	5.38	−22	−96	12
Left	Superior frontal gyrus	6	34	5.2	−14	16	46
Right	Insula	13	250	5.19	32	16	−6
Right	Inferior frontal gyrus	47			30	30	−6
Right	Inferior frontal gyrus	47	10	5.17	34	28	−16
Right	Cuneus	19	34	5.15	16	−80	34
Left	Inferior temporal gyrus	37	18	5.11	−56	−54	−2
Right	Subgyral	32	27	5.09	14	22	40
Left	Cuneus	18	11	4.93	−16	−78	24

Table III.

F‐values obtained from the 2 (complexity) × 2 (distance) × 2 (phase) ANOVA

ROI	Complexity	Distance	Phase	Complexity* distance	Complexity* phase	Distance* phase	Three‐way interaction
BA45/47	2.86	1.31	3.19	<1	2.7	6.49*	<1
BA 44	4.95*	3.72	2.4	9.09**	5.98	6.59*	4.48*
BA 13	<1	3.51	18.64**	<1	<1	2.11	<1
SMA	<1	4.07	13.54**	<1	1.35	3.38	1.68
BA 21	1	3.08	5.15*	<1	<1	3.5	4.77*
BA 22	5.87*	1.81	<1	<1	6.79*	7.18*	1.04
BA 40	2.4	3.98	18.32**	2.85	<1	2.03	<1

^* P < 0.05;

^** P < 0.01.

Figure 4.

Signal change and ROIs. Error bars denote standard error across participants. CA, conjoined active; OR, object‐relative.

Frontal regions

Two ANOVAs were performed on the time course data. One was a 2 (complexity) × 2 (distance) × 2 (processing phase) ANOVA and the other a 2 (complexity) × 2 (distance) ANOVA on just the probe signal change data. There were three regions within the inferior frontal gyrus (i.e., BA 44, BA 45/47, and the anterior insula) examined. These regions show very distinct patterns of activation. BA 44 has been previously associated with syntactic analysis [Fiebach et al., 2005; Friederici et al., 2003; Newman et al., 2003] and is the only frontal region that revealed a significant main effect of syntactic complexity. The region also revealed a significant effect of distance during the processing of the probe (see Table IV and Fig. 5). The region additionally revealed an interaction between complexity and distance because of a larger distance effect for the conjoined‐active compared with the object‐relative condition. BA 44 also revealed a distance by phase and a three‐way interaction between complexity, distance, and phase. These were due to there being a distance effect during the probe phase but not the sentence phase. However, there was no main effect of processing phase, suggesting that the region was significantly involved during both sentence reading and responding to the comprehension probe.

Table IV.

F‐values obtained from the two‐way repeated measures ANOVA (syntactic complexity × distance) only during the comprehension probe phase

ROI	Complexity	Distance	Interaction
BA45/47	3.84	3.0	<1
BA 44	9.02**	5.7*	8.78**
BA 13	<1	3.6	<1
SMA	<1	5.18*	1.55
BA 21	1.16	4.69*	1.21
BA 22	8.97**	4.16	1.58
BA 40	2.58	4.56*	2.58

^* P < 0.05;

^** P < 0.01.

Figure 5.

Syntactic effect in BA 44.

BA 45/47 is another subregion of IFG that has been previously linked to semantic processing [Fiez, 1997; Wagner et al., 2000]. The region revealed an interaction between distance and processing phase (see Fig. 3). This interaction is due to there being no effect of distance during the sentence processing phase but a higher signal change for the long‐distance condition during the probe phase. The region revealed no main effects of complexity, distance, or phase.

The left BA 13, anterior insula, has been previously associated with verbal working memory processes [Awh et al., 1996; Clark et al., 2000; Newman et al., 2001; Smith and Jonides, 1999] and revealed a significant main effect of phase. As shown in Figure 4, the region revealed little activation during the sentence reading portion of the trial compared with the probe phase. The region failed to reveal a main effect of complexity or distance or any interactions.

The SMA revealed a significant effect of phase such that the off‐line phase elicited a higher signal change than did the on‐line phase. In addition, when examining the probe phase alone, a significant effect of distance was observed. This region revealed a larger effect for the long‐distance object‐relative condition than the short‐distance condition.

Temporal regions

Two ROIs within the temporal lobe, posterior temporal cortex (BA 21), and anterior temporal cortex (BA 22) were examined. The posterior temporal cortex has long been implicated in language processing and overlaps with a classical language processing region, Wernicke's area [Cooke et al., 2001; Just et al., 1996]. In this study, this is the only temporal region that revealed a main effect of syntactic complexity. In addition, the region revealed an interaction between complexity and processing phase (because of a larger syntactic effect during the probe phase compared with the sentence reading phase) and an interaction between distance and processing phase (because of a larger signal change for the long‐distance condition during the probe compared with sentence reading phase). The region revealed no main effects of distance (although for the probe phase, the effect was marginally significant) or processing phase.

The anterior temporal cortex, BA 21, revealed a main effect of processing phase because of a larger effect during the probe compared with the sentence reading phase. Additionally, there was a significant three‐way interaction between complexity, distance, and phase. This interaction may be due to a significant effect of distance observed during the probe phase. Unlike other regions, this effect was observed for the object‐relative sentences, not the conjoined‐active sentences.

Inferior parietal cortex

The inferior parietal cortex, BA 40, has been linked to language‐related processing as well as verbal working memory [Awh et al., 1996; Clark et al., 2000; Newman et al., 2001; Smith and Jonides, 1999]. In this study, we found a significant effect of processing phase in which the probe phase elicited a larger signal change than the sentence reading phase, suggesting its increased involvement in off‐line, rather than on‐line sentence processing. In addition, the region revealed an effect of distance during the probe phase. The region failed to reveal an effect of complexity or any interaction with complexity.

Effects of Strategy

After examining the debriefing questionnaire, we noticed that individuals adopted one of two strategies. The first strategy was to read the entire sentence during sentence presentation and then to use that information during the probe phase (10 participants–whole strategy). The second strategy was a little different. Here, participants reported attempting to remember just key words, nouns, and verbs for example, and the order that they were presented (eight participants–word strategy). We are not suggesting that this group of participants did not read the sentences. Instead, we are suggesting that the way in which they read the sentences may have altered the processing and, therefore, would have affected both the behavioral and activation patterns. To examine the effect of strategy, a between‐participants ANOVA was performed with complexity and strategy as factors. Behaviorally, differences in strategy had a significant effect on both error rate and reaction time [F(1,17) = 4.18, P < 0.05; F(1,17) = 3.93, P = 0.052, respectively—see Fig. 6].

Figure 6.

Strategy effect, behavioral.

Strategy also had significant effects on fMRI brain activation levels (see Fig. 7 and Table V). Here, we focus on three of the language processing regions, BA 44, BA 45/47, and posterior temporal cortex, BA 22. All three regions revealed a main effect of strategy [F(1,17) = 84.38, P < 0.0001; F(1,17) = 5.05, P < 0.05; F(1,17) = 7.53, P < 0.01, respectively] as well as an interaction between strategy and complexity [F(1,17) = 5.98, P < 0.05; F(1,17) = 13.83, P < 0.001; F(1,17) = 8.56, P < 0.01, respectively].

Figure 7.

Strategy effect, activation. CA, conjoined active; OR, object‐relative.

Table V.

Strategy × complexity for the on‐line and off‐line processing

ROI	Sentence			Probe
	Complexity	Strategy	Interaction	Complexity	Strategy	Interaction
BA45/47	5.84*	16.67**	8.41**	7.98**	<1	6.94**
BA 44	<1	108.42**	1.44	10.55**	16.02**	4.94*
BA 13	<1	2.49	<1	2.5	14.77**	4.46*
SMA	<1	7.54**	1.69	<1	<1	1.14
BA 21	2.3	3.05	<1	1.93	4.05*	<1
BA 22	3.24	15.14**	1.87	14.32**	<1	7.22*
BA 40	1.76	13.83	<1	5.06*	9.72**	<1

^* P < 0.05;

^** P < 0.01.

The focus of these analyses was the comprehension probe. To summarize the main findings, here, we found main effects of both complexity and distance during the probe. Interestingly, these effects were observed in classically defined language processing regions (i.e., Broca's and Wernicke's areas). Broca's area (BA 44) revealed a main effect of complexity as well as a main effect of distance. Wernicke's area (posterior temporal cortex) revealed only a main effect of distance and no effect of complexity during the probe. In addition to these main effects, BA 44 also revealed an interaction between complexity and distance, such that the complexity effect was significantly reduced for the long‐distance condition.

DISCUSSION

This fMRI study attempted to better characterize the interaction of ancillary processes related to responding to a comprehension probe and syntactic processes associated with sentence reading. This is an important issue in that these off‐line processes may interact with on‐line sentence processes or even blur our picture of the neural network responsible for sentence comprehension. A number of significant findings are reported and discussed later that may provide greater insight into the neural bases of syntactic processing as well as the interaction between syntactic processing and ancillary cognitive processes related to task performance, such as working memory. The fMRI measured activation effects correlate well with the behavioral effects. First, like in the behavioral data, we found effects of distance primarily for the conjoined‐active sentences. These effects were found in BA 44, BA 40, and marginal effects were also observed in BA 22. Second, two regions revealed off‐line syntactic complexity effects, BA 44 and BA 22, these two regions overlap with Broca's and Wernicke's regions, respectively. Third, we found on‐line/off‐line processing differences such that a pair of regions (BA 13 and BA 40), that has been associated with the working memory system [Awh et al., 1996; Clark et al., 2000; Newman et al., 2001; Smith and Jonides, 1999], revealed increased involvement during the off‐line compared with the on‐line processing phase. This suggests that these regions may be more associated with ancillary processes than with sentence comprehension itself. Finally, post hoc analysis revealed strategy differences in both the behavioral and imaging data.

Distance Effects (Long vs. Short)

The distance manipulation has been used previously in a number of studies of on‐line sentence processing. Typically, the distance is manipulated within the sentence and effects of sentence processing are examined [Cooke et al., 2001]. For example, in a study designed to examine working memory processes during sentence comprehension, the antecedent gap (distance between “who” and reference) was manipulated [Cooke et al., 2001]. This study manipulated distance at the level of the probe, not within the sentence, to examine its effect on neural processing. Here, we found that the distance between the verb and its agent significantly affected processing at the probe in four regions. Three of those regions, BA 44, BA 40, and BA 21, revealed effects only for the conjoined‐active sentences, whereas SMA revealed the effect for the object‐relative sentences. In all cases, the long‐distance condition elicited a larger response than the short‐distance condition.

As mentioned earlier, there are differences between the long/short distance manipulation for the conjoined‐active and object‐relative constructions. For conjoined‐active sentences, the manipulation is straightforward; in the long‐distance condition, the verb referenced in the probe simply has more intervening words between it and its agent in the sentence, whereas the short‐distance condition has fewer intervening words. The object‐relative condition is more complicated. Although the difference between long and short distance remains the same, in the short distance condition the agent, verb, and patient are in a noncanonical order but for the long‐distance condition they are in a canonical order. The noncanonical order of the object‐relative, short‐distance condition makes it computationally more demanding than the conjoined‐active short‐distance condition. This difference is observed in both the behavioral and brain activation data, where we found that for the conjoined‐active sentences there is a consistent effect of distance but it is all but eliminated for the object‐relative sentences. In fact, there are more errors for the short‐distance than the long‐distance condition. Therefore, reduced distance effects for object‐relative sentences are not surprising.

What are these distance effects telling us about the underlying processes? During sentence reading, participants decode the syntactic structure to build a representation of the sentence that is used to determine its meaning. When met with the probe they then interrogate that representation to determine the probe's validity. If participants are encoding the semantic relationships between the sentence constituents (i.e., extracting meaning) then the distance between the verb and its agent should have no effect. In other words, asking in 1a and 1b to determine whether the pilot broke the mirror or scared the escort should be equally taxing. Again, if what is being encoded is that the pilot performed both actions why would there be differences if asked about either action? Maybe a more superficial representation, like the constituents of the sentence and their order, is generated. In other words, maybe a more syntactic‐based representation is generated than a semantic representation in this context. If this is the case then accessing information that is represented closer together in memory may be less demanding than accessing information that is farther apart. There is some previous evidence to support such a view [Caplan et al., 2008; Lee et al., 2008]. In the Lee et al. study, syntactic complexity effects were observed during the processing of the probe in BA 44, BA 40, and BA 22. The explanation for the syntactic effect during the probe was that the representation generated during sentence reading is not just meaning‐based but maintains syntactic information, suggesting that the effect may be expected off‐line because of the possible syntactic reanalysis necessary to appropriately respond to the probe. The additional effect of distance in these regions strengthens this hypothesis because it demonstrates the increase in processing demands necessary to access information that was presented farther apart, which implies that it is represented farther apart, in a syntax‐based structure. An alternative explanation for the distance effect observed for the conjoined‐active sentences is that the probe for the short‐distance condition is visually more similar to the sentence than the long‐distance condition. Therefore, participants can visually match the probe with the sentence for the short‐distance probes. This visual matching may be expected to facilitate processing resulting in faster responding and reduced activation. Although these two hypotheses are different, they both suggest that a more superficial representation may be generated and used to respond to the comprehension probe.

One of the aims of this study was to examine the possible effect the probe has on the resulting language‐related activation pattern observed during studies of syntactic analysis. An intriguing result was found in BA 44—an interaction between syntactic complexity and distance when the signal change measure was collapsed across the on‐line and off‐line phases, as well as when examining the probe alone. This interaction was due to there being a significant complexity effect for the short‐distance but not the long‐distance condition. Additionally, the effect appears to be a function of a differential response to distance for the conjoined‐active sentences, not the object‐relative sentences (see Fig. 4). This interaction in BA 44 implies that it is associated with the construction and interrogation of the syntax‐based representation that is generated during sentence processing. This result also has implications for experimental designs and may explain some discrepancies observed in the literature. For example, most imaging studies do not separate sentence processing from probe processing, and therefore, the activation for the two is observed together. Here, we see that if using conjoined‐active sentences with short‐distance probes, the activation observed in those studies would be expected to be less than if the probes were long‐distance probes. This difference may affect the detection of syntactic complexity differences when comparing conjoined‐active with object‐relative constructions. As such, these findings suggest that attention be given to the type of probe used.

The posterior MTG has been linked to memory retrieval processing for lexical‐semantic knowledge [Friederici et al., 2003; Keller et al., 2001] and thematic role knowledge [Chatterjee et al., 1995; Kable et al., 2002; Wu et al., 2007]. Although the effect of distance in this study during the probe was marginally significant, it may be interesting to consider its contribution. In this study, there are both lexical‐level and sentence‐level semantic processing requirements and the region may be involved in both. According to Lee et al, the greater syntactic complexity effect during off‐line is due to the probe directly questioning the thematic relationships in the sentence. Determining those thematic relationships turns out to be a more demanding process in object‐relative compared with conjoined‐active sentences and may also be affected by the number of intervening words between the verb and the agent.

Off‐Line Sentence Comprehension Processing (Off‐Line Minus On‐Line)

On‐line sentence processing has been considered as immediate and automatic processing, whereas off‐line processing is not automatic but task‐dependent and is reliant upon working memory [Caplan and Waters, 1999]. When examining the regions that elicited a greater response to off‐line compared with on‐line processing, a number of regions were observed including the anterior insula, SMA, and the inferior parietal cortex. Interestingly, these regions have been linked to working memory systems [Awh et al., 1996; Clark et al., 2000; Newman et al., 2001; Smith and Jonides, 1999; Wager and Smith, 2003].

There are a number of studies examining the neural bases of verbal working memory. These studies have provided a fairly consistent account of the cortical regions that make up the verbal working memory network: the inferior frontal gyrus and the inferior parietal cortex [Awh et al., 1996; Carpenter et al., 2000; Curtis and D'Esposito, 2003; D'Esposito et al., 1999; Jonides et al., 1998; Postle et al., 1999; Smith and Jonides, 1999]. The role of each of these regions has also been fairly consistently outlined. The maintenance of verbal information has been thought to involve the left inferior frontal gyrus and the left inferior parietal lobe. These two regions are thought to make up Baddeley's phonological loop with the left IPL being the buffer and IFG the phonological rehearsal component of the WM system [Ardila, 1999; Awh et al., 1996; Fiez et al., 1996; Newman et al., 2001; Smith and Jonides, 1998]. A set of neuroimaging studies, for example, that examined verbal storage and rehearsal compared an item recognition task, an n‐back working memory task, and two dual‐tasks (one with a memory task and repetition and one with a memory task and finger tapping) [Awh et al., 1996]. The prefrontal activation, concentrated in IFG, was activated to a greater extent for rehearsal, and the posterior inferior parietal region was activated to a greater extent for storage. In this study, the working memory‐related activation within the IFG was concentrated in the inferior, posterior portion of the IFG that lies in the anterior insula, and these regions, and presumably working memory processes, were more involved during the probe phase.

Effects of Strategy

When examining the debriefing questionnaires, it was found that 44% of our participants reported using a very specific strategy. As mentioned earlier, this group of participants reported that instead of reading the sentence during the on‐line phase they attempted to focus on the nouns and verbs and their order (the word strategy). Again, what seems to be important here is how the information is read, which may be determined by the purpose for which it is being read. In this case, the purpose is to accurately answer who did what to whom. The sentences have a pattern that can help in answering that question. For example, in the object‐relative sentence “The pilot that the escort scared broke the mirror on the closet” because the questions all concern pilot, escort, scared, and broke one strategy that can be developed is to keep track of the order of these words. Then, for object‐relative sentences, the first noun (pilot) is the agent of the second verb (broke) and the second noun (escort) is the agent of the first verb (scared). If you use that strategy, are you still reading? Yes. But how you are reading is different than if you were to simply read the sentence for understanding. It is this that separates the two groups of participants, how they are reading the sentences.

The purpose of examining these individual differences in strategy is to begin to think about how the way in which sentences are read may influence the resulting brain activation pattern observed. We all read text differently depending on the reason we are reading it. Although the sample size is rather small to make group comparisons, it is interesting to examine how strategy differences can potentially influence results. When comparing the behavioral performance of this group with the remaining participants (who did not report using such a strategy) significant differences were observed. The use of the word strategy resulted in faster reaction times and fewer errors, suggesting that the word strategy was more efficient. Additionally, strategy differences affected the activation levels during both the on‐line and off‐line processing phases.

On‐line strategy effects may be expected given that the strategy itself is to alter the processing of the sentence. Here, we see greater activation in BA 44, BA 45/47, and BA 22 during sentence reading when using the word strategy. One of these regions, BA 44, is linked to syntactic processing. This difference in BA 44 is intriguing in that it suggests a possible narrower explanation of its function. These results imply that the region is extremely sensitive to attending to word order.

Off‐line strategy effects were also observed. Like during the on‐line phase, the word strategy typically elicited greater activation. However, during this phase strategy interacted with syntactic complexity in such a way that the word strategy elicited a greater complexity effect than the whole sentence strategy in each of these three key processing regions. This result also supports our hypothesis that the on‐line syntactic complexity effect may be the result of participants generating a superficial, more syntactic sentence representation. These results would be predicted if the whole sentence strategy group generated a more semantic representation while the word group generated a more syntactic representation.

The effect of strategy use on sentence processing is an extremely understudied area. Here, we found significant effects of strategy on both behavioral and imaging data. The fact that there are still complexity effects is intriguing, actually. Because it shows that even though they are reading differently, the keeping track of order and still having to label what is the agent and patient of the verb is still a syntactic process that must be performed. Although it was not the focus of this study, the results presented here reveal that strategy is an important factor that should be taken into consideration.

CONCLUSIONS

The primary aim of this study was to explore the relationship between the comprehension probe and the resulting syntactic activation pattern. Here, we provide evidence that a manipulation of the probe resulted in significant differences in both behavioral measures as well as in levels of brain activation in a number of language processing regions. This was particularly true of BA 44, which revealed an interaction between complexity and distance such that the complexity effect was all but eliminated for the long‐distance condition. Additionally, we replicated our previous finding of syntactic effects during the off‐line phase [Lee et al., 2008]. Finally, post hoc analysis revealed significant effects of strategy use. Although this is an area of sentence processing in particular, but language processing more generally, that is understudied, it is one that deserves more attention. As it relates to this study, it may be the case that the way in which sentences are read is a function of the task, or type of probe used. Given that the strategy employed determines the cognitive processes that are implemented, it is important to control for strategy use or at least be aware that it is a significant factor.