Abstract
To make sense of a sentence, we must compute morphosyntactic and semantic-thematic relationships between its verbs and arguments and evaluate the resulting propositional meaning against any preceding context and our real-world knowledge. Recent electrophysiological studies suggest that, in comparison with non-violated verbs (e.g. “…at breakfast the boys would eat…”), animacy semantic-thematically violated verbs (e.g. “..at breakfast the eggs would eat…”) and morphosyntactically violated verbs (e.g. “…at breakfast the boys would eats…”) evoke a similar neural response. This response is distinct from that evoked by verbs that only violate real-world knowledge (e.g. “…at breakfast the boys would plant…”). Here we used fMRI to examine the neuroanatomical regions engaged in response to these three violations. Real-world violations, relative to other sentence types, led to increased activity within the left anterior inferior frontal cortex, reflecting participants’ increased and prolonged efforts to retrieve semantic knowledge about the likelihood of events occurring in the real world. In contrast, animacy semantic-thematic violations of the actions depicted by the central verbs engaged a frontal/inferior parietal/basal ganglia network known to mediate the execution and comprehension of goal-directed action. We suggest that the recruitment of this network reflected a semantic-thematic combinatorial process that involved an attempt to determine whether the actions described by the verbs could be executed by their NP Agents. Intriguingly, this network was also activated to morphosyntactic violations between the verbs and their subject NP arguments. Our findings support the pattern of electrophysiological findings in suggesting (a) that a clear division within the semantic system plays out during sentence comprehension, and (b) that semantic-thematic and syntactic violations of verbs within simple active sentences are treated similarly by the brain.
Introduction
In order to make sense of a sentence, we must integrate multiple different types of relationships. These include the morphosyntactic and semantic-thematic relationships between its verbs and arguments, and the relationship between the resulting propositional representation of meaning, any preceding context, and the associations between words and events that have been encountered before and that are stored within long-term semantic memory (our real-world knowledge). This study aimed to examine the neural networks that are engaged in processing and evaluating these three different types of relationships – syntactic, semantic-thematic and real-world knowledge – by examining how the brain responds when each are violated. A large behavioral and electrophysiological literature suggests that the introduction of such violations may tax the underlying neurocognitive processes that are engaged in computing and evaluating these types of relationships (Marslen-Wilson, Brown et al. 1988; Osterhout and Holcomb 1995). This approach has led to claims that aspects of semantic and syntactic processing are instantiated by distinct neurocognitive networks: violations of real-world semantic knowledge and syntactic rules each lead to distinct temporal and spatial patterns of neural activity, as indexed using event-related potentials (ERPs) and functional magnetic resonance imaging (fMRI) techniques (Osterhout and Holcomb 1995; Kuperberg 2007; Osterhout, Kim et al. 2007).
Violations of real-world knowledge stored within semantic memory
In ERP studies, words that are incongruous with their preceding context, and/or with real-world knowledge stored within semantic memory, lead to the production of a distinct negative-going waveform, peaking at approximately 400 msec – the N400 (Kutas and Federmeier 2000). An increased N400 is evoked by violations of semantic associative (Bentin, McCarthy et al. 1985; Rugg 1985) and categorical (Deacon, Grose-Fifer et al. 2004; Grose-Fifer and Deacon 2004) relationships in word-pair semantic priming paradigms, as well as by semantic anomalies in sentence paradigms (Kutas and Hillyard 1980; Kutas and Hillyard 1984). An increased N400 is also evoked by words that are either unlikely, e.g. by “plant” in sentences describing an unlikely event such as “Every morning at breakfast the boys would plant …” (Kuperberg, Sitnikova et al. 2003), or untrue, e.g. by “white” in sentences such as “The Dutch trains are white….”, presented to Dutch comprehenders who know that Dutch trains are, in fact, yellow (Hagoort, Hald et al. 2004).
Using fMRI, several studies have demonstrated that a left inferior frontal and temporal network is modulated in numerous paradigms examining the structure and function of semantic memory. The left inferior frontal gyrus (Brodmann Areas, BA 45 and 47) is thought to play a role in the explicit, controlled retrieval (Wagner, Pare-Blagoev et al. 2001) and/or the controlled selection (Thompson-Schill, D’Esposito et al. 1997; Thompson-Schill, D’Esposito et al. 1999) of the meaning of individual words. It is also activated when semantic violations are introduced within sentences (Ni, Constable et al. 2000; Newman, Pancheva et al. 2001; Kiehl, Laurens et al. 2002; Kuperberg, Holcomb et al. 2003; Hagoort, Hald et al. 2004), including violations of what is likely to occur in the world (Kuperberg, Sitnikova et al. 2003) and violations of what we know to be true (Hagoort, Hald et al. 2004), suggesting that it may also mediate the retrieval of real-world semantic knowledge.
In many of these paradigms, although not all (e.g. Hagoort et al., 2004), the left inferior frontal cortex is co-activated with temporal cortices (Van Petten and Luka 2006). The left middle temporal cortex may function together with the left inferior frontal cortex in explicitly retrieving semantic information (Wagner, Pare-Blagoev et al. 2001), while activity within more inferior temporal cortices may reflect a more implicit and automatic activation of stored associative semantic knowledge (Alexander, Hiltbrunner et al. 1989; Chao, Haxby et al. 1999; Martin and Chao 2001; Bar 2004; Wheatley, Weisberg et al. 2005; Gold, Balota et al. 2006, Kuperberg, Lakshmanan et al. 2007).
Violations of syntax
In ERP studies, syntactic violations or ambiguities within sentences produce several different electrophysiological responses that are distinct from the N400 (reviewed by A. Friederici & Weissenbor, 2007). One such waveform is a positive-going deflection that peaks at approximately 600 msec after the onset of a syntactic violation (or ambiguity), e.g. “The spoiled child throw the toys on the floor.” (Hagoort, Brown et al. 1993) and that has been termed the P600 (Osterhout and Holcomb 1992). The P600 effect evoked by syntactic anomalies or ambiguities is thought to reflect a late cost of syntactic processing, although it is debated whether this constitutes a syntactic or thematic reanalysis after a first stage of structural syntactic assignment (Friederici and Weissenbor 2007) or a prolonged attempt at integrating a word into the preceding thematic structure of the context (Kaan, Harris et al. 2000). Given that their polarity and time courses are distinct, what is undisputed is that the P600 reflects neurocognitive processes that are distinct from those reflected by the N400.
There have been several fMRI studies examining the effects of introducing different types of syntactic violations within sentences (Kuperberg, McGuire et al. 2000; Ni, Constable et al. 2000; Moro, Tettamanti et al. 2001; Newman, Pancheva et al. 2001; Friederici, Ruschemeyer et al. 2003; Kuperberg, Holcomb et al. 2003; Ruschemeyer, Fiebach et al. 2005). Of most relevance to the current study are two studies that introduced violations of morphosyntactic infection between a verb and its arguments. Ni et al. (2000, Experiment 2) reported that, in comparison to non-violated sentences, agreement violations such as “Trees can grew” elicited increased activity within bilateral inferior frontal gyri, pericentral cortices, right middle (BA46) and superior frontal gyri (BA 8&9), as well as within the right inferior parietal lobule. Kuperberg et al. (2003a) found that sentences with violations of inflectional morphosyntax such as “Every morning at breakfast the boys would eats toast and jam”, relative to both non-violated sentences as well as violations of what is likely to occur in the real world, led to robust increases in activity within bilateral parietal cortices (BA 40 and 7). At a less stringent significance threshold, these sentences were also associated with increased activity within posterior inferior frontal (BA 44) and adjacent premotor regions (BA 6), as well as within the left posterior occipito-temporal cortex. Intriguingly, much of this network has been implicated in executing and comprehending the meaning of goal-directed action (Buccino, Binkofski et al. 2001; Rizzolatti, Fogassi et al. 2001). Thus, one possibility is that encountering these morphosyntactic violations on verbs lead to a semantic-thematic reanalysis or integration process (Kaan, Harris et al. 2000; Kuperberg, Holcomb et al. 2003) that involves a computation and evaluation of the meaning of the action depicted by the verb in specific relation to the meaning of its arguments.
Violations of semantic-thematic relationships between verbs and arguments
Semantic-thematic relationships between nouns and their arguments lie at the interface between semantic memory and syntactic structure. Despite their semantic nature, they have direct implications for how the combination of words within a sentence plays out in terms of syntactic structure (Chomsky 1981; Jackendoff 1987; Levin 1993; Levin and Rappaport Hovav 2005). They describe those aspects of meaning of a verb and its arguments (a subset of all their meanings) that have a direct bearing on the number and types of morphosyntactic arguments assigned by a verb (Pinker 1989; Levin and Rappaport Hovav 2005). They encompass those aspects of meaning of a NP argument that are particularly relevant to the action described by the verb (Jackendoff 1978; Jackendoff 1987). So, for example, consider the semantic-thematic violation in the sentence, “…at breakfast the eggs would eat…”. This sentence is violated because “eggs” is inanimate in nature and, as such, does not have the semantic properties to be able to perform the action of eating. The sentence would also be anomalous if its subject was “computers” but not if it was “boys” or “dogs”. The same would be true of other verbs such as “kiss” or “kick” that can only assign the role of Agent (performer of the action) to an animate subject NP for a sentence to make sense. Of note, such semantic-thematic relationships between an animate Agent and a verb can be used to describe the event structure not only of verbs describing bodily action (e.g. “eat”), visual motion (e.g. “run”), or verbs of contact (e.g. “hit”), but also of other generalizable semantic classes of verbs, such as verbs of transfer (e.g. “donate”, “invest”).
Interestingly, in ERP studies, animacy violations of semantic-thematic relationships between verbs and arguments (e.g. “…at breakfast the eggs would eat…”) sometimes fail to evoke an N400 effect but rather lead to a Late Positivity effect (Kuperberg, Sitnikova et al. 2003; Hoeks, Stowe et al. 2004; Kuperberg, Kreher et al. 2007; Kuperberg 2007). This Late Positivity is similar in morphology and scalp distribution to the P600 evoked by inflectional morphosyntactic violations between a subject NP and verb (Kuperberg, Caplan et al. 2006). Although different hypotheses have been proposed to explain why these types of semantic violations evoke a P600 effect rather than an N400 effect (Kuperberg, Sitnikova et al. 2003; Hoeks, Stowe et al. 2004; Kim and Osterhout 2005; Kolk and Chwilla 2007; Kuperberg, Kreher et al. 2007; Kuperberg 2007), they tend to agree that the P600 is often triggered by a temporary conflict between the output of more than one processing stream and that it reflects a continued analysis or reanalysis of the critical word with respect to its context. What is debated is the nature of such processing streams and their output representations, as well as the specificity of such reanalysis to a particular level of linguistic processing. We have suggested that the P600 evoked by both semantic-thematic and syntactic anomalies reflects, at least in part, a continued semantic-thematic analysis of the meaning of arguments against more minimal semantic requirements of the action described by the central verb (Kuperberg, Kreher et al. 2007). Consistent with this hypothesis, the P600 evoked by animacy semantic-thematic violations is also similar to the Late Positivity elicited by violations of visual real-world actions depicted within short, silent movie clips (Sitnikova, Kuperberg et al. 2003; Kuperberg 2007; Sitnikova, Holcomb et al. 2007; Sitnikova, Holcomb et al. 2007), suggesting that such a semantic-thematic integration or reanalysis might engage similar neural processes to those engaged in relating people, objects and action during real-world event comprehension (Sitnikova, Kuperberg et al. 2003; Kuperberg 2007; Sitnikova, Holcomb et al. 2007; Sitnikova, Holcomb et al. 2007). It is also possible that such a semantic-thematic analysis is one component of a more general reanalysis of all aspects of the original linguistic input including orthographic, phonological, lexico-semantic and syntactic information (Vissers, Chwilla et al. 2006; Kolk and Chwilla 2007).
The Present Study
Taken together, ERP findings suggest that the brain may differentiate between two broad types of semantic violations: 1) violations of semantic associative, categorical, and real-world relationships that are violated in sentences such as “…at breakfast the boys would plant..”, and that are based mainly on the frequency of co-occurrence of words or events, as stored within semantic memory (Kutas and Federmeier 2000), and 2) semantic-thematic verb-argument relationships, violated in sentences such as “…at breakfast the eggs would eat…”, that may rely on prolonged attempts to semantically-thematically integrate or reanalyze the meaning of arguments against more minimal semantic requirements of the central verb (Kuperberg, Sitnikova et al. 2003; Kuperberg, Kreher et al. 2007; Kuperberg 2007). Moreover, there is some evidence that such semantic-thematic integration or reanalysis may also be engaged upon encountering morphosyntactic violations (Kaan, Harris et al. 2000; Kuperberg, Caplan et al. 2006) and may share commonalities with processes engaged upon encountering violations of action in the visual world (Kuperberg 2007; Sitnikova, Holcomb et al. 2007).
The present study aimed to determine whether similar broad neural distinctions between these sentence types were observable using fMRI. We examined the neuroanatomical regions engaged in response to introducing three types of violation within sentences. First, we introduced verbs that violated our knowledge about what events are likely to occur in the world, given a particular context. For example, in a sentence such as, “Every morning at breakfast the boys would plant…”, the violation lies at the level of integrating the likelihood of boys planting with what we know about events that usually occur at breakfast. We use the term, ‘real-world pragmatic1 violations’ to describe these types of sentences. Second, we introduced verbs whose thematic structure was such that it violated the animacy of its preceding Agent NP. For example, in a sentence such as, “Every morning at breakfast the eggs would eat…”, the violation lies in the relationship between the inanimate subject NP, “eggs”, and the verb of the sentence, “eat” (that has a thematic-semantic structure such that it assigns the role of Agent to an animate rather than inanimate subject), without necessarily considering the preceding context, “Every morning at breakfast”, or even real-world knowledge. We use the term ‘animacy semantic-thematic violations’ to describe these types of sentences. Third, we introduced inflectional violations between subject NPs, an auxiliary when present, and the main verbs. For example, in a sentence such as, “Every morning at breakfast the boys would eats…”, the verb does not agree in number with the subject NP. We use the term ‘morphosyntactic violations’ to describe these types of sentences.
We contrasted each of these types of sentences to non-violated sentences, and to one another. The design was based on our previous fMRI study described above in which we examined the effects of two of these violations – real-world pragmatic and morphosyntactic (Kuperberg, Holcomb et al. 2003). On the basis of this previous study as well as the findings of Hagoort et al. (2003), we predicted that the real-world pragmatically violated sentences (e.g. “…at breakfast the boys would plant…”), in comparison with non-violated sentences and morphosyntactically violated sentences, would lead to increased activity within the left inferior frontal cortex (BA 45/47). On the basis of this previous study as well as the findings of Ni et al. (2000, Experiment 2), we also predicted that the morphosyntactically violated sentences (e.g. “…at breakfast the boys would eats…”), in comparison with both non-violated and real-world pragmatically violated sentences, would recruit inferior parietal cortices and possibly also posterior inferior frontal cortices and the left occipito-temporal cortex.
Our main aim was to examine the pattern of activity evoked by the animacy semantic-thematic violated sentences (e.g. “…at breakfast, the eggs would eat…”), in relation to each of the other sentence types. On the basis of the distinct ERP effects evoked by real-world pragmatic and animacy semantic-thematic violations, we predicted that the animacy semantic-thematic violations would lead to the recruitment of a neuroanatomical network that was distinct from the left inferior frontal cortex recruited by the real-world pragmatic violations. More specifically, given that what was violated was the relationship between the action denoted by the critical verb and the inanimate properties of the Agent carrying out this action, we predicted that these violations would evoke activity within a widespread network engaged in evaluating such action-based, semantic-thematic relationships. This might include premotor and parietal cortices, known to be engaged in processing both verbal and non-verbal representations of goal-directed physical action (Buccino, Binkofski et al. 2001; Rizzolatti, Fogassi et al. 2001). It might also include dorsolateral and superior prefrontal cortices that may be engaged in processing representations of goal and motivation (Ruby, Sirigu et al. 2002).
Based on the similarity of the Late Positivity/P600 evoked by these types of animacy semantic-thematic violations and the P600 evoked by morphosyntactic violations (Kuperberg, Caplan et al. 2006), as well as on the theory that encountering morphosyntactic violations between a verb and its argument might also lead to a prolonged semantic-thematic analysis (Kaan, Harris et al. 2000), we hypothesized that the neural network engaged by animacy semantic-thematic violations would, at least partially, overlap with the network engaged by these types of morphosyntactic violations.
Method
Construction of stimuli
The stimuli have been described in detail elsewhere (Kuperberg, Sitnikova et al. 2003; Kuperberg, Caplan et al. 2006). Two-hundred-and-forty Agent-Theme or Experiencer-Theme verbs were chosen as critical words and 6–10-word sentences providing a constraining context, with an animate subject NP, were constructed for each of them (see Table 1). Most verbs were dynamic (nine were stative) (Dowty 1989). The critical verb fell on the sixth to the ninth word of the sentence and none of the critical verbs were sentence-final words.
Table 1.
Types of Linguistic Violation
| Linguistic violation | Explanation | Example |
|---|---|---|
| (1) None | Baseline condition. | Every morning at breakfast the boys would eat toast and jam.” |
| (2) Real-world pragmatic violation | The critical verb was replaced by another verb taken from another sentence scenario that was incongruous with the entire preceding context, with respect to real-world knowledge. | Every morning at breakfast the boys would plant the flowers.” |
| (3) ^ Animacy Semantic-thematic violation | The animate NP that is assigned the role of Agent by the critical verb was replaced by an inanimate NP. | Every morning at breakfast the eggs would eat toast and jam.” |
| (4) Morphosyntactic inflectional violation | The verb was changed either to violate subject-verb agreement or by using a finite in place of an infinitival verb. | Every morning at breakfast the boys would eats toast and jam.” |
Our use of the term animacy semantic-thematic violation conveys the fact that, in all of these sentences, an inanimate subject NP was used together with verbs that assign the role of Agent (animate in nature) to their preceding subject noun in simple English sentences (Agent-Theme or Experiencer-Theme verbs), thus violating the thematic structure of these verbs.
In the examples, the critical verb to which the hemodynamic response was modeled, is underlined.
The animacy semantic-thematic violated sentences were constructed by replacing the subject animate NP with an inanimate NP, which was in keeping with the overall preceding context, such that the sentence became anomalous on the critical verb. The syntactically violated sentences were constructed by introducing an inflectional morphosyntactic violation between the subject, an auxiliary when one was present, and the main verb (either by violating subject-verb agreement or by using a finite in place of an infinitival verb). The real-world pragmatically violated sentences were constructed by replacing the critical verbs and its subsequent words with verbs and subsequent words that were chosen pseudorandomly from sentences of another list (see below), such that the sentences were rendered relatively less likely or less plausible, with respect to their preceding contexts and real-world knowledge, than the non-violated sentences.
So that no participant would encounter the same word more than once (leading to repetition priming effects) but so that, across all participants, all critical verbs (and words subsequent to the critical verb) would be seen in all four conditions, the sentences were divided into four lists that were counterbalanced across subjects. Thus, in each list, there were 240 test sentences: 60 normal sentences, 60 real-world pragmatically violated sentences, 60 animacy semantic-thematically violated sentences and 60 morphosyntactically violated sentences.
FMRI Study
Participants
Sixteen (11 male and 5 female; mean age: 42, SD: 9) participants were recruited by advertisement. All participants were right-handed as assessed using the modified Edinburgh Handedness Inventory (Oldfield 1971; White and Ashton 1976). Selection criteria required all participants to have normal or corrected-to-normal vision, to be native speakers of English and not to have learned another language before age 5 years. In addition, participants were not taking any medication and were screened to exclude the presence of psychiatric and neurological disorders and to exclude the contraindications for MRI. Written consent was obtained from all subjects before participation according to the established guidelines of the Massachusetts General Hospital Institutional Review Board.
Stimulus presentation and task
During scanning, each trial began with the presentation of a yellow fixation point at the center of the screen for 400 msec, followed by a 100 msec blank screen, followed by the first word of the sentence. Each word appeared on the screen for 400 msec with an interstimulus interval (ISI) of 100 msec separating words. Thus, the critical verb always started after 3 sec from the beginning of the sentence trial. The final word of each sentence was followed by a question mark cue that remained on the screen until the subject made his/her response at which point the next trial started. The subject’s task was to decide whether or not each whole sentence was acceptable by pressing one of two buttons on a response box (using their left hand, with their index and middle fingers, counterbalanced across subjects). Subjects were told that sentences may not be acceptable in different ways and that if a sentence seemed at all “odd” or unlikely or if it had an obvious grammatical error, they should indicate that it was unacceptable. Subjects were instructed to wait until the question mark cue before responding. Accuracy and judgment reaction times (RTs) were recorded.
Sentence trials were pseudo-randomly presented amongst fixation trials in which subjects were asked to fixate on a white “+” symbol for variable durations ranging from 2 through 20 seconds until the next sentence trial. The fixation trials constituted 32% of all trials. The random interleaving of these ‘fixation’ or ‘null-events’ amongst the sentences was critical for the efficient estimation and deconvolution of the entire hemodynamic response (Burock, Buckner et al. 1998).
Behavioral data analysis
Participants’ accuracy (judging a non-violated sentence as acceptable and a violated sentence as unacceptable) and RTs to each sentence type (collapsed across individual sentences) were entered into a repeated-measures ANOVA with subjects as a random effect and sentence type as a within-subject factor. One participant’s average RTs to the animacy semantic-thematic violated sentences was more than three standard deviations above the mean RT to the animacy semantic-thematic violations (across all participants) and was excluded from analyses. Significant effects of sentence type were followed up using planned t-tests comparing the different sentence types. Alpha was set to 0.05. All RT analyses were repeated using correctly-answered responses only and revealed the same pattern of findings.
MRI acquisition and preprocessing
Imaging took place on a 3 Tesla MR scanner (Siemens Trio) with echoplanar (EP) imaging capability. Head motion was minimized using pillows and cushions around the head. Subjects underwent two conventional high-resolution 3D structural scans, constituting a spoiled GRASS (SPGR) sequence (128 sagittal slices, 1.33 mm thickness, TR = 2530 msec, TE = 3.77 msec, flip angle = 7 degrees, bandwidth = 200 Hz, in-plane resolution = 1 mm × 1.33 mm). These scans were followed by the acquisition of T1-weighted anatomic images (30 slices, 3 mm thickness, skip 1 mm between slices) and then a T2-weighted image acquired in plane with the functional images to assist in the manual registration of the functional data to the high resolution anatomical scans.
Each subjects then viewed one of the four counterbalanced sentence lists with the sentence trials and fixation trials, divided over eight functional runs. Each functional run lasted between 420 seconds during which T2*-weighted echoplanar (EP) images were acquired (30 slices covering the whole brain, 210 images per slice, 3 mm thickness, in-plane resolution of 3.125 mm, slices oriented approximately 30\grad axially, 1 mm between slices), using a gradient echo (GR) sequence (TR = 2 sec; TE = 25 msec; flip angle = 100\grad).
MRI Data analysis
Reconstruction of cortical surfaces from structural MRI data
Following motion correction, the two high-resolution structural scans for each participant were averaged to increase the signal:noise, and this high signal:noise resulting volume was used to reconstruct a model of each individual’s cortical surface using FreeSurfer, developed at the Martinos Center for Biomedical Imaging, Charlestown, MA, http://surfer.nmr.mgh.harvard.edu/ (Dale and Sereno 1993; Dale, Fischl et al. 1999; Fischl, Liu et al. 2001). The surface representing the gray/white border was inflated (Dale and Sereno 1993; Fischl, Sereno et al. 1999), differences between individuals in the depth of gyri/sulci were normalized, and, for the purposes of averaging functional data across subjects (see below), each subject’s reconstructed cortical surface was morphed/registered to an average spherical surface representation that optimally aligned sulcal and gyral features across subjects while minimizing metric distortion (Fischl, Sereno et al. 1999; Fischl, Sereno et al. 1999).
Analysis of individual functional MRI data
For each participant, the acquired native functional volumes were first corrected for potential motion using the AFNI algorithm (Cox 1996). Next, the functional volumes were spatially smoothed using a 3-D Gaussian filter with a full-width half-max (FWHM) of 6 mm. Global intensity variations across runs and participants were removed by rescaling all voxels and time points of each run such that the mean in-brain intensity was fixed at an arbitrary value of 1000.
The functional images were then analyzed with a General Linear Model (GLM) using the FreeSurfer Functional Analysis Stream (FS-FAST). The hemodynamic response for each trial was modeled using three components, each constituting a canonical hemodynamic response function (HRF) (Friston, Fletcher et al. 1998), convolved with a box car of an appropriate length. The first component was modeled as a single regressor and lasted between 3500 msec and 4500 msec and corresponded to the onset of each sentence trial until the offset of the word before the critical word. This did not differ between experimental conditions. The second component lasted from the onset of the critical verb until the onset of the question mark and lasted from 620 msec to 1950 msec and was modeled separately for each sentence type. The third component lasted from the onset of the question mark until each participant’s response that moved him or her on to the next trial. In addition, mean offset and linear trend regressors were included to remove low-frequency drift.
Construction of group cortical statistical maps
These GLM parameter estimates and residual error variances of each participant’s functional data were resampled onto his or her inflated cortical surface and then onto the average cortical spherical representation (see anatomical reconstruction above). Each participant’s data were then smoothed on the surface tessellation using an iterative nearest-neighbor averaging procedure (equivalent to applying a two-dimensional Gaussian smoothing kernel with a FWHM of approximately 8.5 mm). Each participant’s functional data were also resampled into Talairach space (Talairach and Tournoux 1988) for averaging of subcortical data. This was carried out by first aligning the functional to the structural data with a six degrees-of-freedom transform using SPM software, and then aligning the structural data to the MNI305 template using a twelve degrees-of-freedom alignment procedure (Collins, Neelin et al. 1994).
A contrast was generated that compared all sentences across the whole trial (all three regressor components) with the baseline fixation condition. In addition, pair-wise contrasts comparing activity to each sentence type relative to every other sentence type were constructed using the regression weights of the second canonical HRF component (that differed across conditions). These contrasts were tested using a t statistic at each voxel on the spherical surface (to identify cortical clusters) and in Talairach space (to identify subcortical clusters) using a random effects model. To correct for multiple comparisons, significant clusters of activated voxels were identified on the basis of a Monte Carlo simulation (Doherty, West et al. 2004) using a cluster size threshold of 300 mm2 (for cortical clusters) and 300 mm3 (for subcortical clusters) using a threshold for rejection of the null hypothesis of p < 0.05. Cortical maps were then overlaid on one another in order to identify regions that were commonly activated across contrasts of interest. Statistical maps were also generated that included only activity to correctly-answered trials. Any difference in findings revealed by these maps and the maps generated from all trials are noted in the Results tables.
In addition to modeling the hemodynamic response for each trial in each individual using an HRF (Friston, Fletcher et al. 1998), functional images were also analyzed using a finite impulse response (FIR) model, that gave estimates of the hemodynamic response every 2 sec (at each TR), allowing us to examine the hemodynamic response at successive intervals without assumptions about the shape of the hemodynamic response (Burock, Buckner et al. 1998; Dale 1999; Burock and Dale 2000). By qualitatively comparing the results yielded by the HRF-modeled and the FIR-modeled datasets, we were able to determine whether any apparent differences in the amplitude of hemodynamic activity between the sentence types yielded by the HRF-modeled dataset actually reflected differences across sentence types in the time course of hemodynamic activity within the same networks.
Results
Behavioral data (Table 2)
Table 2.
Behavioral classifications.
| Non-violated | Pragmatically violated | Animacy violated | Morphosyntactically violated | |
|---|---|---|---|---|
| Accuracy | 88 (7.57) | 89.45 (9.41) | 95.27 (7.09) | 91.81 (9.56) |
| Reaction Times | 826 (228) | 1005 (415) | 796 (348) | 751 (297) |
Means are shown with standard deviations in brackets.
Accuracy: the percentage of responses classified correctly is given.
Reaction times averaged across all trials are shown (for consistency with fMRI data). The same pattern of findings was observed when correctly-answered trials only were examined.
There were significant differences in accuracy across the four sentence types, F(3, 45) = 5.59, p = 0.002. Follow up paired t-tests showed that participants responded more accurately to the animacy semantic-thematic violated sentences than to the non-violated sentences, t(15) = 3.99, p < 0.001, the real-world pragmatically violated sentences, t(15) = 4.14, p < 0.001, and the morphosyntactically violated sentences, t(15) = 2.981, p < 0.009. There were no differences in accuracy between any of the other sentence types (all ps > 0.1).
There were also significant differences in RTs across the four sentence types, F(3, 42) = 3.55, p < 0.037. Follow-up paired t tests showed that RTs to the real-world pragmatically violated sentences were marginally longer than to the non-violated sentences, t(15) = 1.9, p < 0.071, and significantly longer than to the animacy semantic-thematic violated sentences, t(14) = 2.88, p < 0.02, and to the morphosyntactically violated sentences, t(15) = 2.76, p < 0.015. This pattern of RT findings was the same when only the correctly-answered trials were analyzed.
fMRI data
All sentences versus fixation
As expected, a large network distributed across frontal, temporal and occipital cortices showed more activity to the sentences (all four types) than to the low-level baseline fixation condition (Table 3A, Figure 1). In addition, consistent with previous studies, a large network showed less activity to the sentence trials than to the fixation trials; these so-called ‘deactivated’ regions included the middle frontal, medial frontal, inferior parietal, medial parietal and medial temporal cortices (Table 3B, Figure 1).
Table 3.
Contrast between all sentences and fixation
| No. | Region | Lat. | BA | Size | Tal. x, y, z | p |
|---|---|---|---|---|---|---|
| A. All Sentences > Fixation | ||||||
| 1a | Inferior frontal gyrus (anterior) | L | 45/47 | 1084 | −46 26 −7 | 0.0028 |
| R | 1121 | 47 32 6 | 0.00067 | |||
| 1b | Inferior frontal gyrus (posterior) | L | 44/45 | 989 | −55 20 13 | 0.00056 |
| R | 962 | 55 17 9 | 0.00081 | |||
| 2 | Precentral gyrus | L | 6 | 1224 | −51 −2 38 | 0.00016 |
| R | 742 | 51 4 32 | 0.00049 | |||
| 3 | Middle temporal cortex (anterior) | L | 21/20 | 861 | −56 −19 −13 | 0.00091 |
| 4 | Superior/middle temporal cortex (posterior) | L | 21 | 1822 | −47 −43 6 | 4.27e−6 |
| R | 978 | 48 −37 3 | 0.00063 | |||
| 5 | Inferior temporal gyrus (lateral surface) | L | 37 | 1481 | −52 −54 −7 | 0.00016 |
| R | 1211 | 54 −51 −12 | 0.00071 | |||
| 6 | Extrastriate cortex (lateral surface) | L | 19 | 2062 | −45 −80 −1 | 1.91e−5 |
| R | 1722 | 45 −86 0 | 3.4e−5 | |||
| 7 | Superior frontal gyrus (medial surface) | L | 8/32 | 748 | −9 20 41 | 2.45e−6 |
| R | 422 | 11 17 43 | 0.00031 | |||
| 8 | Inferior temporal & fusiform cortex (ventral surface) | L | 37 | 1075 | −51 −49 −16 | 7.59e−5 |
| R | 982 | 48 −55 −8 | 5.01e−5 | |||
| 9 | Extrastriate cortex (ventral surface) | L | 18 | 960 | −31 −91 −1 | 0.00027 |
| R | 822 | 30 −90 0 | 5.62e−5 | |||
| S1 | Globus pallidus | L | 3136 | −12 4 0 | 0.00104 | |
| S2 | Medial temporal lobe | R | 1088 | 16 −5 −17 | 0.00143 | |
| B. All Sentences < Fixation (“Deactivation”) | ||||||
| 10 | Middle frontal gyrus | L | 46/9 | 2343 | −38 28 27 | 0.0017 |
| R | 2124 | 35 30 39 | 0.00013 | |||
| 11 | Insula | L | _ | 688 | −38 −19 0 | 1.38e−5 |
| R | 652 | 38 −5 −2 | 0.00058 | |||
| 12 | Central sulcus and postcentral gyrus | L | 1/4 | 1257 | −40 −18 39 | 0.0023 |
| R | 489 | 42 −11 31 | 1.12e−6 | |||
| 13 | Inferior parietal lobule (superior portion) | L | 40 | 878 | −48 −46 35 | 0.0019 |
| R | 892 | −48 −41 35 | 0.0016 | |||
| 14 | Angular gyrus | L | 39 | 377 | −46 −69 35 | 0.0035 |
| R | 523 | 50 −68 32 | 4.57e−5 | |||
| 15 | Extrastriate cortex (sup. occipital gyrus) | L | 19 | 630 | −19 −90 30 | 5.89e−6 |
| R | 1022 | 25 −87 26 | 0.0006 | |||
| 16 | Anterior cingulate | L | 32 | 1275 | −6 39 −2 | 0.0017 |
| R | 1323 | 8 41 6 | 0.0004 | |||
| 17 | Paracentral lobule | L | 4/5 | 1813 | −8 −31 58 | 0.0016 |
| R | 1945 | 8 −41 55 | 9.77e−8 | |||
| 18 | Precuneus | L | 7 | 1921 | 4 −60 −38 | 0.001 |
| R | 1988 | 3 62 39 | 5.13e−8 | |||
| 19 | Lingual/Parahippocampal gyrus | L | 30/35 | 1847 | −24 −42 −3 | 2.63e−6 |
| R | 2010 | 17 −46 −4 | 4.27e−7 | |||
| 20 | Cuneus | L | 18/19 | 1724 | −3 −74 29 | 6.03e−7 |
| R | 1632 | 5 −79 35 | 2.63e−6 | |||
Abbreviations: BA: Brodmann Area. Lat.: Laterality. Tal.: Talairach coordinates.
No. corresponds directly to cluster labels in Figure 1. Numbers preceded by S refer to subcortical structures.
When clusters or BAs span over more than one region, both regions/BAs are indicated, separated by a slash sign. All clusters indicated in p columns reached cluster-level significance, p < 0.05 corrected across the whole cortex.
Size: for cortical activations, area in mm2; for subcortical activations, volume in mm3
Figure 1.
Cortical statistical maps comparing responses to all sentences (modeled across the whole trial) with responses to the low-level fixation condition. Activation is displayed on an average cortical surface with light gray indicating the gyri, and dark gray indicating the sulci. Top panel: lateral surfaces; middle panel: medial surfaces; bottom panel: ventral surfaces. Yellow-red: more activity to sentences than to fixation. Blue: less activity to sentences than to fixation.
All clusters circled are significant at a cluster-level p < 0.05. Cluster numbers correspond directly to those regions reported in Table 3.
Increased response to real-world pragmatically violated sentences relative to other sentence types
There was significantly more activity in response to the real-world pragmatic violations than to each of the three other sentence types within the left anterior inferior frontal gyrus (Table 4, top row, and Figure 2). An overlap map confirmed that this region was activated in all three contrasts (Figure 2). The contrast between the real-world pragmatically violated and the non-violated sentences also showed increased activity within the bilateral anterior ventromedial temporal cortices (fusiform and parahippocampal gyri) and within extrastriate cortices.
Table 4.
Regions that showed relatively more activity to pragmatically violated sentences (relative to other sentence types).
| No. | Region | Lat | BA | Size | Tal. x, y, z | p | Size | Tal. x, y, z | p | Size | Tal. x, y, z | p |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pragmatic > Normal | Pragmatic > Animacy | Pragmatic > Syntactic | ||||||||||
| 1 | Inferior frontal gyrus (ant.) | ** L | 47 | 515 | −43 25 −10 | .00012 | 116 | −43 22 −10 | ^.013 | 312 | −46 23 −8 | .0003 |
| 3 | Extrastriate cortex | L | 18/19 | 310 | −13 −86 29 | .0013 | NS | NS | ||||
| R | 1218 | 28 −87 30 | .0023 | NS | NS | |||||||
| 10 | Ant. med. temporal cortex (fusiform & parahippoc ampal gyrus) | L | 36 | 144 | −27 −28 −19 | ^.00004 | NS | NS | ||||
| R | 414 | 31 −30 −13 | .001 | NS | NS | |||||||
Abbreviations: BA: Brodmann Area. Lat.: Laterality. Tal.: Talairach coordinates. NS: non-significant. Cluster numbers (No.) correspond to the cluster labels in Figures 2 and 6. When clusters or BAs span over more than one region, both regions/BAs are indicated, separated by a slash sign. All clusters indicated in p columns reached cluster-level significance, p < 0.05 corrected across the whole cortex except for that marked with ^ which reached significance within a left inferior frontal A priori region of interest (small surface correction, p < 0.05).
Cluster shows overlap between all three contrasts (as seen on overlap image, Figure 2, bottom row). Size: area in mm2
Figure 2.
Top row: Cortical statistical maps comparing responses to the pragmatic violations (modeled from the onset of the critical word until the onset of the “?”) with responses to each of the other sentence types. Activation is displayed on an average left lateral cortical surface with light gray indicating the gyri, and dark gray indicating the sulci. Yellow-red: more activity to the pragmatic violations than to the other types of violations and the non-violated verbs. Blue: less activity to the pragmatic violations than to the other types of violations and the non-violated verbs. The circled cluster (1) indicates the left anterior inferior frontal gyrus that showed significantly more activity to the pragmatic violations in all three contrasts (cluster-level p < 0.05).
Bottom row: the left anterior inferior frontal gyrus was the only area that showed overlap between the yellow-red clusters in the three contrasts above.
Increased response to morphosyntactically violated sentences relative to other sentence types
There was relatively more activity in response to the morphosyntactically violated sentences than to both the non-violated sentences and the real-world pragmatically violated sentences within widespread regions (Table 5, Figure 3). Many of the same regions were activated in both these contrasts, and included bilateral posterior inferior frontal and precentral cortices, bilateral middle frontal gyri, the left inferior parietal lobule, bilateral insula, superior temporal gyri, anterior medial prefrontal cortices, the left lateral occipitotemporal cortex and the basal ganglia. An overlap map confirmed that many of the same clusters were activated in both contrasts (Figure 3, bottom row).
Table 5.
Regions that showed relatively more activity to syntactically violated sentences (relative to other sentence types).
| No. | Region | Lat. | BA | Size | Tal. x, y, z | p | Size | Tal. x, y, z | p | Size | Tal. x, y, z | p |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Syntactic > Normal | Syntactic > Pragmatic | Syntactic > Animacy | ||||||||||
| 1. | Posterior inf. frontal/pre-central gyrus | ** L | 44/45/6 | 287 | −49 3 7 | .0064 | 427 | −49 4 10 | .0028 | NS | ||
| ** R | 455 | 29 19 5 | .0025 | 614 | 54 10 28 | .0088 | NS | |||||
| 2 | Superior precentral gyrus | ** L | 4/6 | 1394 | −45 −9 41 | .0004 | 542 | −54 5 20 | .0024 | NS | ||
| R | 367 | 52 3 33 | ^.0104 | 167 | 35 12 48 | ^.0171 | NS | |||||
| 3 | Middle frontal gyrus | ** L | 9/46 | 771 | −19 33 29 | .0033 | 273 | −23 30 29 | ^.013 | NS | ||
| ** R | 1209 | 20 39 21 | .012 | 1457 | 37 39 16 | .002 | NS | |||||
| 4 | Inferior parietal lobule | ** L | 40 | 1137 | −56 −31 38 | .00017 | 1300 | −57 −38 32 | .00016 | 336 | −49 −39 41 | .00046 |
| 5 | Post-central gyrus | L | 1 | 1442 | −51 −19 46 | .0021 | NS | NS | ||||
| 6 | Insula | ** L | _ | 895 | −39 −4 14 | .00071 | 424 | −36 −6 16 | .0013 | NS | ||
| ** R | 135 | 36 −3 14 | .0122 | 272 | 36 5 −7 | .0187 | NS | |||||
| 7 | Superior temporal gyrus | ** L | 22 | 629 | −53 −20 −1 | .0035 | 182 | −51 −27 9 | ^.005 | NS | ||
| ** R | 281 | 58 −19 3 | .0024 | 262 | 63 −25 9 | ^.005 | NS | |||||
| 8 | Anterior medial prefrontal cortex (anterior cingulate & medial frontal gyrus) | ** L | 32/23/10 | 1367 | −6 37 −5 | .0054 | 1300 | −7 38 −8 | .0014 | 833 | −13 47 0 | .0005 |
| ** R | 1151 | 6 49 −8 | .0008 | 517 | 8 −39 28 | 8.9e−5 | 965 | 10 56 2 | .00047 | |||
| 9 | Superior medial frontal cortex (posterior) | L | 6 | 929 | −11 −6 47 | .0087 | NS | NS | ||||
| R | 590 | 9 −2 52 | .0039 | NS | NS | |||||||
| 10 | Fusiform gyrus (anterior) | L | 36/37 | 314 | −35 −42 −6 | .0031 | NS | NS | ||||
| R | 406 | 35 −38 −9 | .001 | NS | NS | |||||||
| 11 | Precuneus | R | 7 | NS | 401 | 12 −72 42 | .001 | NS | ||||
| 12 | Parietal-occipital junction | L | 7/19 | NS | 1237 | −23 −75 40 | .0032 | |||||
| R | NS | 1438 | 41 −75 37 | .0018 | NS | |||||||
| 13 | Posterior middle temporal cortex and extrastriate cortex | ** L | 18/19 | 2240 | −32 −92 17 | .0012 | 2600 | −20 −79 42 | .00046 | NS | ||
| ** R | 1839 | 25 −80 36 | .011 | 945 | 40 −76 32 | .009 | NS | |||||
| S1 | Caudate | R | NS | 3072 | −32 −12 −13 | 0.00112 | NS | |||||
| S2 | Putamen | L | 1024 | −12 11 −4 | 0.00181 | NS | NS | |||||
| R | 1088 | 28 −15 8 | 0.00382 | 4928 | 24 4 7 | 0.00626 | NS | |||||
| S3 | Medial temporal lobe | R | NS | 512 | 16 −1 −17 | 0.01216 | NS | |||||
Abbreviations: BA: Brodmann Area. Lat.: Laterality. Tal.: Talairach coordinates. NS: non-significant. Cluster number (No.) corresponds directly to cluster labels in Figure 3 and are consistent with those used in Table 4 and Figure 2. Cluster number 10 is shown in Figure 6. Numbers preceded by S refer to subcortical structures.
When clusters or BAs span over more than one region, both regions/BAs are indicated, separated by a slash sign. All clusters indicated in p columns reached cluster-level significance, p < 0.05 corrected across the whole cortex.
Cluster shows overlap between the syntactic>normal and the syntactic>pragmatic contrasts (as seen on overlap image, Figure 3, bottom row).
Size: for cortical activations, area in mm2; for subcortical activations, volume in mm3
Figure 3.
Cortical statistical maps comparing responses to the syntactic violations (modeled from the onset of the critical word until the onset of the “?”) with responses to the non-violated verbs (top row) and to the pragmatically violated verbs (middle row). Activation is displayed on an average lateral cortical surface (left) and an average medial surface (right) with light gray indicating the gyri, and dark gray indicating the sulci. Yellow-red: more activity to the syntactic violations than to the non-violated verbs or to the pragmatic violations. Blue: less activity to the syntactic violations than to the non-violated verbs and the pragmatic violations.
Bottom row: areas that showed overlap between the yellow-red clusters shown in the first two rows, i.e. regions that showed more activity to the syntactic violations relative to both the non-violated verbs and the pragmatic violations.
All clusters circled are significant at a cluster-level p < 0.05. Cluster numbers correspond directly to those regions reported in Table 5.
With the exception of the left inferior parietal lobule (BA 40) and bilateral anterior medial prefrontal cortices (BAs 32/23/10), most of these regions did not show significant modulation in a direct contrast between the morphosyntactically violated and the animacy semantic-thematic violated sentences.
Increased response to animacy semantic-thematic violated sentences relative to all other sentence types
The animacy semantic-thematic violated sentences showed more activity than both the non-violated sentences and the real-world pragmatically violated sentences within a widespread network (Table 6, Figure 4). Once again, many of the same regions were activated in both these contrasts, including the bilateral posterior inferior frontal/precentral cortices, bilateral middle frontal cortices, insula, and the left inferior parietal lobule and the basal ganglia. An overlap map confirmed that many of these clusters were activated in both contrasts (Figure 4, bottom row).
Table 6.
Regions that showed relatively more activity to the animacy violated sentences (relative to other sentence types).
| No. | Region | Lat. | BA | Size | Tal. x, y, z | p | Size | Tal. x, y, z | p | Size | Tal. x, y, z | p |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Animacy > Normal | Animacy > Pragmatic | Animacy > Syntactic | ||||||||||
| 1 | Post. inf. frontal/Precentral gyrus | ** L | 44/6 | 85 | −49 11 5 | #^.0126 | 65 | −50 4 5 | ^.012 | 300 | −50 14 21 | .003 |
| R | 222 | 54 7 10 | .005 | 378 | 47 16 9 | .0132 | NS | |||||
| 2 | Precentral gyrus | ** L | 4 | 714 | −41 −8 46 | .0016 | 250 | −32 −14 60 | .0013 | NS | ||
| ** R | 1010 | 51 3 19 | .00017 | 980 | 56 4 17 | .0009 | NS | |||||
| 3 | Middle frontal gyrus | ** L | 46 | 34 | −35 34 20 | #^.015 | 50 | −39 35 22 | ^.0126 | NS | ||
| ** R | 520 | 25 46 22 | .0017 | 680 | 36 45 7 | .0071 | NS | |||||
| 4 | Inferior parietal lobule | ** L | 40 | 40 | −56 −36 34 | ^.0078 | 784 | −59 −36 30 | .0008 | NS | ||
| R | NS | 812 | 59 −30 34 | .006 | NS | |||||||
| 5 | Postcentral gyrus | ** L | 1/2 | 1074 | −49 −15 34 | .0006 | 234 | −58 −20 21 | .003 | NS | ||
| ** R | 867 | 53 −11 26 | 7.9e−5 | 1423 | 35 −27 55 | .0011 | NS | |||||
| 6 | Insula | ** L | _ | 193 | −40 −16 0 | ^.0034 | 171 | −39 −11 5 | #^.0035 | NS | ||
| ** R | 691 | 43 −7 13 | .0076 | 547 | 32 3 10 | .006 | NS | |||||
| 7 | Superior temporal cortex | L | 22 | 582 | −47 −16 3 | .005 | NS | NS | ||||
| R | 537 | 62 −13 1 | .0026 | NS | NS | |||||||
| 9 | Posterior medial frontal cortex | L | 6 | 810 | −1 −13 59 | #.0067 | 922 | −4 −15 49 | #.014 | NS | ||
| 10 | Fusiform gyrus (anterior) | L | 36/37 | 202 | −35 −40 −8 | ^.00064 | NS | NS | ||||
| R | 445 | 35 −39 −8 | ^.00074 | NS | NS | |||||||
| 11 | Precuneus | ** R | 7 | 418 | 7 −54 41 | #.0092 | 842 | 5 −53 41 | .00082 | NS | ||
| 13 | Occipital/Extra-Striate Cortex | L | 18/19 | 1100 | 0 −88 20 | .0035 | NS | NS | ||||
| R | 1700 | 34 −82 5 | .0034 | NS | NS | |||||||
| 14 | Middle temporal gyrus | L | 21/37 | NS | 990 | −50 −55 4 | .0044 | NS | ||||
| S1 | Caudate | L | 1728 | −20 −30 16 | 0.00278 | 2240(3) | −20 −18 19 | 0.00443 | NS | |||
| R | 1088(2) | 20 −18 19 | 0.00472 | 1088(2) | 12 8 14 | 0.00219 | 768 | 12 8 14 | 0.001409 | |||
| S2 | Putamen | L | NS | 1920 | −28 1 11 | 0.00134 | NS | |||||
| R | 448 | 24 12 3 | 0.00894 | 320 | 24 −11 8 | 0.03178 | NS | |||||
| S3 | Medial temporal lobe | L | 1856 | −24 −5 −17 | 0.00127 | 320 | −24 −5 −20 | 0.01049 | NS | |||
| R | 1792 | 24 −8 −10 | 0.0009 | NS | NS | |||||||
Abbreviations: BA: Brodmann Area. Lat.: Laterality. Tal.: Talairach coordinates. NS: non-significant. No. corresponds directly to cluster labels in Figure 4 and are also consistent with the clusters indicated in Figures 2, 3 and Tables 4 and 5. Cluster number 10 is shown in Figure 6. Numbers preceded by S refer to subcortical structures. For the caudate, clusters were combined – shown in brackets are the number of clusters combined.
When clusters or BAs span over more than one region, both regions/BAs are indicated, separated by a slash sign. All clusters indicated in p columns reached cluster-level significance, p < 0.05 corrected across the whole cortex.
Cluster not activated when only correctly-answered trials were analyzed.
Cluster shows overlap between the Animacy > Normal and the Animacy > pragmatic contrasts (as seen on overlap image, Figure 4, bottom row).
Size: for cortical activations, area in mm2; for subcortical activations, volume in mm3
Figure 4.
Cortical statistical maps comparing responses to the animacy violations (modeled from the onset of the critical word until the onset of the “?”) with responses to the non-violated verbs (top row) and to the pragmatically violated verbs (middle row). Activation is displayed on an average lateral cortical surface with light gray indicating the gyri, and dark gray indicating the sulci. Yellow-red: more activity to the animacy violations than to the non-violated verbs or to the pragmatic violations. Blue: less activity to the animacy violations than to the non-violated verbs and the pragmatic violations. All clusters circled are significant at a cluster-level p < 0.05. Cluster numbers correspond directly to those regions reported in Table 6.
Bottom row: areas that showed overlap between the yellow-red clusters shown in the first two rows, i.e. regions that showed more activity to the animacy violations relative to both the non-violated verbs and the pragmatic violations.
Once again, a direct contrast between the animacy semantic-thematic violated and the morphosyntactically violated sentences failed to reveal significant modulation in most of these regions: only the left posterior inferior frontal gyrus showed more activity in response to the animacy than to the morphosyntactic violations.
Regions commonly activated to morphosyntactically and animacy semantic-thematic violated sentences (relative to non-violated and real-world pragmatically violated sentences)
As noted above, many of the same regions were activated in response to the morphosyntactically violated sentences (relative to both the real-world pragmatically violated and the non-violated sentences) as well as to the animacy semantic-thematic violated sentences (relative to both the real-world pragmatically violated and the non-violated sentences). Cortical regions of overlap are shown in Figure 5 and constituted clusters within bilateral posterior inferior frontal gyri, precentral gyri, middle frontal gyri, the left inferior parietal lobule, the left insula and the left lateral occipitotemporal cortex. Consistent with our previous study, some of these regions fell within the network showing less activation to sentences as a whole than to fixation (Figure 1). In addition, subcortically, the basal ganglia (putamen and caudate) were commonly activated to morphosyntactically and animacy semantic-thematic violated sentences relative to the real-world pragmatically violated sentences.
Figure 5.
The image illustrates cortical maps showing regions commonly activated by the syntactically violated and the animacy violated sentences (relative to the normal sentences and the pragmatically violated sentences). Green: regions of overlap that showed more activity to the syntactic violations relative to both non-violated verbs and pragmatically-violations (i.e. those shown in the bottom row of Figure 3). Blue: regions of overlap that showed more activity to the animacy violations relative to both non-violated verbs and pragmatic violations (i.e. those shown in the bottom row of Figure 4). Yellow: regions of overlap between those areas shown in blue and green. The hemodynamic time courses of activation for each of these yellow regions depicting overall overlap are shown. The numbers indicated correspond directly to the regions reported in Tables 5 and 6 and depicted in Figures 3 and 4. Talairach coordinates within of each of these regions is given. Brodmann Areas (BAs) are approximate.
Regions that showed increased activity in contrasting each type of violated sentence with non-violated sentences
The left and right anterior medial fusiform cortices showed increased activity when each of the violated sentences was compared with the non-violated sentences (cluster number 10 in Tables 4, 5 and 6, Figure 6, top row). An overlap map of each of these three contrasts confirmed that this was the only region commonly activated (Figure 6, bottom).
Figure 6.
Top row: Cortical statistical maps comparing responses to the each type of violations (modeled from the onset of the critical word until the onset of the “?”) with responses to the non-violated verbs. Activation is displayed on average ventral cortical surfaces with light gray indicating the gyri, and dark gray indicating the sulci. Yellow-red: more activity to each type of violation than the non-violated verbs. The circled clusters indicate the anterior medial fusiform cortices that showed significantly more activity to the violations than the non-violated verbs in all three contrasts (cluster-level p < 0.05) and correspond to cluster 10 reported in Tables 3, 4 and 5. Bottom row: the anterior medial fusiform gyri were the only areas that overlapped between the yellow-red clusters in the three contrasts above.
Comparison of HRF-modeled analysis and FIR analysis
A qualitative comparison between the statistical maps yielded by the HRF-modeled datasets (as described above) and an FIR-modeled datasets showed very similar patterns of findings. The pattern of activity to each sentence type at each TR did not indicate the same increases or decreases of activity coming into play at different time points (data not shown). When FIR activity across multiple TRs was summed and the same contrasts as described above were examined, the distinctions in the neuroanatomical areas modulated to each sentence type relative to one another were very similar to those reported in the Results. This suggests that the differences in hemodynamic activity described above did, in fact, reflect true integrated differences in the amplitude of hemodynamic activation over the time-range of interest, rather than reflecting differences across sentence types in the time course of hemodynamic activity within the same network.
Summary
Real-world pragmatic violations uniquely activated the left anterior inferior frontal gyrus (BA 47) in comparison with all other sentence types. Both the morphosyntactic and the animacy semantic-thematic violations (relative to the non-violated sentences and the real-world pragmatic violations) activated a common network that included bilateral posterior inferior frontal gyri (BA 44), bilateral precentral gyri (BA 6), bilateral middle frontal gyri (BA 46/9), the left inferior parietal lobule (BA 40), the left insula, the left lateral occipito-temporal cortex and the basal ganglia. Of these, the left posterior inferior frontal gyrus showed relatively more activity in response to the animacy semantic-thematic violations than to the morphosyntactic violations, and the left inferior parietal lobule (BA 40) showed relatively more activity in response to the morphosyntactic violations than to the animacy semantic-thematic violations. Bilateral medial anterior prefrontal cortices (BAs 32/23/10) also showed more activity in response to the morphosyntactic violations than to the animacy semantic-thematic violations. Finally, relative to the non-violated sentences, all three types of violated sentences showed increased activity within bilateral anterior medial fusiform gyri (BA 36/37).
Discussion
We used fMRI to determine which neuroanatomical regions were engaged upon encountering violations of three different types of relationships within simple, active sentences: (a) relationships between the propositional representation of the sentence and knowledge about the likelihood of the event described occurring in the real world, (b) semantic-thematic relationships, based on animacy constraints, between the Agent subject NP and the action described by the main verb, and (c) morphosyntactic inflectional relationships between the subject NP and the main verb. We compared each type of violated sentence with non-violated sentences, and with one another. Real-world pragmatic violations led to increased activity within the left anterior inferior frontal cortex. In contrast, sentences containing inanimate NP Agents that violated the functional requirements of the action denoted by the verb, engaged a widespread frontal-parietal-occipitotemporal network that overlapped with the regions engaged by morphosyntactic violations between verbs and their subject NPs. Finally, bilateral anterior ventromedial fusiform cortices were activated in contrasting all three types of violated sentences with the non-violated sentences.
Below, we begin by considering some of the potential confounds in interpreting these pattern of findings. We then discuss the patterns of activity to the real-world pragmatic and morphosyntactic violations in relation to previous studies. The main focus of the discussion will be on the pattern of neural activity to the animacy semantic-thematic violations and its overlap with activity to the morphosyntactic violations. We then consider activity to all types of violations relative to non-violated sentences. We conclude by highlighting some of the implications of our findings.
Caveats and considerations in interpretation
During self-paced word-by-word reading, previous studies have shown that it takes longer to process morphosyntactic violations (relative to non-violated verbs) at the point of the anomaly, while reading times at sentence-final words are similar in morphosyntactically-violated and non-violated sentences. In contrast, real-world pragmatic violations do not generally lead to increases in processing load at the point of the anomaly, but rather are associated with longer reading times to words following the critical word including the sentence-final word (De Vincenzi, Job et al. 2003; Ditman, Holcomb et al. 2007). Also, in speeded end-of-sentence acceptability judgment or anomaly detection tasks, participants are slower to detect real-world pragmatic violations than syntactic violations (Fodor, Ni et al. 1996; Kuperberg, Holcomb et al. 2003; Kuperberg, Caplan et al. 2006) or animacy semantic-thematic violations (Kuperberg, Caplan et al. 2006). This is consistent with the pattern of end-of-sentence RT judgments observed in the present study: RTs to the real-world pragmatically violated sentences were longer than to both the morphosyntactically violated sentences and the animacy semantic-thematic violated sentences. Taken together, these findings suggest a temporal distinction in processing syntactic, thematic and real-world pragmatic information; the output of a semantic memory-based analysis is delayed relative to that of a syntactic or semantic-thematic analysis, probably because it requires a longer search through semantic memory to determine that a sentence is implausible with respect to real-world knowledge than to decide whether or not a sentence is anomalous based on a more finite set of semantic-thematic or syntactic constraints between the verb and the subject NP.
In interpreting differences in the hemodynamic response between sentence types, our assumption is that the BOLD response captured and integrated differences in both the nature as well as the timing of neurocognitive processes engaged upon both initially detecting the different types of violations, as well as integrating or reanalyzing them with respect to their preceding contexts and the words that followed until the end of the sentence. Before making this assumption, however, we consider two potential confounds.
A first possibility is that the differences in BOLD activity observed across sentence types did not primarily reflect differences in neural activity during the sentences themselves, but rather reflected differences in decision times that occurred after the sentences had ended. We attempted to address this by modeling the hemodynamic activity to each of the four sentence types from the onset of the critical verb until the end of the sentence; the decision itself was modeled separately for each trial. Thus, we factored out any differences between conditions in judgment RTs on a trial-by-trial basis. A second possibility is that, rather than reflecting differences in the amplitude of the BOLD response between sentence types (multiple hemodynamic response functions integrated over the time span specified), the observed differences in the hemodynamic responses across conditions actually reflected differences in the timing of the same neurocognitive processes; in other words, they reflected differences in the latency of the same overall hemodynamic response. We addressed this possibility by conducting additional analyses in which we examined the hemodynamic response to each sentence type at successive TR intervals, without any assumptions about the time course of the hemodynamic response function. When we examined the patterns of activity to each sentence type at each of these TRs, we did not see the same increases or decreases of activity coming into play at different time points. Thus, it seems unlikely that any differences in activity between conditions simply reflected the same cognitive processes being picked up hemodynamically at different temporal stages.
Activity to real-world pragmatic violations
The only region that showed more activity to the real-world pragmatically violated sentences, relative to all other sentence types, was the anterior left inferior frontal cortex (BA 47). This is consistent with previous findings (Kuperberg, Sitnikova et al. 2003; Hagoort, Hald et al. 2004), and with the known role of the left inferior frontal cortex, particularly its anterior portion, in retrieving information from within semantic memory (Wagner, Pare-Blagoev et al. 2001). As discussed above, we suggest that its increased recruitment to the real-world pragmatically violated sentences, relative to the other sentence types, reflected participants’ increased and more prolonged efforts to search and retrieve information from semantic memory in order to determine whether the event described in the sentence matched previously stored information, for example about what events are likely to occur at breakfast (see Kuperberg, Holcomb et al. 2003 for further discussion).
Additional activity observed within bilateral anterior inferior temporal cortices (fusiform/parahippocampal cortices) is also consistent with some previous fMRI (Newman, Pancheva et al. 2001; Kiehl, Laurens et al. 2002) and MEG (Halgren, Dhond et al. 2002) studies that have introduced semantic violations within sentences. It may reflect an implicit automatic activation of prestored semantic associative information (Alexander, Hiltbrunner et al. 1989; Chao, Haxby et al. 1999; Martin and Chao 2001; Wheatley, Weisberg et al. 2005; Gold, Balota et al. 2006; Kuperberg, Lakshmanan et al. 2007). As discussed below, however, activation of the anterior inferior temporal cortex was not specific to this contrast.
Activity to animacy semantic-thematic violations
The animacy semantic-thematic violated sentences engaged a frontal/inferior parietal/basal ganglia network that was distinct from the anterior inferior frontal cortex engaged to the real-world pragmatic violations. This hemodynamic dissociation is broadly consistent with our previous ERP data that demonstrated clear distinctions in the electrophysiological signatures to these two types of conceptual violations: whereas real-world pragmatic violations evoke an N400 effect, thought to reflect, in part, a semantic memory-based process (Federmeier and Kutas 1999; Kutas and Federmeier 2000), these animacy semantic-thematic violations, similar to syntactic violations, fail to evoke an N400 effect, but rather evoke a P600 effect (Kuperberg, Sitnikova et al. 2003; Kuperberg, Caplan et al. 2006; Kuperberg, Kreher et al. 2007). The essential difference between these two sentence types is as follows: In the real-world pragmatically violated sentences, there is no problem in assigning the thematic role of Agent to the animate subject NP at the point of the verb; the violations arise only at the level of relating the event described to its preceding context and real-world knowledge. The animacy semantic-thematic violations, on the other hand, although also violating real-world knowledge, can be detected purely through computing the relationship between the Agent NPs and the actions described by the verbs.
As discussed below, there are several possible interpretations of this pattern of findings. In this section, we discuss the possibility that the engagement of the frontal/inferior parietal/basal ganglia circuitry to these violations reflected a continued analysis (or reanalysis) based, in part, on semantic-thematic constraints related to the action described the verb and the semantic properties of its NP Agent argument2. This interpretation is consistent with the activation of some of these regions (left inferior frontal, left ventral premotor, and parietal cortices) in association with German sentences in which thematic relationships between arguments were relatively difficult to assign because of a mismatch between the thematic argument hierarchy and syntactic word order (Bornkessel, Zysset et al. 2005).
We suggest that such a semantic-thematic analysis proceeds without the reader necessarily referring to semantic real-world knowledge about the likelihood of the event to occur in the world, explaining why, unlike the real-world pragmatic violations, the animacy thematic violations were not associated with an increased recruitment of left anterior inferior prefrontal cortices (see above). Indeed, based on our ERP studies, we have suggested that the increased demand for semantic-thematic analysis to these animacy thematic role violations may even ‘switch off’ a semantic memory-based analysis involving a referral to real-world knowledge (Kuperberg, Sitnikova et al. 2003; Kuperberg, Kreher et al. 2007).
Intriguingly, many of the regions constituting this frontal/inferior parietal/basal ganglia network to the animacy semantic-thematic violated sentences (relative to both non-violated and real-world pragmatically violated sentences) have been previously implicated in both carrying out and visually comprehending action (Rizzolatti, Fogassi et al. 2001), possibly through activity of ‘mirror neurons’ (di Pellegrino, Fadiga et al. 1992; Gallese, Fadiga et al. 1996). This raises the interesting possibility that the semantic-thematic analysis described above may be mediated by similar action-based neural systems that compute relationships between actors, actions and objects, around an event, in the visual world. This explanation would be consistent with cumulating evidence suggesting that visual events continuously influence the assignment of thematic roles during sentence comprehension (Tanenhaus, Spivey-Knowlton et al. 1995; Kamide, Altmann et al. 2003), and with our recent findings suggesting that the electrophysiological signatures of processing these types of animacy semantic-thematic violations and violations of real-world action are similar (Sitnikova, Kuperberg et al. 2003; Kuperberg 2007; Sitnikova, Holcomb et al. 2007; Sitnikova, Holcomb et al. 2007). More generally, this interpretation is also consistent with evidence for links between real-world action and language systems (Glenberg and Kaschak 2002; Arbib 2005; Kemmerer 2006; Fischer and Zwaan 2007).
We speculate that different parts of this action-based network played different roles in evaluating these Agent-Action semantic-thematic relationships. For example, activity within the left posterior inferior frontal gyrus (Broca’s area), together with adjacent motor and premotor cortices, may have reflected access to the meaning of the physical bodily actions denoted by many of the verbs used (Bak, O’Donovan et al. 2001; Hauk, Johnsrude et al. 2004)3. Activity within the posterior lateral occipito-temporal cortex which lies near area MT (specialized for perceiving biological motion) may have mediated the retrieval of representations of motion features of the actions described by some of the verbs in these sentences (Martin, Haxby et al. 1995; Damasio, Grabowski et al. 2001; Kable, Kan et al. 2005). The left inferior parietal activity may have reflected participants’ attempts to relate the action described by the verb with the functional properties or visuomotor affordances (Gibson 1979; Glenberg and Kaschak 2002) of the Agents executing these actions (Chao and Martin 2000; Buccino, Binkofski et al. 2001; Damasio, Grabowski et al. 2001; Fogassi, Ferrari et al. 2005). Activity within bilateral mid-dorsolateral prefrontal cortex may have reflected the computation of functional relationships between Agents and Action that were more complex, e.g. evaluating whether inanimate agents possessed the goals and motivation to carry out the actions described by the verb (see Ruby et al. 2002 for a similar interpretation). Finally, activity within the bilateral basal ganglia that are also known play a role in selecting and sequencing action (Vakil, Kahan et al. 2000) and that may be a source of the P600 effect produced by semantic-thematic violations (Kotz, Frisch et al. 2003), may have reflected participants’ attempts to repair these sentences, see Stowe et al. (2004) for a similar interpretation of basal ganglia function with respect to repairing syntactically ambiguous sentences.
Activity to morphosyntactic violations
The network activated in association with the morphosyntactically violated sentences (relative to the non-violated and real-world pragmatically violated sentences) overlapped largely with that activated by the semantically-thematically violated sentences. We offer several possible functional explanations for the common activation of this network in the next section. At this stage, we note that the increased activity to morphosyntactic violations (relative to non-violated sentences) within posterior inferior frontal and premotor cortices, as well as within the left middle frontal gyrus, is consistent with the pattern of activity to agreement violations reported by Ni et al. (2000). It also partially replicates the findings of our previous study using similar stimuli in which we reported the most robust activity to morphosyntactically violated sentences (relative to non-violated and real-world pragmatically violated sentences) within parietal cortices (Kuperberg, Holcomb et al. 2003)4. Activity within the basal ganglia is consistent with data from patients with subcortical lesions and with Parkinson’s disease that have implicated this region as a source of the P600 evoked by other types of syntactic violations (Friederici, von Cramon et al. 1999; Friederici, Kotz et al. 2003). It also accords with previous neuroimaging and patient studies that have associated the basal ganglia with aspects of syntactic processing (Ullman 2001; Friederici and Kotz 2003).
Overlap of hemodynamic activity to animacy and syntactic violations
The overlapping pattern of hemodynamic activity to the animacy thematic-semantically violated and morphosyntactically violated sentences, and its neuroanatomical distinction from activity to the real-world pragmatically violated sentences, is particularly interesting as it parallels the pattern of previous ERP findings to the same types of anomalies: both produce P600 effects, as opposed to real-world pragmatic violations that produce N400 effects (Kuperberg, Caplan et al. 2006). In the ERP literature, explanations for why a P600 effect is evoked by both these types of anomalies vary in the degree to which the underlying neurocognitive processes are considered domain-general or specific to a particular level of linguistic analysis. They also vary in the degree to which they emphasize neurocognitive processes engaged upon anomaly detection versus continued analysis or reanalysis occurring as a consequence of anomaly detection. Below we consider some of these neurocognitive processes in relation to the neuroanatomical regions activated to these types of anomalies. However, it is important to note that these accounts are not mutually exclusive of one another. In addition, although it is tempting to infer that these hemodynamic responses reflect the underlying neural sources of the P600, just as in our previous parallel ERP/fMRI investigation (Kuperberg, Holcomb et al. 2003), we are very cautious about such an interpretation. This is because ERPs and fMRI index neural activity at very different time scales. As discussed above, BOLD activity reflected neural activity not only to the critical words but also additional neurocognitive processes occurring after the critical words that are not reflected by ERPs.
One explanation for why the semantic-thematic and morphosyntactic violations engaged overlapping regions (relative to the other sentence types) is that, as discussed above, they were both more easily detectable at the point of the critical word. Although the measured hemodynamic response integrated the neural responses across multiple words within the sentence, the immediate detection of the semantic-thematic and syntactic anomalies at the point of the critical word may have captured attention to a greater degree than to the non-violated or real-world pragmatically violated sentences and this attentional shift may have driven some of the hemodynamic response to these sentence types. Specifically, the increased recruitment of the parietal cortex to both types of anomalies may have reflected the known role of this region in allocating attention to popout or novel events (Ipata, Gee et al. 2006) and perhaps updating such events with their surrounding context. This is similar to explanations of neurocognitive processes proposed to underlie the well-known P300 event-related potential (Donchin and Coles 1988) that may be functionally linked to the P600 (Coulson, King et al. 1998; but see Osterhout, McKinnon et al. 1996 and Osterhout and Hagoort 1999) and that is likely to reflect activity within a widespread network, including some of the parietal and prefrontal regions activated here (Daffner, Scinto et al. 2003; Bledowski, Prvulovic et al. 2004)
Another explanation is that the network engaged to the animacy semantic-thematic and morphosyntactic anomalies reflected the detection of conflict. As outlined in the Introduction, the P600 effect may be triggered by a temporary conflict between the output of more than one parallel processing stream (Vissers, Chwilla et al. 2006; Kuperberg 2007). Kolk, Chwilla and colleagues have hypothesized that this detection of conflict is mediated by a non-domain-specific general monitoring network that, during language comprehension, serves the function of preventing comprehension errors (Kolk, Chwilla et al. 2003; Vissers, Chwilla et al. 2006; Kolk and Chwilla 2007). They have proposed that such monitoring is similar or analogous to the types of conflict monitoring that are thought to occur in the action domain and that lead to the production of an error-related negativity ERP component and increased activity within the anterior cingulate cortex (van Veen and Carter 2006; Taylor, Stern et al. 2007). In addition, they have also suggested that activity within the left inferior frontal gyrus may mediate such a general monitoring function, again linking this to its possible role in the resolution of conflict and selection amongst competing alternatives (Novick, Trueswell et al. 2005). Our findings are only partially consistent with this theory: the syntactically violated sentences (relative to non-violated sentences) did engage the anterior cingulate cortex, but the animacy thematically violated sentences (relative to the non-violated and real-world violated sentences) did not recruit this region. Both morphosyntactically and animacy thematic violations recruited the posterior (but not the anterior) portion of the left inferior frontal gyrus.
Other explanations for why overlapping neural systems were engaged to the animacy semantic-thematically and morphosyntactically violated sentences (relative to the other sentence types) focus on the types of continued analyses or reanalyses processes engaged after anomaly detection. Kolk et al.’s monitoring hypothesis holds that such reanalysis is very general in nature, involving all aspects of language processing: phonological, orthographic, semantic, and syntactic (Vissers, Chwilla et al. 2006; Kolk and Chwilla 2007). It is possible that some of the activity observed in association with these violations reflected such a general reanalysis, although this account might predict even more widespread activity to that observed here. In order to test this hypothesis more directly, one would need to determine whether the same network is activated by other types of violations that lead to conflicts between alternative representations, such as orthographic violations within highly predictable sentences (Vissers, Chwilla et al. 2006).
It may be that the regions activated to the animacy semantic-thematic and morphosyntactic violations reflected more specific reanalysis processes that focused on more specific levels of language processing. Above we considered the possibility that encountering the animacy semantic-thematic violations led to an increased or prolonged analysis that was based partially on semantic-thematic constraints related to the action described by the verb and the semantic properties of its NP Agent argument. It is possible that, like animacy semantic-thematic violations, the inflectional morphosyntactic violations also triggered such a semantic-thematic analysis. This explanation is similar to a proposal by Kaan and colleagues (Kaan, Harris et al. 2000) who suggested that participants may attempt to semantically-thematically integrate a morphosyntactically violated verb with the subject of a sentence, but that such integration is rendered more difficult, if not impossible, because of the grammatical number mismatch; see also discussion by Kuperberg et al. (2006). From a theoretical perspective, a link between processing aspects of syntax and processing semantic-thematic relationships would be interesting as the latter lies at the semantic/syntactic interface: although semantic in nature, thematic relationships, based on properties such as animacy, represent the part of the semantic system that has implications for combination of words around a verb (Pinker 1989; Goldberg 1995; Levin and Rappaport Hovav 2005).
Interestingly, as in our previous study, some of the regions modulated by both animacy semantic-thematic and morphosyntactic violations (relative to other sentence types) overlapped with those that showed ‘deactivation’ when the sentences were compared with the low-level fixation condition (see Figure 1)5. The precise significance of this observation remains unclear but it suggests that the types of neurocognitive processes mediating either a more specific semantic-thematic analysis/reanalysis or a more general detection, monitoring and reanalysis, may be suppressed during the first half of the sentence and only come into play upon encountering these anomalies.
All violations versus non-violated sentences
All three types of violated sentences engaged anterior ventro-medial fusiform cortices, relative to the non-violated sentences. There has been much speculation about what role this region plays during language comprehension. It has been touted as playing a particular role in storing and activating semantic associative, categorical and contextual relationships within semantic memory (Mummery, Shallice et al. 1999; Rossell, Price et al. 2003; Bar 2004; Wheatley, Weisberg et al. 2005; Gold, Balota et al. 2006; Kuperberg, Lakshmanan et al. 2007). It may be one source of the N400 effect as indicated by some intracranial electrode studies (McCarthy, Nobre et al. 1995; Nobre and McCarthy 1995). Other studies, however, have emphasized the involvement of anterior temporal cortices more specifically in sentence comprehension beyond the single word (Mazoyer, Tzourio et al. 1993). and some have suggested that parts of this overall region may even be specialized for aspects of syntactic processing (Dronkers, Wilkins et al. 1994; Humphries, Love et al. 2005; Stowe, Haverkort et al. 2005). The ventral anterior temporal cortex may therefore be sensitive to multiple types of linguistic information rather than only one type, and be involved in integrating these multiple sources of information together to construct higher order propositional meaning. Indeed, its common engagement to real-world pragmatic, morphosyntactic and semantic-thematic violations in the present study is consistent with our findings in an early blocked-design study in which we demonstrate that the ventral temporal fusiform cortex was the only region commonly activated to real-world pragmatic, selection-restriction and subcategorization violations between verbs and object NPs, each in comparison with non-violated sentences (Kuperberg, McGuire et al. 2000).
Summary and conclusions
We have demonstrated similarities in the pattern of hemodynamic response to animacy semantic-thematic violations and morphosyntactic violations between verbs and subject NP arguments, and have shown that this response is distinct from that engaged to real-world pragmatic violations of these verbs. This parallels the overall pattern of ERP findings seen to the same types of violation – a P600 effect to animacy semantic-thematic and syntactic violations, but an N400 effect to real-world pragmatic violations (Kuperberg 2007). We have suggested that the left anterior inferior frontal cortex to real-world pragmatic violations acts to search, retrieve and integrate real-world knowledge into an incoming sentence representation. In contrast, we have discussed several possible non-mutually exclusive neurocognitive processes that may be reflected by activity within a widespread frontal/inferior parietal/basal ganglia network to both semantic-thematic and morphosyntactic violations. Most specifically, we have suggested that such activity may, in part, reflect a continued thematic-semantic analysis or reanalysis of such sentences that may involve attempts to relate the action described by the verb to the affordances, goals and other relevant semantic properties of its Agent NP arguments. In keeping with the ERP literature, our assumption is that both the neurocognitive processes of semantic memory-based integration and semantic-thematic prolonged analysis or reanalysis are particularly taxed when their respective relationships are violated. This, however, is an empirical question. Future studies will determine whether the same regions are also engaged at points of processing difficulty during the processing of non-violated sentences (of note, both N400s and P600s are indeed evoked at points of semantic unexpectancy and syntactic ambiguity respectively during normal sentence processing).
Our findings indicate that, at a neuroanatomical level, the brain respects a distinction between two different aspects of meaning: the semantic relationship between the propositional content of a sentence and our knowledge about what is likely in the real world, and the semantic-thematic relationship between a verb and its arguments within a sentence. Thus, at a neural level, a division within the semantic system into components that are relevant and irrelevant for action (Martin, Haxby et al. 1995) appears to play out during sentence processing. Most intriguingly, these data are consistent with ERP findings in suggesting that violations of semantic-thematic relationships, based on animacy constraints, between a verb and its preceding Agent argument are treated by the brain as similar to morphosyntactic inflectional violations between a verb and its subject NP argument (Kuperberg, Caplan et al. 2006; Kuperberg 2007).
Acknowledgments
This research was supported by NIMH (R01 MH071635) and the Institute for Mental Illness and Neuroscience Discovery (MIND). Gina R Kuperberg was also supported by a NARSAD (with the Sidney Baer Trust) and a Claflin Distinguished Scholars Award from Mass. General Hospital. We thank Marianna Eddy and Fujiro Ozawa for their help in data acquisition, and David Caplan, Mante Nieuwland and Tali Ditman for their comments on the manuscript.
Footnotes
The term ‘pragmatic’ is used to describe these violations following Marslen-Wilson, Brown et al. (1988) and Tyler (1992) who originally discussed violations of real-world knowledge in these terms, and in order to be consistent in terminology with our previous studies using the same types of sentences (Kuperberg, Holcomb et al. 2003; Kuperberg, Sitnikova et al. 2003; Kuperberg, Caplan et al. 2006; Kuperberg, Kreher et al. 2006; Kuperberg, Sitnikova et al. 2006; Kuperberg, Kreher et al. 2007; Kuperberg 2007). However, this is an umbrella use of the term, ‘pragmatic’, referring to the fact that such real-world knowledge is situationally based and context dependent, rather than traditionally ‘linguistic’. Note that this descriptive and general use of the term is distinct from ‘pragmatics’ when used to refer to the relationship between sentence meaning and speaker’s meaning and/or to refer to a variety of phenomena at the level of discourse.
Based on a series of ERP studies, we have suggested that such prolonged semantic-thematic processing is triggered by a conflict between the actual syntactic assignment of the subject NP (“eggs”) to the role of Agent at the point of the verb, and two types of semantic processes: (a) a semantic-thematic process (either heuristic or combinatorial) that initially assigns the role of Theme to the subject NP “eggs” based on its inanimate properties (Kuperberg, Sitnikova et al. 2003; Kuperberg, Caplan et al. 2006; Kuperberg, Kreher et al. 2007; Kuperberg 2007), and (b) a semantic memory-based process that recognizes the close semantic associative relationship between “eggs” and “eat” (Kim and Osterhout 2005; Kuperberg, Caplan et al. 2006; van Herten, Chwilla et al. 2006; Kuperberg 2007).
Although, we cannot exclude the possibility that some of this motor activity reflected participants’ button-press responses, it seems unlikely that such motor responses can explain all differential activity within these regions. First, participants also made motor responses to the non-violated and pragmatically violated sentences, but activity within motor cortices was significantly less to both these sentence types than to the animacy and syntactically violated sentences. Second, we modeled hemodynamic activity to each of the four sentence types from the onset of the critical verb until the end of the sentence; the decision itself was modeled separately for each trial. It will, however, be important to determine whether these findings can be replicated as participants simply read the sentences without making any judgments.
In this previous investigation, the posterior inferior frontal gyrus, left premotor cortex and the left posterior occipito-temporal cortex were also activated to morphosyntactically violated sentences relative to non-violated sentences at subthreshold levels of significance, as shown in Figure 6 in G.R. Kuperberg et al. (2003a). One reason for the discrepancy in findings may be the method of analysis used: in our previous study, we focused on the modulation of the hemodynamic response at and straight after its peak. By focusing in on activity at particular time points, we may have diminished power to capture activity within regions that were not modulated precisely at these time points. In the present study, we modeled the data from the onset of the critical word until end of the sentence using a hemodynamic response function. We also averaged data across participants using a random effects model, rather than a fixed effects model (as in our previous study), perhaps making our current findings more generalizable.
In our previous study in which the trials followed on from one another at a fixed rate, we considered the possibility that differential degrees of deactivation may have reflected the relatively longer periods of rest in between trials, arising because participants were generally fastest to make acceptability judgments to the morphosyntactically violated sentences (G. R. Kuperberg et al., 2003a). This relatively trivial explanation for differential degrees of deactivation seems less likely to account for the pattern of findings in the current study as participants’ acceptability decisions moved them on to subsequent trials, eliminating any rest periods in between trials.
References
- Alexander MP, Hiltbrunner B, Fischer RS. Distributed anatomy of transcortical sensory aphasia. Archives of Neurology. 1989;46:885–892. doi: 10.1001/archneur.1989.00520440075023. [DOI] [PubMed] [Google Scholar]
- Arbib MA. From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics. Behav Brain Sci. 2005;28(2):105–24. doi: 10.1017/s0140525x05000038. discussion 125–67. [DOI] [PubMed] [Google Scholar]
- Bak TH, O’Donovan DG, Xuereb JH, Boniface S, Hodges JR. Selective impairment of verb processing associated with pathological changes in Brodmann areas 44 and 45 in the motor neuron disease-dementia-aphasia syndrome. Brain. 2001;124(Pt 1):103–20. doi: 10.1093/brain/124.1.103. [DOI] [PubMed] [Google Scholar]
- Bar M. Visual objects in context. Nat Rev Neurosci. 2004;5(8):617–629. doi: 10.1038/nrn1476. [DOI] [PubMed] [Google Scholar]
- Bentin S, McCarthy G, Wood CC. Event-related potentials, lexical decision and semantic priming. Electroencephalography and Clinical Neurophysiology. 1985;60:343–355. doi: 10.1016/0013-4694(85)90008-2. [DOI] [PubMed] [Google Scholar]
- Bledowski C, Prvulovic D, Hoechstetter K, Scherg M, Wibral M, Goebel R, Linden DE. Localizing P300 generators in visual target and distractor processing: a combined event-related potential and functional magnetic resonance imaging study. J Neurosci. 2004;24(42):9353–60. doi: 10.1523/JNEUROSCI.1897-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bornkessel I, Zysset S, Friederici AD, von Cramon DY, Schlesewsky M. Who did what to whom? The neural basis of argument hierarchies during language comprehension. Neuroimage. 2005;26(1):221–33. doi: 10.1016/j.neuroimage.2005.01.032. [DOI] [PubMed] [Google Scholar]
- Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ. Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur J Neurosci. 2001;13(2):400–4. [PubMed] [Google Scholar]
- Burock MA, Buckner RL, Woldorff MG, Rosen BR, Dale AM. Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport. 1998;9(16):3735–3739. doi: 10.1097/00001756-199811160-00030. [DOI] [PubMed] [Google Scholar]
- Burock MA, Dale AM. Estimation and detection of event-related fMRI signals with temporally correlated noise: a statistically efficient and unbiased approach. Human Brain Mapping. 2000;11(4):249–260. doi: 10.1002/1097-0193(200012)11:4<249::AID-HBM20>3.0.CO;2-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chao LL, Haxby JV, Martin A. Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nat Neurosci. 1999;2(10):913–9. doi: 10.1038/13217. [DOI] [PubMed] [Google Scholar]
- Chao LL, Martin A. Representation of manipulable man-made objects in the dorsal stream. Neuroimage. 2000;12(4):478–84. doi: 10.1006/nimg.2000.0635. [DOI] [PubMed] [Google Scholar]
- Chomsky N. Lectures on Government and Binding. Dordrecht: Foris; 1981. [Google Scholar]
- Collins DL, Neelin P, Peters TM, Evans AC. Automatic 3D intersubject registration of MR volumetric data in standardized Talairach space. J Comput Assist Tomogr. 1994;18(2):192–205. [PubMed] [Google Scholar]
- Coulson S, King J, Kutas M. Expect the unexpected: Event-related brain responses to morphosyntactic violations. Language and Cognitive Processes. 1998;13:21–58. [Google Scholar]
- Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Computational Biomedical Research. 1996;29(3):162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
- Daffner KR, Scinto LF, Weitzman AM, Faust R, Rentz DM, Budson AE, Holcomb PJ. Frontal and parietal components of a cerebral network mediating voluntary attention to novel events. J Cogn Neurosci. 2003;15(2):294–313. doi: 10.1162/089892903321208213. [DOI] [PubMed] [Google Scholar]
- Dale AM. Optimal Experimental Design for Event-Related fMRI. Human Brain Mapping. 1999;8:109–114. doi: 10.1002/(SICI)1097-0193(1999)8:2/3<109::AID-HBM7>3.0.CO;2-W. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis. I. Segmentation and surface reconstruction. Neuroimage. 1999;9(2):179–194. doi: 10.1006/nimg.1998.0395. [DOI] [PubMed] [Google Scholar]
- Dale AM, Sereno MI. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: A linear approach. Journal of Cognitive Neuroscience. 1993;5:162–176. doi: 10.1162/jocn.1993.5.2.162. [DOI] [PubMed] [Google Scholar]
- Damasio H, Grabowski TJ, Tranel D, Ponto LL, Hichwa RD, Damasio AR. Neural correlates of naming actions and of naming spatial relations. Neuroimage. 2001;13(6 Pt 1):1053–64. doi: 10.1006/nimg.2001.0775. [DOI] [PubMed] [Google Scholar]
- De Vincenzi M, Job R, Di Matteo R, Angrilli A, Penolazzi B, Ciccarelli L, Vespignani F. Differences in the perception and time course of syntactic and semantic violations. Brain and Language. 2003;85:280–296. doi: 10.1016/s0093-934x(03)00055-5. [DOI] [PubMed] [Google Scholar]
- Deacon D, Grose-Fifer J, Yang CM, Stanick V, Hewitt S, Dynowska A. Evidence for a new conceptualization of semantic representation in the left and right cerebral hemispheres. Cortex. 2004;40(3):467–78. doi: 10.1016/s0010-9452(08)70140-0. [DOI] [PubMed] [Google Scholar]
- di Pellegrino G, Fadiga L, Fogassi L, Gallese V, Rizzolatti G. Understanding motor events: a neurophysiological study. Exp Brain Res. 1992;91(1):176–80. doi: 10.1007/BF00230027. [DOI] [PubMed] [Google Scholar]
- Ditman T, Holcomb PJ, Kuperberg GR. An investigation of concurrent ERP and self-paced reading methodologies. Psychophysiology. 2007 doi: 10.1111/j.1469-8986.2007.00593.x. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doherty CP, West WC, Dilley LC, Shattuck-Hufnagel S, Caplan D. Question/statement judgments: an fMRI study of intonation processing. Human Brain Mapping. 2004;23(2):85–98. doi: 10.1002/hbm.20042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Donchin E, Coles MGH. Is the P300 component a manifestation of context updating? Behavioral and Brain Science. 1988;11:355–372. [Google Scholar]
- Dowty D. On the semantic content of the notion of thematic role. In: Cherchia G, Partee B, Turner R, editors. Properties, Types and meaning. Kluwer; 1989. [Google Scholar]
- Dronkers NF, Wilkins DP, Van Valin RDJ, Redern BB, Jaeger JJ. A reconsideration of the brain areas involved in the disruption of morphosyntactic comprehension. Brain and Language. 1994;47:461–462. [Google Scholar]
- Federmeier KD, Kutas M. A rose by any other name: long-term memory structure and sentence processing. Journal of Memory and Language. 1999;41:469–495. [Google Scholar]
- Fischer MH, Zwaan RA. Embodied language – a review of the role of the motor system In language comprehension. Quaterly Journal of Experimental Psychology. 2007 doi: 10.1080/17470210701623605. In press. [DOI] [PubMed] [Google Scholar]
- Fischl B, Liu A, Dale AM. Automated Manifold Surgery: Constructing Geometrically Accurate and Topologically Correct Models of the Human Cerebral Cortex. IEEE Transactions on Medical Imaging. 2001;20(1):70–80. doi: 10.1109/42.906426. [DOI] [PubMed] [Google Scholar]
- Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage. 1999;9(2):195–207. doi: 10.1006/nimg.1998.0396. [DOI] [PubMed] [Google Scholar]
- Fischl B, Sereno MI, Tootell RB, Dale AM. High-resolution intersubject averaging and a coordinate system for the cortical surface. Human Brain Mapping. 1999;8(4):272–284. doi: 10.1002/(SICI)1097-0193(1999)8:4<272::AID-HBM10>3.0.CO;2-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fodor JD, Ni W, Crain S, Shankweiler D. Tasks and timing in the perception of linguistic anomaly. Journal of Psycholinguistic Research. 1996;25:25–57. doi: 10.1007/BF01708419. [DOI] [PubMed] [Google Scholar]
- Fogassi L, Ferrari PF, Gesierich B, Rozzi S, Chersi F, Rizzolatti G. Parietal lobe: from action organization to intention understanding. Science. 2005;308(5722):662–7. doi: 10.1126/science.1106138. [DOI] [PubMed] [Google Scholar]
- Friederici A, Weissenbor Y. Brain Research. 2007. Mapping sentence form onto meaning: The syntax-semantic interface. [DOI] [PubMed] [Google Scholar]
- Friederici AD, Kotz SA. The brain basis of syntactic processes: functional imaging and lesion studies. Neuroimage. 2003;20(Suppl 1):S8–17. doi: 10.1016/j.neuroimage.2003.09.003. [DOI] [PubMed] [Google Scholar]
- Friederici AD, Kotz SA, Werheid K, Hein G, von Cramon DY. Syntactic comprehension in Parkinson’s disease: investigating early automatic and late integrational processes using event-related brain potentials. Neuropsychology. 2003;17(1):133–42. [PubMed] [Google Scholar]
- Friederici AD, Ruschemeyer SA, Hahne A, Fiebach CJ. The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. Cereb Cortex. 2003;13(2):170–7. doi: 10.1093/cercor/13.2.170. [DOI] [PubMed] [Google Scholar]
- Friederici AD, von Cramon DY, Kotz SA. Language related brain potentials in patients with cortical and subcortical left hemisphere lesions. Brain. 1999;122(Pt 6):1033–47. doi: 10.1093/brain/122.6.1033. [DOI] [PubMed] [Google Scholar]
- Friston KJ, Fletcher P, Josephs O, Holmes A, Rugg MD, Turner R. Event-related fMRI: characterizing differential responses. Neuroimage. 1998;7(1):30–40. doi: 10.1006/nimg.1997.0306. [DOI] [PubMed] [Google Scholar]
- Gallese V, Fadiga L, Fogassi L, Rizzolatti G. Action recognition in the premotor cortex. Brain. 1996;119(Pt 2):593–609. doi: 10.1093/brain/119.2.593. [DOI] [PubMed] [Google Scholar]
- Gibson JJ. The ecological approach to visual perception. New York: Houghton Mifflin; 1979. [Google Scholar]
- Glenberg AM, Kaschak MP. Grounding language in action. Psychon Bull Rev. 2002;9(3):558–65. doi: 10.3758/bf03196313. [DOI] [PubMed] [Google Scholar]
- Gold BT, Balota DA, Jones SJ, Powell DK, Smith CD, Andersen AH. Dissociation of automatic and strategic lexical-semantics: functional magnetic resonance imaging evidence for differing roles of multiple frontotemporal regions. J Neurosci. 2006;26(24):6523–32. doi: 10.1523/JNEUROSCI.0808-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldberg AE. Constructions: A construction grammar approach to argument structure. Chicago: Univ. of Chicago Press; 1995. [Google Scholar]
- Grose-Fifer J, Deacon D. Priming by natural category membership in the left and right cerebral hemispheres. Neuropsychologia. 2004;42(14):1948–60. doi: 10.1016/j.neuropsychologia.2004.04.024. [DOI] [PubMed] [Google Scholar]
- Hagoort P, Brown C, Groothusen J. The syntactic positive shift SPS as an ERP measure of syntactic processing. Language and cognitive processes. In: Garnsey SM, editor. Special Issue: Event-related brain potentials in the study of language. Vol. 84. Hove: Lawrence Erlbaum Associates; 1993. pp. 439–483. [Google Scholar]
- Hagoort P, Hald L, Bastiaansen M, Petersson KM. Integration of word meaning and world knowledge in language comprehension. Science. 2004;304(5669):438–41. doi: 10.1126/science.1095455. [DOI] [PubMed] [Google Scholar]
- Halgren E, Dhond RP, Christensen N, Van Petten C, Marinkovic K, Lewine JD, Dale AM. N400-like magnetoencephalography responses modulated by semantic context, word frequency, and lexical class in sentences. Neuroimage. 2002;17(3):1101–1116. doi: 10.1006/nimg.2002.1268. [DOI] [PubMed] [Google Scholar]
- Hauk O, Johnsrude I, Pulvermuller F. Somatotopic representation of action words in human motor and premotor cortex. Neuron. 2004;41(2):301–7. doi: 10.1016/s0896-6273(03)00838-9. [DOI] [PubMed] [Google Scholar]
- Hoeks JCJ, Stowe LA, Doedens G. Seeing words in context: the interaction of lexical and sentence level information during reading. Cognitive Brain Research. 2004;19:59–73. doi: 10.1016/j.cogbrainres.2003.10.022. [DOI] [PubMed] [Google Scholar]
- Humphries C, Love T, Swinney D, Hickok G. Response of anterior temporal cortex to syntactic and prosodic manipulations during sentence processing. Hum Brain Mapp. 2005;26(2):128–38. doi: 10.1002/hbm.20148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ipata AE, Gee AL, Gottlieb J, Bisley JW, Goldberg ME. LIP responses to a popout stimulus are reduced if it is overtly ignored. Nat Neurosci. 2006;9(8):1071–6. doi: 10.1038/nn1734. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jackendoff R. Grammar as evidence for conceptual structure. In: Halle M, Bresnan J, Miller G, editors. Linguistic theory and psychological reality. Cambridge, MA: MIT Press; 1978. pp. 201–228. [Google Scholar]
- Jackendoff R. The status of thematic relations in liinguistic theory. Linguistic Inquiry. 1987;18(3):369–411. [Google Scholar]
- Kaan E, Harris A, Gibson E, Holcomb P. The P600 as an index of syntactic integration difficulty. Language and Cognitive Processes. 2000;15(2):159–201. [Google Scholar]
- Kable JW, I, Kan P, Wilson A, Thompson-Schill SL, Chatterjee A. Conceptual representations of action in the lateral temporal cortex. J Cogn Neurosci. 2005;17(12):1855–70. doi: 10.1162/089892905775008625. [DOI] [PubMed] [Google Scholar]
- Kamide Y, Altmann GTM, Haywood SL. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language. 2003;49:133–156. [Google Scholar]
- Kemmerer D. Action verbs, argument structure constructions, and the mirror neuron system. In: Arbib MA, editor. Action to Language via the Mirror Neuron System. Cambridge: Cambridge University Press; 2006. pp. 347–374. [Google Scholar]
- Kiehl KA, Laurens KR, Liddle PF. Reading anomalous sentences: an event-related fMRI study of semantic processing. Neuroimage. 2002;17(2):842–850. [PubMed] [Google Scholar]
- Kim A, Osterhout L. The independence of combinatory semantic processing: Evidence from event-related potentials. Journal of Memory and Language. 2005;52:205–225. [Google Scholar]
- Kolk HH, Chwilla DJ. Late Positivities in unusual situations: A commentary to (a) Kuperberg, Kreher, Sitnikova, Caplan and Holcomb and (b) Kemmerer, Weber-Fox, Price, Zdanczyk and Way. Brain and Language. 2007;100(3):257–262. doi: 10.1016/j.bandl.2006.07.006. [DOI] [PubMed] [Google Scholar]
- Kolk HH, Chwilla DJ, van Herten M, Oor PJ. Structure and limited capacity in verbal working memory: a study with event-related potentials. Brain Lang. 2003;85(1):1–36. doi: 10.1016/s0093-934x(02)00548-5. [DOI] [PubMed] [Google Scholar]
- Kotz SA, Frisch S, von Cramon DY, Friederici AD. Syntactic language processing: ERP lesion data on the role of the basal ganglia. J Int Neuropsychol Soc. 2003;9(7):1053–1060. doi: 10.1017/S1355617703970093. [DOI] [PubMed] [Google Scholar]
- Kuperberg G, Caplan D, Sitnikova T, Eddy M, Holcomb P. Neural correlates of processing syntactic, semantic and thematic relationships in sentences. Language and Cognitive Processes. 2006;21(5):489–530. [Google Scholar]
- Kuperberg G, Kreher DA, Sitnikova T, Caplan D, Holcomb PJ. The role of animacy and thematic relationships in processing active English sentences: Evidence from event-related potentials. Brain and Language. 2007;100(3):223–238. doi: 10.1016/j.bandl.2005.12.006. [DOI] [PubMed] [Google Scholar]
- Kuperberg GR. Neural mechanisms of language comprehension: Challenges to syntax. Brain Res. 2007;1146:23–49. doi: 10.1016/j.brainres.2006.12.063. [DOI] [PubMed] [Google Scholar]
- Kuperberg GR, Holcomb PJ, Sitnikova T, Greve D, Dale AM, Caplan D. Distinct patterns of neural modulation during the processing of conceptual and syntactic anomalies. Journal of Cognitive Neuroscience. 2003;15(2):272–293. doi: 10.1162/089892903321208204. [DOI] [PubMed] [Google Scholar]
- Kuperberg GR, Kreher DA, Goff D, McGuire PK, David AS. Building up linguistic context in schizophrenia: evidence from self-paced reading. Neuropsychology. 2006;20(4):442–52. doi: 10.1037/0894-4105.20.4.442. [DOI] [PubMed] [Google Scholar]
- Kuperberg GR, Lakshmanan BM, Greve DN, West WC. Task and semantic relationship influence both the polarity and localization of hemodynamic modulation during lexico-semantic processing. Human Brain Mapping. 2007 doi: 10.1002/hbm.20419. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kuperberg GR, McGuire PK, Bullmore ET, Brammer MJ, Rabe-Hesketh S, Wright IC, Lythgoe DJ, Williams SC, David AS. Common and distinct neural substrates for pragmatic, semantic, and syntactic processing of spoken sentences: an fMRI study. J Cogn Neurosci. 2000;12(2):321–341. doi: 10.1162/089892900562138. [DOI] [PubMed] [Google Scholar]
- Kuperberg GR, Sitnikova T, Caplan D, Holcomb PJ. Electrophysiological distinctions in processing conceptual relationships within simple sentences. Cogn Brain Res. 2003;17(1):117–129. doi: 10.1016/s0926-6410(03)00086-7. [DOI] [PubMed] [Google Scholar]
- Kuperberg GR, Sitnikova T, Goff D, Holcomb PJ. Making sense of sentences in schizophrenia: electrophysiological evidence for abnormal interactions between semantic and syntactic processing. Journal of Abnorm Psychol. 2006;115(2):243–256. doi: 10.1037/0021-843X.115.2.251. [DOI] [PubMed] [Google Scholar]
- Kutas M, Federmeier KD. Electrophysiology reveals semantic memory use in language comprehension. Trends Cogn Sci. 2000;4(12):463–470. doi: 10.1016/s1364-6613(00)01560-6. [DOI] [PubMed] [Google Scholar]
- Kutas M, Hillyard SA. Reading senseless sentences: Brain potential reflect semantic incongruity. Science. 1980;207:203–205. doi: 10.1126/science.7350657. [DOI] [PubMed] [Google Scholar]
- Kutas M, Hillyard SA. Brain potentials during reading reflect word expectancy and semantic association. Nature. 1984;307:161–163. doi: 10.1038/307161a0. [DOI] [PubMed] [Google Scholar]
- Levin B. English Verb Classes And Alternations: A Preliminary Investigation. The University of Chicago Press; 1993. [Google Scholar]
- Levin B, Rappaport Hovav M. Argument Realization. Cambridge: Cambridge University Press; 2005. [Google Scholar]
- Marslen-Wilson WD, Brown C, Tyler LK. Lexical representations in language comprehension. Language and Cognitive Processes. 1988;3:1–17. [Google Scholar]
- Martin A, Chao LL. Semantic memory and the brain: structure and processes. Curr Opin Neurobiol. 2001;11(2):194–201. doi: 10.1016/s0959-4388(00)00196-3. [DOI] [PubMed] [Google Scholar]
- Martin A, Haxby JV, Lalonde FM, Wiggs CL, Ungerleider LG. Discrete cortical regions associated with knowledge of color and knowledge of action. Science. 1995;270:102–105. doi: 10.1126/science.270.5233.102. [DOI] [PubMed] [Google Scholar]
- Mazoyer BM, Tzourio N, Frak V, Syrota A, Murayama O, Levrier O, Salamon G, Dehaene S, Cohen L, Mehler J. The cortical representation of speech. Journal of Cognitive Neuroscience. 1993;5:467–479. doi: 10.1162/jocn.1993.5.4.467. [DOI] [PubMed] [Google Scholar]
- McCarthy G, Nobre AC, Bentin S, Spencer DD. Language related field potentials in the anterior-medial temporal lobe: I. Intracranial distribution and neural generators. Journal of Neuroscience. 1995;15:1080–1089. doi: 10.1523/JNEUROSCI.15-02-01080.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moro A, Tettamanti M, Perani D, Donati C, Cappa SF, Fazio F. Syntax and the brain: disentangling grammar by selective anomalies. Neuroimage. 2001;13(1):110–8. doi: 10.1006/nimg.2000.0668. [DOI] [PubMed] [Google Scholar]
- Mummery CJ, Shallice T, Price CJ. Dual-process model in semantic priming: A functional imaging perspective. Neuroimage. 1999;9(5):516–525. doi: 10.1006/nimg.1999.0434. [DOI] [PubMed] [Google Scholar]
- Newman AJ, Pancheva R, Ozawa K, Neville HJ, Ullman MT. An event-related fMRI study of syntactic and semantic violations. J Psycholinguist Res. 2001;30(3):339–364. doi: 10.1023/a:1010499119393. [DOI] [PubMed] [Google Scholar]
- Ni W, Constable RT, Mencl WE, Pugh KR, Fulbright RK, Shaywitz SE, Shaywitz BA, Gore J. An event-related neuroimaging study distinguishing form and content in sentence processing. Journal of Cognitive Neuroscience. 2000;12:120–133. doi: 10.1162/08989290051137648. [DOI] [PubMed] [Google Scholar]
- Nobre AC, McCarthy G. Language-related field potentials in the anterior-medial temporal lobe: II. Effects of word type and semantic priming. Journal of Neuroscience. 1995;15:1090–1098. doi: 10.1523/JNEUROSCI.15-02-01090.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Novick JM, Trueswell JC, Thompson-Schill SL. Cognitive control and parsing: reexamining the role of Broca’s area in sentence comprehension. Cogn Affect Behav Neurosci. 2005;5(3):263–81. doi: 10.3758/cabn.5.3.263. [DOI] [PubMed] [Google Scholar]
- Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9(1):97–113. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
- Osterhout L, Hagoort P. A superficial resemblence does not necessarily mean you are part of the family: counterarguments to Coulson, King and Kutas (1998) in the P600/SPS-P300 debate. Language and Cognitive Processes. 1999;14:1–14. [Google Scholar]
- Osterhout L, Holcomb PJ. Event-related potentials elicited by syntactic anomaly. Journal of Memory and Language. 1992;31:785–806. [Google Scholar]
- Osterhout L, Holcomb PJ. Event-related potentials and language comprehension. In: Rugg MD, Coles MGH, editors. Electrophysiology of mind: Event-related brain potentials and cognition. Oxford University Press; 1995. [Google Scholar]
- Osterhout L, Kim A, Kuperberg GR. The Neurobiology of Sentence Comprehension. In: Spivey M, Joannisse M, McRae K, editors. The Cambridge Handbook of Psycholinguistics. Cambridge: Cambridge University Press; 2007. To appear in. [Google Scholar]
- Osterhout L, McKinnon R, Bersick M, Corey V. On the language specificity of the brain response to syntactic anomalies: is the syntactic positive shift a member of the P300 family? Journal of Cognitive Neuroscience. 1996;8:507–526. doi: 10.1162/jocn.1996.8.6.507. [DOI] [PubMed] [Google Scholar]
- Pinker S. Learnability and cognition. Cambridge; 1989. [Google Scholar]
- Rizzolatti G, Fogassi L, Gallese V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci. 2001;2(9):661–70. doi: 10.1038/35090060. [DOI] [PubMed] [Google Scholar]
- Rossell SL, Price CJ, Nobre AC. The anatomy and time course of semantic priming investigated by fMRI and ERPs. Neuropsychologia. 2003;41(5):550–564. doi: 10.1016/s0028-3932(02)00181-1. [DOI] [PubMed] [Google Scholar]
- Ruby P, Sirigu A, Decety J. Distinct areas in parietal cortex involved in long-term and short-term action planning: a PET investigation. Cortex. 2002;38(3):321–39. doi: 10.1016/s0010-9452(08)70663-4. [DOI] [PubMed] [Google Scholar]
- Rugg MD. The effects of semantic priming and word repetition on event-related potentials. Psychophysiology. 1985;22:642–647. doi: 10.1111/j.1469-8986.1985.tb01661.x. [DOI] [PubMed] [Google Scholar]
- Ruschemeyer SA, Fiebach CJ, Kempe V, Friederici AD. Processing lexical semantic and syntactic information in first and second language: fMRI evidence from German and Russian. Hum Brain Mapp. 2005;25(2):266–86. doi: 10.1002/hbm.20098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sitnikova T, Holcomb P, Kuperberg GR. Neurocognitive mechanisms of human comprehension. In: Shipley TF, Zacks JM, editors. Understanding Events: How Humans See, Represent, and Act on Events. Oxford University Press; 2007. In Press. [Google Scholar]
- Sitnikova T, Holcomb PJ, Kuperberg GR. Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. J Cogn Neurosci. 2007 doi: 10.1162/jocn.2008.20143. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sitnikova T, Kuperberg GR, Holcomb P. Semantic integration in videos of real-world events: an electrophysiological investigation. Psychophysiology. 2003;40:160–164. doi: 10.1111/1469-8986.00016. [DOI] [PubMed] [Google Scholar]
- Stowe LA, Haverkort M, Zwarts F. Rethinking the neurological basis of language. Lingua. 2005;115:997–1042. [Google Scholar]
- Stowe LA, Paans AM, Wijers AA, Zwarts F. Activations of “motor” and other non-language structures during sentence comprehension. Brain Lang. 2004;89(2):290–9. doi: 10.1016/S0093-934X(03)00359-6. [DOI] [PubMed] [Google Scholar]
- Talairach J, Tournoux P. Co-Planar Stereotaxic Atlas of the Human Brain: 3-dimensional proportional system: an approach to cerebral imaging. New York: Thieme Medical Publishers, Inc; 1988. [Google Scholar]
- Tanenhaus MK, Spivey-Knowlton MJ, Eberhard KM, Sedivy JC. Integration of visual and linguistic information in spoken language comprehension. Science. 1995;268(5217):1632–4. doi: 10.1126/science.7777863. [DOI] [PubMed] [Google Scholar]
- Taylor SF, Stern ER, Gehring WJ. Neural systems for error monitoring: recent findings and theoretical perspectives. Neuroscientist. 2007;13(2):160–72. doi: 10.1177/1073858406298184. [DOI] [PubMed] [Google Scholar]
- Thompson-Schill SL, D’Esposito M, Aguirre GK, Farah MJ. Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. Proceedings of the National Academy of Sciences U S A. 1997;94(26):14792–14797. doi: 10.1073/pnas.94.26.14792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thompson-Schill SL, D’Esposito M, Kan IP. Effects of repetition and competition on activity in left prefrontal cortex during word generation. Neuron. 1999;23(3):513–522. doi: 10.1016/s0896-6273(00)80804-1. [DOI] [PubMed] [Google Scholar]
- Tyler LK. Verb-argument structures. In: Tyler LK, editor. Spoken Language Comprehension: An Experimental Approach to Disordered and Normal Processing. London: MIT Press; 1992. pp. 107–124. [Google Scholar]
- Ullman MT. A neurocognitive perspective on language: the declarative/procedural model. Nat Rev Neurosci. 2001;2(10):717–726. doi: 10.1038/35094573. [DOI] [PubMed] [Google Scholar]
- Vakil E, Kahan S, Huberman M, Osimani A. Motor and non-motor sequence learning in patients with basal ganglia lesions: the case of serial reaction time (SRT) Neuropsychologia. 2000;38(1):1–10. doi: 10.1016/s0028-3932(99)00058-5. [DOI] [PubMed] [Google Scholar]
- Van Petten C, Luka BJ. Neural localization of semantic context effects in electromagnetic and hemodynamic studies. Brain Lang. 2006;97(3):279–93. doi: 10.1016/j.bandl.2005.11.003. [DOI] [PubMed] [Google Scholar]
- van Veen V, Carter CS. Error detection, correction, and prevention in the brain: a brief review of data and theories. Clin EEG Neurosci. 2006;37(4):330–5. doi: 10.1177/155005940603700411. [DOI] [PubMed] [Google Scholar]
- Vissers CT, Chwilla DJ, Kolk HH. Monitoring in language perception: The effect of misspellings of words in highly constrained sentences. Brain Res. 2006;1106(1):150–63. doi: 10.1016/j.brainres.2006.05.012. [DOI] [PubMed] [Google Scholar]
- Wagner AD, Pare-Blagoev EJ, Clark J, Poldrack RA. Recovering meaning: left prefrontal cortex guides controlled semantic retrieval. Neuron. 2001;31(2):329–338. doi: 10.1016/s0896-6273(01)00359-2. [DOI] [PubMed] [Google Scholar]
- Wheatley T, Weisberg J, Beauchamp MS, Martin A. Automatic priming of semantically related words reduces activity in the fusiform gyrus. Journal of Cognitive Neuroscience. 2005;17(12):1871–1885. doi: 10.1162/089892905775008689. [DOI] [PubMed] [Google Scholar]
- White K, Ashton R. Handedness assessment inventory. Neuropsychologia. 1976;14(2):261–264. doi: 10.1016/0028-3932(76)90058-0. [DOI] [PubMed] [Google Scholar]