Differentiable cortical networks for inferences concerning people's intentions versus physical causality

Abstract

Cortical activity associated with generating an inference was measured using fMRI. Participants read three‐sentence passages that differed in whether or not an inference needed to be drawn to understand them. The inference was based on either a protagonist's intention or a physical consequence of a character's action. Activation was expected in Theory of Mind brain regions for the passages based on protagonists' intentions but not for the physical consequence passages. The activation measured in the right temporo‐parietal junction was greater in the intentional passages than in the consequence passages, consistent with predictions from a Theory of Mind perspective. In contrast, there was increased occipital activation in the physical inference passages. For both types of passage, the cortical activity related to the reading of the critical inference sentence demonstrated a recruitment of a common inference cortical network. This general inference‐related activation appeared bilaterally in the language processing areas (the inferior frontal gyrus, the temporal gyrus, and the angular gyrus), as well as in the medial to superior frontal gyrus, which has been found to be active in Theory of Mind tasks. These findings are consistent with the hypothesis that component areas of the discourse processing network are recruited as needed based on the nature of the inference. A Protagonist monitoring and synthesis network is proposed as a more accurate account for Theory of Mind activation during narrative comprehension. Hum Brain Mapp, 2011. © 2010 Wiley‐Liss, Inc.

INTRODUCTION

Discourse comprehension includes components such as processing at the word‐, sentence‐, and intra‐sentence/discourse‐levels, but it also spans cognitive processing at higher/different levels: memory and inference generation [Haviland and Clark, 1974; Kintsch, 1988; Kuperberg et al., 2006; Myers and O'Brien, 1998; Myers et al., 2000]; problem solving [Noordman et al., 1992]; Theory of Mind [understanding the thoughts of another person, e.g., Castelli et al., 2002; Mason and Just, 2009]; perspective‐taking [Mano et al., 2009]; and social interpretation [Gernsbacher et al., 1998; Mitchell et al., 2006]. For many language researchers the prominence of a social neural network [for a review, see Van Overwalle, 2009] during reading comprehension has come as somewhat of a surprise. The dorsomedial prefrontal cortex, in particular, has been found to be active in many neuroimaging studies of connected text [Bottini et al., 1994; Eviatar and Just, 2006; Ferstl and von Cramon, 2001, 2002; Ferstl et al., 2005; Nichelli et al., 1995; Xu et al., 2005]. This region has also been found to activate in Theory of Mind tasks [Castelli et al., 2002; Gallagher and Frith, 2003; Greene et al., 2001; Martin and Weisberg, 2003; Moll et al., 2002]. Often, in social and developmental psychology, Theory of Mind refers to the ability to think about the mental states of another person [for a review see Saxe et al., 2004]; perhaps Theory of Mind‐related brain activation should have been anticipated for narratives because they do, after all, tell predominantly character‐based stories in which mental states are often central to the unfolding motivations and themes of the narrative. The ability to understand the goals and intentions of a protagonist as well as other characters in a story is essential for a reader to understand, enjoy, or learn from a story. Thus engagement of a social processing system, particularly a Theory of Mind cortical network, should be expected during the reading of any narrative.

The primary goal of this study was to determine the components of the social processing aspects of a discourse‐based cortical network and the conditions under which it differentially activates to support comprehension and inferences during reading. The social processing was contrasted with the comprehension of physical causality and its neural substrate. A second goal was to determine how inferences about physical causality affected the constituency of the cortical network activated during discourse processing. A third goal was to more generally inform a brain‐based theory of discourse processing. To meet these goals, the study investigated brain activity during the comprehension of narratives, contrasting the inference‐making when it was based on the intentions of characters versus when it was based on physical causality.

Neuroimaging techniques are particularly well suited to determining the participation of various subsets of the diverse cognitive processes that constitute discourse processing in a given comprehension episode. Discourse processing is a complex skill that serves as a good prototype for “general thinking.” It requires that many levels of processing apply to a cumulating inflow of organized stimuli, rendering an integrated representation of the text. Understanding a short text may not be very different than observing and understanding everyday events in the world; therefore, discourse processing should engage a set of general‐purpose cortical networks. In addition to the social processing aspects, there are many other component processes of discourse comprehension, such as lower‐level language processing at the word‐, sentence‐, and the intra‐sentence/discourse‐levels [Bookheimer, 2002; Price 2000]. Additionally, both working memory and long‐term memory are utilized in discourse processing [Haviland and Clark, 1974; Kintsch, 1988; Myers and O'Brien, 1998; Myers et al., 2000]. General problem solving processing may also be engaged [Noordman et al., 1992]. Moreover, individual differences or expertise in any of these abilities can affect discourse processing [Calvo, 2001; Linderholm, 2002]. All of these processes have neural signatures that are becoming increasingly identifiable.

Neuroimaging research suggests that approximately five Parallel Networks of Discourse Processing operate on figurative and meta‐sentence level information during discourse comprehension [Mason and Just, 2006]. These networks include a coarse semantic processing network (right middle and right superior temporal areas), a coherence monitoring network (bilateral dorsolateral prefrontal), a text integration network (left inferior frontal/left anterior temporal), a spatial imagery network (left dominant, bilateral intraparietal sulcus), and, most relevant for the current study, a network for interpreting a protagonist's or agent's perspective (bilateral medial frontal/posterior right temporal/parietal). This last network effectively applies Theory of Mind processes to the comprehension of a narrative [Mason and Just, 2009].

Theory of Mind and many “social interpretation” processes may be used to understand protagonists' actions [Castelli et al., 2002; Gernsbacher et al., 1998]. Consider, for example, this set of sentences:

Brad had no money but he just had to have the beautiful ruby ring for his wife. Seeing no salespeople around, he quietly made his way closer to the ring on the counter. He was seen running out the door.

The sentence invites the inference that Brad stole the ring, based on Theory of Mind processing of the information about the manner in which the protagonist approached a desirable valuable object in a retail environment. An everyday understanding of others' minds is clearly necessary for interpreting the intentions, goals, and actions of characters within a narrative.

In contrast to passages that invite inferences based on a character's intention, other types of passages invite an inference based on the physical consequence of an action. An example of a passage that invites an inference based on physical causality is:

While playing in the waves, Sarah's Frisbee went flying toward the rocks in the shallow water. While searching for it, she stepped on a piece of glass. Sarah had to wear a bandage on her foot for a week.

Here the relation between stepping on glass and needing a bandage is based on a chain of causally related events rather than human intentionality. The physical basis of the inference was typically an event that could be easily visualized, leading to the hypothesis that the inference‐making for this type of passage could evoke activation of a spatial network supporting visual imagery [Mason and Just, 2006].

Inference processing has been the access point of choice for many researchers trying to discover the cortical basis of discourse comprehension [see Ferstl 2007 for a review and Ferstl et al., 2008 for a meta‐analysis]. What has become clear is that the cortical basis of inferencing depends in part on the degree of difficulty of making the inference [Kuperberg et al., 2006; Mason and Just, 2004; Robertson et al., 2000; Virtue et al., 2008] as well as: the linguistic nature of the inference, such as its having a figurative interpretation [e.g., Bottini et al., 1994]; the maintenance of coherence [Ferstl and von Cramon, 2001, 2002]; keeping track of a sequence of events [Ferstl et al., 2005]; or representing the emotional states of a protagonist [Ferstl et al., 2005]. This is in contrast to a previous perspective that posited that inferencing is monolithically a right hemisphere‐localized function [Beeman, 1993, 1998; Beeman et al., 1994]. The view strongly associating the right hemisphere with inference‐making originated in neuropsychological research [Beeman 1993; Brownell et al., 1986; although see McDonald and Wales, 1986; Tompkins, 1991; Tompkins et al., 2004] showing that RH damage often led to inferencing difficulties. This view was also consistent with divided visual field experiments [Beeman et al., 2000; Long and Baynes 2002; Long et al., 2005; Prat et al., 2007]. Beeman and coworkers [Jung‐Beeman, 2005; Virtue et al., 2006; Virtue et al., 2008] have more recently modified their right hemisphere coarse coding hypothesis to include a “Bilateral Activation, Integration, and Selection” (BAIS) process reflecting the current view that inferencing is not a simple one‐stage process localized in the right hemisphere.

Recently language researchers have gone beyond generating activation maps to understand the cortical basis of language, extending the research to include considerations of cortical network attributes, particularly the functional connectivity among network centers. Functional connectivity measures the degree to which two cortical regions work together by measuring the pairwise synchronization of the time courses of their activation. Functional connectivity analysis is a powerful tool that can show how two regions with differentiable specializations can work together during discourse comprehension. For example, one relevant finding emerging from this approach is that in people with high‐functioning autism, there is functional underconnectivity (lower levels of synchronization) between frontal and posterior regions of the cortex [Just et al., 2004, 2007], resulting in abnormal Theory of Mind activation during discourse processing [Mason et al., 2008]. In this study, the points in time at which the functional connectivity between two regions increases provide evidence about the time course of a particular discourse process. In particular, we hypothesize that the functional connectivity between the two cortical centers of the Protagonist Network (dorsomedial prefrontal cortex and right temporo‐parietal junction) will increase when an intentional inference is being made.

The overarching goal of the current study is to learn more about the cortical underpinning of discourse comprehension, focusing on the differences between human intention‐ and physical causality‐based inferences, and on the dynamic nature of the cortical activity. The protagonist network is expected to be recruited to support Theory of Mind‐based inferences; this recruitment is expected to occur specifically at the point in which the inference can first be made. A visual‐spatial network for spatial representation of the situation is expected to be active to support the physical causality‐based inferences.

METHOD

Participants

The participants were 16 undergraduate students (10 female) at Carnegie Mellon University. Data from four participants were discarded due to excessive head motion (>3 mm) and from two other participants because of excessive errors on comprehension probes (>20%; all other participants <10%). All were paid for their participation. Each participant signed an informed consent that had been approved by the University of Pittsburgh and Carnegie Mellon University Institutional Review Boards.

MATERIALS

The participants were asked to read 30 stories and answer a simple yes/no comprehension question in response to each story. The stories contained three sentences: the first sentence provided a context for the passage; the second sentence in the inference conditions was written such that it might be possible to generate a predictive inference; and the third sentence constituted a coherence break if no inference had been drawn. The causal inference either involved a direct consequence in the physical passages or was guided by a character's goal in the intentional passages. For example, a context sentence and possible forward inference‐inducing sentence based on a character's intention were:

Context: After being out all night, Donny knew he wouldn't make practice if he didn't get some rest.

Possible Forward Inference‐Inducing Sentence: When he got to his history class he sat in the very back of the room.

After the inter‐trial rest period, this was followed by the concluding, coherence break sentence:

Coherence Break: Donny was surprised when he heard people leaving the class.

An example of each of the four conditions appears in Table I. The control conditions were equated to the inference versions, with two differences: the second sentence did not invite a forward inference, and the third sentence explicitly stated the inference that could have been made in the inference conditions. There were 10 passages in each of the inference versions (intentional and physical) and five passages for each of the matched control versions. Only five items were used in each of the control versions in order to limit time in the scanner. The same random presentation order was used for all participants. Passages were presented using a Latin square design. The physical passages were obtained from Murray et al. [1993]. The intentional passages were constructed de novo to be similar to the sample intentional passage provided in the Murray et al. materials. The causally related events for the physical passages usually described accidents; other examples included falling into a lake, fainting from lack of food, and getting a flat tire. Twelve independent individuals filled out a norming questionnaire to ensure that contents of the passages were clearly either intention‐based or physically caused; the difference in the ratings for the two groups of passages was significant (t(11) = 15.28, P < 0.0001).

Table I.

Sample texts

Intentional texts

Inference version

Brad had no money but he just had to have

the beautiful ruby ring for his wife.

Seeing no salespeople around, he quietly

made his way closer to the counter.

He was seen running out the door.

Control version

Brad had no money but he just had to

have the beautiful ruby ring for his wife.

He couldn't see the price tag from where he

was, so he quietly made his way closer to the counter.

He then stole the ring and ran out the door.

Physical texts

Inference version

While playing in the waves, Sarah's Frisbee

went flying toward the rocks in the shallow water.

While searching for it, she stepped on a piece of glass.

Sarah had to wear a bandage on her foot for a week.

Control version

While playing in the waves, Sarah's Frisbee

went flying toward the rocks in the shallow water.

While searching, she found a beautiful piece of rare glass.

The glass cut her foot and she started to cry.

Procedure

One or two days before scanning, the participants completed a short practice set of six items, two of each condition, to familiarize them with the task. In the scanner the task was run with the timing illustrated in Figure 1. Three 30‐s presentations of the fixation point were interspersed among the test items, one at the beginning, one after 16 trials, and one at the end. Each experimental trial began with the two‐sentence context. The first sentence of the context appeared for 7 s (the onset was time‐locked to the acquisition of the superior‐most slice in prescription). It was followed by the second sentence, which appeared for 5 s. After that 12‐s interval, a rest X appeared on the screen for 4 s. The third sentence then appeared and remained on the screen for 5 s, followed by a second rest X for 6 s. Each trial ended with a question with two answer choices (“yes” and “no”) below it. The subject was given 4 s to answer the question. An X then appeared for a 2‐s rest period. Although a 2‐s rest period was not enough time for the hemodynamic response to return to baseline between the stories, it was long enough to drop from an asymptotic level [Mason et al., 2003; this can also be seen in the time course in Fig. 5]. An example of this entire sequence is shown in Figure 1. During the 2‐s rests and the 30‐s fixations, participants were instructed to “just relax, clear you mind, and wait for the next story to appear.”

Figure 1.

A representation of the timing of a single trial.

Figure 5.

The average time course of activated voxels in two components of the inference network, (A) the left inferior frontal gyrus and (B) the left posterior superior temporal gyrus, indicates that voxels within these regions match the reading of the sentences in all conditions. Importantly, the increase of the inference passages over the control passages occurs at the peak of the activation corresponding to the processing of the end of sentence 2, suggesting that an inference is drawn at the earliest point possible (i.e., a predictive‐forward inference).

fMRI Procedures

The data were collected using a Siemens Allegra 3.0T scanner at the Brain Imaging Research Center jointly established by Carnegie Mellon University and the University of Pittsburgh. The study was performed with a gradient echo planar pulse sequence with TR = 1,000 ms, TE = 30 ms, and a 60° flip angle. Sixteen oblique‐axial slices were imaged, and each slice was 5‐mm thick with a gap of 1 mm between slices. The acquisition matrix was 64 × 64 with 3.125 × 3.125 × 5 mm³ voxels.

fMRI Analyses

Distribution of activation

To calculate the distribution of activation, the data were analyzed using SPM2. Images were corrected for slice acquisition timing, motion‐corrected, normalized to the Montreal Neurological Institute (MNI) template, resampled to 2 × 2 × 2 mm³ voxels, and smoothed with an 8‐mm Gaussian kernel to decrease spatial noise. Statistical analyses were performed on individual and group data by using the general linear model as implemented in SPM2 [Friston et al., 1995].

For each of the four conditions—intentional inference, physical inference, intentional control, and physical control—separate regressors were created for the context sentence, the second sentence (the forward inference window), the third sentence (the integration or backward inference window), and the comprehension question by convolving a boxcar function with the standard hemodynamic response function as specified in SPM. Thus the final analysis contained regressors for four conditions: intentional inference (Iinf), physical inference (Pinf), intentional control (Icon), and physical control (Pcon); and four temporal regions: the context window (the first sentence), the forward inference window (the second sentence), the integration window (the third sentence), and the comprehension question. Because of the small differences between the control versions and the low number of examples in the two control conditions, the two control passage types were combined into a single control condition for subsequent pairwise contrasts.

Group analyses were performed. Statistical maps were superimposed on normalized T1‐weighted images. An uncorrected height threshold of T = 3.73 (P = 0.001) and an extent threshold of six voxels were used. Regions of activation that fell outside grey matter areas, as indicated by the MNI template, were not reported.

Time Course Analysis

Twenty functional ROIs were defined to encompass an a priori definition of the discourse processing network. Several regions selected for these functional networks were developed on the basis of the parallel networks of discourse processing theory [Mason and Just, 2006], along with several traditional language processing areas. The center and extent of these fROIs were defined using data from a different set of participants [Mason et al., 2008] as well as a meta‐examination of several other comprehension experiments from our lab. Importantly, the fROIs were not contaminated by being based on data from the current experiment. Labels for these 20 ROIs [the left dorsomedial prefrontal cortex, inferior and superior (LDMPFCi, LDLPFCs); the bilateral middle frontal gyri (LMF, RMF); the bilateral inferior frontal gyri, inferior and middle (LIFGi, LIFGm, RIFGi, RIFGm); three bilateral superior temporal ROIs, anterior, middle, and posterior with the right posterior STG region being defined as the right temporo‐parietal junction (LSTGa, LSTGm, LSTGp, RSTGa, RSTGm, RTPJ); additionally three bilateral subcortical/inferior temporal ROIS, the fusiform gyrus, the hippocampus and the parahippocampus (LFUS, LHIP, LPAR, RFUS, RHIP, RPAR)] were assigned with reference to the parcellation of the Montreal Neurological Institute (MNI) single subject T1‐weighted dataset carried out by Tzourio‐Mazoyer et al. [2002]. A sphere was defined for each cluster (with a radius ranging from 5 to 10 mm) that best captured the cluster of activation in the map for each contrast of condition versus fixation in the previously referenced experiments. The time course extracted for each participant was over only the activated voxels. For this analysis, activated voxels were defined more liberally, requiring that they be above a T‐threshold of 3.0 in any of the contrasts of an experimental condition versus fixation. Participants who did not have activation in a given fROI were excluded from further analysis involving that fROI. The number of participants included varied across fROIs; the average number of participants included was 14.7. The number of participants included in each fROI analysis is reported for each contrast.

For each participant, the time course of activated voxels was plotted as a percent change over fixation. This allowed for the scale to be similar across fROIs and participants. A percent signal change value was generated at each TR point by averaging over the activated voxels. These time courses were submitted to a mixed model ANOVA comparing the percent signal change in the second and third sentences for the three conditions (i.e., Iinf vs. Pinf, Iinf vs. Icon+Pcon, Pinf vs. Icon+Pcon). The points corresponding to the second and third sentence windows were offset by 6 s from the onset of each sentence. This shift compensated for the sluggishness of the hemodynamic response, allowing the response to be reflective of the asymptotic activity corresponding to each sentence [Mason et al., 2003].

Functional Connectivity

The functional connectivity was computed (separately for each participant) as a correlation between the average time course of signal intensity of all the activated voxels in each member of a pair of fROIs. Two pairs of fROIs were selected for this analysis: (1) a Language Network, consisting of the inferior portion of the left inferior frontal gyrus and the posterior portion of the left superior temporal gyrus (roughly corresponding to Broca's and Wernicke's areas); and (2) a Theory of Mind Network, consisting of the superior aspect of the medial frontal gyrus and the right temporo‐parietal junction. For this analysis a more liberal threshold of T > 3.0 was used to maximize the likelihood of including all fROIs for each participant. Two participants who did not have activation in the medial frontal functional ROI were excluded from further analysis involving that fROI. The correlation was computed separately on the images corresponding to the each of the three sentences in the passages. Fisher's r to z transformation was applied to the correlation coefficients for each participant prior to the averaging and statistical comparison of the two groups. Two‐sample t‐tests were computed for the contrasts of the three sentences in the intentional inference and physical inference passages. These tests were one‐tailed with a P < 0.05, based on the expectation that the functional connectivity would increase when a network became differentially engaged. For the Theory of Mind Network, the functional connectivity was expected to be higher when generating an intentional inference (sentences 2 and 3) than when processing the context (sentence 1). For the Language Network, connectivity was expected to be relatively high and remain unchanged across sentences.

RESULTS

Overview

There were three main findings. First, the recruitment of cortical networks differed depending on the type of inference, such that intentional inferences recruited an additional Theory‐of‐Mind‐like protagonist network and physical inferences recruited some more visually‐based regions. Second, a common set of networks supported both types of inference making. Third, the increase in inference‐related activation was triggered at the earliest possible point in the text, indicating that readers were generating a forward inference in these passages.

Intentional Inferences

Intentional inferences activated a number of discourse processing areas more than physical inferences did, indicated by the intentional‐physical contrasts. In particular, the right temporo‐parietal junction was more active during the reading of both the second and third sentences. Several other right hemisphere regions were more active during the second and third sentences, including right inferior frontal gyrus, right middle frontal gyrus, and right superior frontal gyrus. The additional intentional inference activation in the left hemisphere (left middle frontal and left posterior superior temporal areas) was noticeably smaller than in the right. Some of the effects of reading an intentional passage were specific to a particular sentence location in the passage. For the second sentence, an additional cluster appeared in the superior occipital gyrus, whereas the third sentence evoked an additional cluster in the orbital frontal region. This intentional inference‐based activation is shown in Figure 2, and the entire set of activated regions is listed in Table II.

Figure 2.

Brain areas that show greater activation for intentional inference over physical inference passages separately for each sentence. The activation is greater in the right temporo‐parietal (green ellipses) when the reader has to generate and integrate an inference based on an understanding of a character's intention. This can be seen in sentences two and three. Activation is projected onto the surface rendering. The corresponding cortical regions, cluster sizes, peak T‐values, and MNI coordinates can be found in Table II.

Table II.

Areas of activation for the contrasts of intentional inference passages minus physical inference passages and physical inference passages minus intentional inference passages

Cortical region	Cluster size	Peak T‐value	MNI coordinates
			x	y	z
Intentional—Physical (sentence 1)
Left anterior inferior temporal	76	4.74	−46	−70	28
Left angular	26	4.66	−50	−8	−28
Right precuneus	11	3.87	8	−60	22
Right rectus	7	3.83	4	50	−18
B) Intentional—Physical (sentence 2)
Right temporo‐parietal junction	87	4.74	58	−62	28
Right opercularis/middle frontal	98	5.50	32	10	40
Right superior/middle frontal	35	4.80	24	18	52
Right superior/middle frontal	30	4.68	26	40	50
Left posterior superior/middle temporal	98	4.44	−56	−50	14
Left middle/superior temporal	242	6.11	−42	−36	0
Left middle frontal/precentral	41	4.86	−32	6	46
Left middle frontal	6	3.93	−36	22	36
Right fusiform/hippocampus/parahippocampus	15	4.23	38	−32	−14
Right fusiform/lingual	194	4.52	14	−70	−6
Left fusiform/lingual/occipital	444	5.73	−16	−76	−6
Bilateral cuneus/occipital/calcarine	1070	8.31	14	−90	30
Left inferior occipital	6	3.93	−36	−82	−6
Right superior occipital/calcarine/cuneus	20	4.21	22	−70	18
Left precuneus	7	4.09	−16	−50	52
Right cerebellum	16	5.34	14	−62	−34
Right cerebellum/lingual	34	4.95	14	−52	−8
Intentional—Physical (sentence 3)
Right temporo‐parietal junction	109	4.26	44	−52	32
Right angular	112	4.24	56	−66	32
Right middle temporal	16	4.02	58	−56	16
Right inferior frontal triangularis	18	4.30	54	26	24
Right inferior orbital frontal/insula	10	4.10	38	26	−8
Right inferior frontal opercularis/middle frontal	115	4.63	48	16	46
Right superior/middle frontal	68	4.44	22	16	56
Right middle frontal	7	3.92	42	8	56
Left angular/supramarginal/middle temporal	105	5.03	−58	−60	28
Left middle frontal	10	4.04	−30	16	34
Left superior/middle frontal	67	5.73	−22	54	0
Left caudate	9	3.93	−8	6	14
Right pallidum	16	4.38	16	0	−6
Physical—Intentional (sentence 2)
Left superior temporal	12	5.01	−66	−32	18
Left middle/inferior occipital/calcarine	117	5.86	−14	−106	2
Right middle/superior occipital/calcarine	12	5.92	12	−102	−6

In Tables II, III, and IV the threshold for significant activation was P < .001 for a spatial extent of at least 10 voxels, uncorrected for multiple comparisons. Region labels apply to the entire extent of the cluster. T‐values and MNI coordinates are for the peak activated voxel in each cluster only.

Physical Inferences

Physical inferences (assessed by a physical‐intentional contrast) activated a bilateral portion of the middle occipital gyrus (extending into the calcarine area) more than intentional inferences. This area commonly activates in response to visual processing. Interestingly, there was no right hemisphere area that had more activation for physical than intentional inferences (except the right hemisphere portion of the bilateral middle occipital region). Activation was found in an additional small region of the left superior temporal gyrus extending into the supramarginal gyrus. The extra activation for physical inference passages is shown in Figure 3 and listed in Table II[Physical—Intentional (Sentence 2)].

Figure 3.

Brain areas that show more activation in the contrast of physical inference sentences minus intentional inference sentences in both the left and the right hemispheres. Along with a small region between left superior temporal gyrus and the supramarginal gyrus, the predominant area of extra activation is in the posterior occipital area. The corresponding cortical regions, cluster sizes, peak T‐values, and MNI coordinates can be found in Table II.

General Inference Network

Much of the activation was similar in (common to) the two inference conditions when compared with a fixation baseline. Drawing an inference of either type resulted in activation in the medial and superior frontal areas, bilateral inferior frontal gyri, the left posterior superior temporal gyrus, and the anterior temporal gyri bilaterally; the full set of areas is listed in Table III. For comparison, the control passages, when contrasted with fixation, activated a corresponding area in left superior temporal gyrus and a similar but much smaller area of the middle and anterior right temporal gyrus. Noticeably absent was activation in the left inferior frontal gyrus as well as the superior frontal regions.

Table III.

Areas of activation for the contrast of intentional inference, physical inference, and control passages minus fixation

Cortical region	Cluster size	Peak T‐value	MNI coordinates
			x	y	z
Intentional—Fixation (Sentence 2)
Left inferior frontal triangularis/opercularis	697	7.17	−56	24	14
Left middle frontal/precentral	447	6.22	−44	2	54
Left medial/superior frontal	82	5.77	−4	34	54
Left medial/superior frontal	58	5.44	−8	54	36
Left Inferior/middle/superior temporal/angular	3756	10.40	−50	−54	20
Right inferior frontal triangularis	25	4.84	62	24	12
Right inferior/middle/superior temporal/angular	1864	11.00	56	−8	−26
Right cerebellum	115	5.89	18	−82	−32
Left precuneus/cingulum	34	4.57	−12	−48	38
Physical—Fixation (sentence 2)
Left inferior frontal triangularis/opercularis	251	5.87	−52	22	20
Left medial frontal	52	5.14	−2	52	42
Left inferior frontal orbital	15	4.09	−52	30	−12
Left medial frontal	28	4.24	−8	34	58
Left inferior frontal triangularis	16	4.55	60	26	10
Left inferior/middle/superior temporal/angular	1456	7.35	−58	−8	−20
Right inferior/middle temporal	107	4.96	54	−28	−14
Right inferior temporal	8	4.17	52	−8	−30
Control—Fixation (sentence 2)
Left inferior/middle/superior temporal/angular	1590	8.09	−64	−12	−20
Right inferior/middle temporal	117	4.60	54	−32	−14
Right inferior temporal	7	4.17	52	−6	−32
Left occipital/lingual/calcarine	67	5.02	−16	−92	−10

There was a large area of cortex activated regardless of inference type. Both intentional and physical types of inference‐making are supported by a large shared common network. As noted in Table III, there was inference‐related activation in three focal points of the left hemisphere language network: the inferior frontal gyrus, middle temporal gyrus, and the temporo‐parietal/angular gyrus. Also consistent with previous studies of inference generation [Mason and Just, 2004], there was activation in the right hemisphere homologues of these three areas. The other major cluster of activation was in the medial and superior frontal region. This area has been found to be active in discourse processing tasks [Ferstl and von Cramon, 2002; Ferstl et al., 2005]. It also overlaps with the region in the frontal cortex that has been suggested to be associated with Theory of Mind processing [Castelli, 2002; Gallagher and Frith, 2003].

The constituency of the inference network may be further clarified by contrasting the inference passages with the control passages. Across both types of inference passages, the left inferior frontal gyrus was more active than for the control passages. The areas that were active in the intentional inference passages as compared to control passages were similar to those more active in intention than physical inferences, including the right temporo‐parietal junction, right inferior frontal gyrus, left middle frontal gyrus, and bilateral anterior temporal gyrus. The contrast between the combination of both inferences versus the control passages indicated a network that included bilateral inferior frontal gyri, left superior temporal gyrus (extending into supramarginal gyrus), and small clusters in left middle temporal and left middle frontal gyri. During the processing of the third sentence, additional activation appeared only in the intentional passage versus the control passage. The third sentence activation appeared in left angular gyrus, left orbital frontal gyrus, and left middle and superior frontal gyri (extending into the medial frontal gyrus). This depiction of the inference network for Sentence 2 can be found in Figure 4 and the activated regions for both Sentence 2 and Sentence 3 are described in Table IV.

Figure 4.

Brain areas that increase in activation when processing an inference than when processing the matched control sentences. Inference‐related activation occurs in several of the discourse processing component networks during the second (forward inference) sentence: left IFG (all types); right IFG (intentional and collapsed); bilateral anterior temporal (intentional); left STG; right TPJ; and, left DLPFC. The corresponding cortical regions, cluster sizes, peak T‐values, and MNI coordinates can be found in Table IV.

Table IV.

Areas of activation for the contrast of both types of inference passages minus the control passages

Cortical region	Cluster size	Peak T‐value	MNI coordinates
			x	y	z
Intentional—Control (sentence 2)
Left inferior frontal triangularis/opercularis	431	5.84	−46	22	12
Left middle frontal/precentral	102	4.64	−42	6	54
Left middle frontal	19	4.10	−32	18	42
Left precentral	13	4.01	−36	4	38
Left middle temporal/angular/supramarginal	670	6.98	−48	−52	28
Left middle temporal	130	5.70	−56	−32	−8
Left middle temporal	6	4.12	−66	−18	−20
Left angular	11	4.11	−48	−74	30
Right inferior frontal triangularis/orbital	217	5.19	58	24	−2
Right inferior frontal triangularis/middle frontal	19	4.19	44	24	30
Right inferior/middle temporal	290	6.03	54	−10	−26
Right middle/superior temporal/angular	115	4.47	48	−50	14
Bilateral calcarine/cuneus/lingual/precuneus	1721	6.87	−6	−68	22
Left middle occipital	7	4.32	−38	−82	40
Left lingual/cerebellum	51	4.83	−10	−50	−2
Left caudate	40	4.73	−8	6	8
Left lingual	128	5.83	−16	−74	−6
Physical—Control (sentence 2)
Left Inferior frontal triangularis	64	6.06	−58	22	22
Inference—Control (sentence 2)
Left inferior frontal triangularis/opercularis	378	6.57	−56	20	16
Left inferior frontal orbital	13	4.09	−48	38	−10
Left middle frontal/precentral	10	4.16	−42	6	58
Left middle temporal/angular/supramarginal	196	5.33	−48	−52	26
Left middle temporal	11	4.25	−58	−34	−12
Right inferior frontal triangularis/orbital	121	4.99	60	26	4
Bilateral calcarine/left cuneus	214	4.93	−2	−70	14
Intentional—Control (sentence 3)
Left inferior frontal triangularis/orbital	116	5.56	−34	28	−6
Left middle frontal	42	4.49	−28	6	50
Left superior frontal	16	4.93	−20	−10	48
Left medial frontal	43	4.31	−4	34	44
Left middle temporal/angular/supramarginal	84	6.84	−44	−50	22

Time Course of Activation Supporting Forward Inferences

The time course of the activation for the inference conditions diverges from the control condition at the peak of the response that occurs during the reading of the second sentence. This activation indicates processing related to drawing a forward inference. This pattern can be seen in the time course for two regions from the inference network: (Fig. 5A) the left inferior frontal gyrus and (Fig. 5B) the left superior temporal gyrus.

An interesting comparison can be made by comparing the two component regions of the protagonist monitoring network, dorsomedial prefrontal cortex and right TPJ regions [Mason and Just, 2009]. While the two regions are both associated with Theory of Mind processing, they have very different time courses. The protagonist monitor (the dorsomedial prefrontal region) activated early in the reading of both types of narratives (involving either a physical or intentional inference), probably in response to the need to monitor a protagonist or agent, as shown in Figure 6A. In contrast, the right temporo‐parietal junction showed an increase in activity when an inference was invited, as shown in Figure 6B. Critically, the RTPJ activation also increased at the earliest point at which the hemodynamic response could indicate the processing of the inference.

Figure 6.

The average time course of activated voxels in the two functional ROIs during the reading of the passages inviting intentional inference and of those inviting physical inference indicates that (A) the activation is constant across sentences for the Protagonist Monitor (medial prefrontal gyrus) and (B) the activation increases after the point of the intentional inference for Protagonist Synthesis (right temporo‐parietal junction).

DISCUSSION

Brain activity in the discourse processing network varied with the type of inference being made, with a clear distinction between inferences about human intention versus physical causality. Inferences based on the intention of a protagonist recruited the right temporo‐parietal junction region [protagonist interpreter, Mason and Just, 2006, 2009] more so than inferences based on physical causality. Intention‐based inferences also led to an increase in bilateral middle frontal areas (coherence monitoring) and left superior temporal gyrus (classic Wernicke's area). In contrast, physical inferences activated a small region of left supramarginal gyrus and posterior occipital gyrus to a greater extent than did intentional inferences.

The two types of inferences had commonalities in the activation that they engendered. There was a common set of areas whose activation increased during the generation and integration of either type of inference. The primary region in this common network was the left inferior frontal gyrus; additional common regions included right inferior frontal gyrus and bilateral anterior temporal gyrus (text integration), left middle frontal gyrus (coherence monitoring), and left superior temporal gyrus. The medial/superior frontal gyrus showed nonreliably more activation in the inference conditions, possibly because it is always activated during discourse comprehension (engaged in protagonist monitoring) but slightly more so when an inference is being drawn.

The time course of activation indicated that the increase in the activation in the inference network occurred as soon as the reader encountered the triggering information in the text (as opposed to waiting for information in a later sentence). (The inferences in the stimulus paragraphs could occur as early as the second sentence.) This type of predictive inference‐making has also been seen in a previous neuroimaging study [Virtue et al., 2006]. The time course indicated a divergence in the activation levels for the inference and control passages at the earliest measurable peak activation corresponding to a potential inference. Consistent with the differential recruitment of networks depending on the type of inference, the activation in the right temporo‐parietal junction following the divergence was greater during the reading of intentional inferences than physical inferences. The time courses also demonstrated that some regions, such as the protagonist monitoring areas (dorsomedial prefrontal cortex), were active but not particularly sensitive to the inference making.

The locations of the member nodes of the discourse inference network in the right hemisphere can now be specified more precisely than was previously done [Mason and Just, 2004]. The current analysis averages the activation across individual brains in a normalized space, as opposed to the previous method of measuring the activation volume within an anatomically‐defined region of interest. Instead of simply noting that the right hemisphere plays a role in the inference process, as we previously did, we are now able to specify the three focal areas of right hemisphere activation: the inferior frontal gyrus (primarily in pars triangularis), the anterior portion of the temporal gyrus, and the slightly more posterior portion of the middle temporal gyrus (but anterior to the posterior superior temporal sulcus).

Cortical Networks of Discourse Processing

Different parts of the discourse processing network are differentially activated, depending on the amount and type of the processing needs evoked by a particular passage. A primary region in the process appears to be the left inferior frontal gyrus, but the more demanding an inference, the greater the extent to which other regions are recruited, in particular the right inferior frontal gyrus, the bilateral anterior temporal gyri, and the left middle and superior frontal gyri. These regions may be more specialized, performing functions such as text integration (right anterior temporal), coherence monitoring (left middle frontal), protagonist monitoring (dorsomedial prefrontal cortex), and inference generation (bilateral inferior frontal gyrus) [Mason and Just, 2006]. Perhaps the strongest demonstration of the adaptability of the discourse processing network is the recruitment of the right temporo‐parietal junction (for protagonist simulation) specifically when the inference was based on the intention of a character in the story. Another instance of the phenomenon was the recruitment of a visually‐based region (bilateral middle occipital gyrus) when the inference was based on the consequence of a physical event. This set of results, discussed below, demonstrates the adaptability with which the components of the discourse processing network are recruited.

Intentional Inferences and the Protagonist Network

As expected, a Theory of Mind‐based protagonist network played a role in the processing of intentional inferences. In particular, the RTPJ was sensitive to the intentional inference passages. The principal Theory of Mind areas are the dorsomedial prefrontal gyrus and the right temporo‐parietal junction [Castelli et al., 2002; Saxe et al., 2003; Saxe et al., 2005]. The conception of these two cortical regions as part of a larger protagonist perspective network [Mason and Just, 2006, 2009] was based in part on the consistent activation of these regions in numerous discourse processing tasks as well as in Theory of Mind tasks [Ferstl and von Cramon, 2002; Fletcher et al., 1995; Friese et al., 2008; Mason et al., 2008].

Although the two main components of this network tend to coactivate, they appear to have differentiable specializations. The frontal component of this network, the dorsomedial prefrontal gyrus, appears to have a Protagonist Monitor role, which can be viewed as an executive processor that activates throughout the processing of a narrative, tracking the progress of the characters in the narrative. The posterior network component, the right temporo‐parietal junction, appears to have a Protagonist Synthesis role, which may be to actively generate expectations concerning the protagonist's thoughts and actions, based on an understanding of the intentions of the protagonist and on general knowledge of the world [Mason and Just, 2009]. We intend the Protagonist Synthesis label for this role to be agnostic with respect to the debate in the Theory of Mind literature between Simulation Theory (which proposes that people simulate other's minds internally) and Theory‐Theory (which proposes that people use an intuitive theory of how the mind works to understand other's minds). If anything, the proposed mechanism here may be considered a hybrid of the two positions [see Saxe, 2005 for a description of this debate].

The Protagonist Synthesis is proposed to function by generating expectations based on a synthesis of information in the text and knowledge of similar situations. Thus a generative process is presumed to underpin this aspect of discourse comprehension. This proposed generative process resembles the synthesis part of Neisser's [1967] analysis‐by‐synthesis model of pattern recognition. According to that model, the analysis or recognition of a pattern is accomplished by synthesizing (or generating) an expected pattern based on (a) perceptual input and (b) prior knowledge, and then matching the synthesized representation to the input. In the context of understanding a protagonist's intentions and actions, this synthesis process can be viewed as generating an expectation signaled by the overlap of knowledge of prior experiences with the information in the text. When there is a match between the synthesized representation and the text, the synthesized representation is integrated into the discourse representation.

Whereas protagonist monitoring is presumed to occur throughout the narrative passages, the synthesis of an inference based on a protagonist's intention should occur only during the generation of an intentional inference. The RTPJ activation associated with intentional inferences supported this conclusion. Furthermore, this RTPJ region continues to be activated during the reading of the sentence that follows, supporting integration of the intentional inference.

Functional Connectivity Modulation to Support the Protagonist Network

The functional connectivity between the two components of the Protagonist Network increased when Protagonist Synthesis (centrally involving the right temporo‐parietal junction) was engaged. This supports the conceptualization of the medial frontal gyrus and the right temporo‐parietal junction as components of a network with different specializations. At the precise point at which it would be expected that the two regions work together (i.e., the inference point), the functional connectivity increased, whereas the two regions had very dissimilar time courses and lower functional connectivity before that point. That is, the medial frontal gyrus was active throughout the passage whereas the right TPJ only became active in the second and third sentences. This finding could indicate that the frontal monitoring region provides the initiating signal for the RTPJ synthesis to begin. In contrast, the functional connectivity between two primary language regions (left inferior frontal gyrus, approximately Broca's area, and left superior posterior temporal gyrus, approximately Wernicke's area) is relatively and consistently high throughout the reading of all of the sentences of the passages. The contrast between the functional connectivity in the two networks is depicted in Figure 7. In summary, the synchronization between the components of the Protagonist Network (dmPFC and RTPJ) increases when an intentional inference is required.

Figure 7.

The functional connectivity (z‐score of the correlation of the time course of activated voxels within a pair of functional ROIs) between the pair of Language Network ROIs (left inferior frontal and left posterior superior temporal) and the pair of Protagonist Network ROIS (left medial frontal and right temporo‐parietal junction), indicates that the Protagonist Network increases in functional connectivity when an intentional inference is made as compared to the context sentences. In contrast the Language Network is highly connected throughout reading of the passages. The increase in functional connectivity between sentence 1 and both sentences 2 and 3 is significant, whereas no other contrasts across sentences are significantly different.

Replication of the Recruitment of Protagonist Synthesis Region

The recruitment of the right temporo‐parietal junction for intention‐based inferences provides an important replication in the investigation of narrative comprehension. A subset of the current passages was used in an experiment that examined inference processing within individuals with autism [Mason et al., 2008]. The matched control group in that experiment had a very similar pattern of activation in the RTPJ for intentional inferences. The centroid of the RTPJ activation shown in Figure 2 was at (58 −62 28) in MNI space and the centroid in the Mason et al. paper was at (56 −66 24), a Euclidean distance of 6 mm.

Physical inferences and visually‐based representations. Although the physical inferences elicited less additional activation than did the intentional inferences, the physical inference passages did activate a set of regions commonly implicated in visual processing. The findings provide evidence of the differential cortical bases of the drawing of the two types of inferences. In the case of the inferences concerning physical causality, it is possible that the participants were forming a dynamic visual representation of the chain of physical events as part of understanding the causal relation, giving rise to the activation in visual brain areas. There is considerable evidence from behavioral research for visual imagery playing a role in sentence and discourse comprehension [Glenberg and Kaschak, 2002; MacWhinney, 1999; Zwaan et al., 2002; Zwaan and Madden, 2005] and from neuroimaging research as well [Just et al., 2004]. A speculative account of the occipital activation observed in this study is that it is related to embodied cognition, such that the conceptual representations of the events linked by causality may include information about how one's sensory and motor processes might be involved in the events [Allport, 1985; Barsalou, 1999; Gallese and Lakoff, 2005]. Comprehending sentences referring to visual experiences (e.g., You see the rope.) activates secondary visual areas [Desai et al., 2010] and processing words related to the human body evokes occipital area activation in the extrastriate body area [Rueschemeyer et al., 2010]. Given that so much technical and scientific knowledge is communicated in expository texts that describe phenomena in terms of physical causality, and given that the comprehension of such texts requires inference‐making [Millis and Graesser, 1994; Noordman et al., 1992; Singer and Gagnon, 1999; Wiley and Myers, 2003], it will be important for future discourse comprehension research to further investigate the neural basis of such inference‐making processes concerning physical causality.

General Inference Network

A network of cortical areas activates whenever an inference of either type (intentional or physical) is made. As indicated by the contrast with fixation, this network consisted of the main language network centers (LIFG and LSTG), RIFG, bilateral anterior temporal gyrus, and the medial/superior frontal cortex. This adds to the growing body of literature that shows right hemisphere activation for language tasks that require discourse‐level processing [Bottini et al., 1994; Mason and Just, 2004; St. George et al., 1999]. In particular, the new findings here help specify the locations of the inference‐processing network with greater precision than our previous attempt.

The converging findings suggest the right hemisphere has a key role in the inferencing process. A previous study [Mason and Just, 2004] found greater right hemisphere language network activation in a condition in which readers successfully generated an inference. At that time, we proposed that inference generation and integration was accomplished utilizing a broadly defined right hemisphere language network, including the inferior temporal, temporal, inferior frontal, and inferior parietal areas. We also suggested that the dorsolateral prefrontal cortex guided the inference generation process. That study investigated causal inferences that varied in their causal distance and, therefore, in the degree to which it was necessary for an inference to be drawn. By contrast, the current experiment compared the comprehension of passages that invited an inference of an event to control passages that explicitly described the event. This methodological and data analysis change allowed us to more accurately define the inference‐processing network. The new experimental design revealed that a key LH area, LIFG, was additionally activated during the processing of inference passages in comparison to control passages. The second illuminating contrast between the 2004 study and this study was that in this study the RH inference‐related areas were evoked only in the intentional inference condition. The physical inferences, although they activated the right hemisphere regions, did not recruit them to a level significantly above that of the control passages. It is possible that inferences that do not require consideration of protagonists' intentions may be computed primarily within the basic left hemisphere language network. Consistent with this view, it was only an LH area (left inferior frontal gyrus) that was activated more by the inference sentence in the physical passages than in the control passages. It is possible that the additional processing evoked by the protagonist monitoring and synthesis network for intentional inferences in narratives results in an extra demand on the language network. This pattern is consistent with a view that the RH inference activation is not inference‐related per se, but that its involvement is a function of the amount of demand required for a successful inference.

Summary

The ubiquitous process of inference‐making during narrative comprehension requires the recruitment of several component regions of a cortical discourse network. A primary node of this network appears to be the left inferior frontal gyrus. The more demanding an inference is, the greater the extent to which other regions are recruited, in particular, the right inferior frontal gyrus, the bilateral anterior temporal gyri, and the left middle and superior frontal gyri. These regions may be more specialized within the inference process, performing functions such as text integration (right anterior temporal gyrus), coherence monitoring (left middle frontal gyrus), protagonist monitoring (dorsomedial prefrontal cortex), and inference generation (bilateral inferior frontal gyrus) [Mason and Just, 2006]. Perhaps the strongest demonstration of the adaptability of the discourse network is the differential recruitment of areas as a function of the inference type. This phenomenon was manifested in the recruitment of a region supporting Protagonist Synthesis (right temporo‐parietal junction) when the inference was based on the intention of a character in the story. Further support came from the recruitment of a visual region (bilateral middle occipital gyri) when the inference was based on the consequence of a physical event. This set of results provides an early definition of the discourse processing network and demonstrates the adaptability with which its components are recruited.