Dynamic Causal Modeling of Subcortical Connectivity of Language

Abstract

Subcortical–cortical interactions in the language network were investigated using dynamic causal modeling of magnetoencephalographic data recorded during auditory comprehension. Participants heard sentences that either were correct or contained violations. Sentences containing violations had syntactic or prosodic violations or both. We show that a hidden source, modeling magnetically silent deep nuclei, is required to explain the data best. This is in line with recent brain imaging studies and intracranial recordings suggesting an involvement of subcortical structures in language processing. Here, the processing of syntactic and prosodic violations elicited a global increase in the amplitude of evoked responses, both at the cortical and subcortical levels. As estimated by Bayesian model averaging, this was accompanied by various changes in cortical-cortical and subcortical–cortical connectivity. The most consistent findings in relation to violations were a decrease of reentrant inputs to Heschl's gyrus (HG) and of transcallosal lateral connections. These results suggest that in conditions where one hemisphere detects a violation, possibly via fast thalamocortical (HG) loops, the intercallosal connectivity is reduced to allow independent processing of syntax (left hemisphere) and of prosody (right hemisphere). This study is the first demonstration in cognitive neuroscience that subcortical–cortical loops can be empirically investigated using noninvasive electrophysiological recordings.

Introduction

The neural basis of language comprehension, taken to comprise Broca's area in the left inferior frontal gyrus (IFG) and Wernicke's area in the left superior temporal gyrus (STG), has been modeled mostly without considering deeper, subcortical structures (Friederici, 2002; Hickok and Poeppel, 2007). An exception is the study by Ullman (2004), which considers the involvement of the basal ganglia (BG). From patient studies (Raymer et al., 1997; Metz-Lutz et al., 2000) and functional magnetic resonance imaging (fMRI) studies (Fiebach et al., 2004; Mestres-Missé et al., 2008), it is clear that subcortical structures such as the BG and the thalamus also play a crucial role in language processing. The BG and the thalamus interact with cortical regions through many loops, including prefrontal, premotor, parietal, and temporal cortices (Crosson et al., 1997). While the role of the caudate as part of the BG is seen in language control (Crinion et al., 2006), intracranial recordings have identified the thalamus as being engaged in the detection of syntactic and semantic violations in spoken sentences (Wahl et al., 2008). This suggests that an adequate model of language comprehension should consider subcortical contributions.

In electroencephalography (EEG) or magnetoencephalography (MEG), it is impossible to reconstruct the allocation, orientation and—without taking into account interactions between brain regions—activity of a source that has no direct impact on the scalp sensors (i.e., most deep brain structures). Here, we profit from the latest advances in the modeling of MEG evoked responses using neural mass models to estimate the activity, topology, and effective connectivity of corticosubcortical loops during auditory speech comprehension.

Dynamic causal modeling (DCM) for MEG evoked responses (Kiebel et al., 2006) uses interacting neuronal populations to reproduce the activity of brain regions. Neuronal current densities are projected into the MEG channel space using a source and a head model. This combined model is then inverted to reconstruct brain dynamics with biological constraints explaining the measurement. Most importantly here, we investigate the feasibility of the DCM method to estimate hidden source activity. First, using simulations, we show that DCM distinguishes correctly between situations where no hidden source was present and situations where a hidden source relayed relevant information between cortical regions. DCM can thus be used to detect the presence of hidden sources. Second, we study actual MEG evoked responses during language processing with DCMs that either include an assumed hidden source or not. The hidden source is assumed to correspond to the thalamus and the respective thalamocortical loops based on data from intracranial recordings with similar language material (Wahl et al., 2008). Using model comparison with the MEG data, we demonstrate that a deep source is a crucial part within the language network, probably as a moderator between the other cortical regions.

Materials and Methods

Stimuli.

The study involved four crucial experimental conditions crossing the factors syntax and prosody: (1) CC, prosody correct and syntax correct; (2) CS, prosody correct and syntax incorrect; (3) PC, prosody incorrect and syntax correct; and (4) PS, prosody incorrect and syntax incorrect. Each condition contained 48 sentences. Each sentence consisted of a proper name (e.g., Maria) and the verb weiß [knows] comprising the matrix clause. Then, a subordinate clause was attached that was always introduced by the complementizer dass [that] and followed by the subject (e.g., der Rentner [the pensioner]) of the subordinate clause. The penultimate constituent of the subordinate clause was always a prepositional phrase (PP) consisting of a preposition (e.g., im [in the]) and the critical word (e.g., Alter [seniority]), followed by the verb of the subordinate clause (e.g., kränkelt [ails]) at clause-final position. German words that are ambiguous regarding word category until their ending (either verbal suffix or nominal suffix) served as critical words. A nominal suffix (suffix -r) was highly expected due to the preceding preposition (im [in the]) (requiring a noun) and a frequency-based bias of the critical word stems toward the usage as a noun. Nevertheless, given that there was an equal probability of occurrence for both suffix types in the present experiment, a final successful determination of word category could not be achieved until the suffix of the critical word was encountered. Therefore, at the suffix, the critical word was disambiguated toward being a noun (e.g. marked by the suffix -r), making a syntactically correct continuation of the PP (syntax correct), or toward being an inflected verb (e.g. suffix -rt), which would lead to a word category violation (syntax incorrect) because a preposition must be followed by a noun.

To avoid possible strategic effects resulting from the fact that the occurrence of the critical stem with verb suffix always coincided with a syntactical error in the two experimental conditions, we included two syntactically congruent filler conditions in which the critical word stem occurred together with a verb suffix in the correct context (e.g., filler 1: Maria weiß, dass der Rentner im Rollstuhl altert. [Maria knows that the pensioner in the wheelchair grows old.], filler 2: Maria weiß, dass der Rentner altert. [Maria knows that the pensioner grows old.]). All critical words were disyllabic, carrying the default (trochaic) stress pattern. Both filler conditions were naturally recorded. To create the sentences for the four experimental conditions, we recorded the required words in different source sentences (for details, see Eckstein and Friederici, 2006). The same splicing procedure was used to create the correct and incorrect sentences to avoid a general effect of the splicing itself. The first part of the sentence (e.g., Maria weiß, dass der Rentner im [Maria knows that the pensioner in the]) preceding the critical region as well as the sentence-final verb (e.g., kränkelt. [ails.]) was identical for all four experimental conditions. The number of splicing points and their positions were also identical between the four experimental conditions.

For the acoustic characterization of the stimuli, we decided to examine fundamental frequency and word durations for each sentence and condition; see Eckstein and Friederici (2006) for details. Descriptively, the f0-contour for critical words in the prosodically correct conditions marked for sentence continuation (CC and CS) showed a rise–fall pattern (see supplemental Fig. S1, available at www.jneurosci.org as supplemental material). Prosodic violations (PC and PS) showed the reverse fall–rise pattern.

Participants.

We tested 11 participants (mean age 26 years; range from 20 to 31 years of age; four women) after they had given informed consent. All were right handed and reported no neurological, hearing, or language impairments. Participants were paid 7 €/h. Participants listened to the sentences and had to judge the grammaticality of each of them via button press. A picture with a smiley and a sad face was presented 1500 ms after the end of the sentence. Participants had to press the button on the side corresponding to the smiley if sentences were correct and the other button for incorrect sentences. Assignment of the location of the smiley was random, with the happy face occurring equally often on the left- and right-hand sides.

MEG data recording.

Measurements were conducted using a Vectorview MEG device (Elekta-Neuromag). Data were first cleaned of interference and transformed into a fixed head position via MaxMove-Software Elekta-Neuromag. Trial-based epochs exceeding 80 μV (electrooculogram), 4 pT (MEG), or 200 fT/cm amplitude variation within the epoch from −200 to 800 ms were excluded from further analysis. Subsequently, data were bandpass filtered from 0.8 to 5 Hz. Averages time locked to the onset of the suffix of the critical word were computed for each subject and the four conditions.

DCM specification.

Elaborating on the work of Wahl et al. (2008), we studied language processing DCMs that either include or do not include a deep source possibly representing the thalamus and the respective corticosubcortical loops (Fig. 1). The cortical areas considered are Heschl's gyrus (HG, primary auditory cortex), the mid-to-anterior STG, and the opercular structure in the IFG, because they were identified using a cortically distributed source reconstruction (Maess et al., 2010) (see supplemental Fig. S2, available at www.jneurosci.org as supplemental material). As a control of DCM model complexity that may facilitate data fitting when a hidden source is used, we also investigated the consequence on model evidence of using an additional cortical source that was not primarily identified using source reconstruction instead of the deep source. We used the anterior cingulate cortex (CG) for that purpose.

Figure 1.

Schematic overview of the 22 models tested, distributed into five families (rows). The first family includes the six cortical regions but no deep source. Other model families are characterized by the interhemispheric connection, which is between the left and right counterparts of one of three cortical regions: HG, STG, or FOP. The family CG is used as a control. It is an STG family assuming an anterior cingulate region instead of the deep source.

The cortical interconnections were based on the following findings: (1) a large number of sentence comprehension studies specified the inferior portion of the pars opercularis/frontal operculum (FOP) and STG bilaterally to be involved in the processing of syntactic violations (Friederici, 2002); (2) structural and effective connectivity data have demonstrated that HG connects to the lateral planum polare and the STG as well as to the planum temporale and the posterior STG (Upadhyay et al., 2008) and the STG connects to the FOP (Friederici et al., 2006). The respective areas (HG, STG, FOP) of the left hemisphere (LH) and right hemisphere (RH) are connected via the corpus callosum (Huang et al., 2005), which was shown to be responsible for the functional interplay of the two hemispheres during auditory sentence comprehension (Friederici et al., 2007).

DCM analyses focused on slow evoked components, occurring between 0 and 500 ms peristimulus time, which compose the ELAN waveform as suggested by data from Wahl et al. (2008). Six regions were modeled using an equivalent current dipole (ECD) a priori positioned using the following MNI coordinates: Heschl's gyrus [±48, −9, 7]; anterior superior temporal gyrus [±35, 0, 3]; and frontal operculum [±54, 16, −4]. The hidden source was simply modeled using an ECD, positioned at [0, 0, 0] so that it did not project to the scalp assuming a spherical head model. Alternatively, as a control of DCM overfitting, the hidden source was replaced by the CG [0, 36, 28]. Regions were interconnected with forward, backward, and lateral connections as described by David et al. (2005, 2006).

Other DCM parameters were as follows: eight modes for data selection, one discrete cosine transform (DCT) component to remove slow drift, Hanning windowing to remove the effect of very late responses, a downsampling of 2 to speed up computations, no constraint of symmetry on the orientation of ECDs, and no modulation of intrinsic connections. DCM parameters were fitted on the four conditions at once (CC, CS, PC, PS). Differences between conditions were obtained by allowing all extrinsic connections to be modulated. Extrinsic connections were modified between the different models but respected some basic features (Fig. 2): (1) All models were LH–RH symmetric, and the auditory stimulus input was entered on HG, which was at the bottom of the hierarchy. Earlier stages on the auditory pathway [e.g., the inferior colliculus and medial geniculate nuclei (MGN) (Hackett, 2011)] were covered by the model parameters of the extrinsic input on HG; (2) STG and FOP were second and third in the cortical hierarchy, respectively, as embodied using forward and backward connections; (3) HG, STG, and FOP were laterally connected assuming transcallosal communication between corresponding regions in the two hemispheres; and (4) the deep source, which embodies reentrant cortical–subcortical loops that take place after the first flow of information passing through the MGN and HG, was arbitrarily assumed between HG and the STG in the hierarchy.

Figure 2.

A, Generative model for simulations to investigate the influence of the hidden source (H) and two measurable sources (A and B). Locations of A and B were varied randomly. B, Results of Monte Carlo simulations testing two model families: with and without a deep (hidden) source (H). Monte Carlo simulations demonstrated that independent of the location and the distance between the visible sources, the model posterior probability provides stable information whether or not inclusion of a hidden source constitutes a better model.

Model evidence at the group and family level was estimated using a random effects analysis of the negative free energy obtained for every subject and model (Stephan et al., 2009; Penny et al., 2010). Family level inference removes uncertainty about aspects of model structure other than the characteristic of interest. We used five families (Fig. 1): models without the deep source (No deep), models with the deep source differentiated according to the lateral connections (HG, STG, and FOP), and models with the anterior cingulate cortex (CG). Extrinsic connectivity parameters of the winning families were estimated at the group level using Bayesian model averaging (BMA) (Penny et al., 2010).

DCM simulations.

Monte Carlo simulations were performed with scalp data to demonstrate the possibility of detecting the presence of a hidden source involved in the brain network having generated those data. The head position (306 channels, Elekta-Neuromag system) of a subject taken at random was used for simulations. The first step consisted of generating scalp data. Two sources were positioned randomly in the brain and the associated forward model was computed using a spherical head model. One example of the sources together with their corresponding time courses is displayed in Figure 2A. Cortical sources are labeled as A (blue) and B (red) and an additional hidden relay source as H (green).

The dynamics of the sources were created using the neural model of DCM assuming forward and backward connections between subsequent sources following two configurations: (1) a simple model only composed of the two measurable sources; (2) an augmented model containing a hidden (deep) source between the two measurable sources located at random. For each model, scalp data were obtained by integrating the differential equations of the corresponding neuronal models, with parameter values chosen randomly (using the prior mean value plus a random component of maximum value equal to 50% of the prior variance). Fifty realizations were considered for the Monte Carlo simulations.

In the second step, synthetic scalp data (generated with and without a hidden source) were adjusted under two assumptions: using a DCM composed of the two measurable sources (A and B), or using a DCM that contained the additional hidden source (H). Each model was the true model for only one set of scalp data. For each condition, fixed effect model posterior probability (Stephan et al., 2009) was computed to assess the possibility of DCM inference to accurately detect the presence or absence of the hidden source.

Results

Behavioral data

Overall, performance on the task was highly accurate, with all participants having an average response rate of >90% correct on the trials. Data from trials in which participants gave incorrect responses were not excluded from analysis.

DCM simulations

We found that models including the deep source recover the temporal time series almost perfectly. It should be noted that even the time course of the hidden source was correctly estimated when appropriately assumed. In contrast, the cortical time courses were rather poorly recovered when the hidden source is wrongly assumed to be absent. Additionally, the value of the negative free energy, an approximation of the log evidence of each model, was higher for the true model, the one with the hidden source.

Model posterior evidence computed on randomly generated data (Monte Carlo simulations) shows clearly that, in principle, it is possible to correctly detect the presence or the absence of the hidden source (Fig. 2B).

DCM modeling of language data

The competing DCMs differed in the topology of subcortical–cortical loops and in their direct interhemispheric connections (Fig. 1). Comparing model exceedance probability of all 22 models and of all five families tested, we found clear evidence of the presence of a hidden source whatever the cortical connectivity (Fig. 3A,B). Indeed, exceedance probability summed over the three families with the deep source (HG, STG, FOP) was >0.95. Models 1–6 and their “No deep” family only reached very low values of exceedance probability. As an empirical demonstration of the actual presence of a deep source, the evidence of family CG including an anterior cingulate source was well below the evidence of the other families having the deep source (HG, STG, FOP)—see Figure 3B. The profile of exceedance probability between HG, STG, and FOP families may indicate that transcallosal connectivity at a higher level in the cortical hierarchy is more appropriate to explain the data. However, the three best models, which represented 86% of exceedance probability, belonged to each of those three families (model 10: HG, model 11: STG, model 15: FOP)—see Figure 3C. This suggests that it was not possible to distinguish between HG, STG, and FOP. Therefore, these families were considered winning families and were included in BMA of connectivity parameters.

Figure 3.

Bayesian model selection based on exceedance probabilities for different scenarios. A, Comparison of the models 19–22 (STG including CG) versus models 11–14 (STG including deep). Note that there is just a minor exceedance probability for the models including CG. B, Comparison of all model families investigated: no deep source, deep source + transcallosal connection between left and right counterparts of HG, STG, or FOP, and finally CG + transcallosal connection between the two counterparts of STG. Note that the three winning families together (all including the deep source) have 95% exceedance probability. C, Exceedance probabilities for each model separately. Note that the three best models together pass an exceedance probability of 85%.

Mean brain evoked responses were computed by averaging over subjects and winning families (Fig. 4A). These mean evoked responses showed increased amplitude in response to violations, from 100 ms peristimulus time onwards in almost all conditions and locations. Specifically, syntactic violations appeared to have more impact than prosodic violations on subcortical activity. The link between amplitude (Fig. 4A) and connectivity (Fig. 4B) modulations, however, was difficult to assess because of heterogeneous connectivity over conditions. The most consistent finding in relation to violations was a decrease in cortical (no violation: 1.41; violation: 1.13) and subcortical (no violation: 2.35; violation: 1.14) reentrant connectivity to HG. Transcallosal lateral connections were also modulated and showed a global decrease in connectivity whenever a violation was detected (no violation: 0.71; violation: 0.36). The difference in conditional modulation of connectivity between the correct and the other three experimental conditions is displayed in Figure 4C.

Figure 4.

Time courses and connectivities of winning model families. A, Mean time courses of the winning families (all models that included a deep source). B, Bayesian model averages including models of the winning families (HG, STG, FOP): mean connectivity between all regions. Note the stronger connection (dark red) from the deep source to the cortical region (LHG)—right upper corner of the CC square. Blue means lowest connectivity. The redundant and nonexisting connections are displayed in gray. The panels display the values for the different conditions CC, CS, PC, and PS, respectively. Direction of connectivity is indicated in the CC-CC panel of part C: “from” is displayed at the top/bottom and “to” on the left/right. C, Bayesian model averages based on models of the winning families (HG, STG, FOP): difference in the mean connectivity between all regions to the condition CC. Note that green means difference equals zero. The redundant and nonexisting connections are displayed in gray. The upper left panel displays the values for condition CC-CC, the upper right panel—condition CS-CC, the lower left panel—condition PC-CC, and the lower right panel—condition PS-CC.

Discussion

This study is important for two reasons. First, from a methodological point of view, we have shown that hidden neural activity (i.e., activity that is not recordable using noninvasive methods) can be recovered using parameter estimation of biophysical models of brain networks (dynamic causal modeling) (David et al., 2005, 2006). Second, the present data allow us to make specific inferences on subcortical–cortical connectivity during auditory language processing.

The ability to record deep brain structures in EEG or MEG has been a matter of debate, and several studies have suggested that thalamic activity is actually be recordable in MEG or EEG (Tesche, 1996; Gross et al., 2002; Attal et al., 2007). When comparing intracerebral and scalp data, it is nonetheless clear that scalp recordings do not capture the activity of deep structures well (Ray et al., 2007; Dalal et al., 2009). Here, we made the radical assumption that deep structures (e.g., thalamus) were not recorded (i.e., their activity did not project to the scalp), and we proposed to indirectly estimate depth activity by the means of biophysical modeling of neuronal connections.

Dynamic causal modeling allows the estimation of neuronal parameters (connectivity and time constants) from EEG or MEG evoked responses (David et al., 2006). Using a Bayesian framework, it is possible to make inferences on estimated parameters, and most importantly, to compare different models on the basis of their evidence, which combines the goodness of fit to the data of model dynamics and model complexity. The simple idea we tested to investigate the presumed influence of deep nuclei was to compare models with or without a hidden (i.e., deep) source. Because a model with a deep source is more complex than a model without, given the same number of cortical sources, it is selected as a better model only if it allows the better reproduction of recorded scalp data through the modulation of brain dynamics by the means of subcortical/cortical connections. Using a simulation (Fig. 2A,B), we demonstrated the face validity and potential value of this new method of modeling the sources of MEG or EEG data. This approach is in fact radically new and opens avenues for the noninvasive study of thalamocortical loops under physiological and pathological conditions. We applied it to investigate the possible role of the thalamus and basal ganglia during the processing of syntactic and prosodic violations in auditory sentence comprehension.

Overall, our MEG DCM results suggest the involvement of a deep source in language processing. Although DCM does not provide a definitive answer here, this deep source very likely represents the thalamus, as intracranial recordings in a study using partly similar material indicated increased thalamic responses to syntactic violations in an earlier time window (Wahl et al., 2008). Here, the particular modulations in connection strength suggest different networks react as a function of syntactic and prosodic information. Syntactic information, which is known to be processed in the left hemisphere (Friederici, 2002; Hagoort, 2005; Hickok and Poeppel, 2007), indeed leads to increased responses in the left STG and FOP, but also in the right FOP and in the deep source. Prosodic information is known to be processed in the right hemisphere (Meyer et al., 2004). Not surprisingly, we observed that prosodic violations involved right-sided modulation of cortical activity but also elicited an increase in subcortical activity, but this was weaker than the one evoked by syntactic violations.

In terms of connectivity patterns, contextual modulations are complex. A consistent feature is a decrease in cortical and subcortical reentrant inputs to HG whenever a violation is detected. Given that the syntactic violations investigated in the present study are reflected in the EEG and MEG experiments as an early effect whose dipolar activity is located in the superior temporal gyrus anterior to HG (Knösche et al., 1999; Friederici et al., 2000) or in close vicinity to it (Herrmann et al., 2009), the reentrant loop between HG and the deep source suggests that the information from the secondary auditory cortex is referred to the deep nuclei for further processing, which affects high- and low-level cortical processing. This is backed up by cortical and intracranial recording data indicating that the cortical effect precedes the respective effect in the thalamus (Wahl et al., 2008).

In addition, we observed a global decrease in transcallosal connections in relation to violations, particularly at the level of HG and FOP. The decrease for the interhemispheric connection between the FOP, HG, and STG sources is in line with the assumption that during normal auditory language processing, both hemispheres work in parallel, with the left hemisphere being primarily responsible for syntactic and lexical–semantic processes and the right hemisphere for prosodic processes. In conditions where one hemisphere detects a violation, possibly via fast thalamocortical (HG) loops, the intercallosal connectivity is reduced to allow independent processing of syntax and of prosody.

To sum up, these results can be interpreted within the theory of predictive coding (Friston, 2005, Friston and Kiebel, 2009). During language learning, both hemispheres acquire prior representations of syntactic and prosodic information. In the presence of syntactic or prosodic violations in a given sentence, after early auditory stimulus processing in HG, thalamus, and STG/FOP, error signals are send back by means of decreased reentrant connectivity to HG and decreased transcallosal connectivity. These error signals may serve the purpose to process this sentence using prior representations of language to repair syntactic or prosodic structures if possible, most likely by dissociating both processes to a certain extent. Overall, our MEG DCM results are highly suggestive of where increased thalamic responses to language violations were measured using intracranial recordings (Wahl et al., 2008). DCM was thus successful in demonstrating that the thalamus is a crucial part within the language network, probably as a moderator between the other cortical regions. As a result, our study opens new avenues for studying the role of subcortical structures in human cognition.