Brain activation for language dual‐tasking: Listening to two people speak at the same time and a change in network timing

Abstract

The study used fMRI to investigate brain activation in participants who were able to listen to and successfully comprehend two people speaking at the same time (dual‐tasking). The study identified brain mechanisms associated with high‐level, concurrent dual‐tasking, as compared with comprehending a single message. Results showed an increase in the functional connectivity among areas of the language network in the dual task. The increase in synchronization of brain activation for dual‐tasking was brought about primarily by a change in the timing of left inferior frontal gyrus (LIFG) activation relative to posterior temporal activation, bringing the LIFG activation into closer correspondence with temporal activation. The results show that the change in LIFG timing was greater in participants with lower working memory capacity, and that recruitment of additional activation in the dual‐task occurred only in the areas adjacent to the language network that was activated in the single task. The shift in LIFG activation may be a brain marker of how the brain adapts to high‐level dual‐tasking. Hum Brain Mapp, 2012. © 2011 Wiley Periodicals, Inc

INTRODUCTION

This study investigates the brain activation of participants in a multitasking situation where they try to understand two people speaking at the same time. We examined the cortical activation in a dual task that involved simultaneous comprehension of two streams of spoken input compared with the activation during the comprehension of a single speech stream. Brain activation in dual‐tasking was investigated in terms of (1) the brain areas recruited only for the dual task; (2) changes in cortical network synchronization as measured by functional connectivity; and (3) the individual differences in the brain mechanisms for dual‐tasking in individuals varying in language processing capacity but who can all perform the dual task.

The experimental dual task in this study corresponds to cognitively‐demanding real‐life situations that require the ability to process two simultaneously presented information streams, such as listening to a radio newscast while being spoken to by a conversation partner. Our pilot studies showed that even among college student participants, many of the potential participants could not answer straightforward comprehension questions about two concurrent sentences that they had just heard. Therefore, the study focused on those participants who were successful at dual comprehension, as one might study chess experts. In choosing to investigate the neural basis of this extremely interesting and possibly unusual dual‐tasking expert ability, we have put aside the questions of what occurs in the brains of people who cannot adequately comprehend two concurrent streams of speech, as well as the question of what distinguishes successful from unsuccessful dual comprehenders. Although the analogy to chess expertise is imperfect, it illustrates the extremely illuminating practice in cognitive science and cognitive neuroscience of explaining the basis of outstanding abilities.

It is noteworthy that the automaticity of two tasks may affect how they are performed concurrently. [We refer to tasks as automatic if they do not require appreciable executive control by the frontal‐lobe systems (Chein and Schneider,2005)]. All previous dual tasks that have been studied with fMRI entail at least one nonautomatic task, and most entail two nonautomatic tasks. For example, the dual task examined by D'Esposito et al., (1995) used two decidedly nonautomatic tasks: semantic classification of words and mental rotation. The studies that examined at least one nonautomatic task have shown that one of the ways in which the brain adapts to dual‐tasking is by recruiting additional prefrontal executive brain areas which are not activated when the single tasks are performed (e.g., D'Esposito et al.,1995; Jaeggi et al.,2003). Thus, we chose to examine the concurrent performance of two instantiations of a highly automatic yet complex task, namely listening comprehension, to determine the neural characteristics of multitasking when only automatic tasks are involved.

It is possible that dual‐tasking of two concurrent automatic tasks (namely listening comprehension) may not result in additional activation in executive areas of the brain relative to single tasks, though there might be an increase in activation in task‐specific areas of the language network. [The main task‐specific areas associated with listening comprehension are the left inferior frontal gyrus (LIFG) and the superior and middle areas of the posterior temporal lobe (Constable et al.,2004;Keller et al.,2001; Michael et al.,2001;)]. The hypothesis that a dual task composed of highly automatic tasks would not recruit additional brain activity in areas associated with executive control finds support in the recent literature. Dux et al., (2009) showed that with increased automaticity and improved performance following training in dual‐tasking there was less activation in areas of the executive network and more activation of the task‐specific areas involved in the single tasks (one visual and one auditory sensorimotor task). To our knowledge, there are no previous brain‐imaging studies on dual‐tasking that combine two automatic, high‐level tasks.

Another distinguishing characteristic of comprehending two spoken sentences concurrently is that the task does not easily lend itself to task switching. One cannot ignore one of the sentences for very long without compromising the ability to comprehend the message. In this sense, it seems implausible that participants will be able to attention‐switch between the two sentences, although there appears to be a phenomenological experience of some alternation of attention in which one of the sentences dominates one's conscious experience. By contrast, some dual tasks have consisted of component tasks that each contained a sequence of simple items that were each acted‐upon quickly, say in about 1 sec per item. Dual‐tasking in such cases can often be accomplished by acting on one item at a time and alternating between the items from the two tasks, for example, letter recognition and orientation judgment (Kondo et al.,2004) or luminance and pitch discrimination tasks (Klingberg and Roland,1997). This study focuses on a high‐level dual task composed of two automatic, continuous tasks. The continuous tasks will likely have to be processed simultaneously if participants are to maintain high performance levels.

Although previous fMRI studies of dual tasks have focused on the amount and location of the brain activation, this study additionally examines the effects of dual‐tasking on the relative timing of the activation in various cortical areas. One way to assess relative timing is through functional connectivity, a measure of the degree of synchronization among the active brain areas. Functional connectivity analysis provides another way to examine changes in cortical function in response to dual‐task demands at the network level rather than at level of individual brain areas. Functional connectivity analyses can characterize the differences in internode synchronization of activation in dual task as opposed to single task conditions. Functional connectivity and its adaptiveness is one of the properties of brain systems believed to underlie individual differences in cognitive performance (Prat and Just,2008). It has been shown that increasing the difficulty of a comprehension task results in greater functional connectivity increases among areas of the language network in more‐skilled participants than in less‐skilled participants (Prat et al.,2007). We expected that there would be an increase in the synchronization of activation among the nodes of the language network in the dual task, possibly reflecting a need for more efficient inter‐node communication.

The study also aimed to investigate the individual differences in the brain mechanisms for dual‐tasking in individuals varying in language processing capacity but who can all perform the dual task. We investigated if individual differences in working memory capacity for language (as measured by the Daneman and Carpenter (1980) Reading Span Test) were associated with brain activation characteristics in the dual task. The reading span test is a well known psychometric measure that predicts performance in a number of language comprehension tasks. In this study, the measure of reading span should predict performance in the single‐sentence listening comprehension tasks, as it has done in previous studies (see Daneman and Merikle,1996, for a review). However, previous studies showed that individual differences in working memory capacity were not correlated with individual differences in multitasking ability (Jaeggi et al.,2008). It is possible that for our population of high‐performing participants individual differences in working memory capacity will not predict performance in the dual task; among the individuals who can perform the dual task there may be both lower and higher reading span participants.

The study examined how the brain activation changes in the dual task (relative to single‐message comprehension) in participants who succeed at the dual comprehension. Both the single task and dual task listening comprehension are expected to draw on the same network of neural resources and, since the demand on this network is greater in the dual case, we expected greater activation in the dual‐task. Of central interest was the comparison of the temporal structure (timing) of the activation between the single and dual task conditions, and the presence of systematic individual differences in timing effects of dual‐tasking.

MATERIALS AND METHODS

Participants

Twelve right‐handed college students (8 females) between the ages of 18 and 28 years (M = 20.75; SD = 2.9) were included in the final analyses. All were native speakers of English. Each participant gave signed informed consent approved by the University of Pittsburgh and Carnegie Mellon University Institutional Review Boards. Each participant received 30 to 45 min of practice with the experimental paradigm before performing it in the scanner. The reading span test (Daneman and Carpenter,1980) was administered during the practice session. Only participants who achieved an accuracy rate of at least 75.0% in the dual task during the practice session were tested in the scanner. Moreover, not all of those tested in the scanner maintained a 75.0% accuracy level during the scanning session; the fMRI data were discarded from 10 participants who did not achieve minimum accuracy, despite having done so in the practice session. Data from three additional participants were discarded due to technical problems during the scan.

Procedure

Experimental design

Participants were presented with a single spoken sentence or two concurrent sentences (one to each ear) and they judged the sentences as true or false of the world in an event‐related design. There were three experimental conditions: Dual Message, Left Ear Message, and Right Ear Message. For each condition, there were three epochs, each containing four trials of the same condition, providing 12 trials per condition. A trial consisted of the presentation of a single sentence or, in the Dual condition, a pair of concurrent sentences. The start of each trial was synchronized with the onset of a TR. Each epoch of a given type was preceded by a two‐second visual cue designating condition type (Left, Right, or Dual). The epochs for each condition were distributed according to a Latin square design. In addition to the sentence epochs, there were four presentations of 24‐second fixation epochs to provide a baseline measure of each participant's brain activation: one at the beginning of the experiment, and one after every three epochs.

The stimulus sentences, referring to general world knowledge, often contained two clauses (which were mutually inconsistent in the false items) and were 12 to 16 words long. An example of the true sentence (which constituted 75.0% of the stimuli) is Man‐made insulin is a helpful drug and it decreases the many symptoms of diabetes and an example of a false sentence is During leap years, February has 29 days, and this occurs once every eight years. Level of sentence difficulty (how unfamiliar the relevant world knowledge was) was evaluated in a norming study and balanced across conditions. Sentences were created using TextAloud, a text‐to‐speech program, and were edited for pronunciation and timing with Goldwave. The mean utterance duration of each sentence was ∼ 6 seconds. Half were spoken in a female voice, half in a male voice, and voice gender was balanced across conditions. In the Dual condition, the two messages were spoken by different genders. Each sentence was preceded by one of two prompts: “Ready to go” or “About to start,” spoken in the same voice and ear as the subsequent stimulus, with 1 sec duration. Although the sentences and prompts were presented, an asterisk was displayed on the center of the screen, upon which the participants were instructed to fixate. Each sentence was followed by a three‐second response period in which true/false responses were recorded, which in turn was followed by a ten‐second rest period in which participants fixated on an “x” displayed in the center of the screen. The sentences were presented via Avotech MR‐compatible headphones. Comprehension accuracy was measured as the proportion of correct responses for the true/false judgments. The true/false responses were made using two‐button mice, one in each hand. Participants responded to sentences presented in their left ear with their left hand and to sentences presented in their right ear with their right hand. The index finger on each hand was used to respond “true”, and the middle finger to respond “false”.

One or 2 days before scanning, participants were given out‐of‐scanner practice to screen out people who could not perform the dual comprehension task with adequate accuracy. They were presented 44 practice dual‐task comprehension trials at least once and then with a second iteration of the 44 trials if necessary. Those participants who could not achieve 75.0% accuracy in either iteration were not included in the imaging study.

fMRI Acquisition Parameters

The data were collected using a Siemens Allegra 3.0T scanner with a commercial birdcage, quadrature‐drive radio‐frequency head coil. Data acquisition was conducted at the Brain Imaging Research Center jointly operated by Carnegie Mellon University and the University of Pittsburgh. The study was performed with a gradient echo, echo‐planar pulse sequence with TR = 1000 ms, TE = 30 ms, and 60° flip angle. Sixteen oblique‐axial slices were imaged. Each slice was 5‐mm thick with a gap of 1 mm between slices. The acquisition matrix was 64 × 64 with 3.125 × 3.125 × 5‐mm voxels.

fMRI Analyses

The data were analyzed using SPM2 (Wellcome Department of Cognitive Neurology). Images were corrected for slice acquisition timing, motion‐corrected, normalized to the Montreal Neurological Institute (MNI) template, resampled to 2 × 2 × 2‐mm voxels, and smoothed with an 8‐mm Gaussian kernel to decrease spatial noise. Statistical analysis was performed on individual and group data by using the general linear model as implemented in SPM2 (Friston et al.,1995). The model for each participant included regressors for each of the three conditions of interest and for the fixation condition, convolved with the canonical SPM2 hemodynamic response function. The response periods and rest intervals between trials were not explicitly modeled. To compare the distribution of activation across the three experimental conditions, group t‐test analyses were performed using a random‐effects model (Friston et al.,1999) using sentence versus fixation contrast images (one per participant, per contrast). For the group‐level contrasts between the dual task and each of the single tasks we applied a mask that excluded voxels with negative values in either of the single tasks (deactivation mask). All t‐maps in each contrast were calculated across the entire cortical volume, thresholded at an uncorrected height threshold of P < 0.001 and an extent threshold of 20 voxels. Statistical maps were superimposed on the high‐resolution, normalized, T1‐weighted, SPM2 individual template image for viewing. Labels for coordinates of activation were confirmed in MNI space (Tzourio‐Mazoyer et al.,2002) and the Talairach Daemon (Lancaster et al.,2000), as implemented in AFNI (Cox,1996).

Anatomical Regions of Interest Definitions

To compare the amount of activation across conditions in the cortical regions that are central to spoken language comprehension, five anatomically defined ROIs in each hemisphere covering the language network activation observed in this task were defined in the Montreal Neurological Institute (MNI) space using the parcellation proposed by Tzourio‐Mazoyer and colleagues (2002) for the single‐participant MNI brain. Our study used three temporal lobe regions (T1, T2, and HES, corresponding to the superior temporal gyrus, middle temporal gyrus, and Heschl's gyrus) and two frontal lobe regions (F3OP and F3T, corresponding to the opercularis and triangularis parts of the inferior frontal gyrus).

Analysis of Subsets of Activated Voxels Locations

The analysis for voxels activated only in each single task and in the dual task was carried out using an exclusive mask based on the activation maps of the left‐out task conditions. The mask excludes any voxel that (a) showed a negative contrast value for all the left‐out task conditions relative to fixation at the group level, and (b) showed significant activation for the left‐out task conditions relative to fixation at the group level, at a threshold of T = 4.02 (P < 0.001 uncorrected). The subsets were all three tasks; only Left‐Message voxels; only Right‐Message voxels; only Left Message and Dual Message voxels; only Right Message and Dual Message voxels; only Left Message and Right Message voxels; and only Dual‐Task voxels (New Activation).

Functional Connectivity

Functional regions of interest were defined to encompass the main clusters of activation in the group activation maps at a threshold of T = 3.0 for each of three contrasts: Right Ear Message > Fixation; Left Ear Message > Fixation; and Dual Message > Fixation. For each contrast, spheres were defined that best captured the activation in a particular region. The final set of functional regions of interest included areas in the frontal and temporal regions of the language network: two regions in the left inferior frontal gyrus, one posterior (LIFG1: x = −50, y = 24, z = −8) and one extending anteriorly into the insula (LIFG2: x = −52, y = 18, z = 18); regions in the left middle temporal gyrus (LMTG1: x = −60, y = −42, z = 4, LMTG2: x = −60, y = −20, z = −14); and regions in left and right superior temporal lobes (LSTG1: x = −60, y = −10, z = 4, LSTG2: x = −48, y = −26, z = 6; RSTG1: x = 62, y = −12, z = −10, RSTG2: x = 50, y = −24, z = 8). These functional regions of interest had a radius of 10 mm, except for the RSTG1 and RSTG2 regions, which had a radius of 12 mm. There were no functional regions of interest drawn for DLPFC because there were no significant clusters of activation in this area in the group‐level contrasts.

Functional connectivity was computed separately for each participant in each condition as a correlation between the average time course of signal intensity of the union of the activated voxels in each member of a pair of the functional regions of interest described above. The activation time course extracted for each participant over the activated voxels within each functional region of interest was based on the normalized and smoothed images that had been low‐pass filtered and had the linear trend removed. One participant was excluded from further analysis because the number of voxels activated for that participant in either of the functional regions of interest constituting the pair was less than 12. The correlation between the time courses for each pair of functional region of interest was computed only on the images belonging to the experimental condition and excluded the fixation condition. Therefore, the correlation reflects the relation between the activation in the two areas while the participant was performing the task. Fisher's r‐to‐z transformation was applied to the correlation coefficients for each participant before statistical analysis. To ensure that the signal extraction from the union of the activated voxels in all conditions did not give the Dual Message condition an unfair advantage over the single conditions (there were more activated voxels in the Dual Message condition), a second functional connectivity analysis was carried out. In this analysis, the signal was extracted separately for each condition for the activated voxels in the condition in each member of a pair of regions of interest.

To summarize the results for differences in functional connectivity between single and dual conditions, the regions of interest (ROIs) were grouped into three aggregated functional regions. The first region included the language network areas in the left frontal lobe, consisting of the two LIFG ROIs; the second region included the language network areas in the left temporal lobe, consisting of the two LMTG and two LSTG ROIs; and the third region, the areas in the right temporal lobe, the RMTG and two RSTG ROIs. The functional connectivity analysis for the language network was computed in terms of these three aggregated measures of inter‐region connectivity.

Correlation Among Functional Connectivity, Reading Span, and the Shift in LIFG Timing

Three correlations were measured: (1) Correlation between reading span and the timing shift from single‐ to dual‐tasking in the peak of the hemodynamic response in LIFG. The peak was measured as the center of the Gaussian curve fitted to the percent signal change time course data for LIFG. (2) Correlation between change in LIFG‐LMTG peak time difference and change in functional connectivity from single to dual task. The change in time difference between LIFG‐LMTG was measured as the ratio of the difference between LIFG and LMTG time to peak in the single tasks minus the difference between these peaks in the dual task, divided by the difference between the peaks in the single tasks. The change in functional connectivity was the increase in functional connectivity z scores for the LIFG‐LMTG pairs from single to dual‐tasking (z score dual task minus z score single task). (3) Correlation between reading span and the change in the peak timing difference between LIFG and LMTG from single‐ to dual‐tasking.

Analysis of Time Courses

Additional analyses of the average time course of signal intensity for the language network were carried out. The objective of the analyses was to uncover changes in the temporal pattern of brain activation from single‐ to dual‐tasking, because any such change would affect functional connectivity. The average time course of percent signal change from fixation for each condition, each participant, and each functional region of interest was fitted to a simple Gaussian function using the least‐squares error method in order to quantify the relative timing and duration of the fMRI (hemodynamic) responses. The fit to the observed data thus provided estimates of the time to peak response and the duration of the response, which was estimated as the full‐width at half maximum (FWHM) of the Gaussian fit for values of the estimated Gaussian curve that were above zero percent signal change. To summarize the results for differences between single‐ and dual‐task conditions, the same ROIs used in the functional connectivity analyses were used here, and statistical analyses were performed on the average peak and width value for each individual participant for each aggregate region of interest.

RESULTS

Behavioral Results

The analyses were applied to participants who were able to process two concurrent auditory sentences with good (at least 75.0%) comprehension accuracy, measured as the ability to judge whether the stimulus sentence was true or false of the world. Despite the fact that there was approximately twice as much information per unit time to process in the dual condition, the comprehension accuracy level in the dual condition (Mean Dual Message accuracy = 82.0%; SE = 1.9%) was not reliably different from the single‐message conditions (Mean Right Ear Message accuracy = 87.0%, SE = 3.4%; Mean Left Ear Message accuracy = 85.0%, SE = 4.7%). In the Dual condition, the accuracies were the same for the messages played simultaneously in the left and right ear (82.0% for both; SE Right Ear Message = 3.2%; SE Left Ear Message = 2.6%). This indicates that participants were attending to both sentences equally in the dual task. The reading span scores had a mean of 3.2, SE = 0.18, range 2–4. As expected, the reading span scores were positively correlated with the comprehension accuracy in the single tasks (r = 0.68, t = 2.93, P < 0.01), but not with the comprehension accuracy in the dual tasks.

Response times to the true/false comprehension probes were slower in the dual‐task condition (M = 1272 ms; SE = 312 ms). The response times for each sentence in the dual task were approximately the same: Right Message in Dual M = 1278 ms (SE = 321 ms); Left Message in Dual M = 1266 ms (SE = 315 ms). The response times in the single tasks were faster than the dual: single‐task Right Ear Message M = 852 ms (SE = 231 ms) and single‐task Left Ear Message M = 942 ms (SE = 336 ms), [(F(2, 22) = 35.61, P < 0.001]. Thus, there may be some cost to dual‐tasking in the present task, but it is only found in terms of speed of response. The longer response times in the dual condition may be due to motor response processing rather than language comprehension processes. For example, participants may be making a serial response to the dual task, i.e., first one true/false response to one sentence in one hand, and then the other response in the other hand, rather than both at the same time.

fMRI Results

Core language network activation across conditions

The activation analyses revealed a core network of left inferior frontal and bilateral temporal areas that were activated in all three conditions. Activation clusters common across conditions were found in the left and right middle and superior posterior temporal gyri, the left inferior frontal gyrus, the left supplementary motor area, and the left thalamus (see Fig. 1 and Table I). No reliable activation was found in DLPFC in either the single‐ or dual‐task contrasts with fixation, as indicated by the empty red ovals in Figure 1 (see also Table II).

Figure 1.

Contrasts of dual task > single tasks and of each condition relative to fixation; clusters of voxels significant at P < 0.001, uncorrected, extent threshold = 6 voxels, T = 4.02. The dual task > right, dual task > left, and dual task > fixation contrasts show that there was no more activation in left DLPFC in dual than in the single tasks (red ellipses).

Table I.

Right, left, and dual message activation compared with fixation

Activation for R message	Voxels	T(12)	MNI
			x	y	z
Frontal
L inf frontal + precentral gyri +insula	977	7.83	−50	26	−8
L supp motor area	367	7.68	−8	16	50
L precentral gyrus	49	3.80	−50	6	48
Temporal
L mid + sup temporal gyri	3302	15.17	−62	−20	−12
R sup + mid temporal gyri	1713	11.53	58	−10	−12
Subcortical
L thalamus + parahippocampal gyrus	1067	12.89	−6	−22	−22
L caudate	24	5.54	−10	6	16
Activation for L message
Frontal
L supp motor area	143	7.06	−6	16	52
L inf frontal gyrus	191	6.42	−60	16	18
L inf frontal gyrus	144	6.26	−52	24	−6
R inf frontal gyrus + insula	33	4.87	36	30	−4
Temporal
L mid + sup temporal gyri	2529	13.16	−60	−6	−12
R sup + mid temporal gyri	2326	10.63	62	−12	−10
Subcortical
L thalamus	72	4.97	−2	−20	4
Activation for dual message
Frontal
L + R supp motor area	1080	8.22	2	14	54
L precentral + L inf frontal gyri	519	7.06	−52	6	46
L inf frontal gyrus + insula	572	6.88	−50	24	−8
R insula + inf frontal gyrus	154	5.62	36	24	−2
R precentral gyrus	32	3.58	44	0	46
R sup frontal gyrus	34	4.63	26	−6	56
Temporal
L mid + sup temporal gyri	3605	17.11	−62	−20	−10
R sup + mid temporal gyri	2960	11.03	62	−12	−10
Subcortical
L thalamus	57	5.14	−6	−8	2

Note: Clusters of voxels significant at P < 0.001, uncorrected, extent threshold = 20 voxels. Region labels apply to the entire extent of the cluster with peak maxima designated by first locale cited. T−values and MNI coordinates are for the peak activated voxel in each cluster.

Table II.

Activation for dual > right and dual > left contrasts

Activation for Dual > Right	Voxels	T (12)	MNI
			x	y	z
Frontal
R inf frontal gyrus + insula	176	8.65	40	22	−4
L insula + inf frontal gyrus	400	8.44	−40	14	8
L precentral + sup frontal gyri	136	8.01	−30	0	60
R + L supp motor area	778	7.44	0	8	58
R inf frontal gyrus	46	6.97	52	16	0
L mid + anterior cingulate gyri	93	6.12	−6	28	30
Temporal
L mid + sup temporal gyri	1796	15.05	−64	−20	4
R sup + mid temporal gyri	2447	10.52	54	−22	8
Activation for dual > left
Frontal
L inf frontal gyrus + insula	535	10.30	−42	16	2
R supp motor area	1097	10.30	8	16	50
L precentral gyrus	456	9.40	−28	−2	56
R inf frontal gyrus	696	7.59	36	4	40
R insula	38	5.54	32	20	−2
L inf frontal + precentral gyri	62	5.26	−54	12	24
Temporal
L mid + sup temporal gyri	2938	17.45	−52	−10	−10
R mid + sup + inf temporal gyri	1811	10.67	58	−46	4

Note: Clusters of voxels significant at P < 0.001, uncorrected, extent threshold = 20 voxels. Region labels apply to the entire extent of the cluster, with peak maxima designated by first locale cited. T‐values and MNI coordinates are for the peak activated voxel in each cluster only.

Cortical Activation in Dual‐Tasking vs. Single‐Tasking

Dual‐tasking produced more activation than either of the single‐task conditions in areas associated with language processing: bilateral inferior frontal gyri, bilateral posterior superior gyri, and middle temporal gyri. The greater amount of activation was assessed in two ways: (i) a greater volume of tissue becomes activated, and (ii) the same tissue becomes activated to a higher level (Just et al.,1996). In terms of (i) greater volume of activation, there were more activated voxels in the dual task. Figure 2 plots the number of language‐network activated voxels in the dual task in three anatomically‐defined ROIs that correspond to common areas of activation (see Materials and Methods). In terms of (ii) (the same areas becoming activated to a higher level), our results also show an increase in the level of activation in the contrast between the dual‐task activation with each of the single tasks. There was a reliable increase in the level of activation of the areas of the left‐hemisphere language network and their right‐hemisphere homologues. Figure 3 shows the recruitment in dual tasking of additional cortical areas surrounding the core language network. Dual‐tasking also resulted in more activation than the single tasks in the premotor cortex. Even though the response periods were not explicitly modeled in the analysis, it is possible that participants are planning responses before the end of the 6‐second presentations.

Figure 2.

Activated voxels in single and dual tasks: more voxels activated in language‐network areas in dual‐tasking activated voxels for a threshold of T = 4.02 in the anatomically‐defined regions of the language network. LIFG is the mean of the activated voxels in the opercularis and triangularis parts of the inferior frontal gyrus [F3OP and F3T (Tzourio‐Mazoyer et al., 2002)]. The Temporal anatomical regions are the mean for the posterior and mid sections of T1 and T2, and HES (Tzourio‐Mazoyer et al., 2002).

Figure 3.

Cortical areas activated in dual‐tasking: core network of shared voxels and the recruitment of areas circumscribing the core with increasing task difficulty clusters of voxels significant at P < 0.001, uncorrected, extent threshold = 6 voxels, T = 4.02: (a) surface rendering showing overlap of the three conditions with the core of voxels activated in all three conditions (intersection of voxels activated in Left Message, Right Message, and Dual Message); (b) sagittal view of the right‐hemisphere and axial view showing the subsets of activated voxels that circumscribe the core of commonly activated areas; (c) sagittal view of the lefthemisphere and coronal view showing the subsets of activated voxels. Dual task–only activation shows clusters that circumscribe the subset of voxels activated in the single tasks only and the intersection of voxels activated in the dual task and one of the single tasks (right and dual only or left and dual only).

Interestingly, the contrast between dual and single tasks did not show any extra activation in DLPFC for dual‐tasking, a different finding from other dual tasks that involved component tasks that were less automatic and were amenable to task‐switching (D'Esposito et al.,1995; Jaeggi et al.,2003; Kondo et al.,2004). The absence of activation in DLPFC in the dual‐task condition when compared with single‐task may indicate that participants were able to process two sentences concurrently without the need for controlled task‐switching.

The spatial relationship among the voxels activated in the three conditions shows a three‐stage “nesting” effect of additional activation as a function of condition. First, at the center of the nesting lies the common core of the language network that is activated in all three tasks, consisting of left inferior frontal and bilateral temporal areas (depicted in white in Fig. 3). In the second layer of the nesting, immediately adjacent to the core language network, are voxels that are activated by one of the single tasks and that are also activated in the dual task (depicted in yellow and purple). The third layer, most of which is immediately adjacent to the second layer, consists of voxels activated only in the dual task (depicted in red). The activation that occurred only in the dual task was in left inferior frontal gyrus; left middle, superior, and inferior temporal gyri; right middle and superior temporal gyri; right inferior frontal gyrus; and bilaterally in and around the inferior orbital gyrus, extending into the insula in the left hemisphere.

In sum, the voxels activated only for dual‐tasking are located in regions that circumscribe both the common core of language network areas activated in all three tasks as well as the areas activated in only one of the single tasks and the dual task. With increasing task demand from single‐ to dual‐tasking, the additional activation radiates outward from the core language network to immediately adjacent areas. As discussed above, this increase in the spatial extent of activation may be due to the recruitment of additional tissue to perform the dual‐task condition, or it may simply reflect an increase in the amplitude of activation in the core regions such that subthreshold activity at the edge of the core regions exceeds threshold in the more difficult dual‐task condition. In either case, the striking finding is that dual‐tasking increased activation primarily in the core language network and not in prefrontal regions associated with central executive processes (e.g., attention allocation, task‐switching, response‐monitoring).

The study also shows increased activation in the premotor cortex in the left and right precentral gyri, and the bilateral supplementary motor area (SMA). Activation in these areas of the premotor cortex may be a result of increased demand for response selection when there are two responses. Other clusters of new activation were found in right calcarine (BA 30), left thalamus, and bilateral cerebellum. The number of voxels activated only in the single tasks indicates that there was a very small amount of activation that was exclusive to each single task. These voxels were located mostly in the hemisphere contralateral to the ear of presentation of the sentences (Table III).

Table III.

Subsets of activated voxels

New activation (dual only)	Voxels	MNI
		x	y	z
Frontal
L+R supp mot area	764	4	14	54
L inf frontal gyrus + insula	71	−36	24	−4
L precentral gyrus	172	−32	0	56
R insula + inf frontal gyrus	223	36	24	−2
R precentral gyrus	32	44	0	46
R sup frontal + precentral gyri	34	26	−6	56
Temporal
R mid + sup temporal gyri	693	68	−36	−2
L mid temporal gyrus	22	−52	−2	−22
L sup + mid temporal gyri	289	−66	−24	4
L inf + mid temporal gyrus	22	−52	−2	−22
L inf + mid temporal gyrus	45	−44	−40	−6
Cerebellar
Vermis	71	−2	−50	−30
R cerebellum	52	38	−70	−34
Subcortical
L calcarine	61	30	−58	4
All three tasks (core)
Frontal
L supp motor area	144	2	16	54
L inf frontal gyrus	193	−40	6	28
R inf frontal gyrus + insula	25	32	24	−−8
L inf frontal gyrus	144	−50	24	−8
Temporal
R sup + mid temporal gyri	2267	50	−24	10
L mid + sup temporal gyri	2532	−42	−40	−4
Subcortical
L caudate	20	−10	8	20
L thalamus	57	−4	−8	2
Left and dual only
Frontal
R inf frontal gyrus + insula	22	32	24	−4
Temporal
R sup + mid temporal gyri	530	50	−24	10
Right and dual only
Frontal
L precentral gyrus	29	−52	6	46
L supp motor area	171	−8	6	60
L inf frontal gyrus + insula	201	−34	24	−6
L inf frontal gyrus	106	−42	8	32
Temporal
L sup + mid temporal gyri	685	−66	−24	2
R mid temporal gyrus	147	48	−38	4
Subcortical
L thalamus	24	−8	−8	2
Left only
Temporal
R sup temporal gyrus	188	34	−36	8
Subcortical
L+R Thalamus	44	−2	−20	4
Right only
Frontal
L inf frontal gyrus	236	−56	20	4
L inf + mid frontal + prec gyri	84	−38	12	30
L supp mot area + sup front gyrus	50	−8	26	48
Temporal
L heschl's + sup temporal gyri	28	−42	−24	−6
L sup temp gyrus + rolandic operc	66	−42	−34	14
L mid + inf temporal gyri	26	−58	−16	22
Cerebellar
R cerebellum	29	28	−66	−34
Subcortical
L parahippocampal + thalamus	484	−12	−18	−22
Left and right only
Temporal
R sup + mid temporal gyri	32	40	−28	−2

Note: Extent threshold = 20 voxels. The voxels reported here were obtained from the masking procedure described in the Experimental Procedures section. They represent the voxels activated at a threshold of T = 4.02. Region labels apply to the entire extent of the cluster with peak maxima designated by first locale cited. MNI coordinates are for the peak activated voxel in each cluster.

Functional Connectivity in the Language Network

Functional connectivity analyses indicated an increase in synchronization from single‐ to dual‐tasking between the areas of the language network. The functional connectivity increased in the dual task for the frontal‐temporal lobe pairs LIFG:LMTG (t(10) = 2.20; P < 0.05), LIFG:LSTG (t(10) = 3.14, P < 0.01), and LIFG:RSTG (t(10) = 3.00, P < 0.01) (as noted in the Materials and Methods, data from one participant were excluded from these analyses). The functional connectivity also increased in the dual task for the inter‐hemispheric temporal lobe pairs LMTG:RSTG (t(11) = 3.99, P < 0.001) and LSTG:RSTG (t(11) = 4.67, P < 0.001) (see Fig. 4). The functional connectivity analyses when applied to the signal extracted from the voxels activated in each condition showed the same increase in functional connectivity from the single to the dual condition as the analyses applied to the signal extracted from the union of voxels activated across the different conditions (see Materials and Methods). There was no significant difference between the functional connectivities in the language network in the two single tasks.

Figure 4.

Functional connectivity in dual‐tasking: higher synchronization between areas of the language network. Functional connectivity analyses for the language network regions (LIFG and bilateral mid and superior temporal). Increase in functional connectivity from single to dual condition was observed for the comparison between the FCA z‐score for the dual condition and the average FCA z‐score for the two single conditions (right and left ear message); (*) = P < 0.05; (**) = P < 0.01; (***) = P < 0.001.

The Change in the Timing of Brain Activation for Dual‐Tasking

There are many ways for an increase in functional connectivity to come about. For example, the signal in each area could become less noisy, allowing the correlation to increase. Alternatively, some new component could enter into the waveform of both time series at the same time, providing an additional basis for correlation. To informally determine what change in the waveforms gave rise to the increase in functional connectivity in the dual‐task condition, the plots of the activation time courses from pairs of areas were visually inspected. The visual inspection indicated that what changed from the single to the dual task was that the activation peak of LIFG occurred earlier in the dual task than in the single tasks, bringing its peak into closer correspondence with the time of the activation peak of LMTG and bilateral superior temporal gyri.

To quantify this observation, Gaussian curves were fitted to the fMRI signal data and the locations of the activation peaks were assessed from the midpoints of the Gaussians. Consistent with the informal observation, the peak of the best‐fitting Gaussian curve for LIFG activation was located at an earlier point in time (0.76 sec earlier on average) in the dual task than in the single task (LIFG Dual: M = 9.72 s from sentence onset; LIFG Single: M = 10.48 s; t(10) = 2.68, P < 0.05). In contrast, left and right temporal regions showed no statistically significant changes in LMTG, LSTG, or RSTG peaks from single‐ to dual‐tasking. The activation peak in the temporal regions occurred earlier than the LIFG peak in the single conditions, and remained approximately at the same point in time in the dual task condition. Thus, the change in timing of LIFG activation may be underpinning the higher functional connectivity in the dual task condition. Figure 5 shows the time shift in LIFG activation for the dual task in one participant. The LIFG peak was located at an earlier point in time in the dual task than in the single tasks for 10 of the 11 participants included in the analyses. Thus, the shift is not only statistically reliable but also common to almost all of these high‐performing multitaskers. There were no significant differences in the width of the Gaussian curve for single and dual tasks. The width provides a measure of how long the activation lasted. The findings for this parameter of the activation suggest that there was no difference in the duration of activation for dual‐ and single‐tasking in the language network. Table IV reports the mean values for the peak and width of the Gaussian curves for single‐ and dual‐tasking.

Figure 5.

Analysis of time courses: timing shift in lifg activation underpins synchronization of LIFG‐temporal brain activity time course plots for % signal change for the LIFG and LMTG2 functional regions of interest for one individual participant, averaged over individual trials. The peak coordinates (x‐axis) reported are for the Gaussian curve that represents the best fit calculated for the activation for this specific participant. The distance between the peak for the best‐fitting Gaussian curve for LIFG and LMTG decreases in the comparison between single and dual task.

Table IV.

Parameters for the gaussian curve best fit to the percent signal change data for language‐network regions

	LIFG		LMTG		LSTG		RSTG
	Single	Dual	Single	Dual	Single	Dual	Single	Dual
Peak (s) (SE)	10.48 (0.26)	9.72* (0.14)	8.55 (0.14)	8.39 (0.15)	7.85 (0.16)	7.86 (0.12)	7.72 (0.11)	7.82 (0.11)
Width (SE)	6.60 (0.50)	7.14 (0.50)	7.24 (0.30)	7.14 (0.50)	7.32 (0.35)	7.84 (0.50)	7.25 (0.58)	7.73 (0.45)

Parameters for the Gaussian best‐fitting curve. Parameters reported in bold font show a statistically significant change from single to dual task for LIFG (* = P < 0.05; t(10) = 2.68). The parameter values represent the mean for the two functional regions of interest drawn for each area (LIFG = LIFG1 and LIFG2; LMTG = LMTG1 and LMTG2; LSTG = LSTG1 and LSTG2; RSTG = RSTG1 and RSTG2). Standard t‐tests on the parameters for individual participants showed significant change in the parameter for the peak of the curve only for the LIFG activation.

The increase in functional connectivity from single to dual task was greater among participants with lower working capacity (who were still able to perform the dual task.) Though an increase in functional connectivity was found in all participants, the shift in the timing of LIFG activation was negatively correlated with participants' reading span scores (r = −0.66, t = −2.65, P < 0.02); i.e. the lower the reading span, the more the LIFG timing changed from single‐ to dual‐tasking. Moreover, the change in the distance between the time to peak for the LIFG and LMTG was positively correlated with the change in functional connectivity mean z values (r = 0.54; t = 1.9; P < 0.05). In other words, the more the distance between LIFG and LMTG peaks decreased from single‐ to dual‐tasking, the more the functional connectivity increased from single‐ to dual‐tasking. We furthermore calculated the correlation between reading span and the change in the distance between LIFG and LMTG from single‐ to dual‐tasking. The lower the reading span, the more the distance between LIFG and LMTG peaks decreased from single‐ to dual‐tasking (r = −0.63; t = −2.5; P < 0.02). In sum, these correlations show that the shift in LIFG timing ultimately resulted in a decrease in the distance between LIFG and LMTG peaks, and a greater increase in the synchronization between LIFG and LMTG in the dual task.

DISCUSSION

This first neuroimaging study of processing two simultaneous auditory language streams reveals three characteristics of how the brain activation of successful multitaskers changes from single to dual listening comprehension, without a loss of comprehension accuracy. First, there was an increase in the synchronization of the brain activation of frontal and posterior language areas in the dual task. The dual‐task synchronization increase was brought about primarily by the LIFG activation becoming better synchronized with the activation in the temporal areas of the network. Second, the timing shift in LIFG activation was negatively correlated with reading span scores in the dual task and correlated with a greater increase in connectivity between the areas of the language network. The shift in LIFG activation may be a brain marker of individual differences in the functioning of lower and higher reading‐span participants who were able to perform the dual‐task. Third, the language dual task recruited additional activation compared with single‐message comprehension only in areas surrounding the nodes of the network activated in single‐message processing. This dual‐task did not recruit new activation in DLPFC, which is associated with executive processing. The study also corroborates previous findings that individual differences in working memory capacity (measured here with reading span) were not correlated with individual differences in multitasking ability (Jaeggi et al.,2008).

Increase in Functional Connectivity and Earlier Onset of LIFG Activation

The data show that the LIFG activation changes from single‐ to dual‐tasking in a way that increases its synchronization with left middle and bilateral superior temporal activation. The change in the timing of the activation peak from single‐ to dual‐tasking made the onset and offset of LIFG activation more similar to the temporal lobe activation, increasing the functional connectivity in the dual‐tasking condition. This increase may indicate an attempt to establish more effective communication among the areas of the language network and hence attain a high level of performance in the dual task, especially so in the lower working memory capacity participants.

The time shift in LIFG activation suggests a mechanism of brain adaptation in dual tasking that may enable participants with lower working memory capacity to manage the twofold increase in input in the dual task. The correlation between the time shift in the LIFG activation peak and working memory capacity shows that LIFG brain activation of the lower capacity participants made a greater shift in its timing and consequently synchronized better with the activation in the bilateral middle and superior temporal lobes. This greater shift was also correlated with a greater increase in synchronization between the areas of the language network.

Language comprehension is implemented by means of interactive processes including frontal and temporal areas of the language network. The processing of auditory input is one of the language comprehension tasks commonly associated with activation of the superior temporal areas of the language network (e.g. Constable et al.,2004; Michael et al.,2001; Schlosser et al.,1998). LIFG, in turn, is implicated in several language comprehension processes, including phonological processes like subvocal rehearsal of auditory input, controlled access to semantic representations, and syntactic parsing (Chee et al.,1999; Costafreda et al.,2006; Gabrieli et al.,1998; Henson et al.,2000; Keller et al.,2001; Keller et al.,2003). Because the dual‐listening comprehension task is a highly stimulus‐bound task, it is possible that the twofold increase in information is dealt with by the frontal areas of the brain processing input received from the temporal lobe areas at an earlier point in time. Earlier onset of LIFG activation may be associated with earlier parsing of the information for a clue to the trueness or falsity of each of the two sentences. Earlier onset of prefrontal cortex activation has been reported in association with increased speed of information processing in the prefrontal cortex in a study of dual‐tasking. Dux et al., (2009) showed faster behavioral performance associated with earlier onset of activation in the left inferior frontal junction (LIFJ), an area that includes the posterior lateral prefrontal and anterior premotor cortex (posterior Brodmann area 9).

These high‐performing participants may have been able to eliminate some nonessential comprehension processes (such as making elective inferences) during the dual listening condition, relying instead on what has been referred to as “good enough” processing. “Good‐enough processing” refers here to a set of heuristic comprehension processes which are chosen opportunistically to fulfill the needs of the language task at hand (Ferreira and Patson,2007). Ferreira and Patson suggested that factors such as time pressure, which was an operative factor in the dual task, could influence the extent to which the language comprehension system operates in a good‐enough way.

Dual‐Tasking Effects in the Language Network

High‐level, automatic dual‐tasking in language comprehension evoked the same network of brain areas as the single‐message comprehension tasks, but with enlarged activation volumes and magnitude of activation. The new areas of activation in the dual task took the shape of an annulus, or shell, which circumscribed the core regions activated in all three tasks. The shell of activation showed recruitment of voxels for two quantitatively different levels of task demand: activation only in the dual task, where there are two input streams, and activation only in one of the single tasks, where there is only one input stream. The shell of voxels is recruited to deal with the increased demand in the dual task.

The findings provide persuasive evidence that not all dual tasks require additional executive functioning and are suggestive of which task properties determine how the dual‐tasking is controlled. The results of the dual auditory comprehension tasks do not show activation of the executive network areas, such as DLPFC. Activation of DLPFC was absent in the dual‐task contrast with fixation and in the direct contrasts between dual task and each component task. This suggests that the two concurrent sentences were processed without task‐switching. The automaticity of listening comprehension may have made it possible to achieve seemingly interference‐free, concurrent processing of the two auditory sentences in the task.

It is interesting to consider how the current dual‐task findings are related to the brain activation findings for other pairs of tasks. Previous studies of brain imaging and dual‐tasking have investigated combinations of tasks that require processing of two stimuli, one of which can be maintained in working memory. The challenge of that type of dual task lies in being able to temporarily store and manipulate information in working memory for one task until a response is made to the other. As soon as the response is made, executive control can switch attention to the information stored from the other task. For example, D'Esposito et al., (1995) investigated a dual task in which participants had to perform a semantic judgment of auditorily presented words as well as a mental rotation task. The auditory stimuli were presented every two seconds and the visual stimuli, every three seconds, in a 30‐s block of dual‐tasking stimuli. Jaeggi et al. (2003) investigated a dual n‐back task with briefly‐ and simultaneously‐presented target visual material (abstract random shapes) and target auditory stimuli (the spoken sound of 10 consonants). One of the component tasks in these dual tasks was more automatic than the other: the auditory word processing in D'Esposito et al., (1995), and the auditory consonant processing in Jaeggi et al. (2003). Automaticity may have allowed for one task to be resolved more rapidly while information for the other, less‐automatic task was stored and later manipulated in working memory. Evidence for executive control and increased working memory load in those studies is that there was more activation in dual‐tasking, relative to single‐tasking, in areas not activated in either of the single tasks, particularly the DLPFC (D'Esposito et al.,1995; Jaeggi et al.,2003).

To our knowledge, there are no brain‐imaging studies of dual‐tasking with a combination of automatic, high‐level tasks (sentence listening comprehension) that compete for cortical resources from the same network of areas of the brain. Processing two simultaneously‐presented auditory sentences requires sharing of the same cortical resources between two competing and similar inputs. The continuous stream of linguistic information in the sentences cannot be ignored for long without compromising the comprehension processes. Hence, it seems implausible that participants were attention‐switching between the two sentences, although there may be a phenomenological experience of one or other of the sentences dominating one's conscious experience.

The more automatic the listening comprehension task, the more language‐network resources may be available for dual‐tasking. The ability to maintain simultaneous processing of information from two sentences is dealt with by recruitment of new cortical resources only in areas adjacent to those already activated in the component tasks. This focused recruitment of task‐specific cortical activation may be a signature of automatic processing of two input streams (in this case, two instances of listening comprehension), as opposed to a strategy of task‐switching.

The results of the study apply to those people, less than half of the Carnegie Mellon participant pool, who are capable of comprehending two auditory messages at once with high accuracy. The results of the study may be at the core of individual differences in the ability to perform higher‐level dual tasks. The 12 final participants in the study who were able to attain high comprehension accuracy represent ∼34.0% of the initial sample of students who were screened. Our study does not indicate what distinguishes those participants who succeeded and those who failed at achieving high comprehension accuracy in the dual listening task, but it does provide some initial cues. The successful participants showed a change in the temporal organization of their neural processing, a shift in the timing relation among nodes in the language network, achieving higher functional connectivity in the dual task condition. Moreover, successful participants with lower working memory capacities showed larger time shifts. It is possible that the unsuccessful participants could not make a large enough time shift to accommodate the required degree of synchronization. This hypothesis could be tested in a simpler dual comprehension task that presented either simpler sentences or presented the sentences at a slower rate. In that case, presumably more of the initially screened participants would have achieved a 75.0% accuracy, and the prediction is that their LIFG timing peak would occur earlier than in a single sentence condition. By the same token, if the sentences were more complex or spoken at a faster rate, then presumably some of the 12 successful participants would not be able to achieve a 75.0% accuracy rate. More generally, this account predicts that dual comprehension requires a timing change in the network, and that individual differences in the ability to do dual comprehension arise from individual differences in the network's ability to make a timing change.

It is possible that both the brain mechanism of frontal‐to‐posterior synchronization of brain activation and the availability of resources in the outer shell of activation of the language network are at the source of individual differences between the 12 participants that could perform the task and others who could not perform the task. Further studies of multitasking high‐level cognitive tasks may be able to identify the difference between successful high‐level multitaskers and unsuccessful ones. Understanding the core reasons why some brains can multitask while others cannot may make it possible to develop training programs aimed both at improving multitasking and at helping those who are initially unsuccessful at multitasking to improve their performance.