Training improves multitasking performance by increasing the speed of information processing in human prefrontal cortex

Summary

Our ability to multitask is severely limited: Task performance deteriorates when we attempt to undertake two or more tasks simultaneously. Remarkably, extensive training can greatly reduce such multitasking costs. While it is not known how training alters the brain to solve the multitasking problem, it likely involves the prefrontal cortex given this brain region’s purported role in limiting multitasking performance. Here we show that the reduction of multitasking interference with training is not achieved by diverting the flow of information processing away from the prefrontal cortex, or by segregating prefrontal cells into independent task-specific neuronal ensembles, but rather by increasing the speed of information processing in this brain region, thereby allowing multiple tasks to be processed in rapid succession. These results not only reveal how training leads to efficient multitasking, they also provide a mechanistic account of multitasking limitations, namely the poor speed of information processing in human prefrontal cortex.

Introduction

Although we all have a propensity to undertake more than one task at a time in our day-to-day lives, our ability to perform these tasks rapidly and accurately is severely compromised when we attempt to carry them out simultaneously. Such multitasking costs are ubiquitous, occurring regardless of whether the tasks are simple (e.g., making arbitrary sensory-motor decisions) or complex (e.g., driving and talking on the cell phone), and can even be observed when the competing tasks do not overlap in either sensory input or motor output modality, suggesting a central, amodal source of interference (Marois and Ivanoff, 2005; Pashler, 1994). Remarkably, however, this fundamental limitation of our cognitive system is not immutable: Prolonged training with dual tasks greatly reduces multitasking costs (Schumacher et al., 2001; Tombu and Jolicoeur, 2004; Van Selst et al., 1999).

How does training modify the functional architecture of the brain to solve the multitasking problem? Behavioral studies suggest that multitask training improves the performance of each task, thereby reducing the interference that tasks can exert onto each other (Ruthruff et al., 2001; Ruthruff et al., 2003), but these studies are agnostic to the manner in which these processing changes are neurally implemented. Similarly, the neurobiological literature on the effects of training on cognitive task performance (Erickson et al., 2007; Jonides, 2004; Kelly and Garavan, 2005; Poldrack, 2000; Poldrack and Gabrieli, 2001; Poldrack et al., 2005; Rioult-Pedotti et al., 2000; Rioult-Pedotti et al., 1998; Sakai et al., 1998) does not single out a specific neural mechanism that could account for cost-free multitasking, as several of those mechanisms are consistent with the behavioral findings from multitasking studies. Broadly speaking, these neural accounts can be grouped into those positing that training results in a reorganization of the brain circuits supporting task performance, and those suggesting that training improves the processing efficiency of the pre-existing neural substrates (Jonides, 2004). A prominent theory of neural reorganization proposes that training improves cognitive task performance by reducing the dependence of such performance on brain regions involved in cognitive control and attention, while concomitantly increasing its reliance on task- or process-specific neural circuits (Kelly and Garavan, 2005; Petersen et al., 1998). Consistent with the hypothesis that training induces a switch from slow, deliberative processing in ‘general-purpose’ brain networks to fast, automatic processing in task-specific neural circuits, training is often accompanied with decreased activation in prefrontal cortex (e.g., Erickson et al., 2007; Sakai et al., 1998), a key brain region underlying cognitive control (Dosenbach et al., 2006; Koechlin et al., 2003; MacDonald et al., 2000; Miller and Cohen, 2001) and multitasking performance (e.g., Dux et al., 2006; Marois et al., 2005). However, it has also been argued that training could lead to the recruitment of prefrontal cortex regions to co-ordinate efficient multitasking (Erickson et al., 2007). Most importantly, training-induced changes in brain activation, whether they be down- or up-regulations, reveal little about the neural transformations that enable efficient multitasking, for such activation changes may not necessarily reflect the dropping off or recruitment of brain regions with training, as they could just as well result from functional adaptations within such regions to promote efficient task processing (Jonides, 2004; Kelly and Garavan, 2005; Poldrack, 2000).

The elucidation of the neural dynamics that support successful multitasking with training requires methodological approaches that can distinguish between the candidate processes described above. To achieve this goal, we trained seven subjects daily for a period of two weeks (see Experimental Procedures) in a standard dual-task paradigm (Pashler, 1994; Schumacher et al., 2001; Tombu and Jolicoeur, 2004), and scanned these subjects with functional magnetic resonance imaging (fMRI) on three different occasions during this training regimen. We then applied several analytical tools to the fMRI data to isolate the neural mechanism(s) that bring about efficient multitasking. The training and brain scanning sessions comprised three types of trials: Single auditory-vocal (AudVoc) trials, which consisted of the presentation of one of two auditory stimuli that each required a distinct speeded vocal response; single visual-manual (VisMan) trials, consisting of the presentation of one of two faces that each required a distinct speeded finger-press response; and dual-task trials, which involved the simultaneous performance of both the AudVoc and VisMan tasks (Fig. 1A). We scanned the subjects before training was commenced (pre-training), at the mid-point of training (mid-training), and after training had concluded (post-training; after a subject’s dual-task performance improvement reached asymptote; see Experimental Procedures). The timing of these fMRI sessions ensured that any potential changes in neural information processing with training would be captured by the present experimental design (Kelly and Garavan, 2005; Ungerleider et al., 2002).

Figure 1. Task design and behavioral results.

A) Task design. The task included three trial types: Single Auditory-Vocal Trials (AudVoc), where subjects were presented with one of two auditory stimuli that each required a distinct speeded vocal response; Single Visual-Manual Trials (VisMan) where subjects were presented with one of two faces that each required a distinct speeded finger-press response; and Dual-Task Trials where subjects were presented with both the AudVoc and VisMan tasks simultaneously. Training on these tasks took place over several sessions during a two-week period. B) Behavioral results for the training sessions. Upper panel, task reaction times under single-and dual-task conditions. Lower panel, reaction time costs of performing the tasks under dual-task conditions relative to single-task conditions (calculated by subtracting single-task performance from dual-task performance for each task and then summing these values). C) Behavioral results from the scanning sessions. Upper panel, task reaction times under single- and dual-task conditions. Lower panel, Reaction time costs of performing the tasks under dual-task conditions relative to single-task conditions (see Experimental Procedures). All errors bars represent standard error of the mean.

Results & Discussion

In the behavioral training sessions, training reduced the reaction times to each task under both single- and dual-task conditions (main effect of Training Session, F(7, 42) = 68.2, p < 0.0002; 2[Task: AudVoc vs. VisMan] × 2[Trials Type: Dual-Task vs. Single-Task] × 8[Training Sessions 1–8] repeated measures ANOVA). Importantly, this improvement was greater for dual-task than single-task trials (interaction between Trial Type and Training Session, F(7, 42) = 42.1, p < 0.0002, Fig. 1B), and did not result from either a trade-off in accuracy (as an opposite pattern of results to that found for the reaction time data was not observed for the accuracy data; interaction between Trial Type and Training Session, F>1) nor response grouping, as RTs for the two tasks (under both single- and dual-task conditions) were significantly different even after training (ts>3.4, ps<0.02, two-tailed paired-samples t-test; response grouping would be evidenced by comparable RTs for both tasks, see Ulrich and Miller, 2008). In addition, an identical pattern of behavioral results was obtained in the fMRI scanner (Fig. 1C). Therefore, training was successful in reducing multitasking costs to approximately one tenth of their initial value (from approximately 400 ms to 40 ms), although it did not eliminate such costs altogether as the residual multitasking costs were still significant (p = 0.05, two-tailed, one-sample t-test), as has been found in previous behavioral studies (e.g., Tombu and Jolicoeur, 2004).

To isolate candidate brain regions that may limit multitasking performance, we first searched for areas that responded significantly to both single-tasks in the pre-training session, as would be expected of the neural substrates underlying a central, amodal bottleneck of information processing (Dux et al., 2006; Jiang and Kanwisher, 2003b; Marois and Ivanoff, 2005). This contrast isolated in each subject an extensive network of frontal, prefrontal, parietal and sub-cortical areas (see Supplemental Tab. 1) that have previously been implicated in response selection, decision making, multitasking, and sensory-motor training (Dux et al., 2006; Heekeren et al., 2004; Hikosaka et al., 2002; Jiang and Kanwisher, 2003a; Marois et al., 2005; Poldrack et al., 2005; Sakai et al., 1998; Schubert and Szameitat, 2003; Szameitat et al., 2002). In addition, because sensory and motor cortex have also been shown to be influenced by training (Büchel et al., 1999; Karni et al., 1998; Kelly and Garavan, 2005), we isolated the corresponding sensory and motor regions for our AudVoc and VisMan tasks by directly contrasting their activity (see Supplemental Tab. 1). Finally, because dual-task performance could in principle also be controlled by brain regions specifically recruited to co-ordinate multitasking (D’Esposito et al., 1995), we also contrasted activity between the dual-task and single-task conditions in the pre-training session. This latter analysis revealed no brain regions that were specifically activated by the dual-task condition, replicating previous work (Adcock et al., 2000; Dux et al., 2006). Thus, only ROIs that were isolated using the single-task trials were examined, because there were no brain areas that were exclusively activated by the dual-task trials.

Regions involved in limiting multitasking performance should not only display greater activity under dual-task conditions compared to single-task conditions prior to training (because in the former condition twice as many time-consuming response selection operations are required, Erickson et al., 2007; Marois et al., 2005), they should also evidence a greater effect of training on neural activity in dual-task trials than in single-task trials, as multitasking costs diminish over the two-week training regimen (Fig. 1B and 1C). In order to identify which of the above brain areas showed such a pattern of activity, we extracted timecourses of the blood-oxygen-level-dependent (BOLD) signal from each ROI and examined the multitasking effect across the three fMRI sessions. Paralleling previous fMRI studies of single-task and dual-task training with overlapping modalities (Erickson et al., 2007), several cortical and sub-cortical regions showed general task-related decreases in activity across the training sessions (see Supplemental Tab. 1). However, among all these brain regions, only the left inferior frontal junction (IFJ), located at the boundary of the posterior lateral prefrontal and anterior premotor cortex (posterior Brodmann area 9), showed not only greater activity in dual-task than in single-task trials prior to training, but also a significant reduction in this activity difference as training ensued (Fig. 2A). Specifically, BOLD amplitudes for the pre- and mid-training fMRI sessions were greater in the dual-task than in the two single-task conditions (ts>2.7, ps<.04, two-tailed paired samples t-test), with the latter two conditions not differing from one another (t<1). However, by the final session, when dual-task reaction time costs were now strongly attenuated, there was no significant amplitude difference between dual- and single-task trials (ts<1.7, ps>.13). Importantly, this dual- vs. single-task activation difference observed in left IFJ was significantly larger than that observed in all the other regions tested prior to training (t(6) = 5.9, p < .002, two-tailed paired samples t-test), but not post-training (t < 1), attesting to the preferential association of this brain region with the modulation of multitasking performance with training. Moreover, while there was a strong correlation between individual subjects’ BOLD amplitude differences between dual- and single-task trials and their dual-task reaction time costs in the first two fMRI sessions (rs>.7, ps<.05, one-tailed Pearson correlation), this correlation no longer held by the third session, (r =−.3, p=.6). Finally, it is worth noting that these results were not biased by using the pre-training session for IFJ ROI definition, as the same findings were obtained when the ROI was isolated from the post-training session. Taken together, these results not only suggest that an individual’s multitasking performance costs are related to IFJ activity, they also support prior work indicating that this very same brain region is involved in the capacity-limited central stage of response selection and decision-making (present IFJ mean Talairach coordinates of x = −43, y = 8, z = 29 compared to x = −42, y = 17, z = 28 in Dux et al., 2006).

Figure 2. Effect of training on single- and dual-task BOLD response amplitude.

A) Left panel; Left inferior frontal junction (IFJ) ROI (red circle) on SPM of conjunction of AudVoc open contrast and VisMan open contrast (example subject). Right panels, BOLD timecourses for the AudVoc, VisMan and Dual-task trials in the pre-, mid- and post-training fMRI sessions (group). B) & C) Left panel; ROIs in right vocal motor cortex (VocMC) and right auditory cortex (AC) isolated by contrasting AudVoc and VisMan trial activity (example subject). Right panels, BOLD timecourses for the AudVoc, VisMan and Dual-task trials in the pre-, mid- and post-training fMRI sessions. ROIs were isolated from the pre-training session. An identical pattern of results was observed when ROIs were defined from the post-training session. All error bars represent average within-subject standard error of the mean.

The observation that training reduces multitasking-related activity in IFJ is consistent with the hypothesis that efficient multitasking results from a decreased reliance on brain regions involved in cognitive control and attention. According to this hypothesis, regions initially required to cope with unfamiliar, novel task demands are progressively replaced by more efficient task-specific brain regions or networks with training (Chein and Schneider, 2005; Haier et al., 1992; Jansma et al., 2001; Petersen et al., 1998). However, we found no brain regions that showed increased activity with training for any of the task conditions, either in the isolated ROIs or when using a voxel-based analysis that contrasted activity in the pre- and post-training fMRI sessions, suggesting that no brain regions were recruited anew or more extensively with training. Therefore, these findings provide no evidence to support the notion that the emergence of efficient multitasking necessitates the recruitment of new brain regions.

It has been suggested, however, that functional reorganization with training might take place by affecting the strength of connections between brain regions (Poldrack, 2000). According to this account, efficient multitasking would emerge with training because the flow of sensory-motor information from each task would be progressively routed away from slow deliberative processing in prefrontal cortex, thereby bypassing the neural locus of multitasking limitations. The finding that dual-task specific activity decreased with training in IFJ (Fig. 2A) is generally consistent with this hypothesis. This account further predicts that the prefrontal route may be gradually replaced by more direct and specific sensory-motor connections as multitasking interference wanes with training. We tested this hypothesis by performing an effective connectivity analysis (Büchel and Friston, 1997), using structural equation modeling (Rogers et al., 2004; Rowe et al., 2002), to assess whether the strength of the modeled sensory-prefrontal-motor pathways of the AudVoc and VisMan tasks decreased, while the strength of the direct sensory-motor pathways increased, with training. We focused this analysis on the single-task trials because the functional reorganization account predicts that training reduces multitasking interference by re-orienting the flow of sensory-motor information for each of the individual tasks away from prefrontal cortex to more task-specific networks (though the results described below were qualitatively unchanged when dual-task trials were subject to this analysis). The use of single-tasks is further justified by the finding that the reduction in multitasking interference with training can be largely explained by improved performance on the two single-tasks (see Supplemental Fig. 1 and supporting text), rather than improvement in the dual-task condition alone.

The standardized path coefficients, a measure of the relative influence of one region’s BOLD activity onto another’s, were unaffected by fMRI Session in either sensory-prefrontal-motor network (all main effects and interactions involving fMRI Session ps>0.22, 2 [Task: AudVoc vs. VisMan] × 2 [Network: AC-IFJ-VocMC vs. VC-IFJ-ManMC] × 2[fMRI Session: Pre-training vs. Post-training] repeated measures ANOVA, Fig. 3). In addition, there were also no increases in the path coefficients describing the direct sensory-motor pathway (all main effects involving fMRI Session ps>0.28, 2 [Network Relevance: Task-Network vs. Non-Task Network] × 2[fMRI Session: Pre-training vs. Post-training] repeated measures ANOVA, see Supplemental Fig. 2). These findings were obtained regardless of whether BOLD signal amplitude was equated across fMRI sessions (see Experimental Procedures), and they do not appear to be a result of a lack of model sensitivity, as the path coefficients were significantly larger in a given network (sensory-prefrontal-motor or sensory-motor) when subjects performed the task relevant for that network (sensory-prefrontal-motor connectivity analysis: interaction between Task and Network, F(1, 6)=14.19, p<0.01; sensory-motor connectivity analysis: main effect of Network Relevance, F(1, 6)=16.6, p<0.01). Thus, the present results provide no evidence that increased efficiency in multitasking is achieved by a weakening of a prefrontal cortical route and a reciprocal strengthening of a direct sensory-motor route.

Figure 3. Effect of training on effective connectivity in sensory-prefrontal-motor pathways.

Upper Panel, Effective connectivity model. The AudVoc pathway (red) consisted of auditory cortex projecting to IFJ which then projected to vocal motor cortex (AC-IFJ-VocMC; AudVocNetwork), while the pathway for the VisMan task (green) consisted of the visual cortex projecting to IFJ which then projected to manual motor cortex (VC-IFJ-ManMC; VisManNetwork). According to the ‘macro-scale connectivity’ model, efficient multitasking after training would result from the diversion of sensory-motor information away from the slow, inefficient processing in prefrontal cortex. This hypothesis predicts decreased effective connectivity through IFJ with training. Lower Panel, Strength of the AudVocNetwork and VisManNetwork path coefficients as a function of Training Session and Task. An identical pattern of results was observed when either left or right hemisphere AudVoc sensory-motor ROIs were employed in the model. All errors bars represent standard error of the mean.

The finding that training affects activity levels in prefrontal cortex, but does not significantly modulate the inter-regional connectivity with prefrontal cortex, suggests that it may be neural changes intrinsic to this brain region that lead to efficient multitasking. Specifically, we hypothesized that if multitasking interference results from competition between distinct sensory-motor tasks for processing by a common ensemble of prefrontal cortex neurons, thereby creating a ‘bottleneck’ of information processing (e.g., Dux et al., 2006), then training may lead to efficient multitasking by functionally segregating neurons devoted to each sensory-motor task, thereby resulting in independent, parallel processing pathways within prefrontal cortex. This hypothesis is not only consistent with neurophysiological evidence that training/experience can induce local changes in neural connectivity patterns (Kelly and Garavan, 2005; Rioult-Pedotti et al., 2000; Rioult-Pedotti et al., 1998) and promote the differentiation of stimulus- or task-selective neurons in cerebral cortex (Duncan, 2001; Freedman et al., 2001; White and Fitzpatrick, 2007; Wiesel and Hubel, 1963), it is also the cortical mechanism thought to support odor discrimination learning (Li et al., 2008). Thus, according to this account, multitasking interference would initially occur because the same population of neurons within prefrontal cortex (IFJ) performs sensory-motor translation for both the AudVoc and VisMan tasks, but this interference would dissipate with training as IFJ neurons functionally segregate into distinct ensembles for the processing of each of the two sensory-motor tasks (Fig. 4A).

Figure 4. Effect of training on neural decoding of task identity in IFJ.

A) Functional segregation (‘micro-scale connectivity’) model of successful multitasking. According to this model, multitasking interference results from functional overlap in the neural ensembles processing each of the two sensory-motor tasks in IFJ (pre-training). Training may lead to the functional segregation of the IFJ neural ensembles processing each of the two tasks (post-training), thereby eliminating multitask interference. This hypothesis predicts poor decoding of task identity (AudVoc vs. VisMan) in IFJ prior to training, followed by improved decoding performance with training. B) Performance for decoding of task identity in IFJ, auditory cortex (AC, right hemisphere) and motor (VocMC, right hemisphere) cortex across the training period. Chance is 50%. All errors bars represent standard error of the mean.

To test if increased task selectivity within IFJ leads to multitasking improvement with training, we employed a multivariate pattern classification technique to overcome the spatial limitations of fMRI (Haynes and Rees, 2006; Kamitani and Tong, 2005). In the present context, this method relies on each voxel within an ROI having a weak but true bias for one of the two tasks due to the random distribution of neurons within that region that are selective for that task. By employing a linear pattern classification analysis this information can be pooled together across an ROI to identify an ensemble activity pattern that reflects the particular task subjects are performing in a given single-task trial (see Experimental Procedures). Specifically, we examined the accuracy of the pattern classifier in decoding the identity of each of the two single-tasks in IFJ and control sensory-motor regions (right hemisphere AC and VocMC, although the same pattern was observed for the left hemisphere ROIs) across the training period. If, prior to training, multitasking interference results from the processing of two sensory-motor tasks by a largely overlapping pool of IFJ neurons, then the ability of the pattern classifier to distinguish the activity pattern for each of the two single-tasks is expected to be relatively poor. However, decoding performance should improve with training if it leads to the functional segregation of populations of IFJ neurons processing each of the two sensory-motor tasks.

Prior to training, decoding performance was slightly above chance in IFJ (62.22% accuracy, t(6) = 2.94, p<0.05, one sample t-test), although it was lower than that observed for the sensory and motor ROIs (F(2, 12) = 4.2, p<0.05, repeated-measure ANOVA with the single factor of brain region, Fig. 4b), as would be expected of a central, amodal area that is commonly activated by both tasks compared to sensory and motor regions known to exhibit strong modality specificity. Of primary interest was the influence of fMRI Session on classification accuracy in IFJ (Fig. 4B). Contrary to our expectation that decoding performance would increase with training, it slightly decreased across the sessions (F(1, 6) = 3.75, p = 0.05, one-way ANOVA), suggesting that training may attenuate task selectivity in IFJ. By contrast, training had no discernible effect on decoding performance in sensory and motor cortex (Fs< 1). Taken together, these decoding results do not support the hypothesis that efficient multitasking results from the functional segregation of sensory-motor information processing pathways in prefrontal cortex. If anything, training appears to decrease task selectivity in IFJ. This latter finding cannot be explained by the general decrease in BOLD signal amplitude across training sessions, as these results were obtained regardless of whether amplitude was equated across sessions (see Experimental Procedures). Given that the lateral prefrontal cortex is composed of a heterogeneous mixture of neurons with sensory, motor, and sensory-motor properties (Fuster, 1997) that can adaptively code task-relevant information (Duncan, 2001), a possible explanation for the diminished task selectivity observed across the fMRI sessions is that multitask training may ‘prune out’ neurons coding for modality-specific sensory or motor information, thereby enhancing the proportion of cells that code for task-independent sensory-motor translation (response selection).

While the multi-voxel pattern analysis did not reveal any evidence for the functional segregation of task-related activity in IFJ, it did suggest that training modifies the functional neuro-architecture within this region. How can such modifications lead to efficient multitasking? Behavioral studies have hypothesized that training alleviates multitasking interference by shortening the central stage of response selection for each task, thereby reducing processing overlap between these two tasks at this capacity-limited stage of information processing (Ruthruff et al., 2001; Ruthruff et al., 2003) (Fig. 5A). Given that it is centrally involved in response selection (Dux et al., 2006; Marois et al., 2005) and that it is modulated by multitask training (Figs. 2A, 4B), the IFJ is well positioned to mediate training-induced changes in the efficiency of sensory-motor translation. If this hypothesis is correct, then BOLD signal duration in IFJ should be significantly longer in the dual-task condition compared to the single-task condition prior to training because in the former condition two time-consuming sensory-motor translations must be undertaken serially, while in the latter condition only one such operation must be performed. Moreover, if training considerably shortens the duration of sensory-motor translation for each task (Fig. 5A), then the (absolute) difference in the duration of response selection activity under dual- and single-task conditions in the post-training session may be so small as to be temporally irresolvable with fMRI – i.e., we should no longer observe differences in BOLD duration between dual- and single-task trials.

Figure 5. Effect of training on duration of activity in IFJ.

A) Central stage shortening model of successful multitasking. Upper panel, Pre-Training. According to this model, sensory information proceeds through a series of stages, including stimulus perception, central processing (response selection/sensory-motor translation) and response execution. Behavioral evidence suggests that the central stage of response selection is severely capacity-limited, allowing only one sensory-motor translation operation to be carried out at a time, and resulting in multitask slowing (Pashler, 1994). Colors depict distinct sensory-motor tasks. Lower panel; Post-Training. Training may optimize the efficiency of response selection for each task, thereby reducing central processing time and leading to a negligible delay of the second task. This model predicts considerably longer duration of IFJ activity in dual-task than in single-task trials prior to training, but not after training. B) Left panel; Left IFJ ROI (red circle) on SPM of conjunction of AudVoc open contrast and VisMan open contrast (example subject). Middle and Right panels; BOLD timecourses for the AudVoc, VisMan and Dual-task trials in the pre- and post-training fMRI sessions. In the pre-training fMRI session, multitasking affects both signal amplitude and duration because the BOLD response integrates neural activity over time. However, only signal peak latency can be used as an unambiguous measure of duration of neural activity as amplitude can be affected by neural activity intensity and/or duration (Dux et al., 2006). Another measure of duration of neural activity, BOLD response width at half amplitude maximum (Richter et al., 1997), also suggests longer activity duration in the dual-task condition than in the single-task condition pre-training (t(3) = 3.4, p < 0.04, paired-samples t-test), but not post-training (t < 1, p > 0.7). The early signal peaks near the onset of the timecourses are due to vocal artifacts. These artifacts do not affect the later, main activation peaks (Birn et al., 2004).

To test these predictions, we probed, in four new subjects, the timecourse of activity in IFJ prior to and after extensive training in single- and dual-task conditions. This experiment employed high temporal resolution fMRI (5 Hz sampling rate; see Experimental Procedures) in order to resolve the duration of neural activity in IFJ that could not be inferred in the first experiment due to its low temporal resolution (0.5 Hz sampling rate). The behavioral data of this new experiment mirrored those of the previous experiment, with large multitasking RT costs prior to training (~500ms) that were greatly reduced post-training (~100ms, see Supplemental Fig. 3). A comparison of the BOLD response peak latency (a sensitive measure of the duration of the BOLD signal (Dux et al., 2006; Henson et al., 2002)) in left IFJ indicated that activity peaked approximately 500 ms later in the dual-task condition than in the single-task condition prior to training (t(3) = 4, p <0.03, two-tailed paired-samples t-test, Fig. 5B). After training, however, there were no longer differences in the duration of BOLD activity between the dual- and single-task conditions (t = 1.5, p > 0.2), and these durations of activity for both the single- and dual-task conditions peaked earlier than those prior to training (ts >3.18, p ≤0.05, two-tailed paired-samples t-test). Taken together, these results strongly support the hypothesis that training reduces processing time in IFJ, thereby leading to greatly reduced neural and behavioral dual-task costs after training.

Conclusions

In this study, we distinguished between several neural mechanisms that could account for efficient multitasking with training. One model posited that training leads to a shift in sensory-motor information away from slow, deliberate processing in prefrontal cortex to fast and efficient processing in task-specific pathways. A second model assumed instead that training results in the functional segregation of neuronal ensembles that process each of the sensory-motor tasks in prefrontal cortex, thereby creating independent streams of information processing for each task. Finally, a third model held that efficient multitasking develops as a result of the improved efficiency of information processing through the prefrontal cortex. The results of four distinct analyses performed on two experimental data sets are largely consistent with the latter ‘improved efficiency’ account. Of course, it remains possible that subtle inter-regional changes in connectivity patterns that escaped detection by our analyses nevertheless contributed to the development of efficient multitasking. However, the fact that the effective connectivity analysis was sufficiently sensitive to distinguish the particular tasks that subjects were engaged in, together with the robustness of the peak amplitude, pattern classification and latency results, suggest that the development of efficient multitasking, at least with respect to the current paradigm, is primarily achieved by the shortening of a central capacity-limited stage of information processing in human prefrontal cortex rather than by a functional reorganization of the brain circuits supporting multitasking. It should be noted, however, that the conclusion that increased speed of information processing in prefrontal cortex underlies efficient multitasking does not imply that this type of neural change can only occur in posterior prefrontal cortex, or that it supports all forms of improvement in cognitive and multitasking capacity. For example, it is conceivable that more anterior regions of prefrontal cortex become implicated in limiting multitasking performance as the stimulus-response associations become more abstract and require greater levels of cognitive control (Koechlin and Summerfield, 2007). Likewise, it will be important to determine the extent to which the neural mechanisms limiting performance in the PRP generalize to other divided-attention deficits, including those that occur within modality and at more perceptual stages of information processing (Chun and Potter, 1995; Marois and Ivanoff, 2005; Raymond et al., 1992). The fact that the PRP can be observed with several cognitive processes other than response selection (Carrier and Pashler, 1995; Ruthruff et al., 1995; Ulrich et al., 2006), including those that occur within modality (Chun and Potter; Jolicoeur, 1999), raises the prospect that the present findings could apply to other multitasking domains.

While future studies will determine the extent to which the present results generalize across multitasking situations and brain regions, our findings provide a mechanistic blueprint for the development of efficient multitasking with training. According to this account, multitasking interference results from the funneling of information from distinct sensory-motor tasks onto overlapping neural ensembles in prefrontal cortex, thereby creating a bottleneck of information processing at the central stage of decision-making/response-selection. The effect of training is to speed up information processing through this prefrontal bottleneck, thereby reducing temporal processing overlap of the sensory-motor tasks in this brain region. This account accords very well with neurophysiological data suggesting that learning of arbitrary sensory-motor associations reduces the latency of neural activity in macaque prefrontal cortex (Asaad et al., 2000; Wise and Murray, 2000), and with behavioral studies hypothesizing that training improves multitasking performance by reducing the duration of central processing (Ruthruff et al., 2001; Ruthruff et al., 2003) (see Supplemental Fig. 1). Our findings also argue that decreased prefrontal activity with training, a result frequently observed during the performance of cognitive tasks (Erickson et al., 2007; Kelly and Garavan, 2005), may not signify a lesser role of prefrontal cortex in multitasking with training, but rather a more efficient one (Jonides, 2004; Poldrack, 2000). By the same token, in allowing us to observe how the brain solves the multitasking problem un-confounded by changes in sensory input, motor output or task instructions, the present training study has uncovered a key rate-limiting step in our ability to multitask, and that is the poor efficiency of information processing in human posterior prefrontal cortex.

Experimental Procedures

Experiment 1

Subjects

Seven right-handed members of the Vanderbilt University community (4 females, 23–30 years) with normal or corrected-to-normal vision participated in the experiment for financial compensation. The Vanderbilt University Institutional Review Board approved the experimental protocol and informed consent was obtained from the subjects.

Experimental Overview

Over the course of this experiment subjects performed three types of session; one brief familiarization session, three fMRI sessions and eight to twelve behavioral training sessions conducted over a two-week period (see below). The familiarization session was intended to expose the subjects to the stimulus-response mappings, and was administered immediately preceding the first fMRI session. fMRI sessions occurred prior to the first behavioral training session (pre-training), after the third, fourth or fifth behavioral training session (depending on subject’s performance; mid-training) and after the final behavioral training session (post-training). Subjects typically performed one session per day, although in a few instances two sessions were carried out in a day (morning and afternoon) to accommodate scheduling conflicts. Because subjects performed a varying number of behavioral sessions, session number for each subject was normalized from one to eight using the following formula: ROUND((session number/max(session number)) * 8) in order to facilitate subsequent analyses.

Tasks

For each trial in all the sessions, subjects performed either one (Single-task condition) or two (Dual-task condition) distinct sensory-motor tasks. The Visual-Manual (VisMan) task required a manual response to a visual stimulus, while the Auditory-Vocal (AudVoc) task required a vocal response to an auditory stimulus. These tasks were chosen, as they did not overlap in either sensory or output modalities and followed the “standard pairing” as outlined by Hazeltine et al. (Hazeltine et al., 2006). Both tasks were two-alternative discrimination (2AD) tasks, mapping two stimuli to two responses. The visual stimuli, subtending approximately 6.4° of visual angle (Fig. 1A), were two gray-scale faces with similar skin tone, hair color, neutral facial expression and hairline presented on a gray background. Subjects responded to each face with a distinct button press using the right index or middle finger. The auditory stimuli were two discriminable sounds (a complex tone and an edited natural sound) used previously (Dux et al., 2006). Each sound required a distinct vocal response, consisting of the following pseudo-syllables: “Tay” and “Koo”. The visual and auditory stimuli were each presented for 200 ms and, on dual-task trials, simultaneously. Stimulus-response mapping assignments were counterbalanced across subjects for both tasks. The visual and auditory stimuli were presented with equal frequency and, in dual-task trials, paired in a counterbalanced manner across all familiarization, training, and fMRI sessions.

Familiarization Session

The primary purpose of the familiarization session was to teach the subjects the stimulus-response mappings for each task and to familiarize them with the experimental protocol of the ensuing fMRI and behavioral training sessions. Only accuracy was stressed during the familiarization session, which consisted of a total of five runs. In the first three runs, only single-task trials were performed. The first run consisted of 12 VisMan trials, the second of 12 AudVoc trials and the third of 16 randomly intermixed single-task trials (eight of each). During the fourth and fifth runs subjects performed a combination of randomly intermixed single- and dual-task trials: The fourth run consisted of six AudVoc, six VisMan and 12 dual-task trials, while the fifth run consisted of six AudVoc, six VisMan and six dual-task trials. In total subjects performed 82 trials during the familiarization session, including 18 dual-task trials. The familiarization session lasted approximately twenty minutes, with the experimenter present in the testing room for the entire duration in order to score vocal responses (see below).

Each run of the familiarization session was subject-initiated and began with an instruction screen describing the task(s) that the subject would be performing. For runs involving the VisMan task, the face stimuli were presented on the instruction screen to allow subjects ample study time before beginning the trials. Likewise, for runs involving the AudVoc task, three examples of each sound were played during the instruction period. Throughout each run, subjects were asked to fixate a small black square (0.1° of visual angle) at the center of the screen.

During runs one through four, each trial began with a two second fixation period that ended with the presentation of the stimulus/stimuli for 200 ms. Stimulus onset also marked the beginning of the response period, which lasted four seconds. The response period was either immediately followed by a two second feedback period (for VisMan trials of runs one, three, and four), or by a scoring period during which the experimenter entered the vocal response made by the subject (for AudVoc trials of runs two through four). The scoring period began with a prompt requiring the experimenter to indicate the vocal response made (Tay, Koo, No response) and ended when the experimenter made his response. This scoring period generally lasted less than a second. The scoring period, or the response period for VisMan trials, was then followed by a feedback period that lasted for two seconds. If the response was correct for the VisMan task, the words ‘Face task CORRECT’ were presented in green just above the fixation marker, and if the response was incorrect the words ‘oooh – you got the face wrong!’ were presented in red just above fixation. Similar feedback was provided for the AudVoc task just below the fixation marker, with the word ‘tone’ substituted for the word ‘face’. Following the feedback period, the next trial ensued. Each trial during runs one through four was roughly eight or nine seconds in duration depending on the duration of the scoring period (trial duration = eight seconds + length of scoring period).

Run five familiarized subjects with the run structure of an fMRI session. The trials were carried out as in the first four runs except that there were no scoring and feedback periods following the 4 s response period and the fixation period was extended to 12 s (total trial duration of 16 s). Due to the extended period between stimulus presentations, an alerting cue (doubling of fixation size) was presented for 2 s prior to stimulus onset. By this last familiarization run, subjects had reached high performance accuracy (> 95% accuracy in first fMRI session) in each task under both dual- and single-task conditions.

The familiarization session, as well as the behavioral training sessions, were conducted in a psychophysics lab on a G4 eMac computer operating OSX, running Matlab 7 (7.3 R2006b) and the Psychophysics toolbox version 3.08 (Brainard, 1997; Pelli, 1997). The screen refresh rate was 72 Hz. The presentation of auditory stimuli and collection of vocal responses were performed with a Platronics DSP digital headset with built-in microphone. Manual responses were made using a standard QWERTY keyboard.

Behavioral Training Sessions

There were two types of runs in the behavioral training sessions: short inter-trial interval (short-ITI) runs, and long-ITI runs. The purpose of the short-ITI runs was to maximize the number of training trials subjects were exposed to, whereas the long-ITI runs served to ensure that training effects were also obtained at the trial presentation rate that would be used in the fMRI sessions (see below). Subjects performed four short-ITI and six long-ITI runs per training session, with run types randomly intermixed. Each training session consisted of 540 trials (432 short-ITI and 108 long-ITI trials), lasting about ninety minutes.

Each training session began with the presentation of an instruction screen that reminded subjects of the tasks that they would be performing, and also provided them with both face stimuli to study. In addition, each auditory stimulus was played three times to remind subjects of the stimulus-response mapping for the AudVoc task. Instructions stressed that subjects were to respond quickly and accurately and that equal emphasis should be placed on both tasks. To encourage fast and accurate responding a reward system was employed in which subjects accumulated points for trials in which the RTs for correct responses were lower than a deadline. Points were lost for incorrect responses. At the conclusion of the experiment, points translated into bonus pay (most subjects received ~$16). Deadlines were initially set to 2 s for the initial short-ITI run, but were subsequently adjusted based on single-task performance after each short-ITI run. Specifically, for each task the mean and standard deviation of reaction times were calculated from all single-task trials performed during a given short-ITI run. Normal distributions with these means and standard deviations were then used to calculate the reaction time at the 75^th percentile and this value was used as the deadline for both single-and dual-task trials for the next run. The reward system was explained to subjects prior to the first behavioral session.

Subjects initiated each run by pressing the spacebar. In short-ITI runs, trials consisted of a 2 s fixation of a central black square (1°) followed by stimulus presentation for 200 ms. Stimulus onset marked the beginning of a 2 s response period during which manual and vocal responses were digitally recorded. The response period was followed by a 2 s feedback period that provided subjects with the response times and deadlines for each task, as well as accuracy feedback for the VisMan task (no accuracy feedback could be provided for the AudVoc task because accuracy was scored offline - the auditory responses were recorded as .wav files and the RTs were determined by analysis of the spectrograms of these recordings). Long-ITI trials were identical to short-ITI trials except that the feedback period was replaced with a 2 s post-trial period during which only the fixation square was present and the fixation period was extended to 12 s, with the last 2 s of this period serving as an alerting cue by doubling the size of the fixation square. Trial duration was therefore 6 s and 16 s for the short- and long-ITI trials, respectively. Each short-ITI run consisted of 108 trials (36 trials/condition), while each long-ITI run consisted of 18 trials (six trials/condition).

Each run ended with a screen that provided a summary of the subject’s performance. This information included the deadline in effect for that run for each task, the average response time for both single- and dual-task trials for each task, accuracy for the VisMan task and the deadline that would be in effect for the next run. New deadlines were calculated only after short-ITI runs because there were only 6 trials per condition in the long-ITI runs. The new deadlines would be in effect until after the next short-ITI run. As there was no difference in the pattern of performance between the short- and long-ITI runs across the experiment, we combined these data in Fig 1. and supplemental Fig 3.

fMRI Sessions

fMRI sessions consisted of eight slow-event related runs that were identical to the long-ITI runs performed in the behavioral sessions except that a 12 s fixation period was added at the end of each run. There were 18 trials per run (with an equal number for each Trial Type randomly ordered), for a total of 144 trials per session. Prior to the first run of each fMRI session, an instruction screen reminded each subject of the task, stimuli and the response time deadlines for each task. Response deadlines were set based on the previous behavioral session.

Subjects completed three fMRI sessions. Session 1 (pre-training) was conducted immediately after the familiarization session, when multitasking interference should be maximal. The second session (mid-training) was undertaken once dual-task performance had shown significant improvement from the practice regimen but well before dual-task performance ceased improving. The third session (post-training) was run once subjects’ behavioral data indicated that performance for each of the two tasks in the dual-task condition was no longer improving relative to the single-task trials (in three consecutive sessions).

Data Acquisition

Anatomical 2D and 3D high-resolution T1-weighted images were acquired with conventional parameters on a 3T Philips Intera Achieva scanner at the Vanderbilt University Institute of Imaging Science. The visual display was presented on an Avotec (Stuart, FL) LCD panel and back-projected onto a screen positioned at the rear of the magnet. Subjects lay supine in the scanner and viewed the display on a mirror positioned above them. The auditory stimuli were presented and the vocal responses were recorded using a Commander XG MR compatible headset (Resonance Technology Inc, Northridge CA). Manual responses were recorded using a 5-key keypad (Rowland Institute of Science, Cambridge, MA). Functional (T2*) parameters were as follows: TR 2000 ms, TE 35 ms, FA 79°, FOV 24 cm, 128×128 matrix with 33 slices (3.5 mm thick, 0.5 mm skip) acquired parallel to the AC-PC line. Stimulus presentation was synchronized with each fMRI volume acquisition.

General Data Analysis

Image analysis was performed using Brain Voyager QX 1.4 (Brain Innovation, Maastricht, The Netherlands) and with custom Matlab software (MathWorks, Natick MA). Data preprocessing included 3D motion correction, slice scan time correction and linear trend removal. All functional data were aligned to the first localizer run and anatomical T1-weighted data were transformed into standardized Talairach space (Talairach and Tournoux, 1988).

SPMs were created using a multiple regression analysis, with regressors defined for the VisMan, AudVoc and Dual-Task trials and convolved with a double gamma hemodynamic response function (SPM2, http://www.fil.ion.ucl.ac.uk/spm), consisting of a positive gamma function and a small, negative gamma function reflecting the undershoot. Central processing areas were isolated by identifying brain regions that were significantly activated by both the VisMan and AudVoc tasks (i.e., conjointly activated by AudVoc open contrast and VisMan open contrast) in the pre-training fMRI session, although we ascertained that the same results were obtained when these regions were isolated from the post-training scanning session. Sensory or motor areas were isolated by directly contrasting the two single-tasks (AudVoc-VisMan). For both of these analyses, we used a voxel-wise analysis thresholded at a false discovery rate (FDR) of q < 0.05, except for one subject where a lower threshold of p < 0.005 (uncorrected) was employed because of low activation levels. Exclusion of this subject from the analysis did not alter the pattern of results.

BOLD amplitude analysis

A region of interest (ROI) was defined around the peak voxel of the activated foci by including all voxels above statistical threshold up to a maximum size of 343 mm³. ROIs were defined from fMRI session 1 (pre-training), however an identical pattern of results was observed when we defined ROIs using fMRI session 3 (post-training). Individual subjects’ time courses were extracted from the isolated ROIs and percent signal change was calculated relative to the two time points prior to stimulus/stimuli onset of each trial, and averaged across subjects. The peak volume of a timecourse was defined as the volume with the greatest signal amplitude between stimulus onset and the 8^th volume following this time point (16 s from onset). T-tests on peak volumes were used to assess for differences in response amplitude across conditions, using a random effects model.

Effective Connectivity Analysis

An effective connectivity analysis between sensory, prefrontal and motor ROIs was performed for each single-task condition (the dual-task condition was not used for this analysis). The sensory-prefrontal-motor pathways examined consisted of the right VC, left IFJ, and left ManMC, and of the left AC, left IFJ, and left VocMC. To assess for any laterality effect, the above analysis was repeated with the sensory-motor AudVoc ROIs in the right hemisphere (e.g., right AC, left IFJ, right VocMC). This path model yielded highly similar patterns of results to the former model. In a final connectivity analysis, the strength of the direct sensory-motor connections (i.e., right VC projecting to left ManMC, and left AC projecting to left VocMC, right AudVoc ROIs were also used) were also tested.

Raw time-series data from each ROI were first filtered to remove signals of no interest. Filtered data consisted of the residuals after fitting a linear model containing the global signal and 6 estimated rigid-body motion parameters at each time point, plus a high-pass filter set of discrete cosine basis functions with a cutoff of 150 seconds. These filtered time-series were then converted to percent signal change relative to their mean value over time, and were separated into individual trial segments. Task-specific time series data were then created by concatenating the individual trials for each single-task trial-type (AudVoc and VisMan) (Rogers et al., 2004; Rowe et al., 2002).

For each pair of ROIs, the functional connectivity between them was calculated as the Fisher-transformed correlation coefficient (Z) of the two ROI time series. Effective connectivity in the form of path coefficients was then calculated by fitting the full path model (see above and Fig. 3). Because the models contain no loops or reciprocal connections, ordinary least squares techniques were employed (Berry, 1984). The coefficients for a given path model were then averaged across subjects. As all connectivity measures were calculated within subject and within task, second-level statistical analysis was performed by treating task as a repeated measure and subject as the unit of observation. Finally, given the effect of training on response amplitude (Fig. 2), the connectivity analysis was also carried out after equating trial BOLD response amplitude between sessions 1 and 3 to ensure that gross amplitude changes between these sessions did not drive the connectivity results. There was no difference in the overall pattern of results after performing this correction and we present the amplitude equated data in Fig. 3.

Multivariate Pattern Classification Analysis

Adapting ensemble classification methods developed by Kamitani and Tong (Kamitani and Tong, 2005, 2006), a neural decoding analysis was performed to assess whether training lead to increased task selectivity in IFJ. Although any effect of training on neuronal task selectivity should be manifested in both the single-task and dual-task conditions, only the former was used for MVPA because the individual BOLD responses for each task cannot be resolved in dual-task trials.

ROIs and preprocessing

Voxels used for task decoding were selected from three ROIs: left IFJ, right AC, and right VocMC (identical patterns of results were observed when we used other sensory-motor ROIs: right VC, left AC, left ManMC and left VocMC). The voxels from IFJ were rank ordered according to their conjoined responses on both the AudVoc and VisMan single-task trials, whereas the AC and VocMC voxels were isolated on statistical maps generated from contrasting the AudVoc task and the VisMan task activity as described above. These ROIs were isolated using the pre-training scanning session. We verified, however, that the decoding results were the same when the post-training session was used to define and order the ROIs. ROIs were defined by isolating the peak voxel in a given foci and then selecting voxels around this peak up to a maximum size of 4096 mm³. These ROIs were larger than those employed in the amplitude analysis described above in order to increase variability and therefore our likelihood of detecting patterns of activation across ROIs that distinguished between the two tasks. An identical pattern of amplitude results was obtained in left hemisphere IFJ when we employed these larger ROIs; namely significant differences between dual-task and single-task trials pre- but not post-training.

For each ROI, the 100 voxels with highest t values were selected for the decoding analysis (Kamitani and Tong, 2005, 2006) (average min and max t values for the 100 voxels across subjects: IFJ 1.91 – 6.29; AC 0.27 – 3.80; VocMC −0.47 – 2.88; absolute min and max t values: IFJ 0.76 – 10.69; AC 0 – 7.24; VocMC −3.31 – 4.70). For each selected voxel, the timecourses from the two single-task conditions were extracted and the signal intensity of each voxel was averaged over a six-second time window (from 4 s to 10 s after stimulus presentation) in order to capture the peak of the hemodynamic response related to stimulus presentation and task performance. Percent-signal change for each run was calculated relative to a baseline corresponding to the last six-seconds of each fixation period of each trial, averaged across all trials within the run. The decoding analysis was also performed with and without the application of a spatial normalization procedure that normalized the response amplitudes of individual voxels relative to the average of the entire time course within each run. This was done to minimize baseline amplitude differences across runs and sessions, however this procedure made no difference to the overall pattern of results (the spatially normalized data are shown in Fig. 4). For the classifier data training set, the resulting activity patterns were labeled according to which of the two tasks participants were performing on a given trial and served as the input for the task classifier analysis.

Classification Analysis

fMRI activity patterns from IFJ, AC, and VocMC were analyzed using a linear classifier to predict which of the two behavioral tasks subjects were performing on a given trial. Linear support vector machines (SVM) (Vapnik, 1998) were applied in order to obtain a linear discriminant function that could distinguish between the two behavioral tasks. Mathematically, this function can be expressed by:

$$\text{g}(x)={\mathbf{w}}_{\text{i}}{\mathbf{x}}_{\text{i}}+w_o$$

where x_i is a vector specifying the fMRI amplitude of the voxel i, w_i is a vector specifying the weight of each voxel i, and w_o is the overall bias. For a training data set, linear SVM computes the optimal weights and bias for the discriminant function, such that this discriminant function, g(x) satisfies:

$$\begin{array}{l}\text{g}(x)>0\text{when}\text{fMRI}\text{activity}\text{is}\text{induced}\text{by}\text{one}\text{task}\\ \text{g}(x)<0\text{when}\text{fMRI}\text{activity}\text{is}\text{induced}\text{by}\text{the}\text{other}\text{task}\end{array}$$

To evaluate task classification performance, we performed a leave-one-run-out cross-validation procedure (Kamitani and Tong, 2005, 2006). This technique operates by testing the data from a single run, after training the decoder on the data from all other runs, thereby ensuring that independent samples are used for training and test. This procedure was repeated for all runs and performance was then averaged to produce a mean index of task classification accuracy (% correct classification) for each ROI.

Experiment 2

The purpose of this experiment was to examine the effect of dual-task training on the latency of IFJ activity using time-resolved fMRI. The behavioral paradigm and fMRI data acquisition and analysis for this experiment are as described in Experiment 1 except where otherwise specified below.

Subjects

Four right-handed members of the Vanderbilt University community (3 males, 25–30 years), with normal or corrected-to-normal vision, participated in this experiment for financial compensation. These subjects did not participate to Experiment 1.

fMRI Sessions

Subjects performed identical trials to those of Experiment 1, with the sole difference that participants were only scanned pre- and post-training in Experiment 2.

Data Acquisition

Functional (T2*) parameters: TR 200 ms, TE 35 ms, FA 30°, FOV 22 cm, 64×64 matrix with 3 coronal slices (8 mm thick, 0.5 mm skip) acquired perpendicular to the AC-PC line, with the most posterior slice going through the AC. This slice prescription encompassed the IFJ in all four subjects (BA 9; +/−45 (7.1), 9.6 (1.6), 25.3 (7.8); one subject only had a right hemisphere IFJ ROI).

Data Analysis

Data preprocessing included 3D motion correction, slice scan time correction, linear trend removal and high pass filtering (.01 Hz).

The peak volume of a time-course was defined as the volume with the greatest signal amplitude between 2 seconds post stimulus/stimuli onset and 14 seconds following this time point to avoid confounding peak activations with the early magnetic susceptibility and motion artifacts associated with the vocal response (Fig. 5B). Since the vocal artifact is limited to within the first couple of seconds of responding, it does not affect the later peak hemodynamic response (Birn et al., 2004).