Between-Task Competition and Cognitive Control in Task Switching

Abstract

Cognitive control is required to regulate interactions between brain regions to produce effective, purposeful behavior. We used functional magnetic resonance imaging to investigate the nature of these interactions and the role of prefrontal cortex (PFC) in cognitive control as subjects switched between simple face and word categorization tasks. Face and word stimuli were used because previous research has shown them to activate distinguishable cortical regions, allowing us to measure levels of activity in task-selective brain regions during task switching. We found that activity in brain regions selective for the currently irrelevant task predicted the behavioral cost associated with switching tasks. This finding supports the theory that between-task competition is a critical determinant of behavior. Task switching was also associated with increased activity in a network of regions implicated in cognitive control, including lateral PFC and parietal cortex. Within this network of regions, we observed dissociations between task-selective and general purpose mechanisms. These findings provide support for theories that propose a control hierarchy comprising regions responsible for maintaining task-specific information about rules or goals, and regions involved in the coordination of these goals.

Introduction

Cognitive control is required to guide thought and action in accordance with current goals and intentions (Norman and Shallice, 1986; Desimone and Duncan, 1995; Miller and Cohen, 2001). This control is particularly important when multiple tasks are possible and behavioral demands are continually shifting. The present study used functional magnetic resonance imaging (fMRI) to investigate cognitive control in a situation characterized by such fluctuating demands, in which subjects were required to make frequent, rapid switches between tasks. Our research approach was to design tasks that activated distinguishable brain regions, and then to measure activity in task-selective regions as an index of the degree to which subjects were paying attention to each of the tasks as they switched between them. Using this approach, we addressed two central issues in the study of cognitive control: the degree to which behavior is governed by competitive interactions between relevant and distracting sources of information, and the nature of processing in prefrontal cortex (PFC) that has been proposed to regulate this competition (Desimone and Duncan, 1995; Miller and Cohen, 2001).

We first investigated the hypothesis that task-switching performance is critically affected by the level of interference from task-irrelevant cognitive processing. Such interference has been proposed as an account of the “switching cost” that is observed when subjects are required to change task (Allport et al., 1994). Specifically, it has been proposed that subjects have a tendency to continue to perform the same task from one trial to the next, an effect termed “task-set inertia, ” which results in poor performance when task requirements change (Allport et al., 1994; Wylie and Allport, 2000; Yeung and Monsell, 2003a, b). Here, we tested a key prediction of this hypothesis: that on trials requiring a task switch, there should be residual activity in brain regions selective for the previous, now-irrelevant task. This residual activity (reflecting the tendency to perform the previous task) is presumed to interfere with performance of the current task, resulting in the behavioral switch cost. Evidence consistent with this prediction would support theories of cognitive control that emphasize the importance of competitive interactions in the control of behavior (Desimone and Duncan, 1995; Miller and Cohen, 2001).

Our second aim was to distinguish between task-selective and task-insensitive regions activated during switching. Previous studies have shown that many regions thought to be involved in cognitive control show increased activity during switching, including areas within PFC, anterior cingulate cortex (ACC), supplementary motor area (SMA), pre-SMA, and parietal cortex (Dove et al., 2000; MacDonald et al., 2000; Sohn et al., 2000; Brass and von Cramon, 2002; Dreher et al., 2002; Rushworth et al., 2002; Braver et al., 2003; Serences et al., 2004). However, these studies have typically not used tasks designed to activate distinct cortical regions, and have not attempted to dissociate regions involved in general functions of cognitive control from those involved in operations specific to the tasks performed. The present approach, using task stimuli designed to activate distinguishable regions within PFC, allowed us to investigate such dissociations.

Materials and Methods

Subjects.

Fifteen right-handed subjects (8 females and 7 males; ages, 19–24 years) were recruited from the Princeton University community. They received course credit or monetary payment for their participation in a single 2.5 h session. The subjects gave written informed consent and the study was approved by the local research ethics committee.

Procedure.

The subjects performed two tasks, classifying faces as male or female and classifying words as having two syllables or not. Previous studies have shown that tasks involving face and word stimuli activate distinguishable regions of PFC as well as occipitotemporal cortex (Petersen et al., 1989; Kanwisher et al., 1997; Fiez and Petersen, 1998; Kelley et al., 1998; McDermott et al., 1999; Cohen et al., 2002; Polk and Farah, 2002). The precise functional properties of some of these regions is currently a matter of some debate (Gauthier et al., 1999; Price and Devlin, 2003), and our aim here is not to make theoretical claims about these issues. Nor do we claim that the face- and word-selective activity we observe is restricted to the particular gender and syllable classification tasks we chose. Indeed, as will become apparent, although the classification tasks we chose are quite different from the tasks used in previous research, the network of face- and word-selective regions we observed corresponded well with the results of these previous studies. Rather, our approach was simply to use activity in areas showing selectivity for faces or words as an effective index of the degree to which subjects were attending to face and word stimuli as they switched between tasks involving these stimuli.

Each experimental session began with localizer scans that we used to identify task-selective regions of interest (ROIs) for each subject. During the localizer scans, subjects were presented with separate sequences of faces (with no superimposed word) and words (with no superimposed face). They performed the same two classification tasks as in the main experiment (a gender classification on face stimuli and a syllable judgment on word stimuli) on these unambiguous stimuli. The localizer scans therefore also served to give the subjects practice at the two tasks before the main task-switching runs. To match the general visual properties of the stimuli for the two tasks, face stimuli were presented with an unpronounceable letter string superimposed, while word stimuli were presented on top of noise patterns created by phase-scrambling of face images. Each localizer task sequence began with a verbal instruction indicating the upcoming task for 2.7 s, which was followed after 0.9 s by six trials of the corresponding task. Successive stimuli in the task sequence were separated by a 1.8 s interstimulus interval. After completion of the sequence and an interval of 0.9 s, subjects were given brief feedback for 2.7 s. There then followed a fixed intertrial interval of 7.2 s. Subjects performed two runs of 12 such sequences. Within each run, the tasks were presented in counterbalanced ABBA order across sequences.

The subjects next completed eight task-switching runs. During these runs, subjects were presented on each trial with stimuli comprising a word superimposed on a face stimulus (Fig. 1). The superimposed face/word stimuli were presented in short sequences of four trials, which we refer to as “miniblocks.” Subjects performed a single task within each miniblock, and switched back and forth between the tasks across miniblocks. The principal comparison of interest is between task switch miniblocks (in which the task had changed from the previous miniblock) versus task repeat miniblocks (in which the task stayed the same). As described below, the miniblock switching design was effective in producing robust behavioral switching costs even with the long intertrial intervals needed for our event-related fMRI analyses. Presumably, this was because subjects were motivated to prepare for each miniblock, faced with the prospect of performing several trials of the same task. Subjects performed 20 miniblocks in each of the eight runs, yielding a total of 40 samples in each of the four primary conditions (face switch, face repeat, word switch, word repeat). The sequence of specific stimuli was constrained so that none was ever repeated across trials (within and across miniblocks).

Figure 1.

The task-switching design. Subjects were presented with overlapping face and word stimuli, and were required on each trial to classify the face as male or female (face task) or to classify the word as having two syllables or not (word task). Subjects performed short four-trial sequences, or miniblocks, of the two tasks. They performed only one task in any given miniblock and switched between tasks across miniblocks. A colored patch indicated the task to be performed in the upcoming miniblock, and the task order was randomized across the 20 miniblocks of each of the eight task-switching scans. Brief feedback was provided at the end of each miniblock to motivate subjects to perform with high accuracy. S, Stimulus; FB, feedback; ITI, intertrial interval.

Task order was unpredictable (randomized) across miniblocks. A colored patch appeared before the first stimulus of each miniblock to indicate the task required, with the mapping of color (red or green) to task (face or word) varied across subjects. This task cue was presented 0.45 s before the miniblock. Successive stimuli were separated by 1.8 s within a miniblock. After 0.45 s at the end of the miniblock, feedback was presented for 1.35 s. The next miniblock began after a variable interval of 5.4–9.2 s. The first stimulus in each miniblock was always incongruent (that is, the superimposed face and word stimuli were associated with different responses). On the remaining trials within the miniblock, some of the stimuli were congruent (that is, the face and word stimuli were associated with the same response). Across the experiment, there were equal numbers of incongruent and congruent trials.

There were 25 male and 25 female faces, drawn from a database provided by the Max Planck Institute for Biological Cybernetics (Tuebingen, Germany). We added visual noise to the face stimuli to make the gender discrimination more difficult. Visual noise was created by phase-randomization of a subset of the face stimuli, a process that produced cloudy images that were superimposed on top of the original faces. We created nine separate noise patterns. Combined with the 50 face stimuli, there were thus 450 face/noise stimuli. The set of word stimuli comprised 216 two-syllable and 216 one- and three-syllable words, drawn from a database used in a previous study of phonological processing (Poldrack et al., 1999). Word stimuli appeared at most once for each task, and different word sets were used in the localizer and task-switching scans. The subjects responded by pressing keys with the index and middle fingers of their right hand. The mapping of stimulus categories to response fingers was varied systematically across subjects. Subjects’ behavioral performance was assessed in terms of mean reaction times (RTs) and error rates. The RT and error rate data were subjected to separate ANOVAs using within-subjects factors of task (face/word), switching (switch/repeat), and trial within miniblock (1, 2, 3, or 4).

fMRI data acquisition.

Images were collected with a 3.0 Tesla Siemens Allegra head-dedicated MRI scanner. Anatomical images were collected using a T1-weighted MPRAGE protocol [field of view (FOV), 256 mm; matrix, 256 × 256; 176 1 mm sagittal slices]. Functional images were reconstructed from 30 axial slices parallel to the anterior commissure–posterior commissure plane, acquired using a T2*-weighted echo-planar pulse sequence [repetition time (TR), 1800 ms; echo time (TE), 30 ms; flip angle, 75°; FOV, 192 × 192 mm; matrix, 64 × 64; in-plane resolution, 3 × 3 mm; slice thickness, 4.8 mm]. The presentation of task events was synchronized with TR onsets. Two localizer runs (175 scans each) and eight task-switching runs (180 scans each) of fMRI data were acquired. The first three scans of each run were discarded from the analysis to allow magnetization to reach a steady state.

fMRI data analysis.

Data were preprocessed and analyzed with BrainVoyager 2000 software (Maastricht, The Netherlands). Image preprocessing consisted of three-dimensional motion correction using trilinear interpolation, slice scan time correction using sinc interpolation, spatial smoothing with a 4 mm full-width at half-maximum Gaussian kernel, voxel-wise linear detrending, and high-pass filtering of frequencies below three cycles per time course. Spatial normalization across subjects was performed using the standard nine-parameter landmark method of Talairach and Tournoux.

The primary data analysis focused on ROIs that showed significant task-selective activity during the localizer scans. To identify these regions for each subject, we modeled the blood oxygen level-dependent (BOLD) response in the localizer scans using a general linear model (GLM) with separate regressors for face task trials, word task trials, task instructions, and feedback stimuli. Each regressor was convolved with a standard gamma model of the hemodynamic impulse–response function (Boynton et al., 1996), and the resulting general linear model was corrected for temporal autocorrelation using a first-order autoregressive model. We then used a fixed-effects analysis to identify, separately for each subject, regions showing significantly (p < 0.05) greater activity during face task performance than during word task performance (face-selective ROIs), and regions showing the opposite pattern (word-selective ROIs). A number of face- and word-selective ROIs were observed consistently across all subjects. Because there was a high degree of variability in the size of ROIs across brain areas, tasks, and subjects at a given statistical threshold, we chose to generate 8 × 8 × 8 mm ROIs, separately for each subject, that were centered on voxels showing the greatest task-selective activation during the localizer scans. Activity within these cubic ROIs formed the basis for subsequent analysis.

For each ROI identified in the localizer scan, we calculated the percent magnetic resonance (MR) signal change during task-switching runs separately for each of the four miniblock types (face switch, face repeat, word switch, and word repeat). We excluded from this analysis any miniblock in which the subject’s response to the first (incongruent) stimulus of the miniblock was incorrect, because for these miniblocks it was not certain that the subject was performing the correct task. As is evident in the resulting event-related averages (see Figs. 3 and 5), task- and switching-related changes in activity were extended across many TRs. However, for the purposes of statistical analysis, we required a single measure of activity in each ROI. Rather than using percent signal change in one arbitrarily selected time point or indiscriminately averaging across all TRs, we chose to measure percent signal change as a weighted average of activity after the presentation of the first stimulus of the miniblock. The TRs (1–8) were weighted as follows: 0.00, 0.09, 0.17, 0.20, 0.20, 0.17, 0.11, 0.05, and 0.02.

Figure 3.

Activity in posterior cortical regions during switching. Each panel shows event-related activity (percent signal change) averaged across subjects together with ROI locations from one example subject. Event-related averages are shown separately for each task for switch and repeat miniblocks. Data are shown for the face-selective region in right FG (A), left ITG (B), and extrastriate (exstr.) regions in the right (C) and left (D) hemispheres. The icons along the abscissa of each graph indicate the timings of the presentation of each of the four stimuli in each miniblock. SE bars are shown for the contrast between switch and repeat miniblocks, calculated separately for each task. S, Stimulus; FB, feedback.

Figure 5.

Activity in frontal cortex during switching. Each panel shows event-related activity (percent signal change) averaged across subjects together with ROI locations from one example subject. Data are shown for a face-selective region in the posterior portion of the right IFS (A); a word-selective region in the posterior portion of the left IFG (B); a face-selective region in the anterior portion of the right IFS (C); and a word-selective region in the anterior portion of the left IFG (D). The icons along the abscissa of each graph indicate the timings of the presentation of each of the four stimuli in each miniblock. SE bars are shown for the contrast between switch and repeat miniblocks, calculated separately for each task. Ant., Anterior; Post., posterior; L, left; R, right; S, stimulus; FB, feedback.

The TR weightings were derived using a temporal principal components analysis (PCA) of the individual subjects’ event-related averages. PCA provides a formal method for weighting time points according to their contribution to overall variance, by calculating the degree of covariation in BOLD signal among these time points (Donchin and Heffley, 1978). The first principal component corresponds to the weighting that accounts for the most possible variance in the data, and effectively weights each TR according to its relative level of BOLD signal change. This method is conceptually similar to standard hemodynamic deconvolution methods, but has the advantage that it does not make a priori assumptions about the shape of the hemodynamic response function. Thus, the weightings described above match well the general time course of BOLD activity shown in Figures 3 and 5. The resulting weighted averages of percent signal change were used in the reported ANOVAs and t tests comparing task- and switching-related changes in activity, and in correlations between neural activity and behavioral performance.

A second ROI-based analysis was used to determine whether switch-related effects on neural activity were specific to task-selective regions or whether corresponding effects would be observed in other regions activated during performance regardless of the task. To this end, we used a fixed-effects GLM analysis to identify in each subject bilateral regions in extrastriate cortex that showed the most significant activity during task performance, comparing activity in miniblocks (pooling across tasks and switching conditions) against rest. We defined 8 × 8 × 8 mm ROIs around each subject’s voxels of peak activation and then calculated percent signal change in these ROIs during task-switching performance, separately for each miniblock type. Activity in these ROIs, again calculated using a PCA-based weighted average of activity across the eight TRs after miniblock onset, was assessed for evidence of task- and switching-related activity.

A third, exploratory analysis was run to determine whether brain areas other than the task-selective ROIs would show evidence of switching-related activity. A random-effects GLM analysis was applied to the functional data acquired during the task-switching runs, using separate regressors for each miniblock type (face switch, face repeat, word switch, word repeat). Regions showing switch-related activity were identified using a contrast between switch versus repeat miniblocks (pooling across the two tasks). A number of regions showed greater activity in switch miniblocks than repeat miniblocks (p < 0.0001, uncorrected; minimum cluster size = 432 mm³ = 10 voxels). There were no regions showing greater activation in repeat miniblocks than switch miniblocks, even at a more liberal threshold (p < 0.05, uncorrected). To assess whether the ROIs that were activated during task switching also showed some evidence of task selectivity, we calculated event-related activity in each of the ROIs separately for each subject using the PCA-based method described above, and then subjected the resulting data to an ANOVA with factors of task (face/word) and miniblock type (switch/repeat).

In a final analysis, we investigated the extent to which activity in task-selective ROIs varied as a function of overall RT, as an index of nonspecific effects associated with time-on-task or mental effort. We performed a median split to divide miniblocks into fast and slow subsets, separately for each task. We then calculated percent MR signal change in the task-selective ROIs during task-switching runs separately for each of the four miniblock types (face fast, face slow, word fast, word slow). Activity in these regions, once again calculated using a PCA-based weighted average of activity across the eight TRs after miniblock onset, was assessed for evidence of variance related to overall RT.

Results

Behavioral results

Performance was slower (F_{(1, 14)} = 10.1; p < 0.01) and somewhat less accurate (F_{(1, 14)} = 3.1; p < 0.1) for switch miniblocks than repeat miniblocks (Fig. 2). This switching cost primarily affected performance on the first trial of the miniblock, both for RTs (F_{(3, 42)} = 8.4; p < 0.01) and for error rates (F_{(3, 42)} = 6.5; p < 0.05). A restart cost (Allport and Wylie, 1999) was also evident in terms of increased RTs on the first trial of a miniblock even when the task was repeated (F_{(3, 42)} = 15.0; p < 0.01). Overall, the face gender discrimination was performed faster (F_{(1, 14)} = 20.5; p < 0.01) but less accurately (F_{(1, 14)} = 12.8; p < 0.01) than the word syllable judgment. However, switching costs did not differ consistently between the two tasks.

Figure 2.

Behavioral results. Shown are mean RT and percent errors separately for each of the four trials of switch and repeat miniblocks, separately for the face and word tasks. S, Stimulus.

Switching costs and task-selective neural activity

Our first analysis of the imaging data tested predictions of the task-set inertia hypothesis. According to this hypothesis, the behavioral switch cost reflects a residual tendency to perform the previous task that interferes with performance of the new task. A straightforward prediction of this hypothesis is that on trials requiring a switch of task there should be residual activation in brain regions selective for the previous, now-irrelevant task. For example, on a trial in which subjects are required to attend to word stimuli, we predict that there should be increased activation in face-selective regions if the face task had been performed on the previous trial, relative to the case in which the word task is performed repeatedly. In turn, we predict that the size of the behavioral cost associated with switching to the word task should depend on the magnitude of this residual tendency to attend to the previous, now-irrelevant attribute (faces).

The predictions of the task-set inertia hypothesis regarding neural activity in task-relevant areas during switching is less obvious: One might expect that task-relevant activity should be reduced during switching, reflecting increased competition from (residual) activity in task-irrelevant regions (predicting a negative correlation between task-relevant activity and behavioral cost). Alternatively, it could be that task-relevant activity will increase during switching, reflecting the extra time required to process task-relevant information in the face of this competition (predicting a positive correlation). In fact, computational simulations using an existing model of cognitive control in task switching (Yeung and Monsell, 2003a) suggest that these two effects trade off, such that task-relevant activity is not strongly predictive of behavioral performance (as described in supplemental material, available at www.jneurosci.org). Thus, our theory makes the strong prediction that activity in task-irrelevant brain regions should be the primary measurable correlate of behavioral performance.

Task-selective ROIs were identified in individual subjects on the basis of localizer scans run before the task-switching scans. A number of regions showed task-selective activity consistently across subjects, with individual subjects’ ROIs falling within 15 mm of group average Talairach coordinate locations (Table 1) (see also supplemental Table 1, available at www.jneurosci.org as supplemental material). These regions included the right fusiform gyrus (FG) and left inferior temporal gyrus (ITG), areas commonly found to activate during processing of faces (Kanwisher et al., 1997) and written words (Cohen et al., 2002; Polk and Farah, 2002), respectively. Our first analysis focused on activity in these regions.

Table 1.

Talairach coordinates of regions of interest identified in exploratory analyses

	Side	x	y	z
Localizer: face > word
Fusiform gyrus	R	38	−43	−13
Posterior IFS	R	35	8	27
Anterior IFS	R	41	29	18
Localizer: word > face
Inferior temporal gyrus	L	−48	−49	−9
Posterior IFG	L	−48	7	12
Anterior IFG	L	−45	31	10
Switching: miniblock > rest
Extrastriate	R	18	−79	−2
Extrastriate	L	−24	−81	−6
Switching: switch > repeat
Insula/operculum (a)	R	35	26	3
Insula/operculum (b)	L	−31	22	6
Ventral posterior cingulate (c)	L	−7	−44	12
Anterior cingulate cortex (d)	R/L	4	23	21
Dorsal posterior cingulate (e)	R/L	1	−29	29
Precuneus (f)	L	−10	−65	42
Anterior cingulate/pre-SMA (g)	R	7	9	37
Medial superior parietal (h)	L	−7	−68	42
Inferior parietal lobe (i)	L	−34	−56	43
Inferior parietal lobe (j)	L	−49	−38	36
Middle frontal gyrus (k)	R	29	43	34
Posterior parietal cortex (l)	R	32	−68	28

Task-selective ROIs in individual subjects fell within 15 mm of the average locations presented above. Letters a–l in the switch versus repeat contrast correspond to the regions labeled in Figure 6. R, Right; L, left.

Both regions showed clear evidence of task selectivity during the switching scans, although the same type of stimulus (a word superimposed on a face) was shown for trials in each task (Fig. 3). Activity in the right FG was greater in face task miniblocks than word task miniblocks (F_{(1, 14)} = 35.4; p < 0.01), whereas activity in the left ITG showed the opposite pattern (F_{(1, 14)} = 21.4; p < 0.01). Thus, consistent with previous findings (Wojciulik et al., 1998; O’Craven et al., 1999; Serences et al., 2004), we observed substantial modulation of object-selective neural activity by visual attention. These regions were also affected by task switching, showing greater activity during switch than repeat miniblocks (right FG, F_{(1, 14)} = 35.6, p < 0.01; left ITG, F_{(1, 14)} = 5.83, p < 0.05).

Subsequent pairwise comparisons demonstrated that, consistent with our predictions, activity was increased in regions selective for the currently irrelevant task after a change of task. During performance of the word-reading task, activity in the face-selective ROI was increased if the face task had been performed in the previous miniblock than if the word task had just been performed (Fig. 3A), that is, the right FG was more active during word switch than word repeat miniblocks (t₍₁₄₎ = 3.47; p < 0.01). Similarly, during performance of the face task, activity in the word-selective left ITG was increased if the word task had just been performed (Fig. 3B); this region was more active during face switch than face repeat miniblocks (t₍₁₄₎ = 3.08; p < 0.01). These findings are consistent with the hypothesis that the switching cost is caused by increased interference from recently performed tasks. However, activity in regions selective for the currently relevant task varied in a similar manner during switching, also being increased during switch relative to repeat miniblocks (Fig. 3A, B). Activity in the right FG was greater during face switch than face repeat miniblocks (t₍₁₄₎ = 2.57; p < 0.05); and activity in the left ITG was somewhat greater during word switch than word repeat miniblocks (t₍₁₄₎ = 1.43; p > 0.15).

Although these results are not surprising [one would naturally expect increased task-selective activity in more difficult or demanding conditions, and corresponding effects are observed in our computational simulations of between-task interference effects (supplemental material, available at www.jneurosci.org)], they raise the question of whether there was a general increase in neural activity in switch miniblocks. Observation of a general increase in activity would cast some doubt on our hypothesis that task-irrelevant activity during switch miniblocks is related to the observed behavioral cost. We therefore conducted two further analyses to address this issue. In the first analysis, we assessed directly whether there was a general increase in neural activity during switching. To this end, we performed an exploratory analysis to identify regions that showed activity during task performance (pooling across switching conditions and tasks) relative to rest. This analysis identified bilateral regions in extrastriate cortex that were very consistently observed across subjects (Table 1) (see also supplemental Table 1, available at www.jneurosci.org as supplemental material). Analysis of activity in these regions provides an important test of whether the increased switch-related activity in nearby task-selective regions reflects a general increase in activity in this part of cortex (e.g., because subjects simply spend a longer time visually processing the stimuli). Critically, although the extrastriate regions were very strongly activated during task performance, they did not show increases in activity that were specific to switch miniblocks (F values < 1) (Fig. 3C, D), in contrast to the task-selective regions described above. Thus, there was not a general increase in neural activity in switch miniblocks; such increases were not observed in task-nonselective regions that nevertheless showed a great deal of activity during task performance.

In the second analysis, we tested directly the prediction that increases in task-irrelevant activity during switch miniblocks should be correlated with the observed behavioral cost of switching. This analysis exploited the fact that there was variability across subjects in their susceptibility to switch costs and in the magnitude of their task-related neural activity. We calculated for each subject the extent to which activity in task-irrelevant brain regions was increased during switch relative to repeat miniblocks. We then correlated this measure with behavioral switch costs across subjects. Switch costs were calculated separately for each task as the difference between mean RT in switch miniblocks and mean RT in repeat miniblocks. We calculated behavioral costs using all four trials of each miniblock, because we were correlating this behavioral measure with BOLD signal change recorded across whole miniblocks. As shown in Figure 4, top panels, significant correlations between behavioral costs and task-irrelevant neural activity were observed for both tasks. Thus, the cost of switching to the word task increased as a function of the difference in activity in the face-selective right FG between word switch and word repeat miniblocks (Fig. 4A). Similarly, the cost of switching to the face task increased as a function of the degree of activity in the word-selective left ITG during face task miniblocks (Fig. 4B). (A weakness of the latter relationship is that the correlation is influenced by the leftmost data point, a subject showing no cost of switching to the face task and correspondingly little effect of switching on activity in the left ITG. With outliers removed, the correlation remained in the predicted positive direction but fell just short of statistical significance.) These results provide support for the theory that competition from task-irrelevant information is an important determinant of performance in task switching.

Figure 4.

Correlations between neural activity and behavioral performance. Behavioral task-switching costs are plotted against switching-related changes in activity in task-irrelevant (A, B) and task-relevant (C, D) brain regions, with each point corresponding to the averaged data from one subject.

In contrast to the correlations between behavioral performance and task-irrelevant neural activity, we found no significant relationship between switching costs and activity in regions selective for the currently relevant task (Fig. 4, bottom panels): Activity in the face-selective region did not predict performance of the face task in switch miniblocks; similarly, activity in the word-selective region did not correlate reliably with performance of the word task. Although this lack of correlation may seem surprising, as noted above (and elaborated in Discussion), the results of computational modeling work suggest that these findings, together with those from task-irrelevant areas, provide support for the hypothesis that switching costs are caused by increased interference from recently performed tasks.

Task-selective activity in PFC

In a second set of analyses, we investigated PFC contributions to task-switching performance. According to the “guided activation” theory, a critical function of task-selective regions of PFC is to maintain specific information about relevant rules or goals that guide the flow of activity in other brain regions required to perform the task (Miller and Cohen, 2001). Based on this theory, we predicted that we would observe task-selective regions within PFC that would show increased activity during switching, reflecting the increased need for cognitive control in difficult or demanding conditions.

The localizer scans identified several regions within PFC that were selectively activated by one of the two tasks, including portions of lateral PFC close to those found to be activated in previous studies (Dove et al., 2000; MacDonald et al., 2000; Sohn et al., 2000; Dreher et al., 2002; Braver et al., 2003). Specifically, we found two regions in right PFC that showed face selectivity and two regions in left PFC that showed word selectivity (Table 1) (see also supplemental Table 2, available at www.jneurosci.org as supplemental material). The locations of these regions corresponded well with those of previous studies using face and word stimuli (Kelley et al., 1998; McDermott et al., 1999). The presence of task-selective regions within PFC is broadly consistent with the guided activation theory, which proposes that PFC plays a critical role in maintaining task-relevant information to guide cognitive processing (Miller and Cohen, 2001). Of interest was whether these task-selective frontal regions would show increased activity during switching.

The activity patterns in these frontal ROIs closely resembled those in the occipitotemporal cortex described above (Fig. 5). The regions showed significant modulation by task, with right frontal regions more active during face task performance than word task performance [right inferior frontal sulcus (IFS): posterior, F_{(1, 14)} = 22.5, p < 0.01; anterior, F_{(1, 14)} = 30.6, p < 0.01] and left frontal regions showing the opposite pattern [left inferior frontal gyrus (IFG): posterior, F_{(1, 14)} = 47.4, p < 0.01; anterior, F_{(1, 14)} = 29.3, p < 0.01]. These regions were also more active during switch miniblocks than repeat miniblocks (right IFS: posterior, F_{(1, 14)} = 9.91, p < 0.01; anterior, F_{(1, 14)} = 25.5, p < 0.01; left IFG: posterior, F_{(1, 14)} = 10.2, p < 0.01; anterior, F_{(1, 14)} = 5.79, p < 0.05).

Subsequent pairwise comparisons tested the hypothesis that there should be increased activity during switch miniblocks in PFC regions selective for the currently relevant task. Consistent with this prediction, face-selective frontal regions were more active during face switch than face repeat miniblocks (right IFS: posterior, t₍₁₄₎ = 2.89, p < 0.05; anterior, t₍₁₄₎ = 3.17, p < 0.01) and word-selective regions showed increased activity in word switch relative to word repeat miniblocks (left IFG: posterior, t₍₁₄₎ = 2.15, p < 0.05; anterior, t₍₁₄₎ = 3.18, p < 0.01). In addition, switch-related increases in activity were also observed in the region selective for the currently irrelevant task (right IFS: posterior, t₍₁₄₎ = 3.71, p < 0.01; anterior, t₍₁₄₎ = 4.71, p < 0.01; left IFG: posterior, t₍₁₄₎ = 2.42, p < 0.05; anterior, t₍₁₄₎ = 2.40, p < 0.05). These findings suggest that task-selective representations in frontal cortex were subject to task-set inertia effects similar to those observed in the task-selective occipitotemporal regions described above. However, we did not observe any statistically reliable correlations between behavioral switching costs and activity in task irrelevant regions in PFC (r < 0.43; p > 0.11), although in each area the trend observed was in the predicted, positive direction. The corresponding correlations between task performance and activity in task-relevant ROIs were also not reliable (r < 0.37; p > 0.17). It is difficult to interpret these nonsignificant correlations, particularly given that activity in these frontal regions was smaller and more variable across subjects than was activity in the task-selective occipitotemporal regions described above.

Our findings of task-selective PFC activity contrast, to some degree, with those of a recent study that looked for, but did not find, correlations between activity in task-selective prefrontal and posterior cortical regions (Stephan et al., 2003). However, in our experiment, subjects made rapid switches between tasks and the task to be performed was cued shortly before task performance, whereas Stephan et al. used a blocked design and presented task cues many seconds before task performance. Thus, cognitive control demands are likely to be much lower in the design of Stephan et al. design than in ours, which may explain the different pattern of PFC results observed in the two studies.

Switch-specific neural activity

A number of recent theories have proposed the existence of functional specialization within the network of areas involved in cognitive control. In particular, a distinction has been drawn between areas involved in task-specific operations versus those involved in higher-level, domain-general functions such as the coordination of multiple concurrent goals, such as may be involved in task-switching (Koechlin et al., 1999, 2003; Christoff and Gabrieli, 2000; Fletcher and Henson, 2001; O’Reilly et al., 2002; Braver et al., 2003; Buckner, 2003). To seek evidence of dissociations in control functions, we performed an exploratory analysis to identify regions showing increased activity (p < 0.0001) in switch miniblocks compared with repeat miniblocks independent of task (Fig. 6A, Table 1). We then investigated the nature of activity in these regions during switching.

Figure 6.

Regions showing switch-related increases in activity. A, Regions showing significant activity in initial exploratory analysis (p < 0.0001). Activation foci were observed in the right insula/operculum (a), left insula/operculum (b), ventral posterior cingulate (c), anterior cingulate cortex (d), dorsal posterior cingulate (e), precuneus (f), anterior cingulate/pre-SMA (g), medial superior parietal cortex (h), left inferior parietal lobe (i), and left inferior parietal lobe (j). B, Regions showing significant activity at a more relaxed statistical threshold (p < 0.001). In addition to regions identified in the initial exploratory analysis, activation foci were observed in the right middle frontal gyrus (k) and right posterior parietal cortex (l), as well as in task-selective regions of PFC and occipitotemporal cortex. Talairach coordinates for activation peaks within ROIs a–l are given in Table 1.

Consistent with previous studies, we found activity associated with task switching in a specific network of regions including ACC/pre-SMA, insula/operculum, posterior cingulate, precuneus, and parietal cortex (Dove et al., 2000; Kimberg et al., 2000; MacDonald et al., 2000; Sohn et al., 2000; Brass and von Cramon, 2002; Dreher et al., 2002; Rushworth et al., 2002; Yantis et al., 2002; Braver et al., 2003; Serences et al., 2004). We did not find regions of activity in lateral PFC. However, an analysis using a more relaxed statistical threshold (p < 0.001) identified a region in the right middle frontal gyrus (MFG) in addition to the more posterior and inferior task-selective regions of PFC described above (Fig. 6B, Table 1). At this less conservative threshold, a region of activity was also observed in the right posterior parietal cortex.

Previous studies have implicated the regions identified in this analysis in a variety of general (rather than task-specific) cognitive control functions, such as performance monitoring by ACC (Carter et al., 1998; Botvinick et al., 2001; Yeung et al., 2004), shifting motor set by pre-SMA (Rushworth et al., 2002), and shifting attentional set by parietal cortex (Corbetta and Shulman, 2002; Yantis et al., 2002; Serences et al., 2004). The present methods allow us to test the prediction that these areas should be activated during switching regardless of the task currently required. Consistent with this prediction, we found no evidence of task selectivity in ACC (Fig. 6, region d) (F < 1) and ACC/pre-SMA (g: F_{(1, 14)} = 1.92, p = 0.19). Moreover, in contrast to the strong task selectivity of the regions identified in the right IFS and left IFG, activity in the right MFG did not differ for the two tasks (k: F_{(1, 14)} = 1.59, p = 0.23). The results further indicated a dissociation between regions of parietal cortex: A medial area was activated regardless of the task performed (h: F < 1), whereas areas in the left inferior parietal lobe showed selectivity for the word task (i: F_{(1, 14)} = 2.44, p = 0.14; j: F_{(1, 14)} = 20.9, p < 0.01) and in the right posterior parietal lobe showed selectivity for face stimuli (l: F_{(1, 14)} = 4.58, p = 0.05). Of the remaining areas identified in the exploratory analysis, only the right insula showed a degree of task selectivity (faces > words: a, F_{(1, 14)} = 3.77, p = 0.07; all other regions, F < 2.52; p > 0.13).

To investigate further the nature of activity in these brain regions, we calculated for each subject the degree to which each switch-specific region showed increased activity in switch relative to repeat miniblocks, and then correlated this measure with behavioral switch costs across subjects. Reliable positive correlations were observed for the ACC/pre-SMA (g: r₍₁₄₎ = 0.56, p < 0.03) and posterior cingulate (e: r₍₁₄₎ = 0.70, p < 0.01). There was no reliable correlation between behavioral switch costs and activity in the right MFG (k: r₍₁₄₎ = 32, p > 0.25), medial parietal cortex (l: r₍₁₄₎ = 0.47, p = 0.08), or precuneus (f: r₍₁₄₎ = 0.34, p > 0.21). Our finding that greater ACC/pre-SMA activity was associated with worse performance (larger switch cost) might initially seem to contrast with the findings of Brass and von Cramon (2002), who found that greater pre-SMA activity was associated with better performance (larger benefit of task preparation). However, Brass and von Cramon’s analysis focused on neural activity during task preparation, whereas our analysis focuses on activity during task performance. The combined results are consistent with the hypothesis that pre-SMA is involved in cognitive control (e.g., the representation of task-relevant stimulus-response mapping rules) and the management of between-task interference. Thus, when the area is active during task preparation [as in the study by Brass and von Cramon (2002)], interference will be reduced and performance will improve, so that neural activity will be increased as performance improves. In contrast, when little time is available for preparation (as in the present study), activity in this region will increase as a function of the degree of interference, so that neural activity will increase in parallel with worsened performance. This interpretation of the data leads to the prediction of a negative correlation between levels of pre-SMA activity during task preparation and those observed during task performance, a prediction that might form the basis for future research.

RT-related activity in task-selective regions

In a final analysis, we investigated whether observed effects of switching might be explained in terms of global differences (e.g., in levels of attention or task difficulty) between switch and repeat miniblocks, rather than reflecting a specific effect of task-set inertia. To address this issue, we analyzed whether activity in the task-selective ROIs in posterior cortical regions and PFC varied as a function of overall RT, independently of task-switching requirements. To this end, we performed a median split to divide miniblocks into fast RT and slow RT subsets. Average RTs were 703 ms in fast miniblocks and 963 ms in slow miniblocks. The question of interest was whether activity in task-selective ROIs would vary with RT in a similar manner as they vary across switching conditions.

If the observed association between task-irrelevant activity in posterior cortex and task-switching performance reflected a global effect of task difficulty, we would expect similar effects to be apparent in the contrast between fast and slow RT trials. However, no such effects were observed (right FG: F_{(1, 14)} = 1.26, p = 0.23; left ITG: F_{(1, 14)} = 0.08, p = 0.93). Similarly, if increased activity in task-relevant PFC regions during switching was simply a reflection of slowed RTs on switch trials, we would expect to observe greater activity on slow RT trials than on fast RT trials. However, we observed no such effect in our data: Activity in task-selective PFC regions did not vary reliably as a function of overall RT (left IFS: anterior, F_{(1, 14)} = 0.28, p = 0.78; posterior, F_{(1, 14)} = 1.74, p = 0.10; right IFG: anterior, F_{(1, 14)} = 1.88, p = 0.08; posterior, F_{(1, 14)} = 0.79, p = 0.44). It is worth noting that these null effects were observed, although much greater RT differences were observed between fast and slow RT trials in this comparison (260 ms) than in the comparison between switch and repeat trials (22 ms). Thus, it is very unlikely that the observed effects of switching on task-selective neural activity reflect global differences in attention or task difficulty. Instead, the effects we observed appear to be specifically related to the requirement to switch tasks.

Discussion

We used neuroimaging techniques to investigate between-task competition and cognitive control in task switching. We found that regions selective for the currently irrelevant task showed increased activity during switching, and that performance correlated with this task-irrelevant neural activity. These findings support theories of cognitive control that emphasize the importance of competitive dynamics in the control of behavior (Desimone and Duncan, 1995; Miller and Cohen, 2001). In addition, we found dissociations among regions activated during switching, between regions showing task selectivity and those activated regardless of the task performed. Evidence of task-related PFC activity supports the proposal that this region is involved in maintaining task-relevant information needed for the control of behavior (Desimone and Duncan, 1995; Miller and Cohen, 2001). The observed dissociations within PFC provide important insights into the functional architecture of this region, supporting theories that propose the existence of control hierarchies involving domain-general and task-specific processes (Koechlin et al., 1999; Christoff and Gabrieli, 2000; Braver et al., 2003; Sakai and Passingham, 2003).

Between-task competition

Consistent with predictions of the task-set inertia theory (Allport et al., 1994), we found that switching costs correlate with increases in task-irrelevant neural activity after a switch. It has recently been shown that prolonged practice can lead to lasting increases in neural activity related to a task even when it is no longer required (Wylie et al., 2004). Using a similar experimental logic, we have shown that task-irrelevant neural activity varies dynamically during task switching, and that this activity correlates with behavioral performance. Our findings suggest further that task-set inertia affects PFC representations as well as those in occipitotemporal cortex. It should be noted, however, that our results do not rule out the hypothesis that switching costs may also partly reflect the time consumed by control processes (Rogers and Monsell, 1995; Rubinstein et al., 2001) in addition to effects of increased between-task competition. Nonetheless, the present results strongly support the hypothesis that interference from recently performed tasks is a critical contributor to these costs.

Many features of our data favor the task-set inertia account over the alternative interpretation that there are global differences in neural activity between switch and repeat conditions. First, bilateral extrastriate regions were strongly activated during task performance but showed no effect of task switching, suggesting that switching-related activity in nearby task-selective regions does not reflect a general increase in activity in this part of cortex. Second, if the effects of switching reflected global increases in task difficulty, one would expect similar effects to be apparent in the comparison between fast and slow RT trials. However, no such differences were observed. Finally, a comparison between switch and repeat conditions identified a specific set of cortical regions (including lateral and medial prefrontal and parietal cortices) that corresponded well with previous findings, not a global increase in neural activity. Together, our findings cannot easily be explained as reflecting global changes in activity during switching, but instead strongly suggest a specific effect of between-task competition caused by task-set inertia.

A striking aspect of the present findings was that performance measures correlated more closely with task-irrelevant activity than with activity in regions selective for the required task. The lack of correlation with task-relevant neural activity may initially seem surprising. However, computational simulations of our findings suggest that this lack of correlation may reflect the opposing effects of attentional control and between-task interference on task-relevant neural activity during switching (see supplemental material, available at www.jneurosci.org). Thus, for some subjects, high levels of task-relevant neural activity indicate generally high levels of attention to task-relevant information and correspondingly superior performance, resulting in a negative correlation between brain activity and switching costs. In other subjects, however, high levels of task-relevant activity indicate that processing is prolonged because of high levels of between-task interference, resulting in a positive correlation between neural activity and switching costs. Moreover, within a given subject, fluctuations across trials in attentional control and between-task interference will likewise have opposing effects that tend to reduce the correlation between behavior and task-relevant activity. It follows that there is no simple relationship between these measures. Instead, the primary measurable correlate of behavioral performance is the level of task-irrelevant activity.

A key issue for future research concerns the level of processing at which task selection is implemented, and at which effects of task-set inertia are observed. With the word and face stimuli used in the present study, control may be exerted at many levels of processing, from selection of low-level perceptual features (e.g., high spatial frequency information in words; lower frequency information in faces) to higher-level semantic features (e.g., concepts of number versus gender). The neuroimaging methods developed in the present study might readily be extended in future research; for example, to investigate whether principles of task competition and selection are observed during the response selection process.

Mechanisms of cognitive control

Our second goal was to investigate the organization of regions involved in cognitive control. Consistent with previous findings, areas within PFC, ACC, pre-SMA, precuneus, and parietal cortex showed increased activity during switching (Dove et al., 2000; Sohn et al., 2000; Brass and von Cramon, 2002; Dreher et al., 2002; Rushworth et al., 2002; Braver et al., 2003; Serences et al., 2004). Extending these findings, we observed dissociations between regions showing task selectivity and regions activated regardless of the task performed. Specifically, we found dissociations within PFC [between a region in anterior right MFG that was not task selective and more posterior and inferior regions showing selectivity for faces (right IFS) and words (left IFG)], and within parietal cortex, between a task-insensitive medial region and task-selective lateral regions.

According to guided activation theory, PFC maintains task-relevant information about goals or task rules, and uses this information to guide processing in posterior cortical regions (Miller and Cohen, 2001). On this account, PFC performs integrated functions of working memory (maintaining task-relevant information) and cognitive control (using this information to guide cognitive processing). This hypothesis makes specific predictions about dynamics of neural activity during task preparation and execution. Specifically, task-selective PFC representations should show sustained increases in activity during preparation (reflecting the working memory function of PFC), and this preparatory activity should be predictive of subsequent performance and associated task-relevant activity in posterior cortex (reflecting the cognitive control function of PFC). We have begun to test these predictions using extensions of the present methods (N. Yeung, L. E. Nystrom, S. Polyn, J. A. Aronson, and J. D. Cohen, unpublished observations). Relevant to this issue, it has recently been shown that task-selective activity is evident during preparation for a task switch in PFC (Sakai and Passingham, 2003) and occipitotemporal cortex (Wylie et al., 2006). An important question for future research is whether this preparatory activity is predictive of overt behavior, and is correlated among task-selective regions, as is predicted by the guided activation hypothesis.

In addition to task-selective regions, we identified many regions showing increased activity during switching regardless of the task being performed. Activity in two of these regions, ACC/pre-SMA and posterior cingulate, correlated positively with behavioral switching costs. Additional research is required to determine whether this activity is a cause of the switching cost (i.e., the activity reflects time-consuming control processes that contribute to the cost) or, conversely, is caused by the switching cost (i.e., the activity reflects monitoring or resolution of between-task interference introduced by switching). Moreover, it remains possible that functional subdivisions will become evident within identified “switch-specific” regions in future experiments. For example, the activity we observed in right MFG could in principle reflect task-specific operations that happen to be common to the specific tasks we used. Relevant to this issue, Wylie et al. (2006) have recently replicated our observation of right MFG activation during switching using quite different tasks (color and motion judgments), supporting our interpretation that this region performs domain-general functions. As in the present study, Wylie et al. used tasks that activated distinguishable regions in occipitotemporal cortex, raising the intriguing question of whether right MFG plays a specific role in cognitive control when competing tasks are associated with distinct neural correlates. The observation of similar activity in studies using task pairs not designed in this way (Brass and von Cramon, 2002; Dreher et al., 2002) speaks against this possibility, but the question awaits systematic exploration.

In conclusion, the present findings provide important evidence of functional subdivisions within the network of regions involved in cognitive control. The observation of task-selective PFC regions is consistent with the proposal that this structure plays a key role in maintaining task-relevant information (rules, goals, etc.) to guide processing elsewhere in the brain (Miller and Cohen, 2001). At the same time, the observation of a more anterior area in dorsolateral PFC that activated during switching regardless of task is consistent with previous findings implicating anterior PFC regions in the higher-level coordination of behavior (Koechlin et al., 1999; Christoff and Gabrieli, 2000; Braver et al., 2003; Buckner, 2003). These combined findings suggest that control hierarchies exist within PFC, with anterior regions responsible for regulating the operation of task-specific representations in more posterior regions (O’Reilly et al., 2002; Koechlin et al., 2003).