Shifting set about task switching: Behavioral and neural evidence for distinct forms of cognitive flexibility

Abstract

Task switching is an important aspect of cognitive control and understanding its underlying mechanisms is the focus of considerable research. In this paper, we contrast two different kinds of task switching paradigms and provide evidence that different cognitive mechanisms underlie switching behavior depending on whether the switch is between sets of rules (rule switch) or sets of features presented simultaneously (perceptual switch). In two experiments, we demonstrate that behavioral effects (Experiment 1) and neural recruitment (Experiment 2) vary with the type of switch performed. While perceptual switch costs occurred when the alternative feature set was physically present, rule switch costs were observed even in their absence. Rule switching was also characterized by larger target repetition effects and by greater engagement of the dorsolateral prefrontal cortex. In contrast, perceptual switching was associated with greater recruitment of the parietal cortex. These results provide strong evidence for multiple forms of switching and suggest the limitations of generalizing results across shift types.

The ability to switch tasks rapidly is important for responding flexibly in a changing environment (Miller & Cohen, 2001). Both behavioral and neuroscientific investigators have sought to understand how people disengage from one task in order to undertake another (see Monsell, 2003, for a review). In this set of experiments, we seek to understand the behavioral and neural consequences associated with different forms of task switching. Specifically, we examine whether there are qualitative differences in the cognitive mechanisms used to switch between tasks that require the reorienting of visuospatial attention (e.g., to what or where should I attend?) and switches that involve the ability to deal with changing goal-related information such as rules (e.g., what should I do?). We propose that the “task” used to investigate task switching (or “set” in set shifting) has important implications for understanding the nature of the cognitive process necessary to switch behavior. In this paper, we provide behavioral and neural evidence that a switch of one type of task does not necessarily reflect the same cognitive process as another type. Critically, these processes have been confounded in most studies of task switching.

Many studies of task switching conflate shifts of visuospatial attention (perceptual switching) and contextual rules (rule switching). For example, one paradigm requires participants identify the color or the shape of an object and then apply the correct stimulus-response rule for that particular color or shape (Hayes, et al., 1998). Given that both colors and shapes are presented simultaneously, task switching requires participants to move visuospatial attention away from one set of features in order to selectively attend to the alternative feature set. In addition, participants are required to implement the appropriate set of response rules which were arbitrarily assigned to particular colors and shapes (e.g., left keypress for red or circle, right keypress for blue or triangle). Thus, we would argue that a shift in the target feature also entailed a shift in task-relevant information; that is, the rules for responding to one particular feature became irrelevant and participants had to instantiate the correct rules based on the current context. Our goal was to isolate the effects of each type of switch in order to determine whether these cognitive processes are dissociable.

1.1 Terminology

The ability to change behavioral goals has been referred to by multiple terms including task switching, set shifting, and attention switching. The variety of terms used to refer to this ability suggests that it may incorporate several component processes, but the way in which these terms differ has never been explicitly stated. The term “set shifting” is typically used when there is a change in attentional set, and a “set” is defined as the property of the stimulus that is relevant in a given trial (Rushworth, et al., 2005). For example, in order to report the color or shape of a stimulus, the appropriate feature set must be instantiated. In the motor domain, a set may include the appropriate stimulus-response mappings for a given trial.

A “task” has not been formally defined (Rogers & Monsell, 1995), but may be the equivalent of a change in the goal state (e.g., classify a digit as odd/even or greater/less than 5)(Brass, et al., 2004). A task switch may also include a change in the set if goal achievement depends on different characteristics of the stimuli. However, goal states exist at many levels of abstraction and pinpointing the level at which a goal state constitutes a task is difficult. In the above example of switching between color and shape classifications, one might claim that the task repeats but the set shifts (e.g., color to shape) because in all trials the task is to report the property of the stimulus. On the other hand, one could claim that a task switch is also occurring because the goal of the trial changes (ignore color, attend shape or attend shape, ignore color).

In some instances, the same experimental paradigm has been referred to as both set shifting and task switching. For example, switching between classifying the location of a stimulus as either horizontal or vertical is referred to by some as set shifting (Smith, et al. 2004; Rubia, et al., 2006) and by some as task switching (Brass, et al., 2003; Ruge, et al., 2005). We use the most broad term “task switching” in this paper and discuss studies that examine the behavioral or neural consequences of a change in the goal or the relevant stimulus property regardless of whether it is referred to as a “set shift, “task switch”, or “attention switch”. Our aim is not to determine the differences between these umbrella terms, but to specify the particular cognitive processes needed for switching as this may be more helpful in understanding how task switching or set shifting is performed.

1.2 Dissociable forms of switching – behavioral studies

Only a few behavioral studies have examined whether there are qualitative differences in performance between types of task switches (Allport, Styles, & Hsieh, 1994; Meiran & Marciano, 2002). Allport, Styles, & Hsieh (1994) examined shifts of stimulus dimensions, semantic categories, cognitive operations, and response modes. All types of switches were associated with similar performance costs and there was no interaction of task with other factors such as decision difficulty. Note, however, that the comparison used in these experiments was between pure blocks of a repeated task and blocks of task alternation (between-block design). As such, this study assesses “mixing cost” – a combination of task switching processes and additional working memory processes incurred with keeping both sets active in alternating trials (see Rogers & Monsell, 1995 for a discussion of this issue). Thus, it is difficult to conclude from this study whether shifting performance per se was affected by different forms of task switching.

Meiran and Marciano (2002) assessed the effects of advanced preparation on rule, response, or dimension shifting. According to our classification scheme, we would consider both rule and response switches to be types of rule switching as they involve switches in either the judgment or response rules. In contrast, their dimension switching condition is similar to our idea of perceptual switching and involves reorienting visuospatial stimuli without a switch in the rule set. Interestingly, rule and response shift costs (RT for shift trials – RT for repeat trials) were affected by advance preparation similarly (i.e., both decreased with greater amounts of advanced preparation), and were dissociated from dimensional shifting (i.e, unaffected by advance preparation). While these results support our partitioning of task switching into perceptual and rule shifts, the authors note that differences in advance preparation between conditions are due to the particular paradigm used (i.e., same-different discrimination rather than identification). In other studies, preparation does reduce shift cost associated with perceptual switching (Ravizza & Ciranni, 2002). However, the study by Meiran and Marciano (2002) does demonstrate that perceptual and rule-based shifts can be differentially affected by experimental demands.

If dissociable cognitive processes (i.e, visuospatial attention and rule-based processing) underlie each type of switch, different qualitative effects should be observed depending on whether the experimental manipulation affects those fundamental processes. As we are claiming that perceptual switching involves reorienting visuospatial attention, manipulations that increase the amount of stimulus interference should make perceptual switching more difficult. In contrast, reconfiguring a new set of task rules may be difficult even in the absence of the competing response set as it takes time to retrieve and instantiate the appropriate information from long term memory (Mayr & Kliegl, 2000). Therefore, we predict that the presence of stimulus interference is not necessary in order for a shift cost to be observed in the rule switching condition. In contrast, we claim perceptual switching is defined by the need to switch visuospatial attention from externally-based features or locations, and will only be observed when the competing stimulus set is present.

Behavioral effects that are modulated by rule-based processing should be observed in the rule-switching condition, but not the perceptual. Previous research has shown that, if rule information is afforded by a stimulus, a priming effect will occur when that stimulus repeats (Biederman, 1972; Allport, Styles, & Hsieh, 1994; Meiran, 1996). In the rule switching condition, the target identity signals the correct S-R rule and we predict that an effect of target repetitions will occur. This is not true in the perceptual condition where a change in target identity does not require a change in task rules; that is, participants use the same rule of responding regardless of the particular target they see. Thus, target repetitions should have less of an effect in the perceptual switching condition.

1.3 Dissociable forms of switching – imaging studies

Other studies have assessed whether different types of task switches are associated with activity in separable neural regions. Rushworth and colleagues (2001, 2002) have examined the neural correlates of switches of perceptual attention or visuomotor shifts. In studies focused on either the medial frontal cortex (Rushworth, et al, 2001) or parietal cortex (Rushworth, et al., 2002), separable subregions were associated with different types of switching (whole-brain analyses were not reported). However, Wager et al., found only weak dissociations in neural activity when comparing different types of switches (i.e., object and feature switches); that is, a few regions were more active when switching between objects than features of objects, but no regions were found showing the opposite pattern of activity. Instead, these investigators found common regions to be associated with behavioral switching performance for both features and objects

In addition to these studies, others have found modulation of neural activity in regions affected by a switch (Wylie, Javitt, & Foxe, 2006; Yeung, et al., 2006). For example, when switching between the categorization of faces or words, activity of the fusiform face area could be enhanced or suppressed depending on the task set (Yeung, et al., 2006). However, the goal of these studies is very different from ours in that we are examining activity in regions enabling different kinds of shifts rather than activity that is a consequence of a task switch. In the above example, the fusiform gyrus is active when the current task set involves faces regardless of whether a task switch occurred. In contrast, in the present study we are interested in identifying brain regions that are sensitive to task switching depending on whether the shift is rule-based or perceptual

If different forms of task switching exist, then these mechanisms should map on to separable neural regions. In a second experiment, we employ fMRI to assess whether the pattern of neural recruitment varies depending on the type of switch performed. Rather than focusing on particular subregions (Rushworth, et al., 2001, 2002), we examined regions that were differentially modulated by the interaction of shifting and type of task (rule or perceptual). To preview, we observed greater activity in the dorsolateral prefrontal cortex (DLPFC; note that we define this as Brodmann areas 9 and 46) for rule shifting that perceptual shifting whereas the superior parietal cortex exhibited the opposite pattern. Moreover, we determined whether any region was important for both types of shifting. If so, this would suggest that there may also be a common set of processes that are engaged whenever a task switch is required.

1.4 General Design

An odd-man-out (OMO) design was used to assess differences in task switching between sets of letters and shapes (Figure 1). Participants were asked to find the letter or shape that did not match the other letters or shapes, and a switch trial occurred when the OMO shifted from letter to shape or vice versa. In a baseline condition, only one dimension was present (letters or shapes) and the response rule was intuitive (i.e, S-R mapping was spatially congruent to target location). To invoke perceptual switching, we presented both letters and shapes simultaneously thereby requiring participants to reorient their attention from one set of features to the other set. To invoke rule switching, an arbitrary set of response rules was implemented so that a switch trial involved retrieving and applying the appropriate set of stimulus-response mappings. Thus, the perceptual switching differed from the baseline only in the amount of stimulus interference that was present whereas the rule switching condition differed from the baseline only in the response rule.

Figure 1.

Example of a repeat and switch trial in the baseline condition (left), and examples of the stimulus and response parameters of the rule and perceptual conditions (right). Boxes underneath the stimuli represent the correct keypress.

Note that in the perceptual switching condition, rule information did not switch from trial-to-trial. Responses were always made to the location of the OMO regardless of whether the OMO was a letter or shape. Similarly, switching between sets of locations or features was minimal in the rule switching condition. Stimuli from the alternative task set were absent and so there was no need to reorient spatial attention. While the focus of attention must move to find the OMO on every trial, this demand is equivalent in both shift and repeat trials.

We have tried to match the perceptual and rule switching conditions in every regard except for the type of switching that is required. However, one could argue that the rule condition not only requires rule shifting but the ability to overcome spatial incongruity. For example, participants may want to respond to the location of the OMO rather than to an arbitrary stimulus-response mapping. Given that we are assessing dissociations in difference scores (shift cost or repetition priming) in each condition, this should not affect our interpretation of the results. Response incongruency effects will be present on both repeat and switch trials in the rule condition and so will not affect the switch cost. Indeed, previous studies have found congruency effects and switch costs to be orthogonal (see Meiran, 2000, for a review).

This same argument can be made for concerns that the stimuli are much more complex in the perceptual switching condition than the rule switching condition. Finding the OMO is much more difficult in the perceptual condition, however, it is equally difficult in both repeat and shift trials. While we cannot compare absolute values between the perceptual and rule switching conditions, we can compare variables calculated from differences between repeat and shift trials..

2. Experiment 1

2. 1 Methods

Subjects

Ten undergraduate students (average age = 18.9) from the University of California, Davis participated in this study for subject pool credit. All had normal or corrected-to-normal vision. All participants gave informed consent to the procedures approved by the Institutional Review Board of UC Davis.

Stimuli

Four letters (i.e., b, i, n, v) and four shapes (i.e., cross, hexagon, parallelogram, triangle) comprised the stimulus set in the baseline and perceptual-switching condition. A different set of letters and shapes was used in the rule switching task (i.e., o, s, t, x, circle, diamond, pentagon, square).

Procedure

For all conditions, the task was to determine which stimulus did not match the other stimuli (Figure 1). Three of the stimuli were identical and one did not match. Stimuli were presented until the participant responded and were followed by a 500 ms interval before the next trial began. In all conditions, keys forming a row on a computer keyboard were used to collect responses. The probability of an OMO switching from one feature set to the other was 0.5.

In the perceptual-switching condition, both letters and shapes were presented, and a switch required participants to shift visuospatial attention from one set of features to the other. If the OMO was a letter, then all the shapes were different and vice versa when the OMO was a shape. Participants responded in this condition by pressing the button that corresponded to the spatial location of the OMO.

In the rule-switching condition, only one feature set was present. Once participants identified the OMO, they were required to press a button on the computer keyboard that had been previously memorized for that letter or shape. The index through little fingers of the right hand were mapped to the letters “s”, “x”, “t”, “o” and circle, diamond, pentagon, square, respectively. In this condition, a switch required participants to retrieve and implement the set of stimulus-response rules that was appropriate for the alternative feature set.

The baseline task was similar to the perceptual-switching condition in that responses were spatially congruent to the stimulus location. It was also similar to the rule-switching condition in that only one set of features was present in the display.

Participants completed one block of each condition and one of two block sequences was chosen at random (i.e, baseline, perceptual, rule or rule, baseline, perceptual). The baseline block was composed of 128 trials and the rule - and perceptual-switching conditions contained 256 trials which ensured that the each combination of OMO dimension (2), OMO stimulus (4), OMO location (4), and distractor stimulus (4) was presented at least once.

Immediately preceding the rule-switching condition, participants went through a practice session where they memorized the stimulus-response mappings. In these practice blocks, one stimulus, either a letter or a shape, was presented and participants had to produce the correct response mapping for that stimulus. Participants were required to achieve at least 90% correct before they could perform the rule-switching condition. Participants took between 48 − 128 trials to memorize the key mappings.

2.2 Results

Task switching effects

Shift costs (switch RT – repeat RT) for the baseline, perceptual-, and rule-switching conditions were 6 ms, 383 ms, and 162 ms., respectively (see Table 1). Note that shift cost was negligible in the baseline condition, and a 3 (baseline, perceptual, rule) × 2 (switch/repeat) repeated-measures ANOVA produced a significant interaction term (F(2,18) = 66.71, p < .05). A t-test confirmed that switch and repeat trials were equivalent in the baseline condition (t(9) = .34, p = .745) whereas switch trials were slower than repeat trials in the perceptual (t(9) = 13.06, p < .05) and rule (t(9) = 5.84, p < .05) conditions.

Table 1.

Reaction times for shifts of task or target stimulus (repeat trials only). Standard errors of the mean are in parentheses.

	Baseline		Rule		Perceptual
	repeat	shift	repeat	shift	repeat	shift
Task	859.51 (47.92)	865.39 (49.24)	1379.05 (51.56)	1541.36 (56.73)	1254.82 (41.96)	1637.73 (51.04)
Stimulus	861.22 (53.2)	859.45 (47.21)	1218.05 (70.74)	1425.73 (47.98)	1224.29 (52.45)	1266.82 (41.64)

Shift cost was greater in the perceptual switching condition than the rule switching condition. In a direct comparison of these two conditions, a significant interaction of shift × condition was produced (F(1,9) = 60.1, p < .05).

To ensure that shift cost was not entirely due to a priming effect of stimulus repetitions, we ran the same analyses discarding repeat trials where the stimulus also repeated. The analyses produced exactly the same results with significant shift costs for the perceptual (t(9) = 11.00, p < .05) and rule-switching conditions (t(9) = 5.84, p < .05), and no shift cost in the baseline condition (t(9) = .69, p = .511).

Accuracy was at ceiling for all conditions (>96%) and was not analyzed further.

Stimulus repetition effects

The same analysis was used to analyze RT in “repeat” trials when the target repeated or changed (Table 1). A significant interaction effect was produced (F(2,18) = 10.83, p < .05) and t-tests confirmed that stimulus repetitions affected performance in the rule-switching task (t(9) = 4.43, p < .05), but not the baseline (t(9) = .08, p = .941) or perceptual-switching task (t(9) = 1.13, p = .287).

2.3 Discussion

As predicted, manipulations of visuospatial and rule processing had different effects on switching difficulty depending on the type of switch. When rule information remained relatively static, a shift cost was only evoked if the competing feature set was present. In contrast, a shift cost was observed in the rule-switching condition even when stimulus interference from the other feature set was absent.

The effects of target repetitions were also different in the two switching conditions. Stimulus repetitions were not advantageous in the case of perceptual switching, but had an impact in the rule-switching condition. Target repetitions were advantageous in this condition because targets are tied to a particular S-R rule whereas this is not the case for targets in the perceptual condition.

Shifting difficulty was greater in the perceptual switching condition than in the rule switching condition. We are not contending, however, that perceptual switching is inherently more difficult than rule switching. For example, shift cost in the perceptual condition will increase or decrease depending on the amount of interference from the alternative feature set (Ravizza, 2007) and may, at some point, show equivalent costs compared to the rule switching condition. Our research goal is to provide evidence that separable cognitive processes underlie these two kinds of task switches rather than proving that one process is more effortful than the other.

Detecting targets that switch from letters to shapes does not, in and of itself, produce a decrement in speed (baseline condition). Shift costs are evoked only if perceptual interference is added to the display or if sets of rules change from trial-to-trial. By manipulating these factors separately, we were better able to isolate the source of the shift cost.

3. Experiment 2

A strong test of whether perceptual and rule switching reflect separate cognitive processes would be to observe if dissociable neural regions underlie shifting performance. We tested this idea using a modified version of our previous experiment in the fMRI environment. Moreover, we examined whether there were common regions recruited for both types of shift.

3.1 Methods

Subjects

Twenty participants were recruited for this experiment and paid for their participation. Six participants were excluded for either a) excessive movement (n=2), b) failure to complete the experiment due to claustrophobia/discomfort in the scanner (n = 3), or c) lack of recorded behavioral data (n=1). Thus, we were able to acquire complete data sets from fourteen participants (female = 7, male = 7). The average age of these participants was 27.14 years and all were right-handed with normal or corrected-to-normal vision. All participants gave informed consent to the procedures approved by the Institutional Review Board of UC Davis.

Stimuli

Stimuli were the same as those reported in Experiment 1.

Procedure

Stimulus presentations were paired so that there was a fast inter-target interval (ITI) between pairs and a longer ITI between trials (Figure 2). This was done to ensure a robust shift cost for the second set of stimuli in each pair, and is similar to the more rapid ITI used in Experiment 1 and other behavioral studies of task switching. The long ITI between trials allowed the hemodynamic response to the second set of stimuli to return to baseline. Only data from the 2^nd presentation was analyzed. The first set of stimuli was presented for 2 seconds followed by a fixation cross for 500 ms. The next set of stimuli was also presented for 2 seconds, but was followed by 13.5 seconds of rest.

Figure 2.

An example of the display and timing parameters used in Experiment 2.

Each run was composed of one block of 12 stimulus pairs for both the rule and perceptual conditions. The order of switching conditions was chosen at random and participants were cued at the beginning of each block of 12 paired trials as to the condition that they were about to perform. Participants completed between 7−9 runs. Behavioral data were acquired using the E-prime software package interfaced with a response box and a color LCD projector. Before being scanned, participants went through a practice session of 64 trials where they memorized the stimulus-response mappings for the contextual condition.

fMRI acquisition

Data were acquired on a 3.0 T Siemens Trio magnetic resonance imaging scanner with an 8 channel phased array for brain imaging located in the UC Davis Imaging Research Center. We used an echo planar imaging protocol to acquire 36, 3.4 mm axial slices every two seconds (TE 25 ms, 90 degree flip angle, 22 cm FOV, 3.4375 mm × 3.4375 mm resolution). Thus, 216 scans were acquired during each run. T2-weighted in-plane scans were acquired in each scanning session as well using the same slice thickness as the functional scans but with an inplane resolution of .859 mm × .859 mm.

Motion correction was undertaken using an automated image recognition (AIR) program which implements two iterations of a six-parameter, rigid body realignment, to co-register each image with the first functional image acquired in a session. Data from participants moving more than an average of 3 mm in any direction or rotating their heads more than 2 degrees across the session were excluded from the experiment. Images were spatially smoothed with an 8mm FWHM Gaussian filter and an additive baseline correction was applied to each voxel-wise time course independently. Intensities in each voxel were detrended with a simple linear regression to remove intensity changes due to scanner drift. Structural and functional images for each participant were cross-registered to the MNI common reference brain and the Talairach atlas system using AFNI.

fMRI data analysis

Statistical analyses were implemented using the NIS software suite (http://kraepelin.wpic.pitt.edu/nis). Only data acquired in the second set of each trial were analyzed, and error trials were discarded. Moreover, we excluded trials where the first set of stimuli in each pair switched from the previous trial. This was done because, despite the long ITI, shift costs were still present for the first set of stimuli. This resulted in 25−55 trials per person in each condition after excluding error trials and trials in which the first of the pair was a shift trial (50% probability). Mean intensities of each voxel in our experimental conditions were averaged across participants. Regions of interest were obtained by running a 2 (shift/repeat) × 2 (perceptual/rule) × 7 (timepoint; scans 2−8) ANOVA. Images were threshold using a p-value of .001 (uncorrected) and a contiguity threshold of 16 voxels. Monte-Carlo based simulations (AFNI AlphaSim) confirmed that our threshold produced a low probability of family-wise error (p-value of <.05). Post-hoc tests using a threshold of p< .05 were performed for scans corresponding to 4−6 seconds (timepoints 3−4) after the onset of the second trial in each pair to unpack significant interaction effects in regions produced by our voxel-wise ANOVAS.

One advantage of analyzing the imaging data with an ANOVA is that we do not have to assume the shape of the hemodynamic response function (HRF) (Bandettini & Cox, 2000). Instead, any region that displays a significant 3-way interaction in our voxel-wise analysis (condition × shift × time) will be produced. This is an important consideration given that it is unclear why some regions such as the dorsolateral prefrontal cortex (DLPFC) are not consistently reported across imaging studies of task switching (Wager, et al., 2004). Using this analysis, we are better able to detect activity in regions that may not conform to the standard HRF. Since any region showing a change in activation or deactivation at any timepoint will be produced by this analysis, we will focus on those regions that conform to our a priori predictions. First, a region must show greater activity in shift than repeat trials for one switching condition but not the other. Second, this difference must occur 4 − 6 seconds after the onset of the trial. Third, activity in the region must be above baseline (time 2).

3.2 Results

Behavioral

Before we examined performance on our trial of interest, behavioral data from the first trial of each pair was analyzed to determine whether a shift cost was present despite the long inter-target interval. Given that a small but reliable shift cost was present in these trials (shift cost: rule = 22 ms., perceptual = 92 ms; main effect of shift: F (1,13) = 16. 47, p < .001), the following behavioral and fMRI analyses will exclude trials in which the first trial of each pair was a shift trial.

Reaction time data were subjected to a 2 (condition) × 2 (shift) repeated-measures ANOVA (Figure 3). A main effect of shift (F (1, 13) = 45.97, p < .001) was obtained as well as the interaction of condition × shift (F (1, 13) = 30.96, p < .001). Simple contrasts ensured that a shift cost was present in both the rule (t (13) = 3.03, p = .01) and perceptual (t (13) = 7.32, p < .001) conditions, however, the magnitude of the shift cost was much greater in the perceptual condition.

Figure 3.

Reaction time and accuracy data in Experiment 2.

fMRI

Activity in seven regions displayed a significant 3-way interaction in our voxel-wise analysis (condition × shift × time), but only three of these ROIs displayed selectivity (i.e., significant differences in shift and repeat trials for one of the conditions, but not the other) for shifts depending on the task (Figure 4). A region in the left DLPFC (−53, 41, 19) was more active in rule shifts than repetitions (time 3: t (13) = 2.86, p < .05; time 4: ns), but was not sensitive to shifts in the perceptual condition (time 3: t (13) = 1.4, p > .1; time 4: n.s.). Moreover, shifts of rule information were associated with greater activity in the DLPFC than perceptual shifts (time 3: t (13) = 2.46, p < .05; time 4: n.s.).

Figure 4.

The region in the DLPFC that displayed selectivity for rule shifts with F-values being greater for more yellow than orange colors. This region was most active in the rule shifting condition (timecourses are shown from the onset of the 2^nd trial of each pair) and good shifters had greater activity in this area than poor shifters.

In contrast to the above findings associated with rule shifting, a region in the right superior parietal cortex (11, −74, 54) was more active when shifts occurred than repetitions in the perceptual condition (time 3: t (13) = 3.17, p < .01, time 4: t (13) =2.29, p < .05), but showed no difference in activity in the rule condition (time 3: t (13) = 1.57, p > .1, time 4: t (13) =.12, p > .1)(Figure 5). This region also displayed greater activity for perceptual shifts than rule shifts (time 3: t (13) = 3.88, p < .005, time 4: t (13) =3.45, p < .005).

Figure 5.

The region in the right superior parietal lobe that displayed greater activity for perceptual shifts with F-values being greater for more yellow than orange colors. This region was most active in the perceptual shifting condition (timecourses are shown from the onset of the 2^nd trial of each pair), but good shifters and poor shifters had equivalent levels of activity in this region.

A region in the right premotor cortex (47, −1, 29) showed the same pattern of activity (see Supplementary Figure 1) as our region of interest in the parietal lobe, however, activity in this region was not correlated with shift cost and will not be discussed further.

To determine the relationship between activity in the DLPFC and performance, we compared DLPFC activity between those with the highest and lowest rule shift costs (see Figure 4 and Supplementary Figure 2). We analyzed the data collapsed across time points 3 and 4 to capture the peak of the BOLD response during the trial using the DLPFC ROI obtained in the voxelwise ANOVA analysis. Groups were formed by dividing the sample in half based on shift cost in the rule condition. Good rule shifters had more activity in the DLPFC in shift than repeat trials than poor shifters (t(12) = 3.68, p < .005). If participants were divided into good and poor shifters based on perceptual shift cost, no differences in the DLPFC (t (12) = 1.65, p > .1) were observed. A further correlational analysis showed a strong, but not reliable, trend for a negative association between the rule shift cost (r = −.497, p = .071) and activity in the DLPFC. These analyses suggest that better performance on the rule shifting task is associated with greater shift-related activation in the DLPFC. There was no significant relationship between perceptual shift cost and activity in this DLPFC region of interest (r = −.147, p > .1).

Equivalent levels of activity were observed for good and poor perceptual shifters in the parietal cortex (t(12) = 1.24, p >.1) (see Figure 5 and Supplementary Figure 3). Activity in the superior parietal cortex also tended to be associated with perceptual shift costs (r = .509, p = .063), but in the opposite direction than the DLPFC. This region showed no significant correlation with shift cost in the rule condition (r = −.123, p > .1).

To see whether activity in any region was important for task switching in general, we examined the pattern of performance in the regions produced by the interaction of shift × time in the voxel-wise ANOVA. Regions that were more active in shift than repeat trials are listed in Table 2. None of these regions displayed significant activity in both the perceptual and rule switching conditions. Instead, activity in these regions was primarily driven either by a significant difference in shifts of perceptual attention, rule information, or neither contrast was significant.

Table 2.

Regions produced by the shift × time interaction (R=rule, P = perceptual, s=shift, r=repeat)

Region	x	y	z	Rs>Rr	Ps>Pr
Left occipital (BA19)	−29	−86	25		*
Left cerebellum	−48	−63	−18		*
Vermis	3	−64	32		*
Left DLPFC (BA 9/46)	−50	34	22	*
Right cerebellum	23	−72	−24	*
Left putamen	−25	11	−6	*
Left motor (BA 4)	−44	−16	49
Right premotor (BA 6)	25	−12	65		p=.063
Left parietal (BA 7/40)	−25	−60	46		p = .07
Right occipital (BA18/19)	36	−80	−1		p = .06
SMA/Ant. Cing	6	−12	51	p=.098

3.3 Discussion

Task switching is not a top-down process that is indifferent to the type of representations on which it is working. When attention must be switched between features in the perceptual domain, the right superior parietal cortex is more engaged than when perceptual interference is low as in the rule condition. In contrast, a region in the dorsolateral prefrontal cortex did not differentiate between repeat and shift trials in the perceptual condition where task-relevant information changed very little from one trial to the next. Instead, this region was more actively engaged when the relevance of a set of stimulus-response rules switched from trial to trial. Further, we did not find any region that was active in both types of shifts.

Activity in the parietal and prefrontal cortices was associated with individual differences albeit in different ways. Consistent with its role as a putative control structure, activity in the DLPFC showed a strong relationship to successful performance, suggesting that contributions of this region were important to changes in rule information. The superior parietal cortex, while being modulated by shifting requirements, was not clearly associated with greater shifting abilities and, in fact, the correlation was in the opposite direction. One possible explanation for this result may be that activity of the superior parietal cortex reflects greater stimulus interference rather than attempts to reduce stimulus interference. Parietal cortex activity is higher in shift than repeat trials because it is harder to ignore stimuli that were previously the focus of attention. In the same way, participants who have trouble filtering out stimulus interference also show greater activity in this area than those who are efficient in filtering out such information in the environment. In other words, greater activity in this region may reflect not only greater attention to the relevant dimension but also greater response to the irrelevant dimension. While this explanation is speculative at this point, it is important to note that shifts of perception and rules are associated with performance in two very distinct ways, and supports the notion that these types of switches engage different cognitive mechanisms.

Shift cost was higher in the perceptual than rule condition in both Experiments 1 and 2. Although, we are not claiming that perceptual switching is inherently more difficult than rule switching, this difference in shifting difficulty weakens our claims of a true double dissociation in neural engagement. For example, the superior parietal cortex may not be selective for perceptual shifts of attention but may just show greater activity in more difficult shifting situations. Interestingly, activity of the DLPFC often tracks with general task difficulty, however, in our study, this region was engaged more in the least difficult switching condition. Thus, while we have good evidence that the DLPFC is selectively engaged by rule shifts, more evidence is needed for selective recruitment of the superior parietal cortex in perceptual shifts.

4. General Discussion

The goal of these experiments is to further our understanding of the mechanisms underlying task switching and, specifically, to ascertain whether distinct cognitive processes are engaged depending on the particular task being performed. Our results indicate that distinct forms of task switching are likely to exist and provide some insights into the nature of the cognitive process underlying them. A perceptual switch cost was induced simply by presenting the alternative feature set thereby requiring, in the case of a switch, that attention be reoriented from one set of locations and features to another. Consistent with the strong association between parietal cortex and selective attention (Corbetta & Shulman, 2002, Kanwisher & Wojciulik, 2000), the right superior parietal cortex was engaged when participants were required to perform a perceptual switch.

In contrast, a rule switch cost was present even when the alternative feature set was absent. Switching between sets of response rules requires the ability to retrieve the correct rule set and load it into working memory. One might argue that the crucial difference between the rule and perceptual conditions is that the former requires working memory while the latter does not. Often, rule-guided behavior relies on updating the contents of working memory and as such may represent a specific component of working memory (Kerns, et al., 2004). According to this view selection and maintenance of context-appropriate responses are two reflections of the same underlying neural mechanism. In the present study rule switching was marked by greater activity of the DLPFC, a region strongly linked to rule-guided behavior in other studies and to the maintenance of context for responding across a range of higher cognitive tasks (see Miller & Cohen, 2001, for a review).

While our findings are consistent with functional hypotheses for the DLPFC and superior parietal cortex proposed by others, it is possible that a region in a different part of the parietal cortex is also modulated by rule shifts, but did not reach threshold in our study. Thus, we are not trying to make strong claims that the DLPFC is only involved in rule switches or superior parietal cortex in perceptual switches. Our main claim is that we do not see one neural region that is engaged by both perceptual and rule shifts. As this claim rests on a null result, however, we must interpret this result with due caution.

We believe that the present results may bear on a current debate in the literature concerning the nature of task-switching with some arguing that the time taken to switch tasks reflects an active process of engaging the current task set (Rogers & Monsell, 1995; Meiran, 1996) while others argue that it reflects the time taken for the previous task set to decay (Altmann, 2003; Ruge et al., 2005). One could argue that a minimum requirement for a putative control structure would be a strong relationship between functional activity and successful performance. If task switching is an active process, then “good” shifters should engage neural regions involved in task switching to a greater degree than “poor” shifters. Indeed, in the rule switching condition, shifting speed was related to activity in the DLPFC. However, in the perceptual condition, parietal activity was not modulated by individual differences in switching behavior. It may be that both ideas are correct, and that switching difficulty reflects different processes depending on the type of switching being performed. Rule switching may require top-down control with the shift cost reflecting the time taken for the DLPFC to guide and coordinate other regions that undertake the processing for a particular task. In contrast, the shift cost in the perceptual condition may reflect passive interference from competing stimuli and activity of the parietal cortex reflects this greater interference. In any case, our results do not clearly support perceptual switching as reflecting an executive process important for successful performance.

The results of our fMRI study are broadly consistent with other imaging studies of task switching. In an informal review of task switching studies, we noticed that prefrontal activity was primarily reported in studies where rules switched. Tables 3 and 4 summarize studies of task switching that performed whole-brain analyses and reported regions of interest in stereotactic coordinates. As reporting can vary from study to study, we located each peak coordinate in two ways. To determine whether activity was in the DLPFC, we classified Brodmann's area and frontal gyrus of peak activity according to the closest gray-matter region using the Talairach daemon (http://ric.uthscsa.edu/td_applet/). We classified peak coordinates as being in the DLPFC if it fell within BA 9 or 46, although these locations should be considered approximate. Note, as well, that the Talairach daemon also has a fair bit of imprecision and the reader should interpret BA labels as approximate. In addition to the DLPFC, a region in the inferior frontal junction (IFJ) is observed consistently in studies of task switching (Derrfuss, et al., 2005). We labeled regions as residing in the (IFJ) based on criteria detailed in a meta-analysis of this region in task switching studies (Derrfuss, et al., 2005). Our goal was not to provide an exhaustive meta-analysis of the imaging literature on task switching, but simply to determine whether there were any meaningful patterns that could be observed across studies in regard to PFC activity.

Table 3.

Areas of the frontal cortex reported in studies of viusospatial attention switching. Letters in parentheses refer to the type of switch performed according to Wager, et al., 2004. (L=Location, A = Attribute, T = task, R = rule, O=object). Note that we deviated from Wager by classifying a rule switch as any change in the rule in addition to reversals of S-R mappings.

				Frontal regions
				Left			Right
Study	Task (switch type)	Contrast	BA	x	y	z	x	y	z
Smith, 04	Up/down or left/right decisions (T)	switch-repeat	claustrum				32	14	9
			10	−28	47	17
Schmitz, 06	Up/down or left/right decisions (T)	switch-repeat	claustrum	−26	−14	16
Pollmann.,00a	detect motion or color OMO (A)	switch-repeat	10	−18	50	6
			insula	−39	19	0
Pollmann, 06	detect motion or color OMO (A)	switch-repeat	nothing
Weidner, 02	detect motion or color OMO (A)	switch-repeat	9(SFG)	−11	56	19	14	52	24
			32				4	34	29
Kimberg et al., 00	letter decisions t (C/V) or number decisions (odd/even) and key mapping (O,T,R)	switch-repeat (prep)	6(SMA)	−16	−10	51
		switch-repeat (target)	nothing
Sohn, et al., 00	letter decisions (C/V) or number decisions (odd/even) and key mapping (O,T,R)	cue × switch (prep)	45				53	27	6
		switch-repeat (target)	8				26	23	43
Rubia, 06	Up/down or left/right decisions (T)	switch-repeat	10				27	44	20
			white matter				44	−13	23
Serences, 04	Detect face or house targets (L, O)	shift-hold	6				19	−7	49
Wilkinson, 01	Detect targets on local or global level (A)	Difficult — easy switch	6	−52	2	38

Table 4.

Regions of DLPFC (BAs 9 & 46) and IFJ involved in switches of rule information. Letters in parentheses refer to the type of switch performed according to Wager, et al., 2004 (L=Location, A = Attribute, T = task, R = rule, O=object). Note that we deviated from Wager by classifying a rule switch as any change in the rule in addition to reversals of S-R mappings.

Context Switching				DLFPC and IFJ regions
				Left			Right
Study	Task (switch type)	Contrast	BA	x	y	z	x	y	z
Brass,04	Number decisions (odd/even/ or parity number) (T, R)	switch-repeat	IFJ	−37	5	32
			IFJ	−37	8	35
Braver,03	Decisions about objects being manmade or their size (T, R)	switch-repeat	9(IFG)	−46	15	21
Wylie, 04	Faces (color or famous) & line (motion or frequency) decisions (O, T, R)	Switch (3rd block) — switch(1st block) —grp1	9(MFG)				46	19	32
		same as above — grp2	9(MFG)	−54	11	36
Li, 04	remember # of shapes(i.e., ellipse) (R)	switch-repeat	9(MFG)	−46	17	27
Sylvester, 03	remember # of arrows (left or right) (R)	switch-repeat	46				45	26	22
			9(MFG)	−38	30	26
		High probability switch - low probability switch	IFJ	−41	9	27
Garavan, 00	remember # of shapes (big or small) (R)	switching frequency (High>Med>Low)	9(IFG)	−43	6	25
			IFJ				45	5	31
Cools, 04	Respond to previously chosen object or the opposite (O, R)	object + rule switch - repeat	nothing
Dove, 00	switch key mappings to shapes (R)	switch-repeat	IFJ	−44	5	37
			IFJ				40	8	36
Barber 04	Respond as indicated by stimulus	switch-repeat(cue)	nothing
	(e.g., “l”=left key) or opposite (R)	switch-repeat(target)	nothing
Pollman, 00b	switch key mappings to shape (R)	switch-repeat	9(MFG)				45	16	37
Crone, 05	switch key mappings to objects (R)	switch-repeat	9(MFG)	−32	36	38
Ruge 05	Up/down or left/right decisions and key mapping rules (T, R)	switch-repeat	IFJ	−41	8	34
		switch × cue time	IFJ	−41	6	29
			9(MFG)				40	24	26
Parris, 07	switch key mappings to colors (prep) (R)	switch - repeat		−42	20	21
			46	−38	30	20
Yeung, 06	Faces (gender)/Word (syllable) decisions (L,O,T,R)	switch-repeat	9(MFG)				29	43	34
Wylie, 06	Color (red/blue)/motion (quickly/slowly) decisions (A,T,R)	switch-repeat	9(SFG)				38	43	28
Konishi, 02	WCST (A, R)	switch × feedback	46	−38	38	14

MFG = Medial Frontal Gyrus, IFG = Inferior Frontal Gyrus, IFJ = Inferior frontal junction

Note that Ruge et al., used the same design as the spatial switching studies in Table 2 with the further addition of arbitrary response mapping (left/top = diagonal top left key; right/bottom = diagonal right bottom key). We consider this a type of S-R rule to remember

We have classified switch types into those where location, attributes, tasks, rules, or objects switches as in Wager et al.'s meta-analysis of task switching (see Wager, et al., 2004 for a description of these categories) Table 3 reports studies of perceptual shifting where rule information did not change from trial-to-trial. In general, shifts of visuospatial attention to one or another perceptual aspect of a display did not evoke activity of the DLPFC or IFJ above threshold (see Table 3), although these shifts did engage other frontal regions and the parietal cortex. In one study, participants responded whether a movement-defined or color-defined target was present or absent (Pollmann, 2000). Regions of the parietal cortex and inferior frontal cortex were more active when the target switched dimensions than when they repeated, but neither DLPFC nor IFJ activity was observed. The only study reporting activity in BA 9 was conducted by Weidner et al., (2002), and this activity was in a very medial part of the superior frontal gyrus.

In contrast, the DLPFC or IFJ is more consistently reported when task-relevant information changes from trial-to-trial (see Table 4). The types of rule information that may switch in these studies include stimulus-response rules (e.g., respond in the direction of the arrow or the opposite; respond to the location of the target or the opposite), the relevance of conceptual properties of objects (e.g., whether it is man-made or. larger than a breadbox), or which set of items in working memory should be updated (e.g., add one to either the circle shape counter or the triangle shape counter). While most studies listed in Table 4 report activity in lateral PFC, the location of peak activation in these studies varies widely. For example, regions in the inferior frontal gyrus, middle frontal gyrus, and the IFJ all seem to be associated with rule switching and it is not apparent from the table that there is a clear association between different kinds of rules and PFC regions. For the most part, the peak coordinate appears to reside primarily in the IFJ or left BA 9, either in the inferior or middle frontal gyrus rather than BA 46 (although see Konishi, et al., 2001, Sylvester, et al., 2003 and Parris, et al., 2007). In contrast, we found a more anterior region in BA 46 to be engaged by rule shifts. While it is unclear how our more anterior region is functionally different from the majority of more posterior regions reported in rule switching studies in Table 4, it is clearly distinct from the site found in the parietal cortex for perceptual switching.

One potential exception to the hypothesis that the lateral PFC is important for rule shifts involves imaging studies examining neural activity during a shifting task created by Rogers and Monsell (1995). In this task, letters and numbers are simultaneously presented (e.g., “G3”, “A4”) and participants must either say whether the letter is a consonant or vowel or a number is odd or even. On the surface, the switch required in this paradigm would appear to be rule-based because participants have to determine which task should be undertaken (consonant/vowel or odd/even). However, it could be argued that successful performance in this task relies more on visuospatial switching. Once participants in this study know where to attend – to the left or right of the compound stimulus – they know what to do because a letter cannot be classified as odd or even nor a number classified as a consonant or vowel. Accordingly, imaging experiments using this design have not reported significant DLPFC or IFJ activity (see Table 3). Thus, one reason for the lack of consistent activity in the DLPFC in neuroimaging studies of task switching (Wager, et al., 2004) may be due to the variety of shifting paradigms employed across studies; some entail switches of rule information whereas others do not involve this type of switching.

In these experiments, our goal was to isolate the effects of rule- and perceptually-based switches on performance and neural engagement. However, most studies have not specifically examined the cognitive processes entailed by their particular task switching paradigm. A good example in both the imaging and neuropsychological literatures is the Wisconsin Card Sorting Task which we would classify as both a rule (sorting rules can change from trial-to-trial) and a perceptual switch (number, color, and shape are features that are always present). Interestingly, while this task is thought to be a test of frontal dysfunction, those with posterior lesions can also be impaired on this task (Anderson, et al., 1991). A consideration of the underlying cognition for task switches may help to clarify some of the inconsistent finding in both the behavioral, neuropsychological, and neuroimaging research. Indeed, we were able to demonstrate the clinical utility of our design by demonstrating that patients with frontal lobe abnormalities associated with schizophrenia were impaired on rule switching but showed intact performance with perceptual switching (Ravizza, et al., submitted). While our broad categorization of shift types does not explain every inconsistency, we believe it is a promising method of understanding the essence of a task switch.

The design of our experiment is very similar to that used by Rushworth et al (2001, 2002) who distinguished between switches of attentional (visual) and intentional (response) set. In the attentional condition, participants switched between attending to colors and shapes while detecting rare targets. In the intentional condition, participants were required to switch the S-R mappings associated with two shapes. While formally similar to our experimental paradigm, Rushworth's studies differ from ours in several aspects including the design, results, and interpretation. First, our study was designed to assess shifting differences across the entire brain (excluding the inferior cerebellum) whereas Rushworth and colleagues used a region of interest analysis focusing on either the parietal or medial frontal cortex. Second, Rushworth reported the opposite results for activity in the superior parietal cortex; that is, this region was more active in the visuomotor switching task than the attentional task. While speculative, this may be due to the fact that the motor task required spatial reversals (i.e., square=left hand, triangle=right hand vs. square=right hand, triangle = left hand) that may call upon visuospatial transformation processes computed by the superior parietal cortex. In contrast, our response rules were not reversals but separate sets of S-R mappings for letters and shapes that remained constant. Third, Rushworth distinguishes between switches of attention vs. switches of action. While we do manipulate sets of response rules, we propose that rule switching is not limited to rules of action. Thus, we predict our findings in the PFC to generalize to other kinds of situations where people must switch between sets of rules - motor or otherwise. However, we tested only one type of shift in rule information (i.e., stimulus-response mappings), and it is possible that the behavioral and neural effects we observed in the rule shifting condition will be relevant only for switching between response rules rather than to all type of rule shifts.

Similarly, we are unable in this design to determine whether a perceptual switch involves reorienting attention to the relevant feature or to the relevant location (to the center or the periphery of the compound letter/shape object). There might be important distinctions behaviorally and neurally between shifts of feature and shifts of spatial location that we are unable to observe in this study. This leaves open the question as to whether the right superior parietal cortex is involved in location or feature switches or both, and it is quite possible that separable regions of the parietal cortex could be found for each type of shift.

These results emphasize that a consideration of the task is crucial to understanding the cognitive and neural mechanisms that allow for this form of flexible behavior. In sum, an appreciation of both the “task” and the “switch” in necessary to any study of task switching.