Cross-Hemispheric Complementary Prefrontal Mechanisms during Task Switching under Perceptual Uncertainty

Abstract

Flexible adaptation to changing environments is a representative executive control function implicated in the frontoparietal network that requires appropriate extraction of goal-relevant information through perception of the external environment. It remains unclear, however, how the flexibility is achieved under situations where goal-relevant information is uncertain. To address this issue, the current study examined neural mechanisms for task switching in which task-relevant information involved perceptual uncertainty. Twenty-eight human participants of both sexes alternated behavioral tasks in which they judged motion direction or color of visually presented colored dot stimuli that moved randomly. Task switching was associated with frontoparietal regions in the left hemisphere, and perception of ambiguous stimuli involved contralateral homologous frontoparietal regions. On the other hand, in stimulus-modality-dependent occipitotemporal regions, task coding information was increased during task switching. Effective connectivity analysis revealed that the frontal regions signaled toward the modality-dependent occipitotemporal regions when a relevant stimulus was more ambiguous, whereas the occipitotemporal regions signaled toward the frontal regions when the stimulus was more distinctive. These results suggest that complementary prefrontal mechanisms in the left and right hemispheres help to achieve a behavioral goal when the external environment involves perceptual uncertainty.

SIGNIFICANCE STATEMENT In our daily life, environmental information to achieve a goal is not always certain, but we make judgments in such situations, and change our behavior accordingly. This study examined how the flexibility of behavior is achieved in a situation where goal-relevant information involves perceptual uncertainty. fMRI revealed that the lateral prefrontal cortex (PFC) in the left hemisphere is associated with behavioral flexibility, and the perception of ambiguous stimuli involves the PFC in the right hemisphere. These bilateral PFC signaled to stimulus-modality-dependent occipitotemporal regions, depending on perceptual uncertainty and the task to be performed. These top-down signals supplement task coding in the occipitotemporal regions, and highlight interhemispheric prefrontal mechanisms involved in executive control and perceptual decision-making.

Introduction

Executive control guides flexible adaptation to changing environments, which has been most developed in humans through evolution (Roberts et al., 1998; Fuster, 2015), and is implemented in distributed frontoparietal cortical regions (Miller and Cohen, 2001; Dosenbach et al., 2006; Duncan, 2010). Shifting between different types of behaviors is a representative executive control function (Jersild, 1927; Allport et al., 1994; Rogers and Monsell, 1995; Monsell, 2003), and task switching paradigms have often been used to investigate neural mechanisms of behavioral flexibility. Previous neuropsychological and neuroimaging studies suggest that the posterior part of the left PFC (lPFC) plays a critical role in task switching (Fig. 1A) (Konishi et al., 1998, 2002; Dove et al., 2000; Rushworth et al., 2002; Crone et al., 2006; Brass and von Cramon, 2004; Derrfuss et al., 2005; Yeung et al., 2006; C. Kim et al., 2012).

Figure 1.

Study design. A, Behavioral flexibility involves the engagement of the lateral PFC and requires perceptual decision-making implicated in the occipitotemporal regions where modality-dependent mechanisms are distributed. B, Visual stimuli. Coherence of random dot motion and color were independently manipulated. White dots were presented within a donut-shaped display circle as indicated by the dotted lines. Arrow indicates the overall motion direction of each dot. C, Task sequence. After the presentation of a task cue (MOTION or COLOR), the visual stimulus was presented, and participants judged the overall motion or color of the stimuli and pressed a corresponding button. They continued the judgment along the same dimension until the next cue was presented. Only the trials immediately after the cue presentation involved perceptual uncertainty, and subsequent trials did not involve uncertainty (100% coherent). D, Task switch and task repeat. If the presented cue was different from the former one, the task to be performed was switched (left); otherwise, the task was repeated (right).

On the other hand, goal-relevant sensory information from the external environment is perceived and guides a course of action, which is referred to as perceptual decision-making. Studies of perceptual decision-making have used behavioral tasks that demand discrimination of presented sensory stimuli involving perceptual uncertainty (Newsome and Pare, 1988; Corbetta et al., 1991; Erwin et al., 1992; Romo et al., 1998). In one common task, randomly moving dots are presented visually, and participants are required to judge the overall direction of the movement or the overall color of the dot stimuli. It is known that the middle temporal (MT) region plays an important role in perceptual decision-making for moving stimuli (Britten et al., 1992; Shadlen et al., 1996; Beauchamp et al., 1997; Braddick et al., 2001; Corbetta and Shulman, 2002; Huk et al., 2002; Mazurek et al., 2003; Kayser et al., 2010). On the other hand, it has been suggested that ventral visual areas, including V4 and the ventral visual complex (VVC), are associated with the perception of color stimuli (Schein and Desimone, 1990; Motter, 1994; Bartels and Zeki, 2000; Kayser et al., 2010). The MT and VVC/V4 involvements reflect perception of goal-relevant information rather than simple accumulation of stimulus evidence (Kayser et al., 2010). These results suggest stimulus-modality-dependent mechanisms distributed across occipitotemporal regions for the perception of visual information (Fig. 1A).

Executive control depends on the perceived information of the external environment, and the perceived information should be relevant to the achievement of behavioral goals. In other words, executive control presumes that relevant information is appropriately extracted by the perception of the external environment (Fig. 1A). In our daily life, however, goal-relevant environmental information is not always evident. Although most prior studies of executive control have used sensory stimuli that could be perceived distinctively (e.g., Krystal et al., 1994; Rogers and Monsell, 1995; Casey et al., 1997; MacDonald et al., 2000; Miyake et al., 2000; Braver et al., 2003; Aron et al., 2004; Seeley et al., 2007), several studies of perceptual discrimination used stimuli for which multidimensional visual uncertainties were manipulated within a stimulus set (Kayser et al., 2010; Mante et al., 2013; Zhang et al., 2013; Kumano et al., 2016). However, to our knowledge, task switching has been rarely explored in behavioral situations where environmental information involves perceptual uncertainties.

The current study aimed to elucidate relationships between task switching and perceptual decision-making, and to explore the underlying neural mechanisms. We hypothesized three independent mechanisms: (1) a functionally merged single region implements task switching and perceptual uncertainty (e.g., Shackman et al., 2011; Yarkoni et al., 2011); (2) distinct regions for perceptual decision-making and task switching interact to guide task switching (e.g., Konishi et al., 1998; Egner and Hirsch, 2005; Kayser et al., 2010; Waskom et al., 2014); and (3) a hub-like region links the regions for task switching and perceptual decision-making (e.g., Cole et al., 2013; Osada et al., 2015; Nee and D'Esposito, 2016; Jiang et al., 2018).

To test these three hypotheses, fMRI was administered while human participants alternated tasks in which the motion strength and color strength of randomly moving dots were independently manipulated as perceptual uncertainties (Fig. 1B–D). We first explored the brain regions associated with task switching, motion strength, and color strength. Then whole-brain exploratory multivariate pattern analysis (MVPA) was performed to identify brain regions involving task-relevant information. Finally, to examine functional network mechanisms, effective connectivity was examined among the task-related brain regions during task switching under conditions of perceptual uncertainty.

Materials and Methods

Participants

Written informed consent was obtained from 28 healthy right-handed subjects (age range: 18-23 years; 10 females). The experimental procedures were approved by the institutional review boards of Keio University (Yokohama, Japan) and Kochi University of Technology (Kami, Japan). Participants received 2000 yen for participating in each session. One participant was excluded from the analysis because of low behavioral performance. All participants had normal color vision. The number of participants was determined before the collection of the current data based on the effect sizes in relevant previous and pilot behavioral experiments (α = 0.05, β = 0.95, η² = 0.42).

Experimental design

Behavioral procedures

The experiment consisted of two sessions administered on separate days. The first day was training sessions, in which participants practiced two discrimination tasks (color and motion) and switching between these two tasks. The second day was scanning sessions, and the participants performed the switching task using the motion dot stimuli identical to those used in the training sessions.

Stimuli

All stimuli were generated in MATLAB version 2012a, using the Psychophysics Toolbox extension version 3.0.10 (Brainard, 1997; Pelli, 1997), and were presented visually on a computer screen. The current stimuli were similar to those used in a previous study of perceptual decision-making (Chen et al., 2015) (Fig. 1B), but each dot was colored in red or green. Each motion stimulus involved 75 dots moving inside a donut-shaped display patch with a white cross in the center of the patch on a black background (Fig. 1B). The display patch and cross were centered on the screen and extended from 6 to 12 degrees of visual angle. Within the display patch, every dot moved at a speed of 10 degrees of visual angle per second. Some dots moved coherently in one direction (up or down), whereas the others moved randomly. The percentage of coherently moving dots determined the “motion coherence,” and three levels (20%, 40%, and 80%) of motion coherence were used.

Dot presentation was controlled to remove local motion signals following a standard method for generating motion stimuli (Newsome and Pare, 1988; Britten et al., 1993; Shadlen and Newsome, 1994; Palmer et al., 2005). Namely, on stimulus onset, the dots were presented at new random locations on each of the first three frames. They were relocated after two subsequent frames, such that the dots in frame 1 were repositioned in frame 4, the dots in frame 2 were repositioned in frame 5, etc. When repositioned, each dot was either randomly presented at the new location or aligned with the predetermined motion direction, depending on the predetermined motion strength for that trial. Each stimulus was composed of 18 video frames with 60 Hz refresh rates (i.e., 300 ms presentation).

Each dot was either orange (RGB = [255, 123, 11]; HSV = [27, 95, 100]) or green (RGB = [0, 255, 0]; HSV = [120, 100, 100]), and the two colors were equally bright (V = 100). The percentage of red dots in the dot stimuli was defined as “color coherence,” and three levels (20%, 40%, and 80%) of color coherence were used. Then, the color coherence and motion coherence were independently manipulated for the dot stimuli.

Task procedure

At the beginning of the task, a task cue (MOTION or COLOR) was presented, indicating the dimension of discrimination in subsequent trials (Fig. 1C). Then, the dot patch was presented. In the motion task, participants were required to judge the direction of overall motion of the patch (up or down); in the color task, they were required to judge the overall color (orange or green). They then pressed a corresponding button (right or left) with their right thumb as quickly and accurately as possible. If participants made an incorrect response, or did not respond within 1.8 s from the onset of the dot stimuli, a feedback stimulus indicating the error (X) was presented for 1.0 s, followed by a cue presentation for the same task. The stimulus-response association for the two tasks was identical on days 1 and 2, and was counterbalanced across participants.

For the task switching design, we applied an intermittent cueing paradigm that requires participants to actively maintain a task set until the next cue presentation, which enhances the switching effect (Sakai, 2008). Specifically, after a cue was presented, the dot target trial was repeated 4-6 times, and the participants identified the dot stimulus along the same dimension until the next cue stimulus was presented. Motion and color coherence levels of the dot patch in these trials (second to sixth trials after cue presentation) were set to 100% and did not involve uncertainty, which assured successive correct trials until the next task cue was presented to enhance the task switch effect (see also Behavioral analysis). After the participants gave successive correct responses in 4-6 trials, the next task cue was presented.

One task block lasted ∼90 s, which involved 5-7 task-cue presentations. Fixation blocks were also presented for 20 s between task blocks. One functional run consisted of three task blocks and two fixation blocks. Each run lasted for 305 s.

Practice procedure

On the first day, participants practiced behavioral tasks outside the scanner. They first practiced stimulus discrimination tasks for dot stimuli, where only one task dimension (color or motion) was repeatedly presented (i.e., single-task run). Participants alternatively practiced a motion or color discrimination task for every 1 or 2 practice runs, and completed a total of 8 practice single-task runs.

They then practiced switching tasks, in which a task cue was pseudorandomly alternated (i.e., task-switching run; Fig. 1D). During task-switching runs, the dot target trial that took place immediately after the cue presentation involved perceptual uncertainty manipulated by dot color and motion coherence, as in practice runs; however, after the first target trials until the next cue presentation, the dot target involved no uncertainty (i.e., 100% coherence for both color and motion; Fig. 1C; see also above). Participants performed two practice runs for the task-switching runs.

Behavioral procedure in scanning session

On the second day, after practicing a task-switching run, they performed nine task-switching runs using the same procedure as that was used on day 1 while fMRI was administered.

Imaging procedure

MRI scanning was administered using a 3T MRI scanner (Siemens Verio) with a 32-channel head coil. Functional images were acquired using a multiband acceleration EPI sequence (TR: 0.8 s; TE: 30 ms; flip angle: 45 deg; 80 slices; slice thickness: 2 mm; in-plane resolution: 3 × 3 mm; multiband factor: 8) (Moeller et al., 2010). Each functional run involved 385 volume acquisitions. The first 10 volumes were discarded for analysis to take into account the equilibrium of longitudinal magnetization. High-resolution anatomic images were acquired using an MP-RAGE T1-weighted sequence (TR: 2500 ms; TE = 4.32 ms; flip angle: 8 deg; 192 slices; slice thickness: 1 mm; in-plane resolution: 0.9 × 0.9 mm²]).

Statistical analysis

Behavioral analysis

Two types of target trials were of interest: (1) task-switching trials immediately after cue presentation indicating a task alternation (i.e., switch to the color task from the motion task or vice versa); and (2) task-repeat trials immediately after cue presentation, indicating a repeat of the previous task. These trials were further classified in terms of the performed task (motion and color), and the coherence level of the dot stimulus (20%, 40%, and 80%), entailing a 2 × 2 × 3 factorial model (switch, task, coherence). The trials that occurred after an error were excluded from the entire analysis. Trials with 100% coherence after the task switching and repeat trials were classified as either motion or color trials. Then, accuracy and reaction time were calculated for each condition. Statistical tests were performed based on repeated-measures ANOVAs implemented using SPSS Statistics 24 (IBM).

Image preprocessing

MRI data were analyzed using SPM12 software (https://www.fil.ion.ucl.ac.uk/spm/). All functional images were first temporally realigned across volumes and runs, and the anatomic image was coregistered to a mean image of the functional images. The functional images were then spatially normalized to a standard MNI template with normalization parameters estimated for the anatomic scans. The images were resampled into 2 mm isotropic voxels, and spatially smoothed with a 6 mm FWHM Gaussian kernel.

Imaging analysis: General linear model First level analysis

A GLM approach (Worsley and Friston, 1995) was used to estimate parameter values for task events. The events of interest were correct switch, repeat, and 100% coherent trials, and were coded for motion and color tasks separately. For switch and repeat trials, the normalized (demeaned and divided by SD) coherence level of relevant stimuli (motion coherence for motion task or color coherence for color task) was added as a parametrical effect of interest. Error trials were coded in GLM as nuisance effects. Those task events were time-locked to the onset of random dot stimuli and then convolved with a canonical HRF implemented in SPM. Additionally, six axis head movement parameters, white matter signal, CSF signal, and the parametrical effect of reaction times were included in GLM as nuisance effects. Then, parameters were estimated for each voxel across the whole brain.

Group-level analysis

Maps of parameter estimates were first contrasted within individual participants. The contrasted maps were collected from participants and were subjected to group-mean tests. Voxel clusters were first identified using a voxelwise uncorrected threshold of p < 0.001. The voxel clusters were then tested for a significance across the whole brain with a threshold of p < 0.05 corrected by family-wise error rate based on permutation methods (5000 permutations) implemented in randomize in FSL suite (www.fmrib.ox.ac.uk/fsl/), which was empirically validated to appropriately control the false positive rate in a previous study (Eklund et al., 2016). To identify occipitotemporal cortical regions specifically associated with the color task (color minus motion tasks), a mask obtained from Neurosynth (http://neurosynth.org/) for the search word “color” (z > 3.0, for uniformity test) was used. Peaks of significant clusters were then identified and listed in tables. If multiple peaks were identified within 12 mm, the most significant peak was kept.

Exploration of hemispheric laterality

To examine the hemispheric laterality of the task effects, we explored brain regions showing greater task effects than the contralateral homologous regions. For each participant, contrast maps were flipped along the x (left-right) axis and were subtracted from the original nonflipped maps on a voxel-by-voxel basis (Konishi et al., 2002). Then, the group-level statistical significance was tested within the right hemisphere. Statistical correction was performed within one hemisphere because the subtracted maps show sign-flipped symmetry along the x axis.

We applied this procedure to the maps of the coherence effect (motion and color collapsed) and the switch effect (switch vs repeat). For the coherence effect, the maps were sign-flipped and had a greater signal in the low-coherent trial. Thus, positive values of hemispheric laterality indicate a greater signal in low-coherent trials in the right hemisphere and/or a lower signal in high coherent trials in the left hemisphere. Likewise, for the switch effect, positive values indicate greater activation during switch versus repeat trials in the right hemisphere and/or lower activation switch versus repeat trials in the left hemisphere. The negative values indicate the reverse effects.

ROI analysis for hemispheric laterality

The exploratory analysis above assumes a homology of the brain regions in the right and left hemispheres. We next performed an ROI analysis without assuming hemispheric homology. ROIs were defined as brain regions identified as a multiple demand network (Camilleri et al., 2018). Specifically, ROIs were classified as being in the left or right regions based on the x axis coordinates (Camilleri et al., 2018, their Table 2). ROI images were created with spheres that had a radius of 6 mm and were centered at the coordinates. Then, for each of the left and right ROIs, the estimation contrast of switch effect (switch vs repeat) and the parametric effect of the coherence were extracted from each participant. Group-level statistics were calculated treating participants as a random effect.

To test the robustness of the ROI analysis against the ROI definition, we redefined ROIs. Image maps of meta-analyses were obtained from Neurosynth (Yarkoni et al., 2011) using the terms “switching” (https://neurosynth.org/analyses/terms/switching/) and “task demands” (https://neurosynth.org/analyses/terms/task%20demands/). A threshold of p < 0.01 (uniformity test with whole-brain correction based on false discovery rate) was set for each map. Then, an image map of the logical sum of the two maps with thresholds was created, and ROIs were defined as being in the left or right hemisphere based on the x axis coordinates. Regions with X = 0 were excluded from the analysis. The same statistical tests were performed using these ROIs.

MVPA

To explore brain regions representing task-related information, an MVPA was performed. Bivariate classification based on a support vector machine was used to decode a performed task (motion or color). A searchlight procedure (Kriegeskorte et al., 2006) with a 5 voxel radius was used to provide a measure of decoding accuracy in the neighborhood of each voxel. Training and testing were performed based on the Decoding Toolbox (TDT; version 3.95; https://sites.google.com/site/tdtdecodingtoolbox/).

For each functional run, a single-level GLM analysis was first performed within the standard brain space with regressors identical to those in the univariate analysis. For each run, the classifier in each searchlight was trained using β maps for 100% coherent trials. To calculate the training data, we used the third and later trials after cue presentation and eliminated data from the second trials, to avoid potential confounds derived from cue-related effect.

Next, from β maps for switch and repeat trials (i.e., immediately after cue presentation), image data were extracted within the same searchlight and scanning run. Then, the trained classifier was tested for whether it correctly classified the performed task (motion or color tasks) based on the extracted data (i.e., switch or repeat trials) for each run. It is important to note that training and tested maps were independent. Then, classification performance was collected from all functional runs and averaged within participants for each searchlight. A linear kernel was used and the C parameter was set to 1.0 (default).

For group-level analysis, accuracy maps were first contrasted between switch and repeat trials, and the contrasted accuracy maps were collected from all participants. Then voxelwise group-mean tests were performed, and the significance was tested based on the permutation method (5000 permutations) implemented in randomize in FSL. To exclude brain regions that showed a classification accuracy below the chance level in either of the switch or repeat trials, voxels with t values <0.0 (uncorrected) in the group-level accuracy map of a switch or repeat trial were masked out when testing statistical significance. In other words, tested voxels showed classification accuracy above the chance level for both the switch and repeat trials.

ROI analyses were also performed for classification accuracy. ROIs in occipitotemporal regions were defined as being in the MT or VVC regions identified by univariate analysis (Fig. 2B; see Table 2). ROIs in the frontoparietal regions were independent of the current data to avoid circular analysis (Kriegeskorte et al., 2009) because, in the current study, the frontoparietal regions were identified based on switch and repeat trials involving perceptual uncertainty. Specifically, ROIs were defined based on the multiple demand network identified in a previous study (Camilleri et al., 2018). ROI images were created similarly to another ROI analysis for hemispheric laterality (see ROI analysis for hemispheric laterality), but separately for frontal and parietal regions. To test the robustness of the differential accuracy against the definition of ROI, we redefined ROIs based on the meta-analysis maps of Neurosynth, similarly to the analysis performed above. Then for each participant and ROI, classification accuracy was calculated for switch and repeat trials, and statistical significance was tested treating participants as a random effect.

Figure 2.

Behavioral results. A, Accuracy for task as a function of motion and color coherence. B, Reaction time for task as a function of motion and color coherence. Solid lines with filled circles indicate switch trials. Dotted lines with open circles indicate repeat trials. Red lines indicate motion tasks. Blue lines indicate color tasks. Data of 100% coherence trials were derived from the trials presented after the switch and repeat trials, and not from the switch or repeat trials (Fig. 1C,D). Error bars indicate SEM across participants.

Table 2.

Brain regions showing significant signal increase in the contrast of motion versus color trials^a

Area	x	y	z	z value	BA
Frontal cortex	40	38	12	−4.11	46
	2	−24	62	3.8	4/6
Parietal cortex	26	−60	42	−5.61	7/39
	28	−68	52	−4.56	7
	−6	−40	66	4.39	7
	58	−26	20	4.33	40
	−50	−54	10	4.23	39
	−28	−62	50	−4.23	7
	−52	−66	16	4.13	39
	−54	−34	34	4.02	40
	−22	−62	38	−4.02	7
	32	−34	44	3.86	1/40
	36	−36	60	3.84	1
	58	−30	32	3.8	40
	−30	−70	32	−3.41	39
	−64	−26	14	3.13	40/41
Occipital cortex	−30	−82	22	−4.95	19
	32	−72	18	−4.72	19/39
Temporal cortex	−42	−64	8	4.83	19
	40	−56	4	4.18	19/37
	54	−58	4	4.11	37
	−34	−54	−16	−3.98b	37
	44	−36	14	3.87	22
	56	−40	12	3.83	22
	−56	−42	18	3.48	22

^aPositive z values indicate a signal increase in motion trials, and negative z values indicate a signal increase in color trials.

^bCorrected within a meta-analysis mask of a color-related region. The formats are similar to those in Table 1.

Effective connectivity analysis

A dynamic causal modeling (DCM) (Friston et al., 2003) analysis was performed to examine functional network mechanisms associated with task switching during the perception of a motion or color stimulus. In particular, we hypothesized that functional connectivity among brain regions associated with (1) task switching, (2) perceptual decision-making, (3) motion perception, and (4) color perception is modulated by task manipulations and brain signals.

DCM allows us to explore effective connectivity among brain regions under the premise of the brain as a deterministic dynamic system that is subject to environmental inputs and produces outputs based on the space-state model. The model constructs a nonlinear system involving intrinsic connectivity, task-induced connectivity, and extrinsic inputs. Parameters of the nonlinear system are estimated based on fMRI signal (system states) and task events.

Four ROIs were first defined based on univariate activation contrasts:

1. Motion effect (MT; Fig. 3B) (Newsome and Pare, 1988; Britten et al., 1992, 1993; Shadlen et al., 1996; Beauchamp et al., 1997; Britten and Newsome, 1998; Braddick et al., 2001; Corbetta and Shulman, 2002; Huk et al., 2002; Mazurek et al., 2003; Kayser et al., 2010);

2. Color effect (VVC; Fig. 3B) (Zeki, 1973; Schein and Desimone, 1990; Zeki et al., 1991; Bartels and Zeki, 2000; Kayser et al., 2010);

3. Switching effect (lPFC; Fig. 3A) (Konishi et al., 1998, 2002; Dove et al., 2000; Rushworth et al., 2002; Braver et al., 2003; Crone et al., 2006; Brass and von Cramon, 2004; Jimura et al., 2004; Derrfuss et al., 2005; Yeung et al., 2006; C. Kim et al., 2011, 2012);

4. Perceptual uncertainty effect [right PFC (rPFC); Fig. 3C,D] (Kayser et al., 2010), with 6 mm radius spheres centered in the peak coordinates in group-level activation maps.

Figure 3.

Statistical activation map of univariate analysis. Maps are overlaid onto 3D surface of the brain. Hot colors represent positive effects. Cool colors represent negative effects. A, Switch versus repeat trials. B, Motion versus color tasks. C, Motion coherence effect. Hot colors represent a greater signal in high-coherence trials. Cool colors represent a greater signal in low-coherence trials. D, Color coherence effect. The format is similar to those in C.

To avoid circular analysis (Kriegeskorte et al., 2009), the coordinates of ROIs were defined with a leave-one-subject-out procedure, where ROIs for 1 participant were determined based on other participants. ROIs in MT and VVC were defined based on a group-level statistical map for the contrast of motion and color trials with 100% coherence, respectively. lPFC ROI was defined based on a group-level statistical map for the contrast of switch and repeat trials. rPFC ROI was defined based on a group-level statistical map for the coherence effect of switch and repeat trials.

Then, signal time courses of four ROIs and regressors in the events of interest were extracted from first-level GLMs. The events of interest were correct trials immediately after cue presentation (switch and repeat) and subsequent 100% coherence trials. For switch and repeat trials, the contrast of the two trials and the normalized coherence level of stimuli were also added as a parametrical effect of interest. Nuisance effects of head motion, white matter signal, ventricle signal, functional run, and contrast were subtracted from the ROI time courses.

For each trial effect, causal models were defined as those that differed in external inputs and modulatory effects among ROIs. As the current analysis involved three ROIs, the tested models entailed 512 types (i.e., 2³ inputs and 2⁶ connection effects). Then, connectivity matrices reflecting (1) first-order connectivity, (2) effective change in coupling induced by the inputs, and (3) extrinsic inputs on the MRI signal in ROIs were estimated for each of the 512 models based on DCM analysis implemented in SPM12. A parametric regressor (switch vs repeat/coherence) was used as the extrinsic effect for effective connectivity between ROIs and ROI inputs.

To estimate the effective connectivity strength, a Bayesian model reduction method (Friston et al., 2016) was used. The reduction method enables the calculation of posterior densities of all possible reduced models, which was then inverted to a fully connected model. Then the reduced models were supplemented with second-level parametric empirical Bayes (Friston et al., 2016) to apply empirical priors that remove subject variability from each model.

Subsequently, parameters of these models were estimated based on Bayesian model averaging (Penny et al., 2010) to estimate connectivity changes. Because the current analysis aimed to identify the average effective connectivity observed across participants, we used a fixed-effect estimation assuming that every participant uses the same model, rather than a random-effect estimation assuming that different participants use different models, which has been often used to examine group differences in effective connectivity (Penny et al., 2010).

The significance of connectivity was then tested by thresholding at a posterior probability of 95% CI (uncorrected), as the current analysis aimed to test whether connectivity between two specific brain regions was enhanced depending on task manipulation and brain activity, but did not aim to explore one model involving connectivity among multiple brain regions that best fits to imaging and behavioral data.

Additionally, to test the robustness of the functional connectivity, we also examined Bayesian model averaging-based DCM without model reduction or without empirical prior, and confirmed the overall results were consistent across these estimation methods.

The number of trials for each condition (see Behavioral results) was comparable to those in recently published relevant studies examining cognitive control using DCM (∼16-40 trials for each condition) (Nee and D'Esposito, 2016; He et al., 2019; Bowling et al., 2020). We used a high temporal resolution sequence for functional imaging (Moeller et al., 2010) (TR = 0.8; see also Imaging procedures), which is >2 times higher than these recent studies (TR =1.9–2.0 s), enabling us to collect more scan frames to increase the signal-to-noise ratio of the DCM estimation (Penny et al., 2004; Stephan et al., 2010).

Results

Behavioral results

Participants gave correct responses for 35.56 ± 4.90 (mean ± SD) switch trials and 38.70 ± 6.41 repeat trials in the motion task; in the color task, they gave correct responses for 36.07 ± 3.74 switch trials and 33.70 ± 4.68 repeat trials. In the motion task, participants gave correct responses for 27.89 ± 2.56 high-coherence trials, 25.26 ± 4.53 middle-coherence trials, and 21.11 ± 5.06 low-coherence trials; in the color task, they gave correct responses for 23.30 ± 2.97 high-coherence trials, 25.63 ± 3.54 middle-coherence trials, and 20.85 ± 3.27 low-coherence trials.

Accuracy became lower in the lower coherent (i.e., more uncertain) trials (F_(1,26) = 149.4, p < 0.001, η² = 0.85), and became lower in switch trials than in repeat trials (F_(1,26) = 22.8; p < 0.001, η² = 0.47) (Fig. 2A). The interaction effect of trial type (switch/repeat) and coherence levels was not significant (F_(1,26) = 2.1, p = 0.16, η² = 0.08). Because dots were presented in the periphery visual field (see Materials and Methods), participants needed a higher signal-to-noise ratio compared with the centrally presented dots used in the previous studies (e.g., Palmer et al., 2005; Gold and Shadlen, 2007). This seems to be a major reason why the accuracy levels observed in this study were within the usual range despite very high coherence levels. Accordingly, reaction times became longer in lower coherent trials (F_(1,26) = 126.1; p < 0.001, η² = 0.83), and became longer in switch trials than in repeat trials (F_(1,26) = 19.9; p < 0.001, η² = 0.43) (Fig. 2B). The interaction effect of trial type (switch/repeat) and coherence levels was not significant (F_(1,26) = 3.9, p = 0.058, η² = 0.13). These behavioral results suggest that the current behavioral task successfully manipulated task switching (Dove et al., 2000; Braver et al., 2003; Crone et al., 2006; Jimura et al., 2014) and perceptual decision-making (Shadlen et al., 1996; J. N. Kim and Shadlen, 1999; Mazurek et al., 2003; Palmer et al., 2005). In 100% coherent trials, reaction times also became longer in the color task than in the motion task (F_(1,26) = 10.1, p < 0.005, η² = 0.28).

Imaging results: univariate activation

We first explored brain regions associated with task switching, motion perception, and color perception.

Figure 3A shows brain regions with a significant increase and decrease in brain activity during task switching (switch trials vs repeat trials). Robust activation increases were observed in the left frontal regions, including the inferior frontal cortex (IFC), the dorsolateral PFC (DLPFC), the anterior PFC (aPFC), the inferior frontal junction (IFJ), and the pre-supplementary motor area (pre-SMA), and in the left parietal regions including the posterior parietal cortex (PPC). These left hemisphere-dominant frontoparietal activations are consistent with previous studies of task switching (Konishi et al., 1998, 2002; Dove et al., 2000; Rushworth et al., 2002; Braver et al., 2003; Crone et al., 2006; Brass and von Cramon, 2004; Jimura et al., 2004; Asari et al., 2005; Derrfuss et al., 2005; Yeung et al., 2006; C. Kim et al., 2011, 2012). A full list of brain regions is shown in Table 1.

Table 1.

Brain regions showing significant signal increase in the contrast of switch versus repeat trials^a

Area	x	y	z	z value	BA
Frontal cortex	28	0	60	6.42	6
	0	12	46	5.95	6
	−8	24	34	5.62	32
	6	20	32	5.61	32
	−48	6	32	5.41	6
	−30	−2	58	5.35	6
	−2	8	60	5.34	6
	10	8	68	4.76	6
	−20	10	62	4.72	6
	−44	2	48	4.69	6
	−44	40	12	4.68	46
	−4	34	18	4.65	24/32
	−42	28	22	4.59	46
	−24	8	46	4.43	6
	−8	0	72	4.38	6
	26	40	26	4.2	9
	−58	14	18	4.13	44
	46	18	10	3.78	44
	28	52	26	3.63	10
	−50	6	10	3.55	6/44
	−34	−4	44	3.54	6
	26	−4	48	3.35	6
Parietal cortex	−54	−30	44	5.66	40
	−44	−40	50	5.63	40
	−8	−70	50	4.93	7
	−34	−52	46	4.89	39
	16	−70	52	4.73	7
	−28	−70	36	4.62	39
	−58	−26	30	4.61	40
	−44	−40	32	4.24	39/40
	−28	−68	52	4.04	7
Occipital cortex	−18	−66	26	5.02	19
	16	−66	34	4.68	19
Others	−32	14	2	5.64	Insula
	28	24	6	5.07	Insula
	36	10	−2	4.76	Insula
	28	20	−8	4.67	Insula
	30	−68	−52	4.31	Cerebellum
	18	−74	−50	3.89	Cerebellum
	−44	10	−2	3.68	Insula

^aPositive z values indicate a signal increase in the switch trials. Coordinates are listed in the MNI space. BA, Brodmann areas (approximate).

Next, we explored brain regions associated with motion and color tasks during the task switching and repeat trials (Fig. 3B; Table 2). Activity became greater in the MT area during motion tasks, whereas in color tasks, greater activity was observed in VVC (Glasser et al., 2016). These results are also consistent with prior studies examining perceptual decision-making for motion (Newsome and Pare, 1988; Britten et al., 1992, 1993; Shadlen et al., 1996; Beauchamp et al., 1997; Britten and Newsome, 1998; Braddick et al., 2001; Corbetta and Shulman, 2002; Huk et al., 2002; Mazurek et al., 2003; Kayser et al., 2010) and color (Zeki, 1973; Schein and Desimone, 1990; Zeki et al., 1991; Motter, 1994; Bartels and Zeki, 2000; Kayser et al., 2010).

We then examined coherence effect. Whole-brain activation maps for the coherence effect in motion and color tasks are shown in Figure 3C and Figure 3D, respectively. In both the color and motion tasks, activations became greater in low coherent trials in multiple frontoparietal regions, including the IFC, DLPFC, aPFC, IFJ, pre-SMA, and PPC (Fig. 3C,D; Tables 3 and 4) (Kayser et al., 2010). Interestingly, in both tasks, the coherence effect in frontoparietal regions looked prominent, especially in the right hemisphere.

Table 3.

Brain regions showing significant parametrical effect with motion coherence in motion trials^a

Area	x	y	z	z value	BA
Frontal cortex	−40	6	32	−6.21	6/8
	52	12	24	−6.17	44
	42	8	30	−6.12	44
	48	34	16	−5.99	46
	4	26	40	−5.93	8
	0	12	50	−5.81	6
	−6	50	40	5.75	9
	−52	6	26	−5.52	44
	42	24	26	−5.51	9
	−28	26	50	5.47	8
	22	36	48	5.38	8
	44	4	52	−5.35	6
	46	16	0	−5.34	44
	48	4	40	−5.1	6
	−6	58	20	4.95	10
	34	−14	32	4.81	4
	−44	4	16	−4.78	44
	32	2	46	−4.77	6
	42	32	4	−4.75	45
	12	46	44	4.74	9
	20	50	−10	−4.73	10/11
	10	−28	74	4.72	4
	44	44	−10	−4.43	10/47
	−2	20	−18	4.42	25
	−48	40	18	−4.41	10
	26	−4	30	4.37	6
	−22	22	40	4.28	8
	−40	24	26	−4.26	9
	0	34	−22	4.23	11
	−54	−16	10	4.2	1/40
	−4	32	26	−4.2	32
	−28	−2	56	−4.19	6
	6	34	18	−4.16	32
	−44	−8	22	4.07	6
	−42	42	2	−4.04	46
	24	−20	76	4	6
	10	−44	72	4	5/7
	24	−28	64	3.89	4
	2	−24	64	3.88	6
	48	−16	48	3.84	4
	−46	16	8	−3.82	44
	20	36	−14	−3.8	11
	−10	32	38	−3.77	8
	−24	−22	52	3.63	4/6
	−12	−24	74	3.62	6
	−10	40	52	3.61	8
	6	−22	52	3.47	6
	8	26	−14	3.37	11
	56	−2	14	3.27	4
Parietal cortex	−42	−70	42	6.08	39
	−50	−60	30	5.79	39
	4	−18	32	−5.58	23
	2	−50	36	5.31	23
	42	−42	46	−5.07	7/40
	−14	−70	36	−4.89	7
	8	−30	26	−4.84	23
	−16	−40	64	4.63	7
	−22	−30	38	4.6	1
	−42	−44	44	−4.56	40
	−14	−44	36	4.51	31
	0	−40	62	4.46	1/5
	−2	−52	22	4.43	23
	56	−14	12	4.43	40
	−54	−32	42	−4.43	40
	−58	−28	28	−4.37	40
	54	−30	40	−4.33	40
	22	−44	68	4.31	7
	50	−60	28	4.13	39
	−24	−34	24	4.09	40
	−26	−60	44	−4.07	7
	−12	30	14	−4.02	24
	8	−36	34	4.01	23
	30	−48	44	−4.01	7
	22	−46	56	3.97	7
	−4	−36	72	3.89	5
	24	−32	50	3.65	1/5
	−34	−54	54	−3.57	7
	54	−48	48	−3.5	40
	−12	−70	48	−3.49	7
	60	2	−2	3.4	22
	−24	−34	56	3.39	1
	−26	−34	70	3.34	5
	52	−30	52	−3.31	40
	−16	−50	72	3.3	7
	62	−40	40	−3.24	40
	−22	−62	28	−3.16	7/31
Temporal cortex	−54	−10	−24	5.18	21
	−64	−22	−14	4.72	21
	60	−12	−26	4.6	21
	−60	2	−10	4.27	38
	−60	2	−26	4.23	38
	−62	−8	2	4.01	22
	−64	−26	12	4.01	22
	56	2	−36	4	20
	−66	−34	2	3.95	21
	−46	−38	−6	3.81	21
	−60	−42	−6	3.63	21
Occipital cortex	2	−74	4	−4.82	18
	−4	−86	−2	−4.62	18
	12	−78	12	−4.61	17
	16	−64	36	−4.49	19
	24	−84	40	4.24	19
	10	−82	0	−4.07	17
	−12	−48	0	3.83	19
	12	−88	34	3.73	19
	−10	−70	8	−3.29	17
	10	−94	8	−3.26	17
	0	−66	22	3.25	18/31
Others	−32	18	6	−6.57	Insula
	34	16	10	−6.55	44/Insula
	32	22	−4	−5.99	Insula
	0	−48	−32	−5.8	Cerebellum
	−8	2	8	−5.67	Caudate
	12	8	16	−5.54	Caudate
	10	−12	12	−5.48	Thalamus
	−18	−6	−24	5.46	Amygdala
	12	2	4	−5.36	Thalamus
	−32	−60	−30	−5.06	Cerebellum
	−26	−14	−16	4.9	Hippocampus
	−8	−12	4	−4.86	Thalamus
	−16	−76	−26	−4.86	Cerebellum
	22	−10	−20	4.78	Hippocampus
	2	−16	−12	−4.72	Brainstem
	4	−28	−2	−4.71	Brainstem
	−32	−36	−38	−4.67	Cerebellum
	−20	−70	−50	−4.64	Cerebellum
	42	−14	14	4.59	Insula
	−20	−60	−30	−4.27	Cerebellum
	16	8	−8	−4.12	Putamen
	32	6	−6	−3.88	Putamen
	2	−60	−28	−3.84	Cerebellum
	−16	−18	18	−3.47	Thalamus
	14	−10	−8	−3.34	Brainstem

^aPositive z values indicate a signal increase in high-motion coherence trials, and negative z values indicate a signal increase in low-motion coherence trials. The formats are similar to those in Table 1.

Table 4.

Brain regions showing significant parametrical effect with color coherence in color trials^a

Area	x	y	z	z value	BA
Frontal cortex	44	30	20	−6.2	9
	10	24	34	−5.98	8
	−2	18	48	−5.91	8
	−40	2	32	−5.54	6
	8	32	20	−5.13	32
	4	10	28	−5.13	24
	0	30	−14	5.08	11
	50	20	−8	−5.05	47
	50	20	−8	−5.05	47
	2	36	36	−4.77	8
	6	14	58	−4.72	6
	2	52	−8	4.57	10
	46	4	30	−4.52	6
	−26	38	48	4.5	8
	−14	48	−2	4.46	10/32
	52	12	22	−4.31	44
	6	−24	60	4.28	6
	−26	18	38	4.21	8
	6	−6	30	−4.15	24
	10	42	−12	3.97	11
	−52	12	42	−3.92	6/8
	54	16	40	−3.75	44
	−6	42	−16	3.68	11
	24	−8	34	3.62	6
	−14	42	34	3.56	9
	−54	18	−2	−3.51	45
	−6	14	−18	3.47	25
Parietal cortex	50	−70	32	5.41	39
	30	−68	30	−5.4	7
	−22	−62	40	−5.28	7
	18	−40	20	4.9	23
	−38	−54	18	4.96	39
	−26	−50	44	−4.93	7
	34	−28	24	4.91	40
	56	−62	24	4.82	39
	30	−58	52	−4.76	7
	4	−44	64	4.69	1/5
	24	−36	30	4.58	23
	38	−50	56	−4.55	7
	34	−74	20	−4.46	39
	−38	−78	38	4.38	39
	46	−24	22	4.38	40
	50	−32	48	−4.33	40
	−14	−72	40	−4.31	7
	−2	−34	44	4.21	31
	−14	−36	20	4.08	23
	16	−44	66	4.08	7
	−32	−40	18	3.96	23
	−20	−48	22	3.92	23
	−50	−70	34	3.92	39
	−42	−44	52	−3.92	7/40
	2	−58	62	3.83	7
	−2	−22	30	−3.82	23
	34	−46	24	3.79	39
	−52	−60	22	3.76	39
	40	−22	42	3.74	1
	6	−44	32	3.73	23
	−42	−38	36	−3.49	40
	40	−14	26	3.45	1
	−8	−46	38	3.17	31
Temporal cortex	54	−14	−30	4.41	20
	12	−84	10	−4.25	17
	64	−4	−8	4.09	22
	58	−8	−20	4.08	21
	60	−12	0	4.01	22/41
	8	−52	16	4	23
	32	−36	14	3.98	22
	−58	−64	12	3.97	39
	−24	−20	−20	3.76	36
	20	−24	−6	−3.64	36
Occipital cortex	−30	−78	22	−5.03	19
	−8	−86	−2	−4.05	18
	−2	−72	0	−3.93	18
	60	−60	8	3.92	37
	−10	−52	10	3.86	23
	−12	−74	14	−3.74	17
	12	−66	8	−3.61	17
Others	32	24	−2	−6.51	Insula
	−10	−76	−22	−6.44	Cerebellum
	−32	22	0	−6.1	Insula
	−8	−12	−2	−5.66	Thalamus
	−28	−66	−26	−5.61	Cerebellum
	36	16	6	−5.52	Insula
	−10	4	4	−5.3	Caudate
	8	−8	8	−5.26	Thalamus
	6	−22	−2	−5.23	Thalamus
	0	−50	−34	−5	Cerebellum
	8	4	4	−4.99	Thalamus
	−20	−8	−18	4.89	Amygdala
	10	−76	−26	−4.77	Cerebellum
	4	−20	−16	−4.69	Brainstem
	28	−10	16	4.39	Putamen
	−36	−46	−34	−4.38	Cerebellum
	4	−62	−28	−4.26	Cerebellum
	−40	−66	−30	−4.03	Cerebellum
	18	2	24	3.85	Caudate
	22	14	18	3.84	Caudate
	−12	−20	18	−3.78	Thalamus
	20	−78	−34	−3.52	Cerebellum

^aPositive z values indicate a signal increase in high-color coherence trials, and negative z values indicate a signal increase in low-color coherence trials. The formats are similar to those in Table 1.

To test this possibility, we first examined conjunction and disjunction of univariate maps for switch effect and negative motion coherence effect (motor and color tasks collapsed) (p < 0.05 corrected; Fig. 4A). Frontoparietal regions in the right hemisphere showed the coherence effect predominantly, but the switch effect was almost absent in the right hemisphere. On the other hand, in the left frontoparietal regions, both of the switch and coherence effects were observed, but the switch effect showed more broadly. These results suggest a double dissociation in the frontoparietal regions, with switch and coherence effects involving the left and right frontoparietal regions, respectively.

Figure 4.

Hemispheric laterality. A, Conjunction and disjunction maps. Statistical significance maps for switch and negative coherence effects (p < 0.05 corrected across the whole brain) were mapped on the 3D surface image of the brain, and then color-coded. Orange represents switch effect only. Blue represents negative coherence effect only. Yellow represents both of switch and negative coherence effects. B, Exploratory analysis procedure for hemispheric asymmetry. Contrast maps were flipped along the x axis (left-right), then subtracted from the original nonflipped maps before being entered into voxelwise group-level analyses. C, Statistical map of brain regions showing a differential coherence effect between the right and left hemispheres (top). Hot colors represent a greater signal in low-coherent trials in the right hemisphere and/or a lower signal in high-coherent trials in the left hemisphere. Cool colors represent the reverse effect. Statistical map of brain regions showing differential activation for switch versus repeat trials between the right and left hemispheres (bottom). Hot colors represent greater activation for switch versus repeat trials in the right hemisphere and/or smaller activation for switch versus repeat trials in the left hemisphere. Cool color represents the reverse effect. Formats are similar to those in B. D, ROI analysis. ROIs in the frontoparietal cortex were defined independently of the current data, and then switch and negative coherence effects were compared between the right and left hemispheres. ***p < 0.001. L, Left; R, right.

Then, we explored brain regions to show the double dissociation. We contrasted the maps of the coherence effect between the right and left hemispheres on a voxel-by-voxel basis by flipping maps along the x axis, and explored the brain regions showing a differential coherence effect between hemispheres by calculating group-level statistics (Fig. 4B; see also Materials and Methods). Figure 4C (top) and Table 5 show brain regions with the differential effect. Positive effects were observed in multiple frontoparietal regions, including the DLPFC, pre-SMA, and PPC. Technically, a positive effect indicates greater activation in low-coherent trials in the right hemisphere, and/or lower activation in high-coherent trials in the left hemisphere. These positive hemispheric laterality effects can be considered to be derived from greater activation in low-coherent trials in the right hemisphere, which was weaker in the contralateral (left) hemisphere, given the prominent coherence effects (greater activations in low-coherence trials) in those frontoparietal regions in the right hemisphere (Fig. 3C,D; Tables 3 and 4).

Table 5.

Brain regions showing significant difference in the low-coherence effect for the right hemisphere versus left hemisphere in switch and repeat trials^a

Area	x	y	z	z value	BA
Frontal cortex	0	28	42	6.98	8
	0	12	52	6.01	6
	8	26	54	5.10	8
	0	36	24	4.89	32
	38	6	52	4.84	6/8
	44	26	2	4.55	45
	10	−14	50	−4.53	6
	10	40	36	4.31	9
	48	24	−10	4.29	47
	14	20	44	4.25	8
	10	16	62	4.19	6
	30	−14	58	−4.18	6
	52	22	30	4.14	44
	34	−8	46	−3.44	6
Parietal cortex	36	−70	34	4.71	39
	38	−54	18	4.70	39
	40	−74	46	4.65	39
	48	−76	30	4.22	39
	48	−44	54	3.95	40
Temporal cortex	50	−26	−2	4.76	22
	48	−40	10	4.67	22
	60	−24	−12	4.34	21
Occipital cortex	0	−82	0	4.53	18
	14	−82	12	4.45	17
	42	−70	20	3.45	19
Others	0	−48	−34	5.91	Cerebellum
	28	−62	−24	−4.82	Cerebellum
	30	−66	−56	−4.73	Cerebellum
	0	−62	−28	4.41	Cerebellum
	34	20	8	3.84	Insula
	42	−62	−42	−3.75	Cerebellum

^aPositive z values indicate a signal increase in the right hemisphere, and negative z values indicate a signal increase in the left hemisphere. The formats are similar to those Table 1.

Similarly, to confirm the left-lateralized switch effect in frontoparietal regions, we contrasted the maps of the switching effect (switch vs repeat) between the right and left hemispheres. Figure 4C (bottom) and Table 6 show a prominent negative effect in multiple frontoparietal regions, including the IFC, DLPFC, aPFC, IFJ, and PPC. Technically, again, a negative effect indicated greater switch-related activation in the left hemisphere and/or weaker switch-related activation in the right hemisphere; however, given the greater switch related activation in the left hemisphere (Fig. 3; Table 1), the negative effects were derived from greater switch-related activity in the left hemisphere, which was weaker in the contralateral hemisphere. This result was consistent with prior studies (Konishi et al., 2002; Jimura et al., 2004).

Table 6.

Brain regions showing significant difference for the right hemisphere versus left hemisphere in contrast of switch versus repeat trials^a

Area	x	y	z	z value	BA
Frontal cortex	0	12	46	5.94	6
	0	24	32	5.37	32
	0	8	60	5.28	6
	44	42	10	−5.17	46
	48	6	18	−4.59	44
	46	8	36	−4.52	6/8
	0	36	20	4.49	32
	42	28	26	−4.38	9
	46	28	40	−4.28	9
	8	16	64	3.94	6
	28	−20	64	−3.94	6
	12	28	58	3.84	6
	0	10	30	3.78	24
Parietal cortex	56	−22	30	−5.02	1/40
	28	−70	36	−5.00	39
	56	−30	44	−4.68	40
	34	−52	46	−4.58	39
	46	−44	46	−4.47	40
	10	−76	38	−4.30	7
	18	−76	56	−3.99	7
	38	−34	36	−3.96	1/40
Temporal cortex	50	−30	−8	4.17	21
	58	−58	−6	−4.00	37
Occipital cortex	30	−72	20	−3.64	19
Others	30	−70	−48	4.15	Cerebellum

^aThe formats are similar to those in Table 5.

Finally, we performed ROIs analysis to further demonstrate the double dissociation. Frontoparietal ROIs in the left and right hemispheres were defined based on a previous study (Camilleri et al., 2018) (see also Materials and Methods); and for each of the left and right ROIs, the estimation contrast of the switch effect (switch vs repeat) and the parametric effect of the coherence were extracted for each participant. The switch effect was significant in both hemispheres (left: t₍₂₆₎ = 8.5, p < 0.001, r² = 0.74; right: t₍₂₆₎ = 5.2, p < 0.001, r² = 0.51), but was greater in the left hemisphere (left > right: t₍₂₆₎ = 5.3, p < 0.001, r² = 0.52) (Fig. 4D). The negative coherence effect (greater signal in low coherence trial) was also significant in both hemispheres (left: t₍₂₆₎ = 12.8, p < 0.001, r² = 0.86; right: t₍₂₆₎ = 10.64 p < 0.001, r² = 0.81), but conversely, the coherence effect was greater in the right hemisphere (right > left: t₍₂₆₎ = 4.7, p < 0.001, r² = 0.46). To test the robustness of the hemispheric dissociations against the definition of ROIs, we redefined ROIs based on the meta-analysis maps (Neurosynth; see Materials and Methods) (Yarkoni et al., 2011). The switch effect was significant in both hemispheres (left: t₍₂₆₎ = 8.5, p < 0.001, r² = 0.74; right: t₍₂₆₎ = 5.5, p < 0.001, r² = 0.54), but was greater in the left hemisphere than the right hemisphere (left > right: t₍₂₆₎ = 3.7, p < 0.01, r² = 0.34). The negative coherence effect was also significant in both hemispheres (left: t₍₂₆₎ = 9.8, p < 0.01, r² = 0.79; right: t₍₂₆₎ = 10.7, p < 0.001, r² = 0.81), but was greater in the right hemisphere (right > left: t₍₂₆₎ = 6.4, p < 0.001, r² = 0.61).

These collective results clearly demonstrate hemispheric asymmetries in the frontoparietal regions: task switching and perceptual decision-making are associated with the left and right hemispheres, respectively, suggesting distinctive neural mechanisms across hemispheres. No regions showed an interaction between the switch and coherence effects.

Searchlight MVPA

Given the brain regions associated with tasks to be performed, switching tasks, and perceptual demand identified by univariate activation analysis, we next explored brain regions involving task-related neural representation that may reflect appropriate neural coding of the task to be performed. More specifically, brain activity patterns were examined if the pattern involved discriminable information of motion and color tasks based on searchlight MVPA (Kriegeskorte et al., 2006). Importantly, in the current analysis, training and test data were independent; training and test data were based on 100% coherent trials and switch/repeat trials, respectively (Fig. 1B; see also Materials and Methods).

Occipitotemporal regions showed greater accuracy in the switch trials (Fig. 5A, left) but not in the repeat trials (Fig. 5A, middle). Then, to identify brain regions showing enhanced task coding during task switching, classification accuracy maps were contrasted between switch and repeat trials on a voxel-by-voxel basis (Jimura et al., 2014); and a group-level analysis was performed. The occipitotemporal regions showed a differential effect (Fig. 5A, right; Table 7). As stated above, in this region, there is a greater amount of information about performed tasks in switch trials relative to repeat trials. Of interest, this region is located spatially between the MT region (showing greater activity during motion tasks; Fig. 5B, blue) and the VVC region (showing greater activity during the color task; Fig. 5B, green), which were identified in a univariate activation analysis (Fig. 2B; Table 2). Thus, the occipitotemporal region adjacent to the stimulus-modality-dependent MT/VVC areas showed enhanced task coding during task switching. It is possible that enhanced task coding in this middle region reflects enhanced task coding in the MT/VVC areas during task switching (see Discussion). To examine this possibility, an ROI analysis was performed. ROIs were defined as MT and VVC regions that were identified by univariate analysis (Fig. 2B; Table 2); the classification accuracy was averaged for each of the ROIs for switch and repeat trials. The MT region showed greater classification accuracy in both the switch and the repeat trials than the chance level (switch: t₍₂₆₎ = 6.3, p < 0.001, r² = 0.60; repeat: t₍₂₆₎ = 3.1, p < 0.01, r² = 0.27), but the accuracy was greater in the switch than in the repeat trial (t₍₂₆₎ = 5.8; p < 0.001, r² = 0.42). In the VVT region, classification accuracy was greater in the switch trial (t₍₂₆₎ = 7.1, p < 0.001, r² = 0.66), but not in the repeat trial (t₍₂₆₎ = −0.2, p = 0.83, r² = 0.00), compared with the chance level. The accuracy difference was also significant (t₍₂₆₎ = 4.7, p < 0.001, r² = 0.46).

Figure 5.

Searchlight MVPA. A, Statistical significance maps of classification accuracy for motion and color tasks in searchlight MVPA. Switch trial (left), repeat trial (middle), and switch versus repeat trials (right). Hot colors represent greater accuracy in switch and repeat trials (relative to chance; left and middle) and in switch relative to repeat trials (right). Cool colors represent the reverse effect. B, The statistical maps were overlaid on a 2D section as indicated by the y axis on the left. Black solid square on the section (left) represents the area magnified on the right. Hot colors represent brain regions showing greater classification accuracy for switch trials relative to repeat trials in the searchlight MVPA. Cool and green colors represent brain regions with greater activation during motion relative to color tasks, and during color relative to motion tasks in univariate analyses, respectively. Yellow arrowheads indicate major anatomic landmarks on the 2D section at left. ITC, Inferior temporal sulcus; OTS, occipitotemporal sulcus; CLS, collateral sulcus. C, ROI analysis. For switch (SW) and repeat (RP) trials, voxelwise accuracy was averaged within MT and VVC identified by the univariate analysis in A (top), and also within the PFC and PPC ROIs defined independently of the current data (bottom). Error bars indicate SEM across participants. *p < 0.05; **p < 0.01; ***p < 0.001.

Table 7.

Brain regions showing a significant difference in accuracy to classify task dimension (motion/color) between the switch and repeat trials^a

Area	x	y	z	z value	BA
Frontal cortex	−30	0	42	4.67	8/6
	−34	−14	52	3.63	6
	−26	−4	54	3.56	6
Parietal cortex	34	−68	24	4.69	19/39
	42	−76	20	4.37	19
	46	−28	42	4.25	40
	−40	−82	22	4.10	19
	−44	−34	34	4.00	40
	58	−30	34	3.99	40
	−28	−84	22	3.92	19
	22	−74	30	3.79	19/39
	−26	−70	14	3.63	19
	−42	−28	52	3.60	40
	−34	−32	44	3.47	40
	46	−72	34	3.33	39
	−28	−40	32	3.29	40
	−40	−82	38	3.13	39/19
Temporal cortex	−52	−56	−4	4.86	37
	50	−52	−10	4.65	37
	−40	−70	6	4.43	19
	−34	−56	−16	3.91	37
	−48	−46	6	3.88	21/37
	−40	−56	−2	3.87	37/19
	−48	−62	−16	3.73	37
	56	−52	2	3.51	37/21
	42	−58	−2	3.45	19/37
	−40	−68	−8	3.38	37/19

^aPositive z values indicate higher accuracy in the switch trial. The formats are similar to those in Table 1.

We also performed an ROI analysis for the frontoparietal regions. ROI images were created similarly to the hemispheric laterality analysis, but for the frontal and parietal regions, separately (Fig. 4D; see also Materials and Methods). In the frontal ROI, the accuracy was higher than the chance level in the switch and repeat trials (switch: t₍₂₆₎ = 9.0, p < 0.001, r² = 0.75; repeat: t₍₂₆₎ = 2.3, p < 0.05, r² = 0.17), but was higher in the switch trial than in the repeat trial (switch > repeat: t₍₂₆₎ = 4.7, p < 0.001, r² = 0.45). In the parietal ROI, the accuracy was higher than the chance level in the switch and repeat trials (switch: t₍₂₆₎ = 8.4, p < 0.001, r² = 0.72; repeat: t₍₂₆₎ = 4.5, p < 0.001, r² = 0.43), but was higher in the switch trial than in the repeat trial (switch > repeat: t₍₂₆₎ = 2.3, p < 0.05, r² = 0.17). To test the robustness of the differential accuracy against the definition of ROI, we redefined ROIs based on the meta-analysis maps of Neurosynth, similar to the analysis above. Consistent results were obtained in the frontal ROI (switch: t₍₂₆₎ = 9.1, p < 0.001, r² = 0.76; repeat: t₍₂₆₎ = 1.9, p = 0.07, r² = 0.12; switch > repeat: t₍₂₆₎ = 4.4, p < 0.001, r² = 0.43) and parietal ROI (switch: t₍₂₆₎ = 10.3, p < 0.001, r² = 0.80; repeat: t₍₂₆₎ = 5.3, p < 0.001, r² = 0.51; switch > repeat: t₍₂₆₎ = 3.7, p < 0.001, r² = 0.35). These results suggest that the frontoparietal regions implicated in task switching and perceptual uncertainty also enhanced task coding in the switch relative to the repeat trials.

Effective connectivity analysis

The whole-brain exploratory analyses identified three types of brain regions: (1) occipitotemporal MT and VVC regions associated with motion and color perception, respectively; (2) lPFC associated with task switching; and (3) rPFC associated with perceptual uncertainty. In addition, searchlight MVPA identified an occipitotemporal region between MT and VVC where task information was increased during task switching. One possible mechanism to comprehensively explain these results is that the MT/VVC regions received task-switching signals from the lPFC, and the task information was complemented by signaling from the rPFC when task-relevant information involved uncertainty. To test this hypothesis, we performed interregional effective connectivity analysis for switch and coherence effects during task switching based on DCM (Friston et al., 2003), which allowed us to examine the directionality of task-related functional connectivity via the state-space model (see also Materials and Methods).

As shown in Figure 6A, the connectivity was enhanced from lPFC toward MT in the switch-to-motion trials. By contrast, connectivity in the reverse direction (i.e., from MT toward lPFC) was enhanced in repeat-motion trials. On the other hand, in color tasks, the connectivity was enhanced from lPFC toward VVC in the switch-to-color trials, and the connectivity was enhanced in the opposite direction during the repeat-color trials (Fig. 6B). These results suggest that modality-specialized occipitotemporal regions receive and send signals toward the switch-related prefrontal region, depending on whether the task was switched or repeated.

Figure 6.

Effective connectivity analysis based on DCM. A, Effective connectivity and extrinsic inputs modulated in switch trials relative to repeat trials during motion tasks. Red arrows indicate enhancements of connectivity in switch trials. Blue arrows indicate enhancements of connectivity in repeat trials. Solid and dotted lines indicate effective connectivity that is statistically significant (p < 0.05; uncorrected) and insignificant, respectively. Positive values indicate connectivity enhancements in switch trials. Negative values indicate connectivity enhancements in and repeat trials. B, Effective connectivity and extrinsic inputs modulated in switch relative to repeat trials during color tasks. The formats are similar to those in A. C, Effective connectivity and extrinsic inputs modulated by motion coherence during motion tasks. Red arrows indicate enhancements in high-coherence trials. Blue arrows indicate enhancements in low-coherence trials. Solid and dotted lines indicate effective connectivity that was statistically significant (p < 0.05; uncorrected) and insignificant, respectively. Positive values indicate connectivity enhancements in high-coherence trials. Negative values indicate connectivity enhancements in low-coherence trials. D, Effective connectivity and extrinsic inputs modulated by color coherence during color tasks. The formats are similar to those in C.

In low coherence motion trials, the connectivity was enhanced from rPFC toward MT, and from rPFC toward lPFC (Fig. 6C). In contrast, in the high-coherence motion trials, the connectivity was enhanced from MT to lPFC and rPFC, and also from lPFC to rPFC. On the other hand, in the low-coherence color trials, the connectivity was also enhanced from rPFC toward VVC, from rPFC toward lPFC, and from V4 toward lPFC (Fig. 6D). In the high-coherence color trials, the connectivity was enhanced from lPFC toward VVC and rPFC. These results suggest that rPFC has a key role in signaling the switch-related lPFC and modality-specialized occipitotemporal regions when perceptual demand increases because of the increased the perceptual uncertainty.

Together, during task switching in a situation where external perceptual information involves more uncertainty, the top-down signal from the prefrontal regions associated with task switching and perceptual uncertainty becomes stronger toward the stimulus-modality-dependent occipitotemporal regions. On the other hand, the bottom-up signal from the occipitotemporal regions to the prefrontal region becomes stronger during repeat trials. These results collectively suggest complementary prefrontal mechanisms during task switching, with perceptual uncertainty with the lPFC and rPFC for task switching and perceptual decision-making, respectively. Then, the prefrontal mechanisms increase task-related information in the occipitotemporal regions.

Discussion

The current study examined neural mechanisms during task switching in a situation where goal-relevant information involved perceptual uncertainty. The lPFC, MT/VVC, and rPFC were associated with task switching, perception of the target stimulus of an engaged task, and perceptual demands, respectively. During task switching, the lPFC signaled to the MT and VVC regions, and task-related neural pattern information in the cortical region increased in an occipitotemporal region between MT and VVC. On the other hand, when goal-relevant information involved more perceptual uncertainty, the lPFC and MT/VVC received a signal from the rPFC. These collective results suggest distributed cortical mechanisms during task switching under perceptual uncertainty, where cross-hemispheric prefrontal mechanisms complementarily signal to task-dependent perceptual regions in the occipitotemporal cortex (Fig. 7).

Figure 7.

Schematic diagrams of neural mechanisms among the lPFC, rPFC, MT, and VVC during task switching with perceptual uncertainty. Arrows indicate signal directions.

Behaviorally, the interaction effect of switching and coherence levels was not strong, as it failed to reach statistical significance. In contrast, both of the two main effects (switching and coherence) were statistically reliable. In univariate imaging analyses, the two main effects were predominantly observed across the whole brain, but the interaction effect was absent. These behavioral and imaging results suggest that the main effects of switching and coherence are dominant. Thus, for putative mechanisms for successful task switching under perceptual uncertainty, the current study suggests that distinct mechanisms for perceptual decision-making and task switching cooperatively guide successful task switching. More specifically, when target information involves uncertainty, activation in the right prefrontal region becomes greater, and the cross-hemispheric signal from the right prefrontal region toward the left prefrontal region is enhanced (Fig. 7). These results may suggest that the perceptual decision-making mechanism involves the right frontoparietal regions and helps to achieve task switching involving the left frontoparietal regions.

Previous studies of task switching have focused on perceptual decision-making and demand, where the target stimulus involved uncertainty, and revealed occipitotemporal and frontoparietal mechanisms (Kayser et al., 2010; Mante et al., 2013; Zhang et al., 2013; Kumano et al., 2016). More recent studies examined task switching involving perceptual uncertainty where the relevant task was not explicitly cued and the switch to an alternative task was guided by feedback to the response (Purcell and Kiani, 2016), and identified medial prefrontal mechanisms critical for voluntary switching (Sarafyazd and Jazayeri, 2019). In contrast, the current study focused on cue-guided switching when a perceptually ambiguous stimulus was presented. Two neural mechanisms (one associated with task switching involving the lPFC and the other associated with perceptual decision-making involving the rPFC, MT, and VVC) cooperatively guide flexible adaptation to changing environments.

DCM analyses revealed a top-down signal from the lPFC to the MT and VVC regions during task switching, depending on the task that was switched. More specifically, the lPFC signaled to the MT during a switch from the color to motion tasks, and to VVC during a switch from the motion to color tasks. This top-down mechanism may reflect supplemental attention to a visual stimulus required to collect task-related information to make a decision, with the prefrontal region complementing stimulus-modality-dependent perceptual information in the occipitotemporal region (Desimone and Duncan, 1995; Tomita et al., 1999; Kastner and Ungerleider, 2000; Miller and Cohen, 2001; Corbetta and Shulman, 2002). The current study extends this view that the top-down signal from the prefrontal to occipitotemporal regions is enhanced when the stimulus involves perceptual uncertainty, and the enhanced prefrontal signal was supplemented from the contralateral hemisphere (Fig. 7).

A searchlight MVPA analysis identified an occipitotemporal region showing enhanced task coding in the middle of the MT and VVC. It can be speculated that, during task switching, task coding was enhanced in the MT and VVC regions, which reciprocally increased the amount of discriminable task information in this middle region, which is consistent with the follow-up ROI analysis (Fig. 5B). This interpretation is also compatible with DCM results showing an enhanced top-down signal from the lPFC to the MT or VVC depending on the task to be switched. It is well known that occipitotemporal regions involve stimulus-modality specialized patchy regions (e.g., Shadlen and Newsome, 1994; Treisman and Kanwisher, 1998). The exploratory searchlight MVPA (Fig. 5A) result is dependent on these patchy characteristics; and for this result to occur, it may be important for the MT and VVC to be located relatively close (∼25 mm apart; Table 2) within the occipitotemporal region.

In the current study, the decoding accuracy in the occipitotemporal regions may be lower for visual sensory regions. One possible reason for this is that the visual stimulus was identical in the motion and color tasks, which may have attenuated the sensitivity of visual cortical areas that are sensitive to visual evidence. Another possibility is that the classifier was not optimized to classify the trials immediately after cue presentation involving perceptual uncertainty because the classifier was trained based on 100% coherence trials that do not involve perceptual uncertainty.

A recent MVPA study of task switching suggested that task coding in frontoparietal regions is attenuated in a switch trial (Qiao et al., 2017), whereas another study suggested that task coding is independent of task switching (Loose et al., 2017). On the other hand, in the current study, the classification accuracy in the frontoparietal regions was greater in the switch trial. The variability of accuracy patterns in the previous and current studies may be because of the differences in behavioral design and cognitive demands of task switching. In previous studies, a task-cueing paradigm was used, where a task cue was presented in each trial (Loose et al., 2017; Qiao et al., 2017); conversely, the current study used an intermittent cue paradigm in which the switch trial occurred after successive correct trials for the alternative task without presenting a cue (Fig. 1; see also Materials and Methods). In the latter paradigm, the cue stimulus is presented less frequently, and participants actively maintain the task set for a longer period until the next cue is presented. Thus, in the switch trial, the effect of a cue presentation of the previous task is diminished, but the interference of the previous task set becomes greater (Sakai, 2008). The greater demand in the switch trial and the weaker effect of cue presentation of the previous task may be attributable to the greater classification accuracy of the switch trial observed in the current study.

Several previous studies have shown that switching alternative tasks was associated with the frontoparietal region in the left hemisphere (Konishi et al., 1998, 2002; Dove et al., 2000; Rushworth et al., 2002; Braver et al., 2003; Crone et al., 2006; Brass and von Cramon, 2004; Asari et al., 2005; Derrfuss et al., 2005; Yeung et al., 2006; Jimura and Braver, 2010; C. Kim et al., 2011, 2012). In the current study, when a stimulus involved perceptual uncertainty, prefrontal activity in the contralateral hemisphere became stronger, which may supplement top-down signaling to stimulus-modality-dependent occipitotemporal regions (Fig. 7), suggesting a compensatory mechanism across hemispheres. Indeed, such contralateral compensation has been reported in neuropsychological studies of patients with hemispheric stroke (Buckner et al., 1996; Cramer et al., 1997; Gold and Kertesz, 2000), which may be compatible with the complementary activation and top-down signaling under perceptual uncertainty in the current study. Interestingly, it is suggested that the right prefrontal and parietal regions are associated with stimulus-driven attention (Corbetta and Shulman, 2002), consistent with increased perceptual demand during low-coherence trials in the current study.

Previous electrophysiological studies have shown that MT activity becomes greater when subjects are perceiving random dot motion with greater coherence (Britten et al., 1992, 1993; Britten and Newsome, 1998; Mazurek et al., 2003). On the other hand, similarly to the current study, under conditions where visual stimuli involved both color and motion perceptual uncertainty, MT activity increased in high-motion coherence trials when subjects did not attend to the motion (Kayser et al., 2010); this was consistent with the previous electrophysiological studies (Britten et al., 1992, 1993; Britten and Newsome, 1998; Mazurek et al., 2003). However, when subjects did attend to motion, the direction of motion coherence modulation reversed, with MT activity becoming greater in low-motion coherence trials, suggesting top-down signaling from the frontoparietal regions (Kayser et al., 2010). The current study suggests that the top-down signal is enhanced during task switching in low-coherence trials, and that the top-down signal is associated with supplemental attention to task-relevant information when the stimulus involves multiple dimensional uncertainties (Fig. 7). Notably, consistent with a previous study (Kayser et al., 2010), in the current study, such enhanced MT activation was not observed during high-coherence trials, but rather MT activity increased in low-coherence trials (p < 0.05 corrected within small volume in the MT region defined with contrast of motion vs color trials in a univariate analysis).

In conclusion, the current study elucidated one aspect of executive control. Specifically, behavioral flexibility and perceptual decision-making interact to guide goal-directed behavior, which is implemented as a distributed cortical network in the frontal and occipitotemporal regions.