Multi-level decoding of task sets in neurophysiological data during cognitive flexibility

Summary

Cognitive flexibility is essential to achieve higher level goals. Cognitive theories assume that the activation/deactivation of goals and task rules is central to understand cognitive flexibility. However, how this activation/deactivation dynamic is implemented on a neurophysiological level is unclear. Using EEG-based multivariate pattern analysis (MVPA) methods, we show that activation of relevant information occurs parallel in time at multiple levels in the neurophysiological signal containing aspects of stimulus-related processing, response selection, and motor response execution, and relates to different brain regions. The intensity with which task sets are activated and processed dynamically decreases and increases. The temporal stability of these activations could, however, hardly explain behavioral performance. Instead, task set deactivation processes associated with left orbitofrontal regions and inferior parietal regions selectively acting on motor response task sets are relevant. The study shows how propositions from cognitive theories stressing the importance task set activation/deactivation during cognitive flexibility are implemented on a neurophysiological level.

Graphical abstract

Highlights

• •Stimulus-related, motor, and response selection aspects of task set were decoded
• •Activation of task rule information occurs at multiple neurophysiological levels
• •Activation and deactivation of rule sets contributes to cognitive flexibility

Biological sciences; Neuroscience; Behavioral neuroscience; Cognitive neuroscience

Introduction

To achieve higher level goals, it is not only essential to keep focus but cognitive flexibility is another important aspect which has extensively been investigated in cognitive neuroscience using various experimental approaches. One prominent approach is to use task switching paradigms (Allport and Wylie, 2000; Jersild, 1927; Monsell, 2003). The canonical finding in this task is that whenever a response has to be switched, the response times increase and responses become more error-prone than non-switch responses (Monsell, 2003). There has been an intense debate about the cognitive mechanisms underlying these switch costs (Kiesel et al., 2010). Some studies propose that switch costs reflect additional time needed to reconfigure the current task set; other findings suggest that interference between simultaneously active task sets plays a role (Allport and Wylie, 2000; Allport et al., 1994; Goschke, 2000; Grange and Houghton, 2014; Meiran, 2000; Wylie and Allport, 2000). Moreover, such interference in task sets can emerge at stimulus processing levels as well as response selection and motor levels (Goschke, 2000; Kiesel et al., 2010; Koch and Allport, 2006; Philipp et al., 2007; Rogers and Monsell, 1995; Steinhauser and Hübner, 2009; Vandierendonck et al., 2010). The common theme in all these possible explanations is that the degree of activation of the task set at a particular point in time is central to cognitive flexibility. The rapid activation of a task set during switching may require the inhibition (deactivation) of the no longer relevant task representation (Dajani and Uddin, 2015; Klimesch, 2011; Mayr and Keele, 2000; Wolff et al., 2017a; Zhang et al., 2016a, 2016b). Therefore, it is crucial to understand the temporal activation/deactivation of task sets and how this is implemented on a neurophysiological level.

Notably, the question of how the activation/deactivation of task rules are implemented on the neurophysiological level is still mostly elusive and therefore reflects a critical theoretical gap in knowledge. Only using more recent multivariate pattern analysis (MVPA, originally known as multi-voxel pattern analysis when applied to fMRI data, Haxby et al., 2001), the precise representational dynamics (i.e., the activation and deactivation of rules) can be analyzed (Carlson et al., 2019; Fahrenfort et al., 2018; Grootswagers et al., 2016; King and Dehaene, 2014). This is because these methods allow time-resolved decoding of mental representations from neural activity, including changes of patterns observed in the EEG (King and Dehaene, 2014). Specifically, classifiers can be trained by using the difference of the neurophysiological codes between experimental conditions (Fahrenfort et al., 2018; King and Dehaene, 2014). When decoding performance is represented as a function of time, it allows interpretation about the timescale of the neurophysiological activity related to the encoded information (King and Dehaene, 2014). That is, applying this type of analysis to the time series data of EEG, it is possible to determine when the information is represented in the brain and how long it is maintained. Then, the nature of the identified representation can be specified by testing the generalizability of the classifier to other time points. This so-called temporal generalization analysis (Fahrenfort et al., 2018; Grootswagers et al., 2016; King and Dehaene, 2014) seems especially useful when examining the neural dynamics of task set activation/deactivation, because this method provides a metric for the stability of mental representations over time. For instance, gradual changes in the strength of the decoded pattern or recurring representations can be detected (King and Dehaene, 2014). When depicting temporal generalization along the axes of training and testing times, the diagonal represents training and testing on the same time points (Fahrenfort et al., 2018; King and Dehaene, 2014). A diagonal pattern has been considered as the representation of a series of processing stages (King and Dehaene, 2014). Moreover, other representational dynamics can be found as decoded patterns of different training and testing time points (hence, off-diagonal decoding). Importantly, significant below-chance decoding can be evident in MVPA results, which has been suggested to reflect a representation’s deactivation (Carlson et al., 2011; King and Dehaene, 2014). Therefore, the MVPA decoding pattern will explain how the activation and deactivation are implemented on the neurophysiological level and its temporal dynamics.

In the current study, we use EEG-based MVPA to examine the dynamic of task set activation/deactivation on a neurophysiological level and combine this with source localization analyses. Since MVPA requires a comparison of conditions (Fahrenfort et al., 2018), it is essential to maximize the difference in the neuronal dynamics between conditions. One way of doing so when interested in the importance of executive control in the activation/deactivation of task sets is to compare switching between trials requiring either ‘endogenous’ switching or ‘exogenous’ switching (Arrington and Logan, 2004, 2005; Terry and Sliwinski, 2012). During exogenous task selection, participants receive external cues before each trial indicating the rule to be applied or activated (Meiran, 1996). Endogenous task switching is more complicated/effortful since task sets retrieval processes are not supported by external stimuli (i.e., cue information). Instead, one has to use endogenous prompts to retrieve and activate relevant task sets. This procedure requires more active control to perform switching (Arrington and Logan, 2004, 2005; Emerson and Miyake, 2003; Kray, 2006; Mayr and Bell, 2006) and makes task switching difficult and slower (Gajewski et al., 2011; Wolff et al., 2017b). Therefore, decoding switching between exogenous (cue-based) and endogenous (memory-based) trials should be particularly useful when examining the neural dynamics of the activation/deactivation of task sets during task switching.

As mentioned, evidence suggests that relevant dynamics in task sets can emerge at the stimulus processing level, the response selection level, or the motor levels (Goschke, 2000; Kiesel et al., 2010; Koch and Allport, 2006; Philipp et al., 2007; Rogers and Monsell, 1995; Steinhauser and Hübner, 2009; Vandierendonck et al., 2010). This theoretical distinction is critical when examining the activation/deactivation dynamics on a neurophysiological level using EEG because EEG-signals reflect a mixture of different activity sources (Huster et al., 2015; Nunez et al., 1997; Stock et al., 2017). Especially in tasks taxing executive control, stimulus processing, response selection, and motor processes are intermingled in the EEG signal at virtually the same time and in almost overlapping brain structures (Chmielewski et al., 2018; Folstein and Van Petten, 2008; Mückschel et al., 2017a). To examine the dynamics of the task set activation/deactivation using MVPA we, therefore, do this after decomposing the EEG signal into theoretically relevant fractions of informational content. Residue iteration decomposition (RIDE) (Ouyang and Zhou, 2020; Ouyang et al., 2011, 2015, 2017) can be used to distinguish different aspects of information processes in the neurophysiological signal that is processed simultaneously in overlapping functional neuroanatomical structures in a theoretically meaningful way (Mückschel et al., 2017a, 2017b; Opitz et al., 2020; Takacs et al., 2020a). RIDE decomposes the EEG into three clusters of dissociable significance: an S-cluster reflecting stimulus-related processes, an R-cluster reflecting motor execution processes and a C-cluster reflecting response selection mechanisms (Ouyang et al., 2011, 2015). Combining RIDE-decomposition with MVPA, it should be possible to examine the dynamics of the task set activation/deactivation in the neurophysiological signals at theoretically relevant levels relating to the role of stimulus processing, response selection level, or motor responding during task switching.

Based on the above considerations, we suppose that the task set activation/deactivation dynamics relevant to task switching are evident in all decomposed clusters of activity, but the C-cluster and the R-cluster may reflect most pertinent information. These clusters should particularly reveal temporal stability of activated task sets as indicated by significant off-diagonal activity (King and Dehaene, 2014) and evidence for deactivated task sets as indicated by below-chance decoding performance. Since the task set’s inhibition is only critical during switching, such deactivation patterns should only emerge when decoding endogenous (memory-based) versus exogenous (cue-based) switching trials, but not when distinguishing between repetition trial types. For time periods yielding relevant activation/deactivation dynamics, we conduct source localization analyses to examine what functional neuroanatomical structures are likely to be associated with them.

Results

We describe behavioral and MVPA results obtained from an experiment that required cognitive flexibility. Participants had to perform a task switching (Gajewski et al., 2011; Petruo et al., 2018, 2019; Wolff et al., 2016), which consists of two different blocks, a cued and a memory-based block. In both blocks, three task rules must be applied interchangeably. For this purpose, either an informative cue (cued block, indicated with ‘NUM’, ‘PAR’, or ‘FS’ depending on the task rule at hand) or a non-informative cue (memory-based block, indicated with a‘XXX’ dummy cue) was presented (see Figure 1). In each trial, one of eight numbers between 1 and 9 (except 5) were shown, to which a motor response had to be executed according to the corresponding pre-defined rule. The task rule ‘numeric’ (indicated with a ‘NUM’ cue) required a decision on whether the presented digit is less than or greater than 5. The task rule ‘parity’ (indicated with a ‘PAR’ cue) required whether the given digit is an even or odd number. The task rule ‘font-size’ (indicated with an ‘FS’ cue), required the question of whether the digit is written in a small or large font. In the memory-based block, participants were always instructed to follow the same order of task rules of 3 × ‘NUM’, 3 × ‘PAR’, and 3 × ‘FS’, i.e., NUM, NUM, NUM, PAR, PAR, PAR, FS, FS, FS, NUM, NUM, NUM, PAR, etc. Here, the order of the rules must be retrieved independently by the participant. Participants were asked to respond to target stimuli within 2500 ms. Responses were followed by a display of a black screen for 500 ms. Then, a feedback stimulus was displayed for 500 ms. A plus sign indicated a correct response while a minus sign indicated an incorrect response. The cues (and, correspondingly, the task rules) were presented in a randomized order. For detailed information about the task structure, see STAR Methods.

Figure 1.

Schematic illustration of the trials in the cue-based and the memory-based conditions

Behavioral results

The behavioral data were analyzed using repeated measures ANOVAs with trial type (cued versus memory), rule (repetition versus switch), and motor response (repetition versus switch) as within-subject factors. The mean and SEM are given. For the response accuracy data (percent of correct responses), there was a main effect rule repetition versus switch (F(1,85) = 21.54; p< .001; η_p²> .202) showing that there were more correct responses in rule repetition trials (94.83% ± 0.46) than rule switch trials (93.62% ± 0.54). The trial type by motor response interaction was significant (F(1,85) = 4.18, p = .044, η_p² = .047). There was also an interaction “rule x motor response” (F(1,85) = 6.15, p = .015, η_p² = .068) where it was shown that accuracy was higher when the response was repeated compared to switched responses when the rule was also switched (t(85) = 3.06; p = .003). No such response switch effect was obtained when the rule was repeated (t(85) = 0.66; p = .507).

For the reaction times (RTs), there was a main effect of trial type (F(1,85) = 24.70, p< .001, η_p² = .225). Participants responded faster for cued (720 ms ± 19) than for memory-based trials (760 ms ± 18). The main effect rule was also significant (F(1,85) = 123.60, p< .001, η_p² = .593), showing that responses were faster for rule repetitions (709 ms ± 17) than for rule switches (771 ms ± 19). The main effect motor response (F(1,85) = 4.54, p = .036, η_p² = .051) revealed that RTs were shorter when the motor response was repeated (736 ms ± 18), compared to when it was switched (744 ms ± 18). There was an interaction “trial type x rule” (F(1,85) = 39.98, p< .001, η_p² = .320). We compared switch costs (i.e., the difference between task repetition and task switch) between cued and memory trial types to analyze the interaction effect. Switch cost was larger in memory (87 ms ± 7) than in cued trials (36 ms ± 6) (t(85) = 6.32, p< .001). Thus, participants showed behavioral cost of alternating rules. This was larger when they had to remember the task’s rules instead of having cues to remind them. The behavioral results pattern replicates finding from multiple previous studies using the same task switching paradigm (Gajewski et al., 2011; Petruo et al., 2018; Wolff et al., 2017b).

Again, there was also an interaction “rule x motor response” (F(1,85) = 11.26, p = .001, η_p² = .117). It is shown that RTs were shortest when rule and motor response were repeated (698 ms ± 17). When the rule was repeated but the motor response switched, RTs were significantly longer (719 ms ± 17) (t(85) = 3.81; p< .001). RTs were longest when the rule was switched with no difference depending on whether the motor response was repeated (773 ms ± 20) or switched (769 ms ± 19) (t(85) = 0.81; p = .418). This is important for the interpretation of the MVPA findings, as outlined in the discussion section. Figure 2 presents RTs as a function of trial type and rule.

Figure 2.

Individual and group reaction times as a function of trial type (cue versus memory) and rule (task repetition versus task switching)

Group level RTs are presented as box plots, where the central vertical bar denotes the median and the black cross the mean. The bottom and top edge denote the 25^th and 75^th percentile, the whiskers depict 1 times the interquartile range, and values outside of this range are marked by a red plus. Individual RTs are presented as scatterplots. Individual data points are organized according to the four subsamples (marked as red, blue, green, and purple dots, for details, see participants section).

Multivariate pattern analysis results

We first introduce the decoding accuracy results of the classification performance. Then, we describe the temporal generalization matrices. Classification and temporal generalization analyses were performed consecutively for the task repetition in C-cluster, task repetition in R-cluster, task repetition in S-cluster, task switching in C-cluster, task switching in R-cluster, and task switching in S-cluster, respectively. Thus, we provide the results separately for the C-, R-, and S-cluster data for each classification (Cued task repetition versus Memory-based task repetition and Cued task switching versus Memory-based task switching). When significant above- or below-chance decoding was detected, the corresponding data was investigated for localizing the signal. That is, if MVPA indicated an above-chance classification between 100 ms and 200 ms after the target presentation in the S-cluster data of task repetition, the same time window in the averaged S-cluster task repetition was used for sLORETA analysis. Similarly, the time windows indicated by significant decoding at the group level were used to compare decoding accuracy between classifications (task repetition versus task switching) and to investigate correlations between neurophysiological classification and behavioral results. Specifically, individual AUC values averaged over significantly above-chance time windows have been compared between task repetition and task switching classifications, separately for each RIDE cluster. Finally, correlations between individual AUCs averaged over significantly above-chance or below-chance time windows and behavioral measures of switch cost have been reported.

Task repetition

Figure 3 presents the decoding performance and temporal generalization matrices. In the C-cluster, the classification was significantly above chance from −200 ms before the stimulus presentation to 75 ms after that, and anterior cingulate (BA32) and extrastriate visual areas (BA18) were associated with that. Additionally, the classification was significantly above the chance level from 150 ms to 1000 ms with superior parietal cortex (BA7) activity being associated with that. In the R-cluster, the classification was significantly above chance from −200 ms before the stimulus presentation to 75 ms after that, similarly to the initial successful classification time window in the C-cluster. Supplemental motor areas were associated with it (BA6). Moreover, in the R-cluster, a second above-chance activity started later, from 250 ms after the stimulus presentation, and it lasted until the end of the trial with the insular cortex being associated with it (BA13). In the S-cluster, decoding above-chance level was possible in the time windows of −200 ms–660 ms relative to stimulus presentation, and extrastriate visual areas (BA18) were associated with that. Thus, in all three RIDE-clusters, the MVPA provided successful classifications, suggesting that sustained neural activity can be observed at all three isolated coding levels in the EEG signal.

Figure 3.

MVPA results for repetition condition

RIDE decomposed C-, R-, and S-cluster EEG data for the classification of cue versus memory block. Left: Temporal generalization plot. The plots show the degree to which the classifier when trained on a given time point (yaxis) generalizes to time points in the trial (xaxis). The colors indicate the classifier performance. The diagonal (bottom left to top right) shows classification performance when the classifier is trained and tested simultaneously—middle: Area under the curve (AUC) decoding accuracy across time. Time zero denotes the presentation of the target stimulus. Thicker lines indicate significant time windows (p < 0.05; two-sided cluster-based permutation). Right: results of sLORETA source localization for significant time windows as indicated by roman numbers in AUC plots. For the C-cluster, activity differences (against zero) were found in BA18 and BA32 for the time window I and BA7 for time window II. For the R-cluster, sources in BA6 for time window I and BA13 for time window II were found. For the S-cluster, the sLORETA revealed sources in BA18 and BA19.

Considering the temporal generalization matrix, the C-cluster showed that above-chance decoding performance ranging from −200 to 75 ms had a squared shape with the diagonal axis in its center, and this decoding accuracy had faded after the stimulus presentation, as indicated by the jittered edges. At the same time, this activation did not include the diagonal axis itself (i.e., when training and testing on the same time points). It was followed by another above-chance decoding starting from 150 ms after the stimulus presentation. This chain-like pattern along the diagonal was extended with jittered edges and showed the most considerable activity between 200 and 600 ms before it continued declining toward the trial’s end. Similarly, the R-cluster's first above-chance decoding was characterized by a diagonally centered shape whose decoding performance attenuated after the stimulus was presented. Also, this pattern did not include the area right above the diagonal axis. Then, a chain-like above-chance decoding was observed after 250 ms on the diagonal axis. This activation was extended further from the axis with a gradually changing generalizability. In the S-cluster, above-chance decoding of a single time window was characterized by a combination of three identified patterns–a chain-like decoding between −200 ms and 660 ms along the diagonal axis and two extended generalization patterns: from −200 ms to 75 ms and from 200 ms to 600 ms.

The first of these diagonal clusters showed the same fading after the stimulus presentation as it was already observed in the C-cluster and R-cluster data. The second diagonal pattern showed the most extensive activation in the same window as in the C-cluster data. Thus, the C-, R-, and S-cluster data showed complementary but distinctive activity patterns.

Task switching

Figure 4 presents the decoding performance and temporal generalization matrices. In the C-cluster, the classification was significantly above chance in the 200 ms interval before the stimulus presentation and parietal regions (BA7), as well as inferior parietal areas (BA40), were associated with this activity. Additionally, the classification was significantly above the chance level from 190 ms until the end of the trial and inferior parietal regions (BA40), and medial frontal areas (BA10) revealed activity. In the R-cluster, the classification was significantly above the chance level from −200 ms to 25 ms associated with the supplementary motor area (SMA, BA4), and from 300 ms to the end of the trial, which was again associated with activity SMA activity (BA6). Interestingly, there was a significantly below-chance (negative) activation between 75 ms and 190 ms after the stimulus presentation that was associated with activity in the inferior parietal cortex (BA40), left orbitofrontal activity (BA11), and right temporo-occipital activity (BA21). In the S-cluster, the activity was above chance level in the time window of −200 ms–575 ms relative to stimulus presentation, associated with superior parietal cortex (BA7) activity. Together, the MVPA showed successful classifications for the RIDE clusters, indicating that task sets used during task switching can be decoded at multiple levels in the neurophysiological signal.

Figure 4.

MVPA results for switching condition

RIDE decomposed C-, R-, and S-cluster EEG data for the classification of cue versus memory block. Left: Temporal generalization plot. The plots show the degree to which the classifier when trained on a given time point (yaxis) generalizes to time points in the trial (xaxis). The colors indicate the classifier performance. The diagonal (bottom left to top right) shows classification performance when the classifier is trained and tested simultaneously—middle: Area under the curve (AUC) decoding accuracy across time. Time zero denotes the presentation of the target stimulus. Thicker lines indicate significant time windows (p < 0.05; two-sided cluster-based permutation). Right: results of sLORETA source localization for significant time windows as indicated by roman numbers in AUC plots. For the C-cluster, activity differences (against zero) were found in BA7 and BA40 for the time window I and BA10 and BA40 for time window II. For the R-cluster, sLORETA revealed sources in BA4 for time window I, in BA11, BA21, BA40 for time window II and BA6 for time window III. For the S-cluster, activity differences were found in BA7.

Following the decoding of the accuracy results, the C-cluster temporal generalization matrix showed that the first time window of decoding (−200 to 0 ms) developed around but did not include the diagonal axis. While fading of decoded information was visible in the matrix, this was already outside of the successfully decoded interval. The C-cluster’s temporal generalization pattern in the second above-chance decoding followed a chain-like pattern from 190 ms after stimulus presentation with an extended diagonal shape. This extended activation was the largest between 300 ms and 500 ms; then it declined toward the trial’s end. The task switching R-cluster's temporal generalization showed a diagonal pattern from −200 ms prior stimulus presentation to 25 ms after that. This above-chance decoding did not include the diagonal axis and started fading after the stimulus was presented. Additionally, a second above-chance decoding was detected from 300 ms to 1000 ms. A consistent chain-like pattern over the diagonal axis was extended in a ramping fashion. Notably, between the two above-chance decodings of the task switching R-cluster, activation time window (75 ms–190 ms after the stimulus presentation) with below-chance decoding was also detected with a chain-like pattern over the axis. The single time window of above-chance decoding in the S-cluster can be described as a combination of three patterns: a chain-like decoding between −200 ms and 575 ms, with two jittered extensions from −200 ms to 0 ms and from 300 ms to 600 ms.

Comparison between task repetition and task switching

The time windows identified above were selected for each participant and for each classification. Then, within these intervals, the individual AUC values were averaged to facilitate comparisons between different MVPA results. The rule (task repetition versus task switching) by time window (first versus second) ANOVA on the individual AUCs of the C-cluster data showed that the main effect of rule was significant (F(1,85) = 84.79, p< .001, η_p² = .499). Decoding performance was higher in task repetition (.84 ± .01) than in task switching (.70 ± .02). Similarly, the main effect of time window was significant (F(1,85) = 54.89, p< .001, η_p² = .392). Decoding performance was higher in the second time window (.80 ± .01) than in the first one (.74 ± .02). Finally, the rule by time window interaction was also significant (F(1,85) = 6.89, p = .010, η_p² = .075). Decoding accuracy was higher for task repetition than for task switching both in the first time window (.80 ± .02 vs .68 ± .02, p< .001) and in the second one (.87 ± .01 vs .72 ± .02, p< .001). Additionally, for both rules, the second time window yielded higher accuracy than the first one (ps < .001). The rule (task repetition versus task switching) by time window (first versus second) ANOVA on the individual AUCs of the R-cluster data showed that the main effect of rule was significant (F(1,85) = 70.79, p< .001, η_p² = .454). Decoding performance was higher in task repetition (.80 ± .02) than in task switching (.68 ± .02). Similarly, the main effect of the time window was significant (F(1,85) = 78.73, p<.001, η_p² = .481). Decoding performance was higher in the first time window (.77 ± .02) than in the second one (.71 ± .02). However, the rule by time window interaction was not significant (F(1,85) = 1.93, p = .169, η_p² = .022). In the S-cluster, AUCs were higher for task repetition (.73 ± .01) than for task switching (.65 ± .01) (t(85) = 3.81; p< .001,d = CI[.073; .107]). Thus, the C-, R-, and S-cluster data could be distinguished from each other based on their decoding performance (and temporal generalization matrices, see Figures 3 and 4), indicating that the temporally decomposed clusters are sensitive to different aspects of the task set dynamics. To analyze the exact relationship between task dynamics and MVPA results, correlations were run between AUC values and behavioral switch cost measures. Importantly, switch cost in the cued block correlated negatively with decoding performance in the first (r = −.274, p = .011, CI [-.462;−.062]) and second (r = −.287, p = .007, CI [-.474;−.069]) above-chance decoding of task switching rule in the R-cluster. Additionally, switch cost in the memory block correlated negatively with decoding performance in the first (r = −.340, p< .001, CI [-.527;−.131]) and second (r = −.379, p< .001, CI [-.549;−.193]) above-chance decoding of task switching rule in the R-cluster. Thus, higher above-chance decoding accuracy of task switching in the R-cluster was related to smaller switch cost both in the cued and memory blocks. None of the other pairs showed significant correlations between individual AUCs and switch cost measures (p> .105), for detailed results, see Figure S2).

Discussion

The current study’s goal was to investigate the task set activation/deactivation dynamics during cognitive flexibility. The study was motivated by considerations that the content of task sets and the deactivation of no longer relevant content is a central element in theoretical accounts on task switching and cognitive flexibility, but has nevertheless not directly been examined in neurophysiological (EEG) data thus far. To close this theoretically significant knowledge gap, the study combined temporal EEG signal decomposition (RIDE) with MVPA, here, also focusing on temporal generalization analysis (King and Dehaene, 2014). Signal decomposition was used because theoretical considerations suggest that task sets’ relevant activation/deactivation dynamics can occur at the stimulus processing level, the response selection and motor levels (Goschke, 2000; Kiesel et al., 2010; Koch and Allport, 2006; Philipp et al., 2007; Rogers and Monsell, 1995; Steinhauser and Hübner, 2009; Vandierendonck et al., 2010). Using RIDE, it is possible to distinguish such coding levels in neurophysiological data in a theoretically meaningful way (Mückschel et al., 2017a, 2017b; Opitz et al., 2020; Takacs et al., 2020a; Wolff et al., 2017b).

‘Quasi-parallel’ activation of task sets at different neurophysiological levels

Participants either had to memorize a simple sequence of task rules or recall the relevant rule based on cue information. The first one corresponds to endogenous task switching in which flexibility of response selection is supported by existing knowledge. The second one can be described as exogenous task switching which requires a more dynamic, trial-by-trial adaptation. We have used these two types of conditions for classification, separately for task repetition and task switching, and for each temporally decomposed clusters, respectively. Thus, decoding results should depict the difference in exogenous and endogenous task representations when a mental set had to be maintained (repetition) or altered (switching). Significant MVPA decoding was evident in all isolated RIDE-clusters of neuronal activity during repetition and switch trials; i.e., in the S-cluster, the C-cluster, and the R-cluster. According to Ouyang et al. (2011, 2015), the S-cluster reflects stimulus-related processes; the R-cluster reflects motor execution processes, and the C-cluster reflects response selection mechanisms. Thus, the MVPA data show that relevant task sets can be decoded at multiple coding levels in the neurophysiological signal; i.e., during stimulus-related processing, response selection, and motor response execution. This is of particular theoretical importance since it has been suggested that interferences between task sets can emerge at the stimulus processing level, or the response selection and motor levels (Goschke, 2000; Kiesel et al., 2010; Koch and Allport, 2006; Philipp et al., 2007; Rogers and Monsell, 1995; Steinhauser and Hübner, 2009; Vandierendonck et al., 2010). The current data suggest that task sets dynamics at all of these levels are relevant to task switching. As shown in Figures 3 and 4 (cf. AUC plots), the C-cluster and the R-cluster show highly similar time courses of significantly decoded activity (cf. AUC plots). Both, the C-cluster and the R-cluster reveal task set activations before stimulus presentation and 200 ms after stimulus presentation until the end of the trial. This can be interpreted that the task set is not only evident at different levels of information coded in neurophysiological activity, but that activated task sets at these different neurophysiological coding levels are evident at least partly in parallel in time. However, the temporal generalization results provide further information about this dynamic. This data revealed a combined diagonal and the off-diagonal activity pattern. The C-cluster showed the largest activity between 200 and 600 ms in repetition and switching trials and revealed a jittered diagonal and off-diagonal shape fading ∼400 ms after stimulus presentation. Opposed to this, the R-cluster showed a more ramping diagonal and off-diagonal shape activity pattern from ∼400 ms onwards in repetition and switch trials. Several lines of evidence suggest that the C-cluster reflects stimulus-response translation or response selection processes in the neurophysiological signal (Ouyang et al., 2015, 2017; Takacs et al., 2020a; Verleger et al., 2014; Wolff et al., 2017b). The R-cluster has been conceptualized to reflect processes associated with the motor execution (Ouyang et al., 2015). Therefore, it is reasonable that the temporal generalization results show a pattern in which the task rule on the stimulus-response mapping is activated before the task set detailing the motor execution process and that the motor execution task set activation starts ramping in when the activity of the task set detailing stimulus-response mapping fades out. Hence, the content of task sets at different neurophysiological coding levels is activated partly in parallel in time, but the intensity with which these various aspects of task sets are activated dynamically fades in and out. Thereby ‘quasi-parallel’ activations of response selection and motor execution task sets emerge at the neurophysiological level. An off-diagonal activity pattern in the R-cluster and the C-cluster was evident for about 300–400 ms in repetition and switch trials and suggested that the activation is relatively stable. Opposed to this, in the S-cluster off-diagonal activity was much less pronounced suggesting visual stimulus task set activations are not held online to the same extend as it is the case for stimulus-response relations or motor execution plans. This may be interpreted that stimulus information is only relevant to trigger processes reflected in the C-cluster and R-cluster activity pattern. Corroborating this interpretation, diagonal activity suggesting for a chain-like processing of information (King and Dehaene, 2014) was only evident for about 300 ms (between ∼200 and 500 ms). Diagonal activity in the C-cluster and the R-cluster was evident much longer showing this activated content is processed for longer time periods. Notably, the S-cluster main activity pattern was observed in a time period where also the C-cluster activation was strongest (i.e., between ∼200 and 500 ms). As mentioned, processes in the C-cluster have been suggested to reflect stimulus-response translation or selection processes (Ouyang et al., 2015, 2017; Takacs et al., 2020a; Verleger et al., 2014; Wolff et al., 2017b). This, however, necessitates that stimulus information is available, which is clearly evidenced by the activity pattern observed in the S-cluster.

The dynamics of quasi-parallel activations of the C-cluster and the R-cluster task sets discussed above different brain structures are essential. For the C-cluster, mostly superior and inferior parietal regions (BA7, BA40) and medial frontal regions (BA10, BA32) were activated in repetition and switch trials during the pre-stimulus time window and the time window from ∼200 ms onwards. These source location results seem reasonable considering R-cluster and C-cluster activity’s functional relevance and fit with the broader literature on task switching. The C-cluster has previously been shown to be associated with task switching effects in the very same paradigm (Wolff et al., 2017b), which is corroborated by the current study’s findings. Since the C-cluster likely reflects stimulus-response translation (response selection) processes (Ouyang et al., 2015, 2017; Verleger et al., 2014) the source localization findings are also in line with a theoretical proposition derived from imaging data that (inferior) parietal regions are central for mechanisms updating internal task sets during response selection (Geng and Vossel, 2013). Moreover, other evidence shows that inferior and posterior parietal cortical regions are involved in task switching (Armbruster et al., 2012; Cooper et al., 2016; Kubanek and Snyder, 2015; Liu et al., 2015; Petruo et al., 2019; Philipp et al., 2013; Vallesi et al., 2015; Yin et al., 2015; Zhang et al., 2016a, 2016b). Since response selection processes are a relevant function of medial frontal areas (Ridderinkhof et al., 2004; Rushworth et al., 2004, 2005; Shenhav et al., 2013), that finding is reasonable. In the R-cluster, particularly superior frontal pre-motor/supplementary motor areas (BA4, BA6) were activated in repetition and switch trials during the pre-stimulus time window and the time window from ∼200 ms onwards. This is well in line with the R-cluster's conceptualization to reflect execution processes of motor programs (Ouyang et al., 2015) and with studies reporting involvement of these regions during task switching and cognitive flexibility (Cutini et al., 2008).

Besides the activations of task-related representations after the target stimulus has been presented, all three clusters showed significant decoding that started before the trial. In case of the S-cluster, pre- and post-stimulus activities could not be separated from each other. However, in the C-cluster and R-cluster data, the representations from the pre-stimulus time window were not sustained after the start of the trial, as evidenced by the steep drop in decoding accuracy. It is conceivable that information decoded in the inter-trial time windows represent rather general differences between cued and memory-based task switching, such as additional cognitive processes in one condition. For instance, the difference between maintaining the sequence of task rules (in memory block) and monitoring the cue for selecting the appropriate mental set (in cued block) might be reflected by the pre-stimulus decoding. However, any interpretation of the pre-stimulus decoding remains tentative at this point, especially considering the fact, that a pre-stimulus time-window might be sensitive to filtering criteria (van Driel et al., 2021). The fact that this general activity was separable from post-stimulus decoding in the C-cluster and R-cluster data lends support to interpret the post-stimulus representations in these two clusters as the activation of task set.

Finally, the MVPA results of the undecomposed EEG data (see Figure S1) showed both similarities and dissimilarities with the RIDE-clusters’ classification. Specifically, while pre-stimulus decoding was not identified, broad above-chance decoding in the post-stimulus period suggested stable task-related representations both in task repetition and switching conditions. Interestingly, the temporal generalization matrices resembled features of all the three RIDE-clusters albeit with less specificity. Moreover, given the lower decoding accuracy for undecomposed than for temporally decomposed data, the decomposed MVPA results can provide a more precise picture of the dynamics of task representation during cognitive flexibility.

Decoding of motor task sets: The role of inhibition and response strategies

A critical aspect reflected by the temporal generalization matrices is that the off-diagonal activity pattern in the time period from 200 ms after stimulus presentation onwards was highly comparable for switch and repetition trials. That is, maintaining an already active task set (in repetition trials) and recalling an alternative task set (in switch trials) show many similarities in terms of when and for how long a mental task representation is available. At the same time, repetition trials were decoded more accurately than switching trials. Please note that this difference could be influenced by the trial numbers entered into the MVPA (see STAR Methods). Interestingly, switch cost at the behavioral level correlated negatively with both pre- and post-stimulus above-chance decoding performance in task switching of the R-cluster but not with decoding in task repetition. Thus, at least in motor execution, the decoded neurophysiological activity in task switching was associated with inter-individual differences in cognitive flexibility. Moreover, in this regard, the time period between ∼75 ms after stimulus presentation and ∼200 ms after stimulus presentation might be of particular relevance. In this time period, the MVPA was only able to decode significant R-cluster processes in switch trials (cf. Figure 4). Importantly, and unlike all other time periods in all RIDE-clusters, below-chance activity was decoded. The source localization suggests that regions in the left orbitofrontal areas (BA11) and left and right inferior parietal regions (BA40) partly extending to superior temporal areas were active. Until now, the meaning of such below-chance decoding patterns has been a bit contentious (Carlson et al., 2011; King and Dehaene, 2014). On a descriptive level, a stable below-chance decoding indicates a reversal of the decoded neurophysiological pattern, that is, the difference between the classes is the opposite of the ones behind the above-chance decoding patterns. Here we offer two tentative explanations of how below-chance decoding in the R-cluster is related to the neurophysiological dynamics of task-switching.

It has been argued that a below-chance decoding may reflect the deactivation of a representation (Carlson et al., 2011; King and Dehaene, 2014). That is, when a no longer relevant content needs to be suppressed, a reversal is expected between the initial emergence of that representation (above-chance decoding) and the inhibitory phase (below-chance decoding). This interpretation makes sense considering cognitive theoretical propositions on processes occurring during switching, which may contribute to a behavioral difference in switching between the decoded cue-based and memory-based switching trials. Some conceptual approaches state that the activation of a new task set necessitates the inhibition of the no longer relevant task set (Dajani and Uddin, 2015; Kiesel et al., 2010; Klimesch, 2011; Mayr and Keele, 2000). Such inhibition is not required during repetition trials since the response rule from the n-1 trial is still valid. This scenario predicts that possible benefits of repeating stimulus, response, or both should be restricted to task repetitions. Note that our behavioral findings are consistent with this prediction. We obtained an interaction between rule repetition/switching and motor response repetition/switching, indicating that significant response repetition benefits were obtained if the rule/response was repeated, but all of these benefits were eliminated when a switch was included. This observation is consistent with findings from Rogers and Monsell (1995) and others. They suggest that codes of just-performed responses might still be activated in the next trial to some degree if the task repeats, but are inhibited by the process of switching to another task. The obtained MVPA data pattern lends support to this idea because significantly below-chance decoding was only evident in the switch trial decoding. This interpretation does not contradict accounts according to which the control of inference between task sets is important during switching (Kiesel et al., 2010; Vandierendonck et al., 2010) since inhibition is one means to control interference effects (Cisek and Kalaska, 2005; Friedman and Miyake, 2004; Klein et al., 2014; Ocklenburg et al., 2011; Stürmer et al., 2000; Tandonnet et al., 2011; Taylor et al., 2007). Therefore, the obtained data suggest that inhibitory control of task sets is accomplished in the period between ∼75 and ∼200 ms after stimulus presentation during switching. Intriguingly, however, this inhibition of task set was specific for the R-cluster and hence a fraction of the neurophysiological signal carrying motor-response related information. Therefore, the data might suggest that the no longer valid motor response task set is inhibited during switching.

Importantly, however, the current below-chance decoding occurred in a diagonal analysis (i.e., training and testing on the same time points (Fahrenfort et al., 2018). In this training-testing scheme, all trials were divided into equal-sized folds; into five parts in the current study. Then, one fold was used for training and the remaining four folds were used for testing to ensure that training and testing sets are independent from each other (Fahrenfort et al., 2018). This procedure was iterated five times to cover the entire dataset for training. Thus, the identified below-chance decoding is not necessarily the reversal of the two above-chance decoding patterns that were identified in task switching in the R-cluster. However, it is possible that the decoded difference between cued and memory-based switching was reversed as the task progressed. This is in line with the observation that task-switching behavior is characterized by large intra-individual variability that is even more pronounced in memory-based than in cued task-switching (Petruo et al., 2018; Wolff et al., 2017b). Moreover, in a previous study that investigated prolonged testing, the variability difference grew further between the two types of task-switching (Petruo et al., 2018), suggesting that internal switching is more sensitive to time-on-task effects, such as fatigue, than external switching. On a related note, phasic modulatory effects of the norepinephrine system was shown in memory-based but not in cued task-switching (Wolff et al., 2018a), which might contribute to the large variability in internal switching. Thus, variability effects in previous studies (Petruo et al., 2018; Wolff et al., 2017b, 2018a) point to the direction that participants' behavior was more diverse in the memory blocks than in the cued blocks, and this variability involves time-on-task effects as well (Petruo et al., 2018). How could variability differences between classes lead to below-chance decoding? During the task, it is likely, that participants maintained the order of task rules by repeating them internally (Petruo et al., 2018, 2019). However, with more practice, the active maintenance of the order is not necessary any more, and more automatized processes might guide the response preparation. Thus, early and late trials in the memory blocks might differ from each other in terms of what type of memory processes (i.e., intentional versus automatic) they involve. Consequently, the difference between internal and external switching might also differ across consecutive folds, which then contributes to a pattern reversal (below-chance decoding). Nonetheless, any change in response strategy during the task could be more reliably detected in longer testing times and with a mixed presentation of memory-based and cued trials (Petruo et al., 2018) instead of the current block-wise design. At this stage, below-chance decoding as a signal of task-set inhibition seems theoretically feasible; however, it does not completely explain the mechanisms of the pattern reversal. On the other hand, time-on-task effects, such as changes of response strategy provide a potential explanation of a reversed difference between classes; however, the proposed connection to behavioral markers remains elusive. Nevertheless, the two interpretations (task set deactivation versus changes of response strategies) are not necessarily independent from each other. Namely, if memory processes involved in task switching become more automatic, it can result in a more effortless task-switching and, therefore, it can make the deactivation of previous task sets easier.

Of note, decoding performance in the below-chance time window did not show correlation with the behavioral measures of task switching. Thus, this inhibitory activity alone cannot explain the effect of task switching on response times. In contrast, the above-chance decoded representation that directly follows it has a significant and linear relationship with the executed response. As such, the RIDE-decomposed neurophysiological data provide clear-cut evidence for theoretical propositions according to which processes in the cascade of motor response preparation and execution, detached from cognitive response selection mechanisms, affect switching (Koch and Philipp, 2005; Philipp et al., 2007; Schuch and Koch, 2003; Steinhauser and Hübner, 2006, 2008; Verbruggen et al., 2005). These insights would not have been possible with prior EEG signal decomposition. The R-cluster dynamics seem to relate to orbitofrontal regions (BA11) and left and right inferior parietal regions (BA40) as suggested by the source localization results. Interestingly, parietal contributions to motor processes have been suggested to be inhibitory (Bernier et al., 2012; Beste et al., 2009; Cisek and Kalaska, 2002; De Jong et al., 2001; Desmurget et al., 2018; Jaffard et al., 2008; Sulpizio et al., 2017), which nicely fits the above line of arguments. In a similar vein, also orbitofrontal regions have been associated with (motor) inhibitory control processes (Casey et al., 1997; Chikazoe et al., 2009; Majid et al., 2013; Rubia et al., 2005), even though strong evidence also suggests a role of right inferior frontal regions (Aron et al., 2004) (but see(Hampshire et al., 2010)).

Conclusions

In summary, we show that the relevant activation of task sets occurs parallel in time at multiple levels in the neurophysiological signal containing aspects of stimulus-related processing, response selection and motor response execution, and different brain regions. The intensity with which these task sets are activated decreases and increases dynamically. The temporal stability of these activations could, however, only explain behavioral performance on a limited scale. Instead, inhibitory control processes associated with left orbitofrontal regions and inferior parietal regions selectively acting on invalid motor task sets are relevant. The study shows how propositions from cognitive theories stressing the importance of the activation/deactivation of task sets during cognitive flexibility are implemented on a neurophysiological level.

Limitations of the study

The study shows how task sets activation and deactivation is implemented on a neurophysiological level and how these processes unfold in time on distinct neurophysiological levels coding for different information relevant to cognitive flexibility. The applied EEG methods provide high temporal resolution and thus detailed insights into the temporal aspects of activated and deactivated task set representations. However, when comparing the obtained results of task repetition and task switching, the difference between trial numbers used in the separate analyses have to be taken into account. Similarly, decoding accuracy might not reflect the different task rules with the same weight across the analyses, since the number of trials has been balanced between memory and cue conditions but not between the different rules or stimulus types. Furthermore, using EEG, the study is limited regarding the information on the functional neuroanatomical structures likely playing a role. Although the presented study also presents data using EEG source localization methods, the question which functional neuroanatomical structures are implied in the identified dynamics requires further verification using fMRI methods.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Software and algorithms
Matlab 2019a	https://de.mathworks.com/products/matlab.html	RRID:SCR_001622
BrainVision Recorder	https://www.brainproducts.com/productdetails.php?id=21	RRID:SCR_016331
BrainVision Analyzer	http://brainproducts.com/productdetails.php?id=17	RRID:SCR_002356
Deposited data
Raw and analyzed data	This paper	https://osf.io/xuqah/
Participant information	This paper	https://osf.io/xuqah/

Resource availability

Lead contact

Further information and request for resources should be directed and will be fulfilled by the lead contact, Christian Beste (Christian.Beste@uniklinikum-dresden.de).

Materials availability

There are no newly generated materials.

Experimental model and subject details

Participants

Datasets from N=86 healthy young adults (42 males, 44 females, M_age = 24.6 years SD_age = 2.8) were used. Part of the data was re-used from previous studies using the same experimental task (Petruo et al., 2017; Wolff et al., 2017b, 2018b) (N = 11, N = 24, N = 17) and other data were collected newly (N = 34) to increase the sample size and to obtain reliable results for the MVPA analysis representing a data-driven method. All studies used an identical version of the paradigm, including stimuli, timing, conditions, and trial numbers. For a general overview of comparability across the four subsamples, see Figure 2. The previous studies did not use any form of MVPA decoding and data-driven methods, but were designed to test more specific hypotheses referring to event-related potentials (ERPs). The inclusion criteria for the current study were normal or corrected-to-normal vision, no psychiatric or neurological disorders, and no usage of centrally acting medication. All participants provided written informed consent before they participated in the studies. The participants were treated in accordance with the declaration of Helsinki and the study was approved by the ethics committee of the TU Dresden.

Methods details

Task

We applied a well-studied switching task (Gajewski et al., 2011; Petruo et al., 2018, 2019; Wolff et al., 2016) for assessing cognitive flexibility. It consisted of two different blocks, a cued and a memory-based one, each with 198 trials. In both blocks, participants were instructed to follow three task rules interchangeably. First, either an informative cue (cued block) or a non-informative cue (memory-based block) was presented. The task cue was presented in white letters 3 mm below the fixation cross.

In each trial, one of 8 numbers between 1 and 9 (except 5) were shown, to which a motor response had to be executed according to the corresponding pre-defined rule. The task rule ‘numeric’ required a decision on whether the presented digit is less than or greater than 5. The task rule ‘parity’ required a decision whether the given digit was an even or odd number. The task rule ‘font-size’ required the question of whether the digit is written in small or large font. Both the left and right control key (Ctrl keys on a stand QWERTZ-keyboard) had to be used for the responses. Responses indicating ‘smaller than 5’, ‘odd number’ or ‘small font-size’ had to be given by pressing the left Ctrl key. For the respective other options, i.e., ‘greater than 5’, ‘even number’ or ‘large font size’ the response had to be given by pressing the right Ctrl key. Switches or repetitions of the ‘rule’ were therefore varied orthogonal to switches or repetition of the motor response. In the current study, we used a within-subject design with the factors of trial type (memory versus cued) and rule (repetition versus switch) for the main analyses (see sections multivariate pattern analysis and statistics). Thus, the three types of task rules were not decoded from each other. This was necessary to increase the number of features (i.e., trials) when applying MVPA (see sections multivariate pattern analysis).

The experiment began with the cued block. Each trial started by presenting a fixation cross, which was followed by one of three randomly presented cues; they are ‘NUM’ (abbreviation for ‘numeric’), ‘PAR” (abbreviation for ‘parity’), and ‘FS’ (abbreviation for ‘font-size’) presented 1300ms before the target stimulus. In each trial, the cue remained visible during the following presentation of one of the eight digits. Participants were asked to respond within a period of 2500 ms. Once the response was executed, the screen turned black for 500 ms. Then, a feedback stimulus was displayed for 500 ms: A plus sign indicated a correct response while a minus sign an incorrect response. The cues (and correspondingly, the task rules) were presented in a randomized order.

In the memory-based block, the cue was replaced by a non-informative dummy cue ‘XXX’. This was necessary to keep the experiment's visual structure and timing comparable between the cue-block and the memory-block. Participants were always instructed to follow the same order of task rules of 3 × ‘NUM’, 3 × ‘PAR’, and 3 × ‘FS’, i.e., NUM, NUM, NUM, PAR, PAR, PAR, FS, FS, FS, NUM, NUM, NUM, PAR, etc. Here, the order of the rules must be retrieved independently by the participant. As soon as the participant produceed three consecutive errors, the task was paused, and the order of the rules was presented to them again. The corresponding informative cue (‘Num’, ‘Par’, or ‘FS’) was displayed over the next three trials to help get back into the task. After these three trials, the dummy cue was shown for each trial, and the participant continued to work independently on the task. In the memory-based block, the participants thus had to rely on working memory information and they also had to use internal prompts such as inner speech to retrieve the currently relevant task set (Emerson and Miyake, 2003; Kray, 2006). As with the cue-based block, there was a balanced proportion of each rule (33.33%), yet the frequency of switching was reduced to 33.3% of all trials due to the fixed order of task rules in the memory block.

While a fixed order of rules was necessary in the memory-based block to rely on working memory, we deliberately did not use a fixed order in the cue-based block for the following reasons: Within-subject manipulation, such as the introduction of memory and cued blocks, is subject to ‘carry-over effects’(Petruo et al., 2018). For instance, participants could apply their strategy from the memory block to the cued block without the need to process the cue information. Furthermore, a fixed trial order in the cued block could cause a mixture of memory-and cue-based strategies, and consequently, inflate the inter-individual variations. Thus, to rule out potential carry-over effect, trials of the cued block did not follow a predetermined order. This ensured that participants had to process the cue information, and update their behavior flexibly on a trial-by-trial basis. In sum, successful task switching had to be carried out either with an anticipated order of task rules (memory block) or in a dynamically changing environment (cued block). Since the difference between blocks should have influenced of how participants recalled (‘activated’) or switched between (‘suppressed’) task rule representations, memory and cued block trials were used for subsequent classification (see multivariate pattern analysis).

EEG recording and analysis

EEG activity was recorded at 500 Hz sampling rate from 60 Ag/AgCl electrodes and with using the BrainVision Recorder and a QuickAmp amplifier (BrainProducts, Germany). Electrodes were mounted on elastic caps (EasyCap, Germany) in equidistant positions. During EEG recordings, electrode impedances were kept below 5 kΩ. The ground and reference electrodes were placed at the coordinates of θ = 58, φ = 78 and θ = 90, φ = 90. Offline EEG analyses, including filtering, ICA-decomposition for ocular and cardiovascular artifact correction, segmenting, and residual artifact rejection, were conducted in BrainVision Analyzer 2 software package (Brain Products, Germany). The steps and parameters of EEG pre-processing followed the pipeline of previous studies that used the same task (Petruo et al., 2018, 2019; Wolff et al., 2016). Comparable data processing across studies can facilitate more direct comparisons between studies and between different EEG analyses. The offline data pre-processing began with down-sampling the EEG recordings to 256 Hz. The data was filtered using a band-pass filter of 0.5–20 Hz and a notch filter at 50 Hz—both, with a slope of 48 dB/oct. After a manual raw data inspection, ICA decomposition (infomax algorithm) was applied to remove blinks, horizontal/vertical eye movements and cardiac artifacts. These artifacts were identified by visual inspection of the components and the ones showing these artifacts were discarded before back-projecting the data to the EEG sensor space. Next, segments were formed using all channels for corresponding conditions. Segments were stimulus-locked, separating switch from repetition trials, and only included trials with correct responses. This was performed for the cue-based and the memory-based blocks separately. The segments' length added up to 1200 ms, starting at −200 ms before the locking point and ending 1000 ms after the stimulus presentation. Subsequently, an automated artifact rejection was run, that discarded all segments with signal amplitudes higher or lower than ±200 μV. Segments were also discarded if an activity smaller than 0.5 μV was registered for at least 200 ms. Additionally, if amplitude differences between two consecutive peaks in a time frame of 200 ms were higher than 200 μV, the segment was removed. To discard the reference potential from the data, current source density (CSD) transformation was carried out using the spherical Laplace operator (parameters for this procedure were n = 4 splines and m = 10 Legendre polynomials; Lambda = 1 x 10-5). Lastly, the baseline was corrected by calculating new locking points using the period between −200 ms and the locking point for each segment. This baseline-corrected EEG data of all 60 channels have been exported from BrainVision Analyzer as single trial, single-subject time-locked data.

Quantification and statistical analysis

Residue iteration decomposition (RIDE)

Residue Iteration Decomposition (RIDE) was applied to the single-trial level, single-subject data. RIDE decomposition was performed following a previous study that used the same experimental paradigm (Wolff et al., 2017b). Although RIDE was initially developed to control and correct intra-individual variability in the data, it is also used to distinguish intermingled coding levels in neurophysiological data (Mückschel et al., 2017a, 2017b; Opitz et al., 2020; Takacs et al., 2020a). This is the primary reason why we applied RIDE (the toolbox and manual of RIDE available on http://cns.hkbu.edu.hk/RIDE.htm). Comprehensive mathematical details on the RIDE method can be found in Ouyang et al. (Ouyang et al., 2011, 2015). Briefly, RIDE uses a self-optimizing iteration scheme, which decomposes single-trial ERPs into static latency and variable-latency components at each electrode separately. These components are thought to reflect different and partly overlapping stages of cognitive processing. The separation of these stages in the EEG signal leads to three RIDE-components (clusters of components) with similar time-locking properties (Ouyang et al., 2011, 2015): the S-cluster refers to processes related to the stimulus (such as perception and attention processes), the R-cluster relates to the processes associated to the response (such as motor preparation and execution processes), and the C-cluster refers to intermediate processes between S and R (such as stimulus evaluation and response selection processes) (Ouyang et al., 2011, 2015; Verleger et al., 2014). The different components within a RIDE cluster correspond to standard ERP components with different peaks, topographies, and functional significance (Ouyang et al., 2011, 2015). For the decomposition, the RIDE algorithm employs a time window function to obtain each RIDE component’s waveform. It is assumed that the time frames include each component's occurrence (Ouyang et al., 2011, 2015). Therefore, the search windows' parameters must be adjusted to fit the experimental task’s data. In the present case, the following time windows were defined: 0 to 600 ms (including the target stimulus) for the S-cluster, 200 to 900 ms for the C-cluster, and 300 ms before and after the response for the R-cluster. The RIDE procedure created the following types of separate datasets for each condition and each subject: C-cluster averaged data, C-cluster single-trial data, R-cluster averaged data, R-cluster single-trial data, S-cluster averaged data, S-cluster single-trial data, and averaged latency-corrected data of the undecomposed EEG. The single-trial single subject data were used for subsequent MVPA, while the single subject average data were used for source localization (see Source localization analysis (sLORETA)). Prior to the MVPA, the single-trial datasets of all task conditions needed to be concatenated for each subject, separately for the C-cluster, R-cluster, and S-cluster data.

Multivariate pattern analysis

Temporally decomposed, single-trial single-subject datasets were used for MVPA. The MVPA was performed using the ADAM toolbox (version 1.05, Fahrenfort et al., 2018) in Matlab (Mathworks). For a similar approach, see Takacs et al. (Takacs et al., 2020b). First, the EEG data was down-sampled offline to 55 Hz (Fahrenfort et al., 2018). A linear discriminant classifier was trained and tested on each time point. The classifier was trained on 80% of the data, and tested on the remaining 20% of the data, iterating this process until all data points had been tested (5-fold training). The average of the consecutive test folds was used as a final performance index. Classification accuracy was quantified as Area under the ROC curve (AUC). Larger AUC indicates better (more accurate) classification performance (Fahrenfort et al., 2018). The classifier was trained on the following categories, separately for the C-cluster, R-cluster and S-cluster data:

• (1)Task repetition: cue repetition versus memory repetition; (2) Task switching: cue switch versus memory switch. Thus, decoding performance with these categories should reflect the difference between following the task rule after receiving a cue versus remembering that. That is, decoding was expected to reveal the activation/deactivation of the appropriate response under memory load. This allowed us to investigate how neural patterns differ from each other while following the same task rules with external reminders (cue conditions) or relying on internal ones (memory conditions), separately for trials where task switching was either necessary or not. Specifically, decoding categories of task repetition was expected to reveal the difference between being reminded or recalling the continuously present task set. On the other hand, the decoding results of task switching categories was expected to show the difference between neural patterns for being reminded or being recalled an alternative (i.e., competitive) task set. Altogether, the potential differences between task repetition and task switching MVPA results should be informative on the dynamics of neural implementations during upholding vs switching mental sets of task rules. The decoding of task repetition denotes appropriate response activation in a continuous manner. In contrast, task switching represents response activation when participants had to change their mental set from one rule to another between two consecutive trials. Skewed classification was avoided by balancing the categories with ADAM (Fahrenfort et al., 2018). If trial numbers in the categories were unbalanced, the majority class was down-sampled (that is, between cue repetition and memory repetition, and between cue switch and memory switch, respectively). The following trial numbers entered the analyses for each classification (mean, minimum and maximum given): Task repetition in the C-cluster (121.9; 36; 378), Task switching in the C-cluster (89.2; 30; 234), Task repetition in the R-cluster (151; 36; 378), Task switching in the R-cluster (90.9; 30; 234), Task repetition in the S-cluster (150.2; 36; 378), Task switching in the S-cluster (90.2; 30; 234). Classification features consisted of EEG amplitude data at the sixty single channels in all stimulus classes. A backward decoding model (Fahrenfort et al., 2018) was used for training and testing. After the calculation of the AUCs, temporal generalization matrices were computed based on cross-classification across time. Temporal generalization matrices are commonly used to visualize how accurately classification performance for a given time window generalizes to all other time points of the analysed data (Fahrenfort et al., 2018; King and Dehaene, 2014). Good (i.e., highly accurate) classification performance across samples suggests stability of the identified neural pattern. Higher stability was visualized as either darker colours on a heat map (see Figures 3 and 4) or as a larger extension of the generalized pattern in comparison to the diagonal angle. The stability of the C-, R-, and S-clusters activity patterns was examined over time by training the model in one time point and testing its discriminatory power in the other time points. Cross-classification was iterated at every time point. In the final AUC, performance significantly above or below chance level indicates sustained neural activity. Statistical analyses for the MVPA, including corrections for multiple comparisons, were performed in ADAM (Fahrenfort et al., 2018). Two-sided t-tests against chance level defined as AUC = .05 were performed for each time sample across subjects. Cluster-based permutation was used to correct multiple comparisons as implemented in ADAM (Fahrenfort et al., 2018).

Source localization analysis (sLORETA)

Source localization analysis using standardized low resolution brain electromagnetic tomography (sLORETA) (Pascual-Marqui, 2002) was used to examine what functional neuroanatomical structures were associated with MVPA-decoded neuronal activity in time periods showing significantly above/below chance level classification performance. Mathematical details on the sLORETA algorithm can be found elsewhere (Pascual-Marqui, 2002) and the validity of this approach has been corroborated by combined fMRI/EEG and TMS/EEG studies (Dippel and Beste, 2015; Sekihara et al., 2005). Briefly, sLORETA uses a three-shell spherical head model (MNI152 template) in which the intra-cerebral volume is partitioned into 6239 voxels using a spatial resolution of 5mm. The standardized current density is calculated for every voxel in this head model. The algorithm provides a single linear solution for the inverse problem without localization bias (Marco-Pallarés et al., 2005; Pascual-Marqui, 2002; Sekihara et al., 2005). We used the sLORETA contrast against zero for the statistical analysis, and we performed voxel-wise randomization tests with 2,500 permutations and statistical non-parametric mapping procedures (SnPM) to correct for multiple comparisons. Locations of voxels that were significantly different (p < .05) are shown in the MNI-brain www.unizh.ch/keyinst/NewLORETA/sLORETA/sLORETA.htm, with color coding reflecting t-values.

Statistics

Statistical analyses of the behavioral data were performed using IBM SPSS Statistics (IBM Corp., Armonk, NY). Mean accuracy (percentage of correct responses) and means of reaction time (RT) data (for correct responses) were calculated for each participant and each condition. To examine task switching effects, accuracy and RT data were analysed repeated measures ANOVAs with trial type (cued versus memory), rule (repetition versus switch) and motor response (repetition versus switch) as within-subject factors. Switch cost was quantified as the difference between task repetition and task switch RT or accuracy, separately for cued and memory conditions. Mean AUC in the time windows of significant above- or below-threshold classifications was calculated for each participant and each condition. To analyse the difference between MVPA decoding results, repeated measures ANOVAs with time window (as indicated by a successful decoding) and rule (repetition versus switch) were conducted separately for the C- and R-cluster data. In the S-cluster, only since a single time window was identified, decoding results between repetition and switch rules were compared with a paired sample t-test (bootstrapped: sample size of 5000, confidence interval [CI] level of 95%). Here we report η_p² effect size for ANOVA main effects and interactions and Cohen`s d for t-tests. All post-hoc tests were Bonferroni-corrected. Finally, to analyse the relationship between behavioral and neurophysiological markers of task switching, bootstrapped (sample size of 5000, confidence interval [CI] level of 95%) Pearson’s correlations were run between the individual AUCs averaged over significant above- or below-threshold classifications and task switching.

Published: December 17, 2021

Data and code availability

De-identified human behavioral data and neurophysiological datasets for classification have been deposited at Open Science Forum: https://osf.io/xuqah/. Neurophysiological datasets in different stages of processing (e.g., raw, pre-processed, undecomposed, temporally decomposed) are available after specification of requested format upon reasonable request by the lead contact.

Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.