Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Sep 23.
Published in final edited form as: Neuron. 2015 Sep 23;87(6):1357–1368. doi: 10.1016/j.neuron.2015.08.026

The necessity of rostrolateral prefrontal cortex for higher-level sequential behavior

Theresa M Desrochers 1, Christopher H Chatham 1, David Badre 1,2
PMCID: PMC4630805  NIHMSID: NIHMS717961  PMID: 26402612

Summary

Frontal neocortex is thought to support our highest intellectual abilities, including our ability to plan and enact a sequence of tasks toward a desired goal. In everyday life, such task sequences are abstract in that they do not require consistent movement sequences and are often assembled “on the fly”. Yet, remarkably little is known about the necessity of frontal sub-regions for such control. Participants repeatedly completed sequences of simple tasks during fMRI scanning. Rostrolateral prefrontal cortex (RLPFC) activation ramped over sequence position, and reset at the initiation of each new sequence. To establish the necessity and function of RLPFC in this task, participants performed the sequential task while undergoing transcranial magnetic stimulation (TMS) of the RLPFC versus two prefrontal control regions. Across two independent experiments, only RLPFC stimulation increasingly disrupted task performance as each sequence progressed. These data establish RLPFC as necessary for uncertainty resolution during sequence-level control.

Introduction

Routine tasks in everyday life require complex sequences of sub-tasks (Lashley, 1951). Consider a task like taking a shower. Showering is sequential in that the goal state (being clean) requires completing subgoals that unfold in time and in a prescribed order, like washing hair, then face, and so forth. Psychologists have long held that people accomplish sequential tasks by maintaining goals at both the sub- (wash face) and superordinate (take shower) levels at the same time (Lashley, 1951). This ability to simultaneously pursue immediate goals while holding higher-order goals in mind is termed hierarchical cognitive control.

Sequential tasks involve unique demands that are not entailed by non-sequential tasks or simpler motor sequences. First, though a task may require a particular subgoal (like washing hair), the specific motor actions may not be identical every time that subgoal is selected. Thus, the sequence representation exists at a level more abstract than the motor response. Moreover, because action unfolds over time, the prevailing action-relevant state is often only partially observable through the senses. For example, when enacting the sequence of subgoals required to take a shower, there is little in the environment to indicate whether it is time to wash one’s hair or face. Rather, one must internally specify position in the sequence. Because of these unique demands, previous work examining the brain areas serving motor-only or cued sequences (Barnes et al., 2005; Fujii and Graybiel, 2003; Jin and Costa, 2010; Smith and Graybiel, 2013) cannot fully elucidate the neural mechanisms for task sequences. Further, though people may represent tasks sequentially, this does not require that a hierarchical control system is involved (Botvinick and Plaut, 2004). Thus, a fundamental question concerns not only what control systems support sequential tasks, but indeed, whether control systems are necessary for such tasks at all.

Despite their ubiquity in everyday life, we know little about how the brain controls task sequences (Farooqui et al., 2012; Koechlin and Jubault, 2006; Koechlin et al., 2000). The frontal lobes are broadly known to support goal-directed behavior (Passingham and Rowe, 2002). Moreover, neuroimaging (Badre and D’Esposito, 2007; Koechlin et al., 2003) and neuropsychological evidence (Badre et al., 2009) suggest that the frontal lobes may be functionally organized along their rostro-to-caudal axis to support non-sequential forms of hierarchical cognitive control, with more rostral regions supporting more abstract forms of control. The rostrolateral prefrontal cortex (RLPFC), in particular, has been implicated in settings that have elements in common with sequential hierarchical control (Badre and D’Esposito, 2007; Badre and Frank, 2012; Badre and Wagner, 2004; Braver and Bongiolatti, 2002; Braver et al., 2003; Daw et al., 2006; DiGirolamo et al., 2001; Dosenbach et al., 2006; Dreher et al., 2008; Gilbert et al., 2006; Kim et al., 2012; Koechlin et al., 1999, 2003; Kovach et al., 2012; Nee et al., 2013; Orr and Banich, 2013; De Pisapia et al., 2012), and so this region is a prime candidate to support this capacity. However, there have been no direct tests of this region during sequential control. Indeed, few studies have examined the causal necessity of this area for any form of higher-order control using selective disruptive methods such as TMS (though see Bahlmann et al., 2015; De Pisapia et al., 2012).

Here, we provide convergent evidence from fMRI and TMS that RLPFC is necessary to resolve task-position uncertainty while performing internally-monitored task sequences (Schneider and Logan, 2006). In other words, RLPFC provides a momentary check necessary to keep us “on track” during performance of a sequence. We leveraged two key features of sequential tasks: the need to pursue a prescribed ordering of subtasks (not movements), and the need to perform that sequence without the benefit of external cues in the following task. On each trial of both the fMRI and TMS experiments, participants categorized a visual stimulus based on its color or shape (Figure 1A). In a block of trials, participants continuously performed an instructed four-task sequence of the two categorizations (e.g., color, color, shape, shape; Figure 1B). Thus, as no external cues indicated what task to perform or where a four-task sequence began or ended, participants must internally represent the sequence in order to stay on track throughout a run. The tasks and transitions between them (switch or repeat) occurred with equal frequencies across sequences (Figure 1D). As a consequence, there was a hierarchical relationship between subordinate “task level” and superordinate “sequence level” elements of the task in that the sequence representation fully determined the task identity, but the task identity or status had little bearing on the sequence. Relationships between sequence and task level control can therefore be selectively assessed as the effect of Position, when controlling for other factors such as Sequence Complexity (defined here as the frequency of task switches within a sequence; Figure 1D), local task switching across sequence position, and the sequence of motor responses.

Figure 1.

Figure 1

Open in a new tab

Task schematic and behavioral results. (A) Example trial. (B) Example block with the task that should be executed on each trial (as remembered from the instruction screen). (C) Example ordering of different sequence blocks for one run. (D) Sequence Complexities. A and B represent different tasks (e.g. color and shape judgments). Simple sequences contain one task switch (underlined) and Complex sequences contain two in the interior of the sequence. Across sequence repetitions the number of task switches is the same for both Sequence Complexities. (E) RTs and (F) ERs (mean ±SEM) in the fMRI task. The generic task designation (A or B) indicated at each data point.

Results

RLPFC activation ramps with Sequence Position

Response times (RT) provided evidence that participants were performing the task sequence as instructed (see Supplemental Results A and Figure 1F for detailed analysis of error rates [ER]). Most importantly, RT at the first position of each four-task sequence repetition was slowed (t55 = 8.7, P < 0.0001; Figure 1E), regardless of whether it was a Switch or Repeat from the last task performed in the preceding sequence (t27’s > 6, P’s < 0.0001). Thus, the elevated RT at this position can only reflect costs of crossing an “internal” sequence boundary from Position 4 of one sequence to Position 1 of the next.

Given our a priori focus on this region, we selected an unbiased ROI in RLPFC (Koechlin et al., 1999). Strikingly, activation in RLPFC increased progressively from Position 1 to 4 within a sequence (Figure 2A; F3,81 = 2.78, P < 0.05). This observation was confirmed by a whole brain voxelwise analysis of a parametric ramping function that reset at each Position 1 and increased to Position 4 (Figure 2B; Table S1 & Figure 3A,B). Interestingly, the whole brain contrast also revealed regions outside of the RLPFC exhibiting ramping activation. Moreover, three of these regions; the medial frontal, parietal, and lateral anterior temporal cortices; survived follow-up control analyses (see Figure S1), suggesting that RLPFC may coordinate with these other regions during sequence-level control (Supplemental Results D, Figure S2). We return to such network-level considerations in the Discussion, but focus our subsequent analyses and experiments on our a priori ROI within RLPFC.

Figure 2.

Figure 2

Open in a new tab

Results from the fMRI experiment. (A) Mean percent signal change (+SEM) from the peak (6 s) of the FIR from the voxels included the unbiased RLPFC ROI. (B) Voxelwise contrast of the Parametric Ramp regressors (see Experimental Procedures, Figure 3B) over baseline (extent threshold 172 voxels, note lateral views rotated ~50°). (C) Same as (A) for pre-PMd ROI. (D) Voxelwise contrast of all Complex > Simple sequences (extent threshold 185 voxels). All contrasts were family-wise error cluster corrected for multiple comparisons at P < 0.05. Outline of the RLPFC, pre-PMd, and SMA/pre-SMA (see SI) ROIs in black shown in A and D. See also Figures S1, S2.

Figure 3.

Figure 3

Open in a new tab

Regressors. (A) Onsets. Separate, instantaneous (zero duration) regressors for each Position. (B) Parametric ramp. Instantaneous onsets for all positions that linearly increased from Position 1–4. (C) Sustain. Constant square wave from the onset of Position 1 to the offset (response) of Position 4. (D) Ramp. Linearly increased from the onset of Position 1 to the offset of Position 4. (E) Parametric Sequence Dwell Time. The height of each instantaneous onset was proportional to the elapsed time from the onset of Position 1 for each sequence. Separate regressors were included for each Position, except for Position 1 as no time had yet elapsed within the Sequence. Note differences in mean height of the Position 2–4 regressors are for illustration purposes only. All regressors shown were separated by Sequence Complexity (e.g. a model containing Onsets would contain 8 regressors: Simple Positions 1–4 and Complex Positions 1–4).

Importantly, prior studies using non-sequential tasks have routinely observed a sustained function in RLPFC (Braver et al., 2003; Dosenbach et al., 2006; Koechlin et al., 1999, 2003) rather than the ramping function observed here. As these two functions (ramp versus sustain) are correlated (Figure S3), it was important to establish that ramping explained variance in the RLPFC signal beyond what could be accounted for by a sustained function. Follow up analyses (see Supplemental Results B) found that although ramping activation accounted for RLPFC activation beyond what could be accounted for by a sustained function, the opposite was not the case (Ramp: t55 = 3.3, P < 0.01; Sustain: t55 = 1.8, P > 0.05; Figures 3C,D, 4A,B).

Figure 4.

Figure 4

Open in a new tab

FMRI results. (A) Voxelwise contrast of the Ramp regressors over baseline from the Sustain vs. Unique Ramp model showing significant activation in the RLPFC (family-wise error (FWE) cluster corrected for multiple comparisons at P < 0.05, extent threshold 207 voxels). (B) Voxelwise contrast of Sustain regressors over baseline from the Unique Sustain vs. Ramp model. Note the absence of activation in the RLPFC (P < 0.001 uncorrected). Left and right lateral views rotated ~50° in A and B. (C) Right SMA/pre-SMA indicated in the voxelwise contrast of all task Switches > Repeats in Positions 2–4 (FWE cluster corrected P < 0.05, extent threshold 140 voxels). The outline of the RLPFC, pre-PMd, and SMA/pre-SMA ROIs in black shown in A, B, and C. (D) Mean percent signal change (+SEM) in SMA/pre-SMA ROI. Contrast in (C) shown in red/yellow on coronal section. See also Figures S2, S3.

Additionally, we tested whether ramping activation was strictly related to sequence position or other factors, specifically time. Notably, ramping activation in the RLPFC signal was further explained by the time elapsed since the beginning of the sequence (Sequence Dwell Time), above and beyond that explained by Position (t27 = 2.1, P < 0.05; Figure 3A,E). Further, Sequence Dwell Time positively correlated with RT after variance due to other factors had been removed (t27 = 3.1, P < 0.01; Table S2), supporting the behavioral relevance of this factor.

We next sought to control for sequence and task level effects. First, we tested for effects of Sequence Complexity and local Task Switching (i.e., switching independent of sequence position). Sequence Complexity is defined by the number of switches within each sequence (Complex = 2, Simple = 1; Figure 1D) and therefore only differs between these conditions if the tasks are represented in sets of four; otherwise, the number of switches is equated across the entire run of trials for each condition.

Sequence Complexity is important to control for three reasons. First, prior work has highlighted temporal and spatial groupings in sequence planning, and so preparation may be influenced by the number of upcoming task transitions (Lien and Ruthruff, 2004). Second, prior behavioral work indicated that initiation times were influenced by sequence complexity (Schneider and Logan, 2006), an observation we replicate here (Supplemental Results A). Third, work on mental effort and costs of cognitive control have indicated that participants avoid initiating a run of trials they know will involve a higher task switching frequency (Kool et al., 2010), presumably because these runs will be more costly on the control system.

RLPFC did not show effects of Sequence Complexity or Task Switching, in whole brain or ROI analysis (F1,27’s ≤ 0.7, P’s > 0.2). By contrast, pre-dorsal premotor cortex (pre-PMd) was more activated for Complex than Simple sequences in ROI (F1,27 = 8.19, P < 0.01, Figure 2C) and whole brain contrasts (Figure 2D). These functional differences dissociated RLPFC from pre-PMd (Region × Sequence Complexity: F1,27 = 6.04, P < 0.05; Region × Position: F3,81 = 3.43, P < 0.05). The supplementary motor area/pre-supplementary motor area (SMA/pre-SMA) was distinguished from pre-PMd and RLPFC based on their sensitivity to local Task Switching (Figure 4C,D). SMA/pre-SMA showed a significant ROI × Switch/Repeat interaction with pre-PMd (F1,27 = 15, P < 0.001), as well as RLPFC (ROI × Switch/Repeat: F1,27 = 7.15, P < 0.01).

To summarize, RLPFC exhibited a ramping pattern of activation within each sequence that was distinguishable from a sustained function, was not influenced by sequence complexity or task switching, and depended on both position and time within a sequence. At least three processes could be consistent with a ramping function in RLPFC: (A) an evolving process that is actively carrying out the sequence by ordering each task in turn and so is uniformly necessary throughout the sequence; (B) a decision process that marks the sequence boundary and is crucial for initiating the current or next sequence; or (C) a reactive, top-down process that transiently resolves sequence position uncertainty and so is increasingly necessary as one moves away from sequence initiation (and uncertainty grows). Because the fMRI data cannot distinguish these alternatives, we next conducted a brain stimulation experiment to assess the necessity of RLPFC during sequential control.

The Necessity of RLPFC for Sequential Control

To elucidate the causal role of RLPFC in sequential control, we conducted a single-pulse TMS (spTMS) experiment and then replicated and extended these effects in a second TMS experiment. Stimulation was delivered at only one of the Positions (1 to 4) within each four-task sequence (Figure 5A) or was not delivered at any position, which served as a no-stimulation control (“Position 0”). The Position stimulated (0 to 4) was randomized and unpredictable. In the first TMS experiment (TMS1), when stimulation was delivered, it could occur at one of ten possible stimulus latencies following the stimulus onset (SOA; 90–450 ms in 40 ms bins). The target coordinates for stimulation were defined from the fMRI experiment as the locus of ramping activation in RLPFC (xyz = −32, 58, 12) and the Sequence Complexity effect in pre-PMd (xyz = −48, −6, 26).

Figure 5.

Figure 5

Open in a new tab

TMS design and results. (A) SpTMS was delivered at one of 10 SOA at most once in each 4-trial sequence. (B&C) Mean ER (±SEM) across all Positions for no-stimulation (ns) and stimulation trials at 10 different SOA for (B) RLPFC and (C) pre-PMd. Yellow oval indicates points included for analysis at the peak stimulation SOA. (D) Mean difference in ER (±SEM) due to stimulation at peak SOA for RLPFC and for pre-PMd in TMS1. ER differences shown over the course of sequences: beginning (Position 1), middle (Positions 2 and 3), and end (Position 4). Asterisk indicates significant difference in the effect of stimulation at Position 4 (F1,32 = 6.7, P < 0.01). (E) Same as D but for TMS2. Asterisk at Position 1 indicates a reliable difference between RLPFC and pre-PMd (F1,28 = 6.2, P < 0.02). At Position 4, tilde indicates a marginal difference between RLPFC and pre-PMd (F1,28 = 2.9, P < 0.1), and asterisk indicates a reliable difference between RLPFC and RMPFC (F1,14 = 4.4, P < 0.05). (F) Mean post-test questionnaire ratings (±SEM) of the amount of distraction (1–5, left) and discomfort (0–5, right) in TMS2. See also Figures S4, S5.

For each region, we identified a target SOA bin (plus/minus one bin) that showed the peak effect of TMS on error rates (ER), independent of sequence Position and Complexity (peak = 130 ms in RLPFC, 410 ms in pre-PMd, Figure 5B,C; see Supplemental Results E,F and Figure S4 for RT analyses). Additionally, we used the level of stimulation delivered as a covariate.

Stimulation of RLPFC resulted in progressively increased ER over Position relative to the no stimulation control (Position × Stimulation linear contrast F1,16 = 4.6, P < 0.05; Figure 5D orange, Figure S5C). By contrast, pre-PMd showed a quantitatively diminishing effect of stimulation on ER over Positions (Figure 5D blue, Figure S5D) with a reliable difference between stimulation and no-stimulation at Position 1 (t16 = 2.1, P = 0.05) but only marginal over all positions (F1,16 = 3.4, P > 0.08). The opposite effects of stimulation over Position resulted in a crossover interaction between these frontal regions (Position × Stimulation × Region: F2,64 = 4.3, P < 0.02). The Region interactions were not due to a difference in baseline, as there were no differences between the no stimulation conditions in RLPFC and pre-PMd (Position × Region: F2,64 = 1.2, P > 0.3), and there was a significant Position × Region interaction when comparing the error rates on the stimulation trials in RLPFC and pre-PMd (F2,64 = 4.3, P < 0.02). Participants’ self-reported ratings of the amount of discomfort (see Experimental Procedures) did not significantly differ between the two groups (Wilcoxon rank sum test, z = 1.7).

Additionally, we performed a second TMS experiment (TMS2) were spTMS pulses were delivered during the task just as in the first experiment with only two changes. First, only the peak SOA bins (plus/minus one bin) from TMS1 were used for each region stimulated. Second, in addition to stimulating RLPFC and pre-PMd as in TMS1, we added a second control region, rostromedial prefrontal cortex (RMPFC, xyz = 0, 61, 19; SOA bin 130 ms). Based on the fMRI data this region was not expected to show effects of stimulation as a function of position but would be highly similar in terms of peripheral sensations to the RLPFC.

We replicated all the major findings of TMS1 in TMS2. Stimulation of the RLPFC resulted in a progressively increased ER over Position relative to the no stimulation control (Position × Stimulation linear contrast F1,14 = 9.8, P < 0.01), whereas stimulation of pre-PMd again showed a diminishing effect across Position (Figure 5E). These opposite effects replicated the crossover interaction between frontal regions (Position × Stimulation × Region: F2,56 = 5.4, P < 0.03). This Region interaction was again not due to a difference in baseline, as there were no differences between the no stimulation conditions in RLPFC and pre-PMd (Position × Region: F2,56 = 1, P > 0.3), and there was a significant Position × Region interaction when comparing the error rates on the stimulation trials in RLPFC and pre-PMd (F2,56 = 7.8, P < 0.001).

Further, the effects of stimulation on RLPFC were distinct from the RMPFC, as there also was a crossover interaction between these two areas (Position × Stimulation × Region: F2,28 = 3.4, P < 0.05). RMPFC showed an interaction between the effect of stimulation and brain region with pre-PMd (F1,28 = 4.4, P < 0.05). There was no effect of stimulation on RMPFC itself, as the difference between stimulation and no stimulation conditions did not differ from zero at any position (t’s > −1.2, P’s > 0.14). Neither the amount of participants’ distraction (z = 0.97, P > 0.3) nor the amount of discomfort (z = −0.1, P > 0.9) differed between RLPFC and pre-PMd, or between RLPFC and RMPFC (distraction: z = −0.6, P > 0.5; discomfort: z = 0.74, P > 0.4) as shown in Figure 5F (see Supplemental Results F for further analyses).

Discussion

These results provide evidence that RLPFC and its associated network are a necessary component of the neural system that internally guides sequential behavior. Together, the fMRI and TMS experiments specifically demonstrated that RLPFC is increasingly necessary as a sequence progresses. This pattern of increasing necessity resulted in a rare dissociation of RLPFC function from that of pre-PMd and RMPFC, and indicates that RLPFC may be crucial to resolve uncertainty during the course of a task sequence.

What process is supported by RLPFC during sequential performance? The observations from fMRI and TMS are most consistent with the hypothesis that RLPFC supports a control process necessary to “keep us on track” by transiently resolving uncertainty about what task to perform in the context of a position in sequence. This interpretation requires that uncertainty generally increases over the course of the four-task sequence. Assuming that sequence representations are only refreshed at initiation, then at this step, there is no uncertainty about where one is in sequence. However, each subsequent step through the sequence (either through subgoals or dwell time) is associated with some small probability that a transition is made to the incorrect sequence position. This entropic process would yield growing uncertainty about the current position in sequence. As such, there would be an increasing likelihood at each step that this uncertainty will require resolution to select the appropriate task. On this view, RLPFC acts as a source of top-down, sequence-level control signals that help resolve uncertainty as necessary. As a consequence, RLPFC is more often necessary for performance near the end of the sequence (when uncertainty about position is greatest) than at the beginning (when we found no evidence that it is necessary for normal levels of performance). This account is consistent with previous non-sequential studies implicating RLPFC in tracking alternative courses of action (Badre et al., 2012; Boorman et al., 2009; Koechlin et al., 1999), tracking and updating reward contingencies (Kovach et al., 2012), and monitoring superordinate goals to provide a top down superordinate signal over the course of several trials (Badre and Wagner, 2004; Braver and Bongiolatti, 2002; Dreher et al., 2008; Nee et al., 2013; De Pisapia et al., 2012) (see SI for further discussion).

In contrast, the results from TMS allow us to rule out at least two other processes that might have been supported by RLPFC and would have been consistent with a ramping pattern of activation. First, the TMS results are inconsistent with the hypothesis that RLPFC is necessary for routinely carrying out the sequence, per se, such by deciding what the current position in the sequence is and what task to do at that position on every trial. Such a process would be equally necessary throughout the sequence and would have been subject to disruption throughout, accordingly. However, stimulation of the RLPFC increasingly induced errors over the course of a sequence, and there was no evidence of an increase in error rates following RLPFC stimulation at position 1. Thus, it is unlikely that RLPFC participates in a process that is necessary at every step to actively determine the position and task in the sequence. Rather, as we state above, RLPFC only steps in when necessary to keep performance on track.

Relatedly, RLPFC’s ramping function also seems unlikely to reflect processes related to planning, ordering, or retrieval of upcoming sequences, or anticipation of sequence switches. All these accounts predict that the RLPFC ramping signal will impact sequence transitions or initiations. More specifically, a broad class of such ‘boundary demarcation’ accounts predicts that disruption of RLPFC in one sequence should have an increasing impact on the successful termination of that sequence and/or the initiation of the next sequence, but would not have an effect on subtask performance within the current sequence. However, the opposite pattern was observed in the results. Stimulation of RLPFC affected the current sequence (Figure 5D). But, as described in Supplemental TMS Stimulation results (section F), there was no evidence of such carry-over to the initiation of the next sequence in either RT or error rates, even following stimulation at Position 4 (Figure S5H). Thus, the TMS data are also inconsistent with any boundary demarcation account.

The lack of carryover within sequence is intriguing as one might assume that failure to resolve uncertainty at one point in the sequence should impact subsequent task positions in that sequence and/or initiation of the next sequence. However, the lack of carryover is consistent with a core feature of real world sequential behavior, namely that the status of individual substeps within a sequence does not affect performance of the overall sequence. As was first noted by Lashley (1951), sequences are not carried out as action-trigger chains. Rather, performance of real world sequential tasks often includes slips and errors, such as omitting, repeating, or reversing steps in a sequence. Yet, the occurrence of these slips does not impact subsequent steps in the sequence. Similarly, errors introduced by TMS here did not carry over within sequence. This underscores the limited, transient nature of cognitive control during sequential behavior that we highlight here. Thus, it is likely that sequences routinely proceed through other, perhaps more automatic, mechanisms. But, regular checks by the control system keep sequential performance on track and avoid slips and errors at points of vulnerability, such as when uncertainty is high. This observation suggests that the timing of these checks may be particularly crucial for effective, error-free sequential behavior and is potentially an important avenue for future basic and translational research.

As discussed so far, our results permit us to draw conclusions regarding the process carried out by RLPFC and its dynamics of action during a sequence. However, a separate but unresolved question concerns the nature of the representation in RLPFC. In this regard, it is informative to consider whether the ramping signal in RLPFC reflects an event-related response occurring at each trial (Figure 3B) or a continuous, progressive ramp of activation over the course of the sequence (Figure 3D). The fMRI design used in this study did not permit us to unambiguously distinguish these signals in the BOLD response (see Supplemental Results section B), and thus this remains an open topic of investigation. As discussed above, the TMS results lead us to propose that processing in RLPFC is likely punctuated by transient active events that “gate” out a top-down context, as needed, to resolve uncertainty about task-sequence position. This type of output gating has been shown to be important for hierarchical control of this type (Badre and Frank, 2012; Chatham and Badre, 2015; Chatham et al., 2014; Frank and Badre, 2012). Yet, transient gating events of this type only require that RLPFC supports a process that is necessary in an event related way. Either transient or continuous BOLD changes could accompany such a dynamic.

Consider that if activation is event-related, increasing only at the event where RLPFC provides a top down input, then the ramping activation function might reflect the aggregate likelihood of these events at each sequence position or an uncertainty code that comes about due to the activation of multiple competing action paths (e.g., Badre et al., 2012). Alternatively, if a context representation of the sequence position in RLPFC evolves continuously over the course of the sequence, it might yield a continuous BOLD change. Such time-varying signals have been commonly employed by computational models that attempt to address temporal coding problems. For example, these types of signals are widely used for temporal order memory and serial recall (Anderson and Matessa, 1997; Anderson et al., 1998; Houghton and Hartley, 1995). Likewise, hierarchical control systems intended to solve sequential tasks commonly include positional codes of this type (e.g., Miller et al., 1960; Schneider and Logan, 2006). Again, however, our results indicate that were this context representation to be represented within RLPFC, it would nevertheless be referenced in a transient, event-related way when it is needed to resolve uncertainty. And, it is only in this transiently active gating state that is vulnerable to disruption. Perhaps consistent with this latter view, recent working memory experiments using univariate as well as multivariate decoding methods in both humans and non-human primates support that contextual information can be sustained by an evolution of neural states throughout a delay period that do not necessarily require sustained neural activity (D’Esposito and Postle, 2015; Riggall and Postle, 2012; Stokes et al., 2013). Moreover, recent studies have suggested that such maintained signals may be robust to disruption, and can be brought back into and out of the focus of attention (Lewis-Peacock et al., 2012). A similar dynamic could unfold here which could paradoxically lead to a continuous change in BOLD, as a context representation passively evolves, yet with only transient necessity, as exposed by TMS. Directed future research will be necessary to distinguish these alternatives.

The present work builds on a small body of prior studies of task-level sequential control (Farooqui et al., 2012; Koechlin and Jubault, 2006; Koechlin et al., 2000). In Farooqui et al. (2012), participants monitored a stream of individual letters for targets from pre-specified sequences of different lengths. The primary result was that a broad network of frontal and parietal areas, including RLPFC, showed increased activation at the sequence termination. These results provide an informative complement to the present results, and taken together, offer another clue regarding the role and dynamics of RLPFC and its associated network during sequential control. First, the observation that activation was greater at the monitored sequence bound is roughly consistent with the present findings, in that the ramping activation also produced the greatest activation at the termination of the task sequence. This consistency is particularly striking because the individual sequence steps in Farooqui et al. (2012) (i.e., the target letters) were distributed unpredictably among a stream of distractors. Thus, unlike the present experiment, progress through the monitored sequence was not tied deterministically to the presentation of new events. This might provide evidence against the hypothesis that signal in RLPFC is ramping monotonically as a function of time, per se, but rather is driven by factors that correlate with progress through the sequence.

It is notable, however, that in potential contrast to our results, Farooqui et al. (2012) did not provide evidence of a ramping activation over intervening events in the sequence in RLPFC. Though no direct test of a monotonically increasing signal through the letter sequences was reported, pairwise comparisons only provided evidence of greater activation at a sequence termination in RLPFC. This difference may have arisen from a potential key difference between the experiments. Specifically, Farooqui et al. (2012) required sequence monitoring, but did not require selecting a new task depending on sequence position (local task switching). Rather, the task level change was always at the sequence boundary. Thus, our tasks may have placed more demands on task selection as a function of sequence position during the intervening sequence events than did Farooqui et al. (2012). Nevertheless, the general pattern of activation across these two studies is consistent with the view that RLPFC provides a transient top down signal to keep performance on track during task sequences.

Finally, Farooqui et al. (2012) reported a broader network of frontal and parietal regions beyond RLPFC that also showed greater activation to the final target letter in the sequence. We likewise reported a larger network of areas exhibiting a ramping pattern over the task sequence. However, our ramping network comprised only a subset of the regions reported by Farooqui et al. (2012). And, indeed, we observed dissociations among some frontal regions in both fMRI and TMS. Though further experimentation is warranted to understand these functional differences, one straightforward possibility is that our task required control at two levels, both local task switching and higher order sequence-level control, that was not required by the sequence monitoring task of Farooqui et al. (2012). Thus, this manipulation at two levels allowed us to expose differences within the fronto-parietal control network.

Another notable prior experiment scanned human participants with fMRI while they performed a sequence of choice RT tasks versus a simple motor sequence (Koechlin and Jubault, 2006). Unlike the present experiment, the task sequence was performed only once, and its initiation and termination were cued externally. In this context, there was no ramping or sustained activation reported in RLPFC or elsewhere in the brain. Thus, one intriguing possibility is that RLPFC activation becomes particularly important when task initiation and monitoring must be internally maintained, as in the present experiment. Other studies of complex sequential motor control involving uncertainty (Yoshida and Ishii, 2006), or the learning of simple sequential tasks (Koechlin et al., 2002) report effects in RMPFC rather than RLPFC. In the present study, we did not see ramping activation in RMPFC and further showed that this region was not necessary for task performance in TMS2. Thus, the RLPFC may function uniquely within task sequences (see Supplemental Discussion section H).

The distinction between the RLPFC and pre-PMd replicates prior dissociations among rostral cortical areas insofar as it is consistent with the observation that these regions are functionally distinct (Badre and D’Esposito, 2007; Koechlin et al., 2003). Further, more recent conceptions of the rostro-caudal organization of lateral frontal cortex have come to emphasize working memory gating and the demand for multiple contingent gating responses arising from fronto-striatal interactions (Badre and Frank, 2012; Chatham et al., 2014; Frank and Badre, 2012). Thus, the current interpretation of a transient top-down influence of RLPFC during moments of high uncertainty is also consistent with this view. Nevertheless, the differentiation between rostral position and caudal sequence complexity effects is less easily accommodated within prior theories regarding an abstraction or temporal gradient, and underscores the importance of testing sequential/temporal factors directly as they relate to hierarchical control. Yet, one potential account of this experiment in an abstraction framework is that the more posterior sequence complexity effect localized in pre-PMd still derives from local task switching, as a function of the frequency with which a top down signal is required to shift from one task to another. Thus, it is important for readying the system for more or less task switching, but it does not necessarily relate to deciding what task to do based on sequence position. Hence, more caudal regions, such as pre-PMd, are engaged by this local switching context, but not by position at the sequence level as with RLPFC. Moreover, the SOA effects that indicated RLPFC was necessary earlier in a trial than pre-PMd (Figure 5B,C) are consistent with a temporal cascade from regions with higher to lower order control. However, these data do not uniquely support this hypothesis, and further work would be required to show such a hierarchical relationship between these networks.

Finally, the fMRI experiment located ramping activation in areas outside of RLPFC, including regions of medial PFC, superior temporal cortex, and superior parietal cortex. This observation touches on an important and timely issue in neuroscience, namely the tension between a focus on the distributed, network-level origins of behavioral phenomena and the known heterogeneity of regions within these networks. The observation that more than one region of the brain exhibited ramping activation cautions that our focus on RLPFC using TMS does not indicate that other regions of the brain may not coordinate with RLPFC in serving this uncertainty resolution function.

Importantly, however, these results do not reflect a lack of specificity in the sense that the whole brain or any region involved in the task shows this same pattern and is equally necessary for this function. Rather, the distributed set of regions showing the specific ramping pattern is fully in line with growing evidence that specific but large scale distributed networks of regions tend to correlate with each other during task performance, and that these network boundaries may reflect the macro-level functional topography of the brain (Yeo et al., 2011). As evidence of this, the medial frontal and parietal regions show a striking correspondence to regions known to have functionally connectivity with RLPFC at rest (Figure S2).

Furthermore, ramping was not characteristic of every task-active region in the brain or even in the whole prefrontal cortex. For example, pre-PMd was task active, showed sequence level effects (Sequence Complexity), and was necessary at sequence initiation. Yet, this region did not show ramping activation. Pre-PMd was dissociable from RLPFC both using fMRI and TMS. Thus, while the ramping effect in RLPFC is not fully localized to one and only one region (in a classical cognitive localization sense), neither is ramping activation universal or unpredictably distributed throughout the brain. Rather, it is evident that ramping activation is restricted to a distributed, yet specific, set of regions.

The ramping activation in superior temporal cortex was outside of the network that correlates with RLPFC at rest. Nevertheless, this observation may be consistent with recent evidence implicating this region in identifying boundaries within latent statistical structures, a potential link to its role in semantic representation (Schapiro et al., 2013). In this prior study, participants were presented a series of visual stimuli wherein trial-to-trial transition probabilities derived from an underlying community structure that included boundaries between sub-communities. Among other regions, activation increased at community boundaries in the superior temporal gyrus, a site that overlapped with ramping activation in the present study. To the degree that task sequences represent a type of community structure, the pattern observed here may similarly reflect encoding of the sequence boundary by temporal cortex during sequential behavior, as opposed to the uncertainty resolution function we ascribe to RLPFC.

Thus, our focus in the present fMRI and TMS experiments on RLPFC should not be taken to indicate that RLPFC is the only region to perform an uncertainty resolution function during this task. Further, we cannot rule out the possibility from this study alone that the impact of stimulation may occur downstream from RLPFC or that it is disruption of some dynamic between RLPFC and other regions that is impacted by spTMS. Rather, follow up work, such as using the TMS approach employed here, will be required to distinguish whether other specific regions of medial frontal, parietal, and lateral temporal cortex support the same or different functions than we have specified for RLPFC here.

To conclude, we have presented evidence that the RLPFC is necessary to overcome uncertainty and “keep us on track” during a task sequence, differentially so relative to other frontal cortical areas. These previously unobserved dynamics were exposed by requiring performance of a sequence at an abstract task level (rather than motor) and without substantial prior training. Given the ubiquity of sequential action of this type in daily life, these results provide insight into how the brain solves a problem at the very basis of independent, adaptive behavior.

Experimental Procedures

Participants

Twenty-eight (18 female) right-handed adults (ages 18 to 28; mean 20) participated in the fMRI study. Six of the fMRI study participants also participated in TMS1, along with 15 additional participants for a total of 21 participants (12 female; ages 18 to 27; mean 21). All had normal or corrected-to-normal vision, and were screened for the presence of psychiatric or neurological conditions, the use of CNS affecting drugs, and for contraindications for MRI. TMS participants were also screened for contraindications specific to TMS. All participants gave informed, written consent as approved by the Human Research Protections Office of Brown University, and they were compensated for their participation.

Procedure

On each trial, a stimulus was displayed with a particular color (red or blue), shape (circle or square), and size (small [3.5 × 3.5 cm] or large [7.0 × 7.0 cm]) (Figure 1A). All stimuli were presented on a black background and all text was white. Depending on the current task, the participant classified either the image’s color or shape by pressing one of two keys within 4 s. The stimulus remained on the screen until the response, and immediately after the response the fixation cross was shown and the jittered ITI began (fMRI: 0.25 – 8 s, mean 2 s; TMS: 0.25 – 5, mean 0.93 s).

The set of stimulus-response mappings (i.e., which finger mapped to which stimulus feature) remained the same throughout the experiment, was presented with each stimulus, and was counterbalanced across subjects. Response Congruency was controlled across conditions. The size of each stimulus though variable was not task relevant in this experiment.

Stimuli were presented in blocks (fMRI: 24–27 trials; TMS: 48–51 trials; Figure 1B). An instructed sequence of four tasks was repeated continuously throughout each block of trials. Each block began with a 4 sec instruction screen followed by fixation (fMRI: 16 s; TMS: 1 s). The instruction screen indicated the sequence of judgments the participant should make throughout the block, e.g. COLOR, COLOR, SHAPE, SHAPE. This would indicate that the first and second trials should be color judgments, and the third and fourth trials shape judgments. Following the fourth task in the sequence, the participant started the sequence over on the next trial, and repeated the instructed sequence of four judgments continuously until the end of the block of trials. Note that in order to know what judgment to make, the participant had to keep track of where they were in sequence. No cues were provided during the block to cue the appropriate task or position in the sequence.

Blocks could be terminated at any position within a sequence with equal frequency. At the conclusion of a block, participants were probed for which task in the sequence they would have performed next to test whether they had tracked the correct sequence for the duration of the block. Following their response, a fixation cross was present until the 5 s response interval elapsed (mean time remaining = 3.2 s), there was an occasional additional inter-block-interval (see run structure below), and the next block began with a new instruction.

There were two types of sequences that could be performed in a given block (Figure 1D). “Simple” sequences were of the form AABB, where A and B are generic labels for the two different tasks, i.e. color-color-shape-shape (CCSS) and SSCC. These sequences were considered Simple because they had only one task switch in the interior of each sequence (A to B in AABB). By contrast, Complex sequences were of the form ABBA (CSSC and SCCS), and so contained two task switches (A to B and B to A in ABBA). Though there are different numbers of task switches interior to the sequences, the number of switches across the entire block is equivalent as participants repeat each sequence. Thus, the local probability of a Switch or Repeat trial is equal between blocks of Simple and Complex sequences (i.e., 50%).

Each run consisted of four blocks, one of each of the four sequences (Figure 1C), and the order was counterbalanced across the runs (fMRI: n = 5, TMS: n = 8). The 8 possible stimuli, formed from the combination of color, shape, and size, appeared an approximately equal number of times across the course of the experiment, were not allowed to repeat on adjacent trials, and were counterbalanced for response repeats and switches. In the fMRI experiment only, there was 16 s of fixation time before the first block and after the last block of each run. To provide additional baseline time, block 1, 2, or 3 was chosen at random (counter balanced across runs) to have an extra 16 s of fixation inserted before the next block began. Prior to performing the behavioral task in the scanner or with TMS, participants were given instructions and practice on all the tasks and sequences to eliminate effects due to initial learning.

fMRI data analysis

Behavior was analyzed and functional images were acquired and preprocessed using standard procedures (see Supplemental Experimental Procedures).

The 5 general linear models (GLMs) applied to the data using SPM8 were as follows: Onsets Model. To test the basic univariate effects of local task switching, sequence complexity, and sequence position, we constructed a model using instantaneous stimulus onset regressors based on the crossing of Sequence Complexity (Simple/Complex) × Sequence Position (1–4) (Figure 3A).

Parametric sequence position ramp model

This model explicitly tests for ramping activation that increased with sequence position. A parametric regressor of Sequence Position (1–4) was added as a modulator of trial onsets for all positions (Figure 3B). Note that the four Position stimulus onsets were modeled with the same regressor (unlike the Onsets Model) to enable the search for patterns of activations spanning the four positions in the sequence. As implemented in SPM8, this parametric regressor was estimated hierarchically (i.e., after the onsets). Thus, any activation related to the parametric regressor is what can be explained by a ramp function above and beyond what is explained by stimulus onset.

Sustain vs. Unique Ramp model

This model contained Sustain and Ramp regressors (separated for each sequence type) that would compete for variance, in addition to a single regressor for the stimulus onsets at all positions. These regressors started at the stimulus onset of each sequence position 1 and ended at the stimulus offset (response) of sequence position 4 (Figure 3C,D). As the Sustain and Ramp functions share variance, we sought to identify what variance was uniquely explained by each function. This first of a pair of models sought to determine the variance uniquely explained by the Ramp regressor. We orthogonalized (spm_orth.m) the Sustain and Ramp regressors within each sequence type to remove the shared variance from the Ramp regressors (and assign it to the Sustain regressors).

Unique Sustain vs. Ramp model

This second model of the pair sought to identify any variance uniquely explained by the Sustain regressor (independent of Ramp). Specifically, we removed the shared variance from the Sustain regressor (and assigned it to the Ramp regressor). All other aspects of the model were the same as the Sustain vs. Unique Ramp model above.

Parametric sequence dwell time ramp model

This final model tested for the effect of the elapsed time at each sequence position since the initiation of a given sequence. Specifically, parametric regressors were included in the Onsets Model for the total elapsed time since the onset of Position 1 in each sequence separately for Positions 2–4 (Figure 3A,E). Hence, these parametric regressors tested what variance could be explained by the length of time spent in a sequence above and beyond that accounted for by trial onset and by sequence position.

Region of interest analysis

Region of interest (ROI) analyses complemented whole-brain analyses. Two ROIs were defined in an unbiased manner, with respect to the specific conditions in this experiment, from significant peaks of activation found in the onsets model voxelwise contrast of All Stimulus Onsets > Baseline. The supplementary motor area/pre-supplementary motor area (SMA/pre-SMA; All Stimulus Onsets > Baseline center of mass at xyz = 6, 14, 48 mm; total volume of 2056 mm) ROI was chosen for the proximity to other task switching studies (e.g. Cools et al., 2002; Dove et al., 2000; Kenner et al., 2010; Monchi et al., 2001; Rushworth et al., 2002). The pre-dorsal premotor cortex (pre-PMd, xyz = −46, 4, 28; vol. 2056 mm) ROI was chosen for the proximity to ROIs in a previous study of non-sequential hierarchical control (Badre and D’Esposito, 2007). All significant voxels within an 8 mm sphere around the peak were taken for each ROI. Rostrolateral prefrontal cortex (RLPFC) did not show reliable activation in the All > Baseline contrast. Thus, in order to test effects in RLPFC, we chose ROI from a prior study of branching (Koechlin et al., 1999) (xyz = −36, 57, 9). We used an 8 mm sphere centered on the previous study’s peak of activation. All ROIs were masked with each individual participants brain to ensure activations would only arise from the participant’s brain and not noise space.

The time course of the activity for Sequence Complexity × Sequence Position conditions was extracted using an 8-timepoint (16 sec) finite impulse response (FIR) model (using MarsBar SPM toolbox) that contained the same regressors as the standard Onsets Model. We chose to use an FIR model rather than using the canonical HRF because it was unknown how the temporal demands of this task might affect the canonical response function (that is typically established by transient event-related responses). Thus, our approach did not bias our results towards either an event related or sustained dynamic. A mean time course was obtained for each participant across the five runs. The mean percent signal change was taken at the peak of the resultant time course, which was the same for all conditions and ROIs (time point 3, 6 sec), and the resultant data were subjected to RM-ANOVA or paired t-tests.

To compare estimates for the regressors in the Sustain vs. Unique Ramp and Unique Sustain vs. Ramp Models specifically in the RLPFC, we first estimated the model with respect to the mean time course from all voxels within the RLPFC (using MarsBar SPM toolbox). Beta values (regressor estimates) were then extracted for each regressor. Mean beta values across all five runs were obtained for each participant and subjected to RM-ANOVA or paired t-tests.

TMS procedure

Participant-specific placement of the coil over target regions was determined using an MRI image-guided stereotaxic system (Rogue Research Inc.) and the subject’s own high-resolution MRI image. All stimulus output and timing parameters were within established safe ranges (Rossi et al., 2009). We first applied single-pulse TMS (spTMS) to primary motor cortex to determine motor threshold. EMG electrodes were affixed to the first dorsal interosseus and the first proximal interphalangeal joint, and a reference electrode to the elbow. We determined resting motor threshold (RMT), defined as the minimum stimulator intensity needed to elicit a 50 μV or greater EMG response in a target muscle in 50% of pulses while the target muscle is voluntarily relaxed. We adjusted the intensity of stimulation incrementally until the MT is achieved. We used a Magstim Bistim2 stimulator and Magstim D702 coil to deliver timed spTMS to a target region at 96%–121% of resting MT while the participant performed the task. The experimental computer triggered pulses. Calibration with a photodiode/oscilloscope ensured millisecond precision timing.

Target coordinates were determined from the peak activation on the group level in the Parametric Ramp > Baseline contrast for RLPFC (xyz = −32, 58, 12), and in the Complex > Simple contrast for pre-PMd (xyz = −48, −6, 26). For the six participants who had also participated in the fMRI experiment, the target coordinates were taken as the peak of activation in the same respective contrasts that was within 10 mm of the group peak.

The target stimulation level for RLPFC was 110% RMT, and 120% RMT for pre-PMd. The difference was in order to compensate for the increased discomfort of stimulation at the RLPFC site. Stimulation levels were adjusted for participant comfort before starting the experimental session. Stimulation ranged from 96% to 112% (mean 107%) in RLPFC and ranged from 110% to 121% (mean 118%) in pre-PMd. The TMS coil handle was held approximately vertically for RLPFC stimulation and at approximately a 45-degree angle relative to the midline for pre-PMd stimulation. This angle was adjusted on a per-participant basis in an attempt to minimize superficial muscle twitches around the site of stimulation.

SpTMS pulses were delivered on no more than one trial during a four-trial sequence. With at least 48 trials per block, there were a total of 12 complete four-trial sequences per block. The first sequence per block was not included for analysis (see Behavioral analysis section); therefore, stimulation was never delivered in the first four trials of each block. Over the course of the entire experiment, there were 11 sequences per block available for stimulation, times 4 blocks, times 8 runs, equals 352 total sequences. Twenty of those sequences were set aside as pure no-stimulation controls. The remaining 332 sequences each had stimulation delivered at only one Position in the sequence spread evenly across the positions for a total of 83 stimulations at each position (82 at Position 1).

TMS stimulation was delivered at 10 different stimulus onset asynchronies (SOA), spanning 90 ms to 450 ms after stimulus onset in 40 ms bins. Approximately 8 TMS pulses were delivered in each bin at Position 1. Across Positions 2, 3, and 4, 25 samples at each SOA bin were collected. The number of TMS pulses delivered was balanced across Sequence Complexity, local task switches and repeats, and the identity of the stimulus presented. TMS pulses were never delivered in less than 5 s of the last TMS pulse for a maximum rate of 0.2 Hz.

For the second TMS experiment (TMS2), spTMS pulses were delivered during the task just as in the first experiment with only two changes. First, instead of 10 different SOAs, the three peak SOAs from the first TMS experiment (TMS1) were used for each region stimulated: 90, 130, and 170 ms in RLPFC; 370, 410, and 450 ms in pre-PMd. This necessitated relatively fewer trials than TMS1, and therefore, 5 runs were used instead of 8.

Second, in addition to stimulating RLPFC (xyz = −32, 58, 12) and pre-PMd (xyz = −48, −6, 26) as in TMS1, we added a second control region, rostromedial prefrontal cortex (RMPFC, xyz = 0, 61, 19). This control region was included to be highly similar in terms of peripheral sensations to the RLPFC, but based on the fMRI data this region was not expected to show effects of stimulation as a function of position. The target stimulation level for RLPFC and pre-PMd was again 110% and 120% of RMT, respectively. The stimulator output for RMPFC for each participant was matched to RLPFC according to the scalp-to-cortex distance (Stokes et al., 2007) using the following formula:

Outputsite2=Outputsite1+2.8×(Dsite2−Dsite1)

where the percent of stimulator output for Site 1 and Site 2 are Outputsite1 and Outputsite2, respectively; and the cortical distances are Dsite1 and Dsite2. Stimulation ranged from 102% to 111% (mean 107%) in RLPFC, ranged from 108% to 122% (mean 116%) in pre-PMd, and ranged from 106% to 127% (mean 118%) in RMPFC. The same stimulation timings used for RLPFC were used in RMPFC.

Post-test questionnaires were administered to all participants using online survey tools (Qualtrics). It included a question that asked participants to rate the amount of discomfort they experienced as a result of the stimulation. Responses ranged from zero (no discomfort) to 5 (a lot of discomfort). To further enhance our self-report measures, for TMS2 we added a question to the post-test questionnaire so that in addition to asking participants to rate the amount of discomfort they experienced as in TMS1, we asked participants to rate the amount of distraction they experienced on a scale of 1–5, where 1 was a little and 5 was a lot.

TMS1 data analysis

Two TMS sessions and one run from three participants were excluded from analyses: one session and one run for explicitly not following task instructions, and one session for excessively poor performance on the task (23% error). Of the remaining participants/sessions, 15 participated in two separate sessions, one for RLPFC and one for pre-PMd stimulation. The order of the sessions was counterbalanced across participants in an alternating manner to maintain approximately equal numbers of participants in each group. Only one session was included from 5 participants (3 RLPFC and 2 pre-PMd). This yielded a total of 18 sessions for analysis in RLPFC and 17 in pre-PMd.

Given the mixed within- and between-participants design, we performed statistical analyses on the group level conservatively (Russo et al., 1998) by treating the dataset as an entirely between-participants design within SPSS (IBM) and submitting RTs and ERs to RM-ANOVA and t-tests where appropriate. For RM-ANOVA across stimulation sites, the level of stimulation delivered as a percentage of RMT was entered as a covariate. All other trial trimming and analysis procedures were the same as described in Behavioral analysis.

TMS2 data analysis

Thirty-six participants participated in TMS2. Two sessions were excluded for technical difficulties: one session the pulses were not triggered correctly and one where the response mappings were not correctly assigned. Three TMS2 participants were excluded for not following task instructions based on their self-report during a post-test questionnaire. Of the remaining participants, 15 participated in two separate sessions, one for RLPFC and one for RMPFC stimulation with the order of the sessions counterbalanced across participants. A separate set of 16 participants participated in a single session of pre-PMd stimulation. A total of 31 participants were included in the analysis. Thus, for analyses across RLPFC and RMPFC the dataset was a within-participants design and RM-ANOVA and t-tests were used as appropriate. For analyses across RLPFC and pre-PMd or RMPFC and pre-PMd the dataset was a between-participants design and the level of stimulation delivered as a percentage of RMT was entered as a covariate.

Supplementary Material

Suppl

Acknowledgments

The authors would like to thank Kathryn Graves, Sarah Master, Ashley Wu, and Jeffrey Lipton for their contributions to this work. We also thank members of the Badre Lab for many helpful discussions during the preparation of this manuscript. Research reported in this publication was supported by NINDS of the NIH (R01NS065046, D.B. and C.H.C.; F32NS080593, T.M.D.), the NIMH of the NIH (T32MH019118, T.M.D), Alfred P. Sloan Foundation (D.B.), the James S. McDonnell Foundation (D.B.), and the Brown Institute for Brain Science (TMS equipment).

Footnotes

Author Contributions

T.M.D. designed the experiment, performed the research, analyzed the data, and wrote the manuscript. C.H.C. designed the experiment, performed the research, and edited the manuscript. D.B. designed the experiment and wrote the manuscript.

References

  1. Anderson JR, Matessa M. A production system theory of serial memory. Psychol Rev. 1997;104:728–748. [Google Scholar]
  2. Anderson JR, Bothell D, Lebiere C, Matessa M. An Integrated Theory of List Memory. J Mem Lang. 1998;38:341–380. [Google Scholar]
  3. Badre D, D’Esposito M. Functional magnetic resonance imaging evidence for a hierarchical organization of the prefrontal cortex. J Cogn Neurosci. 2007;19:2082–2099. doi: 10.1162/jocn.2007.19.12.2082. [DOI] [PubMed] [Google Scholar]
  4. Badre D, Frank MJ. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 2: evidence from fMRI. Cereb Cortex. 2012;22:527–536. doi: 10.1093/cercor/bhr117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Badre D, Wagner AD. Selection, integration, and conflict monitoring; assessing the nature and generality of prefrontal cognitive control mechanisms. Neuron. 2004;41:473–487. doi: 10.1016/s0896-6273(03)00851-1. [DOI] [PubMed] [Google Scholar]
  6. Badre D, Hoffman J, Cooney JW, D’Esposito M. Hierarchical cognitive control deficits following damage to the human frontal lobe. Nat Neurosci. 2009;12:515–522. doi: 10.1038/nn.2277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Badre D, Doll BB, Long NM, Frank MJ. Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron. 2012;73:595–607. doi: 10.1016/j.neuron.2011.12.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bahlmann J, Beckmann I, Kuhlemann I, Schweikard A, Münte TF. Transcranial magnetic stimulation reveals complex cognitive control representations in the rostral frontal cortex. Neuroscience. 2015 doi: 10.1016/j.neuroscience.2015.05.058. [DOI] [PubMed] [Google Scholar]
  9. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature. 2005;437:1158–1161. doi: 10.1038/nature04053. [DOI] [PubMed] [Google Scholar]
  10. Boorman ED, Behrens TEJ, Woolrich MW, Rushworth MFS. How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action. Neuron. 2009;62:733–743. doi: 10.1016/j.neuron.2009.05.014. [DOI] [PubMed] [Google Scholar]
  11. Botvinick M, Plaut DC. Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. Psychol Rev. 2004;111:395–429. doi: 10.1037/0033-295X.111.2.395. [DOI] [PubMed] [Google Scholar]
  12. Braver TS, Bongiolatti SR. The role of frontopolar cortex in subgoal processing during working memory. Neuroimage. 2002;15:523–536. doi: 10.1006/nimg.2001.1019. [DOI] [PubMed] [Google Scholar]
  13. Braver TS, Reynolds JR, Donaldson DI. Neural mechanisms of transient and sustained cognitive control during task switching. Neuron. 2003;39:713–726. doi: 10.1016/s0896-6273(03)00466-5. [DOI] [PubMed] [Google Scholar]
  14. Chatham CH, Badre D. Multiple gates on working memory. Curr Opin Behav Sci. 2015;1:23–31. doi: 10.1016/j.cobeha.2014.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Chatham CH, Frank MJ, Badre D. Corticostriatal output gating during selection from working memory. Neuron. 2014;81:930–942. doi: 10.1016/j.neuron.2014.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cools R, Clark L, Owen AM, Robbins TW. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 2002;22:4563–4567. doi: 10.1523/JNEUROSCI.22-11-04563.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. D’Esposito M, Postle BR. The Cognitive Neuroscience of Working Memory. Annu Rev Psychol. 2015;66:115–142. doi: 10.1146/annurev-psych-010814-015031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ. Cortical substrates for exploratory decisions in humans. Nature. 2006;441:876–879. doi: 10.1038/nature04766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. DiGirolamo GJ, Kramer AF, Barad V, Cepeda NJ, Weissman DH, Milham MP, Wszalek TM, Cohen NJ, Banich MT, Webb A, et al. General and task-specific frontal lobe recruitment in older adults during executive processes: a fMRI investigation of task-switching. Neuroreport. 2001;12:2065–2071. doi: 10.1097/00001756-200107030-00054. [DOI] [PubMed] [Google Scholar]
  20. Dosenbach NUF, Visscher KM, Palmer ED, Miezin FM, Wenger KK, Kang HC, Burgund ED, Grimes AL, Schlaggar BL, Petersen SE. A core system for the implementation of task sets. Neuron. 2006;50:799–812. doi: 10.1016/j.neuron.2006.04.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Dove a, Pollmann S, Schubert T, Wiggins CJ, von Cramon DY. Prefrontal cortex activation in task switching: an event-related fMRI study. Brain Res Cogn Brain Res. 2000;9:103–109. doi: 10.1016/s0926-6410(99)00029-4. [DOI] [PubMed] [Google Scholar]
  22. Dreher J-C, Koechlin E, Tierney M, Grafman J. Damage to the fronto-polar cortex is associated with impaired multitasking. PLoS One. 2008;3:e3227. doi: 10.1371/journal.pone.0003227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Farooqui Aa, Mitchell D, Thompson R, Duncan J. Hierarchical organization of cognition reflected in distributed frontoparietal activity. J Neurosci. 2012;32:17373–17381. doi: 10.1523/JNEUROSCI.0598-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Frank MJ, Badre D. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb Cortex. 2012;22:509–526. doi: 10.1093/cercor/bhr114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Fujii N, Graybiel AM. Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science. 2003;301:1246–1249. doi: 10.1126/science.1086872. [DOI] [PubMed] [Google Scholar]
  26. Gilbert SJ, Spengler S, Simons JS, Steele JD, Lawrie SM, Frith CD, Burgess PW. Functional specialization within rostral prefrontal cortex (area 10): a meta-analysis. J Cogn Neurosci. 2006;18:932–948. doi: 10.1162/jocn.2006.18.6.932. [DOI] [PubMed] [Google Scholar]
  27. Houghton G, Hartley T. Parallel models of serial behaviour: Lashley revisited. Psyche (Stuttg) 1995;2 [Google Scholar]
  28. Jin X, Costa RM. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature. 2010;466:457–462. doi: 10.1038/nature09263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kenner NM, Mumford Ja, Hommer RE, Skup M, Leibenluft E, Poldrack Ra. Inhibitory motor control in response stopping and response switching. J Neurosci. 2010;30:8512–8518. doi: 10.1523/JNEUROSCI.1096-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kim C, Cilles SE, Johnson NF, Gold BT. Domain general and domain preferential brain regions associated with different types of task switching: a meta-analysis. Hum Brain Mapp. 2012;33:130–142. doi: 10.1002/hbm.21199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Koechlin E, Jubault T. Broca’s area and the hierarchical organization of human behavior. Neuron. 2006;50:963–974. doi: 10.1016/j.neuron.2006.05.017. [DOI] [PubMed] [Google Scholar]
  32. Koechlin E, Basso G, Pietrini P, Panzer S, Grafman J. The role of the anterior prefrontal cortex in human cognition. Nature. 1999;399:148–151. doi: 10.1038/20178. [DOI] [PubMed] [Google Scholar]
  33. Koechlin E, Corrado G, Pietrini P, Grafman J. Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. Proc Natl Acad Sci U S A. 2000;97:7651–7656. doi: 10.1073/pnas.130177397. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Koechlin E, Danek A, Burnod Y, Grafman J. Medial prefrontal and subcortical mechanisms underlying the acquisition of motor and cognitive action sequences in humans. Neuron. 2002;35:371–381. doi: 10.1016/s0896-6273(02)00742-0. [DOI] [PubMed] [Google Scholar]
  35. Koechlin E, Ody C, Kouneiher F. The architecture of cognitive control in the human prefrontal cortex. Science. 2003;302:1181–1185. doi: 10.1126/science.1088545. [DOI] [PubMed] [Google Scholar]
  36. Kool W, McGuire JT, Rosen ZB, Botvinick MM. Decision making and the avoidance of cognitive demand. J Exp Psychol Gen. 2010;139:665–682. doi: 10.1037/a0020198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kovach CK, Daw ND, Rudrauf D, Tranel D, O’Doherty JP, Adolphs R. Anterior prefrontal cortex contributes to action selection through tracking of recent reward trends. J Neurosci. 2012;32:8434–8442. doi: 10.1523/JNEUROSCI.5468-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lashley KS. The problem of serial order in behavior. In: Jeffress LA, editor. Cerebral Mechanisms in Behavior. New York: John Wiley & Sons; 1951. pp. 112–146. [Google Scholar]
  39. Lewis-Peacock Ja, Drysdale AT, Oberauer K, Postle BR. Neural Evidence for a Distinction between Short-term Memory and the Focus of Attention. J Cogn Neurosci. 2012;24:61–79. doi: 10.1162/jocn_a_00140. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Lien M-C, Ruthruff E. Task switching in a hierarchical task structure: evidence for the fragility of the task repetition benefit. J Exp Psychol Learn Mem Cogn. 2004;30:697–713. doi: 10.1037/0278-7393.30.3.697. [DOI] [PubMed] [Google Scholar]
  41. Miller G, Galanter E, Pribram K. Plans and the structure of behavior. New York: Rinehart and Winston, Inc.; 1960. [Google Scholar]
  42. Monchi O, Petrides M, Petre V, Worsley K, Dagher a. Wisconsin Card Sorting revisited: distinct neural circuits participating in different stages of the task identified by event-related functional magnetic resonance imaging. J Neurosci. 2001;21:7733–7741. doi: 10.1523/JNEUROSCI.21-19-07733.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Nee DE, Jahn A, Brown JW. Prefrontal Cortex Organization: Dissociating Effects of Temporal Abstraction, Relational Abstraction, and Integration with fMRI. Cereb Cortex. 2013 doi: 10.1093/cercor/bht091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Orr JM, Banich MT. The neural mechanisms underlying internally and externally guided task selection. Neuroimage. 2013;84C:191–205. doi: 10.1016/j.neuroimage.2013.08.047. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Passingham RE, Rowe JB. Principles of Frontal Lobe Function. In: Stuss DT, Knight R, editors. Principles of Frontal Lobe Function. New York: Oxford University Press; 2002. pp. 221–232. [Google Scholar]
  46. De Pisapia N, Sandrini M, Braver TS, Cattaneo L. Integration in working memory: a magnetic stimulation study on the role of left anterior prefrontal cortex. PLoS One. 2012;7:e43731. doi: 10.1371/journal.pone.0043731. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Riggall AC, Postle BR. The Relationship between Working Memory Storage and Elevated Activity as Measured with Functional Magnetic Resonance Imaging. J Neurosci. 2012;32:12990–12998. doi: 10.1523/JNEUROSCI.1892-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Rossi S, Hallett M, Rossini PM, Pascual-Leone A. Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research. Clin Neurophysiol. 2009;120:2008–2039. doi: 10.1016/j.clinph.2009.08.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Rushworth MFS, Hadland KA, Paus T, Sipila PK. Role of the human medial frontal cortex in task switching: a combined fMRI and TMS study. J Neurophysiol. 2002;87:2577–2592. doi: 10.1152/jn.2002.87.5.2577. [DOI] [PubMed] [Google Scholar]
  50. Russo R, Cullis AM, Parkin AJ. Consequences of violating the assumption of independence in the process dissociation procedure: a word fragment completion study. Mem Cognit. 1998;26:617–632. doi: 10.3758/bf03211382. [DOI] [PubMed] [Google Scholar]
  51. Schapiro AC, Rogers TT, Cordova NI, Turk-Browne NB, Botvinick MM. Neural representations of events arise from temporal community structure. Nat Neurosci. 2013;16:486–492. doi: 10.1038/nn.3331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Schneider DW, Logan GD. Hierarchical control of cognitive processes: switching tasks in sequences. J Exp Psychol Gen. 2006;135:623–640. doi: 10.1037/0096-3445.135.4.623. [DOI] [PubMed] [Google Scholar]
  53. Smith KS, Graybiel AM. A dual operator view of habitual behavior reflecting cortical and striatal dynamics. Neuron. 2013;79:361–374. doi: 10.1016/j.neuron.2013.05.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Stokes MG, Chambers CD, Gould IC, English T, McNaught E, McDonald O, Mattingley JB. Distance-adjusted motor threshold for transcranial magnetic stimulation. Clin Neurophysiol. 2007;118:1617–1625. doi: 10.1016/j.clinph.2007.04.004. [DOI] [PubMed] [Google Scholar]
  55. Stokes MG, Kusunoki M, Sigala N, Nili H, Gaffan D, Duncan J. Dynamic coding for cognitive control in prefrontal cortex. Neuron. 2013;78:364–375. doi: 10.1016/j.neuron.2013.01.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Yeo BTT, Krienen FM, Sepulcre J, Sabuncu MR, Lashkari D, Hollinshead M, Roffman JL, Smoller JW, Zöllei L, Polimeni JR, et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol. 2011;106:1125–1165. doi: 10.1152/jn.00338.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Yoshida W, Ishii S. Resolution of Uncertainty in Prefrontal Cortex. Neuron. 2006;50:781–789. doi: 10.1016/j.neuron.2006.05.006. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Suppl
NIHMS717961-supplement-Suppl.pdf (3.9MB, pdf)

RESOURCES