Corticostriatal output gating during selection from working memory

Summary

Convergent evidence suggests that corticostriatal interactions act as a gate to select the input to working memory (WM). However, not all information in WM is relevant for behavior simultaneously. For this reason, a second “output gate” might advantageously govern which contents of WM influence behavior. Here, we test whether frontostriatal circuits previously implicated in input gating also support output gating during selection from WM. FMRI of a hierarchical rule task with dissociable input and output gating demands demonstrated greater lateral PFC recruitment and frontostriatal connectivity during output gating. Moreover, PFC and striatum correlated with distinct behavioral profiles. Whereas PFC recruitment correlated with the mean efficiency of selection from WM, striatal recruitment and frontostriatal interactions correlated with its reliability, as though such dynamics stochastically gate WM's output. These results support the output gating hypothesis and suggest that contextual representations in PFC influence striatum to select which information in WM drives responding.

Introduction

Cognitive control requires a balance between two incompatible demands: flexibly updating goals versus robustly maintaining them. One solution to this dilemma is to separate the neural mechanisms for working memory (WM) maintenance from those that update the information that is to be maintained, i.e. a WM “gate”. This computational division of labor is thought to have anatomical correlates, with prefrontal cortex (PFC) supporting maintenance and basal ganglia (BG) supporting gating (Braver and Cohen, 2000; Frank et al., 2001; Gruber et al. 2006; Frank and O'Reilly, 2006; Cools et al., 2007). From this perspective, disinhibition of cortico-striato-thalamic loops enables the selective updating of task-relevant information into WM. Once maintained, information supported by PFC is available to exert a top-down bias on posterior neocortex (Desimone and Duncan, 1995). This type of selective control over the input to WM – termed “input gating” – relies on dopaminergic corticostriatal systems (Frank and O'Reilly, 2006; Moustafa, et al., 2008; Cools et al., 2010; Murty et al., 2011; McNab and Klingberg, 2008).

However, not everything in WM will be relevant for behavior at any one point in time. Rather, it is also adaptive to control which representations within WM can influence attention and behavior, and when. Such selection from within WM or “singling out” of WM representations (Oberauer and Hein, 2012) is resource-demanding and PFC-dependent (e.g. Rowe et al., 2000; Bunge et al., 2002; Hester et al., 2007; Tamber-Rosenau et al., 2011). Nevertheless, relative to control over the input to WM, little evidence exists regarding this type of selection process and its relationship to cognitive control.

One hypothesis is that selection from within WM can be conceived of as a gating function analogous to that described above for WM updating. From this perspective, an “output gate” may control the flow of information within WM between an actively maintained but inert state to one that is capable of exerting a top down influence on behavior. In other words, for any given WM representation, when the output gate is closed, that representation would be maintained but would not have a top down influence. Conversely, when the output gate is opened, the maintained representation provides a top down contextual signal. Output gating is a shared solution to the problem of selection from within WM across many computational models (Hochreiter and Schimidhuber, 1997; Hazy et al., 2007; Brown et al., 2007; Kriete and Noelle, 2011; Chatham et al., 2011; Eliasmith et al 2012; Frank and Badre, 2012; Collins and Frank, 2013; Huang et al., 2013).

Like input gating, output gating may be hypothesized to arise from cortico-striato-thalamic loops, wherein candidate contextual representations maintained in PFC act as input to dorsal striatum, which in turn amplifies one of these representations via its pallado-thalamic disinhibitory loop (Hazy et al., 2007; Kriete and Noelle, 2011; Chatham et al., 2011; Collins and Frank, 2013; Kriete et al., 2013). This putative output gating dynamic for selection of a WM representation is an extension of more established interactions between cortex and dorsal striatum during selection of a candidate motor plan (e.g., Mink 1996; Graybiel, 1998; Gurney et al. 2001; Brown et al., 2004). According to this general class of accounts, some potential stimulus-response action plan represented by premotor cortex is “gated out” by striatum, and thereby becomes the motor plan executed by primary motor cortex. However, the hypothesis that similar frontostriatal interactions could support selection of a WM representation currently lacks support. The present study seeks to fill this gap.

To test output gating of WM, we focus on hierarchical control tasks requiring the use of conditional rules (e.g., the shape of a stimulus cues whether to attend to the size or color of a stimulus, and size or color then determine which response must be made.) In the brain, simpler rules that directly map a stimulus to a response tend to recruit more caudal frontal cortex than rules involving higher-order contingencies (Koechlin et al., 2003; Badre and D'Esposito, 2007; Badre et al., 2009; 2010), with a potentially similar rostral-to-caudal gradient of function in striatum owing to its parallel rostrocaudal organization (Badre et al., 2010; Badre and Frank, 2012; Verstynen et al., 2012; Mestres-Missé et al., 2012). Importantly, such conditional rules rely on higher order contexts (e.g., shape) to select which lower order context (e.g., the color or size) should drive action. Thus, following such rules might rely on output gating, if these lower order representations must be selected from within WM. Indeed, a recent computational model predicts performance advantages arise from rostrocaudal frontostriatal organization specifically if rostral frontal areas modulate the output gating of more caudal frontostriatal circuits (Frank and Badre, 2012). Although this model fits behavioral and fMRI data during hierarchical rule learning (Badre and Frank, 2012), its predictions regarding the priority of output gating dynamics for hierarchical frontostriatal interactions are untested.

To this end we used a novel second-order control task to separately manipulate demands on output or input gating, while also controlling for differences in WM load. Subjects based their responses on one of two possible letters and/or two possible “wingding” characters depending on the identity of a number cue (Fig 1). The number acted as a higher-level context by specifying which set of lower-level items (letters or wingdings or both) would be relevant to the response for that trial. The final stimulus event for each trial also included a set of response mappings. Subjects pressed a button corresponding to the side of the response mappings (left vs. right) on which the relevant item(s) appeared (Fig. 1a).

Figure 1. Task rules and example trials.

(a) Numbers acted as higher-order context, specifying which of the lower-level items (2 possible letters and 2 possible “wingdings”) would be relevant on each trial. The numbers “1” and “2” indicate that only the wingdings or letters, respectively, would be relevant for responses; these are “selective” contexts. The number “3” is termed a “global” context, because it indicates that both the letter and the wingding would be relevant. (b-e) All trials conclude with response mappings, to which subjects must indicate (using a left or right button press) where the relevant item (or items, in the case of the global context) from that trial appears on the mappings screen. (Correct answer above is always “left”). Critically, the order of stimuli is rearrangeable. When a selective or global context appears first (“Context First” events; b and d) subjects can use it to selectively input only relevant items into WM. By contrast, if context appears last (“Context Last” events; c and e), subjects must have updated both items in WM and can only use these cues for output gating. In the behavioral version only (not illustrated), these response mappings were presented at the bottom of the screen containing the final token.

Importantly, each stimulus was presented in an unpredictable serial order on each trial. Thus, higher order context was presented either before (Context First) or after (Context Last) the lower order contexts had appeared. Under Context First conditions, subjects could use the number to select which of the subsequent items to update into WM. Thus, the Context First conditions allow subjects to use an “input gating” strategy. Conversely, during Context Last conditions, subjects would need to input each lower order item as it was presented because they could not know which was going to be relevant. Then, upon presentation of the Context in the last position, subjects would need to select from among the items maintained in WM that which would be allowed to influence the response. Hence, the Context Last condition requires subjects to use an “output gating” strategy.

Because the successful employment of an input gating strategy would also serve to reduce WM load when the context was presented first and only the letter or wingding was relevant (termed the Context First, Selective [CF-S] condition), we included a “global” context cue (e.g., the digit “3” in Fig. 1a) that specified that both lower-level items were relevant for determining the correct response. Thus, in contrast to the “selective” context cues (e.g., the digits “1” or “2” in Fig. 1a) that marked only a subset of lower-level items as relevant, a “global” context cue required subjects to utilize two items in memory, regardless of whether it was presented first (Context First, Global [CF-G]) or last (Context Last, Global [CL-G]). These global trials therefore acted as a strong load-matched control condition for the Context Last, Selective (CL-S) condition, wherein only one of the two items in memory was relevant for behavior, and therefore had to be selectively output gated.

This design allowed us to test the hypothesis that corticostriatal output gating mechanisms support selection from within WM. Specifically, by contrasting load matched Context Last with Item Last conditions, we could test the prediction that corticostriatal systems support selection from within WM. Moreover, contrasting Context Last with Context First conditions, we can assess whether regions previously identified with second-order hierarchical control are indeed preferentially activated and coupled with striatum during second-order control over output relative to input gating. We focus here both on regional differences in BOLD activation between these conditions and changes in functional connectivity across frontostriatal circuits.

Furthermore this design allows us to test the maintenance versus gating division of labor between frontal cortex and striatum hypothesized to underlie output gating. In particular, the impact of each component to output gating should be distinguishable in separable behavioral correlates that can, in turn, be related to our brain measures. To illustrate this logic, we draw an analogy to the effects on driving duration of how direct a route is versus how frequently one encounters red lights along the way (Figure 2).

Figure 2. Behavioral signatures of output gating and a clarifying analogy.

(a) The influence of output gating on execution of a WM-dependent stimulus-response (S-R) mapping can be understood by allusion to the influence of traffic on driving along a route. For a given route (condition), the shortest trips (trials) are the least likely to have involved red lights (output gating); trips of increasing duration are more likely to have been “gated” by red lights. Thus, the rate at which trip durations (RTs) increase when rank ordered can index the prevalence of lights (output gating). (b) By contrast, more general factors like the efficiency of the route (e.g., a WM-mediated S-R mapping) will manifest as a constant offset across ranks, and be captured by mean differences. (c) As this analogy implies, mean RT was increased in the CL relative to CF conditions (e.g., where the stimulus-response mapping was more indirect by virtue of requiring selection from WM). This difference was apparent even when comparing the CL conditions to the load-matched CF-G condition, though it was particularly pronounced in the CL-S condition (putatively involving selective output gating). However, these shifts in mean RT were also accompanied by changes in the shape of the RT distribution (d). Similar to the disproportionate influence of traffic lights on the longer trips taken along a route, the CL-S condition showed disproportionate increases in response latency across deciles. Within-participant estimates of the slope of RT as a function of decile (e) confirm the disproportionately greater RTs observed at later portions of the RT distribution in the CL-S condition, relative to the load-matched global conditions. Error bars reflect the SEM. See also Figure S1.

Consider each trial in our task as a trip driven along a route from stimulus to response via representations in WM. Taking a more direct route to a location reduces all transit times along that route equally, relative to less direct routes. Similarly, maintaining a good rule or “policy” in WM – e.g., one that most directly and effectively maps a stimulus to the response – will also yield more efficient performance. Thus, during Context Last conditions, to the degree that strong prefrontal representations of context support more efficient responses, activation in frontal cortex should predict speeded response times (RTs), and this speeding should be reflected by a constant shift across the entire RT distribution.

In contrast, stochastic influences like traffic will fundamentally reshape, rather than simply shift, the distribution. Consider that traffic lights can at best impose no delay to the fastest transit time (i.e., when all lights are green), and otherwise impose delays proportional to the total number of lights (i.e., when all lights are red). Thus, the gating of traffic by red lights will directly correlate with the rate by which transit times increase when trips are rank-ordered from fastest to slowest – that is, the slope of latency across ranks. Similarly, to the degree that striatum relates to gating of a particular stimulus-response mapping, striatal activation and/or corticostriatal interactions should relate to this slope estimate during conditions that put greater demands on selective output gating (e.g, CL-S).

Our results show evidence consistent with the dynamics of a corticostriatal output gating mechanism for selection from within WM. Specifically, we observed that (1) demands on output gating (whether global or selective) are associated with differential recruitment of the dorsal anterior premotor cortex (PrePMd), the cortical area most strongly identified with second-order control in prior work (Koechlin et al., 2003; Badre and D'Esposito, 2007; Badre et al., 2009; 2010; Badre and Frank, 2012); (2) that coupling between PrePMd and striatum increases specifically when output gating is required; and (3) that PrePMd, striatum, and their functional coupling correlate with separate behavioral signatures of output gating that are, potentially consistent with a maintenance versus gating division of labor within this circuit. That is, PrePMd BOLD specifically and uniquely predicts mean shifts in RT during selective output gating, whereas BOLD in caudate and its coupling with PrePMd uniquely predict the slope of the RT distribution during selective output gating.

Results

Behavioral Dissociations

The behavioral results revealed that the CL conditions were associated with characteristic performance costs distinguishing them both from the CF conditions and from costs related to WM load. Fig. 2b plots the effect of experimental conditions on mean RT; similar condition effects were evident in terms of error rates though performance was satisfactory on the task overall (12% errors; see also Figure S1). Specifically, the CL conditions yielded increased RT relative to the CF conditions overall (F(1,21)=250.54, p<.001). This difference is important, as subjects could conceivably have always waited until the final stimulus to decide which lower order item to select. To the contrary, the behavioral facilitation on CF relative to CL trials suggests that subjects took advantage of the input gating strategy available to them on CF trials, and that CL trial imposed a cost associated with selection from WM.

These mean RT costs were particularly evident in the CL-S condition (F(1,21)=6.02, p=02), even when comparing this to the CF-G and CL-G conditions with equivalent WM loads (Fig. 2a; t's>4.07, p's≤.001). These effects were not due to insufficient practice with each contextual token, as the differences were stable with experience (see Fig. S1a-c). Thus, a robust performance decrement is associated with the CL-S condition relative to CF conditions and is dissociable from costs related to greater WM load.

Only partial information about the nature of this difference is conveyed by mean RT. Indeed, and as implied by gating effects in our traffic analogy, separable effects of condition were evident in the RT distribution's shape (Fig. 2c; Balota and Yap, 2011; see also Ratcliff and Frank, 2012). Relative to the CF-S condition, the greater WM load in the CF-G condition was disproportionately apparent in the tail of the RT distribution. Relative to the CF-G condition, however, the load-matched CL conditions yielded dissociable effects on the shape of the distribution. Whereas the CL-G condition increased RT equally across the entire distribution relative to CF-G – akin to a mean shift due to demand on selecting information from WM – only the CL-S condition yielded an additional elongation of the distribution. This elongation is hypothesized to relate to inefficiencies in selection from WM due to output gating, (akin to the effect of traffic lights on the distribution of trip durations), an assumption we relate to fMRI data below.

These changes in shape were quantified as the rate at which RTs increased across deciles of the distribution (“RT Slope”, Fig. 2d). The CL-S condition yielded a larger increase in RT slope than both the CL-G and the CF-G conditions (t(21)=6.54, p<.001; t(21)=4.20, p<.001), and both of these differences were greater than that observed between the CL-G and CF-G conditions (t(21)=3.24, p<.005; t(21)=6.54, p<.001). Thus, and in contrast to the combined effect of WM load on both the mean and shape of the RT distribution, the CL conditions yielded a change in the mean of the RT distribution, with an additional elongation of the distribution only in the CL-S condition. In contrast to the mean shift evident between CF-G and CL-G, the effect of CL-S on RT slope is presumably attributable to the unique, load-independent demands on selective output gating required by this condition.

Performance was comparable in the scanned version of the experiment, where we imposed a delay between the final stimulus event and the response mappings (see above and Fig. 1a). This delay was imposed to ensure that BOLD responses to stimuli appearing last were unrelated to any demands imposed by the active selection and execution of a motor response, which may involve mechanisms similar to output gating. Despite this change, accuracy remained high across all conditions (means of 90.9-93.2%). The only reliable performance effect of condition was increased RT in the global conditions relative to both selective conditions (F(1,21)=22.7, p<.001), perhaps reflecting the increased WM load in the global conditions. See Fig. S1(e-h) for behavioral results from the scanned session.

Univariate fMRI Contrasts

Widespread regions in the PFC were recruited during task performance, both upon presentation of higher-level context (Fig. 3, red regions) and lower-level items (Fig. 3, blue regions; overlap in purple) relative to baseline, averaging across presentation orders (first vs. last). A contrast of these effects revealed that a subset of lateral frontal cortex exhibited a stronger response to context information (i.e., numbers), relative to items (i.e., letters or wingdings; black outlined regions in Fig. 3). The frontal peak of this response fell within the previously identified locus of second-order control, the left dorsal pre-premotor cortex (MNI: -50, 8, 30; Badre and D'Esposito, 2007).

Figure 3. Univariate recruitment to context and items.

BOLD responses to context>fixation (red) and items>fixation (blue) regardless of position (First vs. Last) were largely overlapping (magenta), but significantly larger for context than items (black outlined regions) within lateral PFC. Z>2.3, corrected to p<.05 via GRF. The differential response to context vs. items peaked in frontal cortex within the vicinity of the pre-PMd.

To assess whether frontal cortex might be differentially involved in second-order control over output gating relative to input gating, we contrasted the BOLD responses to contexts appearing last vs. first, relative to the analogous contrast of items appearing last vs. first. This higher-order contrast yielded increased recruitment throughout lateral frontal cortex, including the aforementioned PrePMd region as well as the adjacent inferior frontal sulcus (IFS) and dorsal premotor cortex (PMd), ruling out general influences of stimulus order or WM load (see Experimental Procedures for further explication of this logic). And, like the simpler contrast of context vs. items reported above, this differential response also peaked in frontal cortex within the vicinity of the PrePMd (MNI: -52, 6, 38; Fig. 4a-b).

Figure 4. Univariate brain-behavior correlations.

(a) The differential BOLD response of frontal cortex to context appearing last vs. first, as compared to load-and order-matched items (i.e., items appearing first, or last* in the global context), peaked within the vicinity of the PrePMd (here shown following the initial voxelwise threshold of z>2.3, and GRF-correction to p<.05, then subject to a further increase in the voxelwise threshold equivalent to p<.05 × 10^-7). (b) This peak fell partially within an ROI centered on the PrePMd observed in prior work (black outlined region). PSC extracted from the outlined region (c) and the equivalent location on the right hemisphere (d) during the CL-S condition negatively correlated with mean RT in CL-S. (e) This correlation was independent of PSC observed in the other conditions and of mean RT observed in these other conditions (R=-.43, p<.05). Univariate recruitment of left (f) and right caudate (g) was negatively correlated with the slope of RT during the CL-S condition, and this was independent of caudate recruitment and RT Slope in all cases except for caudate recruitment in CL-G, controlling for which the correlation was marginally significant (h). *The item last trials used in this contrast are limited to those in the global condition, where WM load is matched to that when context appears last (see text). See also Figure S2.

Given the proximity of this univariate effect to prior observations of PrePMd during hierarchical cognitive control tasks, we replicated the above analysis using an unbiased region of interest (ROI) defined from prior work (Badre and D'Esposito, 2007), and its right hemisphere homologue. A flexible model was used to characterize the hemodynamic response function (FLOBS; Woolrich et al., 2004) and estimated using FSL's Expert Analysis Tool (FEAT; see also Experimental Procedures for further details). The reconstructed time courses (Fig. S2) showed a peak in the BOLD response at approximately 4s post-stimulus across all conditions. Average PSC in the 2s surrounding each peak was then subjected to a 2 (Stimulus order: First vs. Last) × 2 (Stimulus type: Context vs. Items) × 2 (hemisphere: left vs. right) RM-ANOVA. We found a greater BOLD response to Context appearing Last vs. First, relative to the same contrast of items (i.e., the interaction of order and type; F(1,21)=4.2, p=.05), with no effect of hemisphere (F's<1, p's>.3; see also Fig. S2). This pattern also replicated in terms of PSC as estimated from the canonical HRF model, suggesting that the canonical HRF accurately captured this key aspect to the hemodynamic differences among our events.

Univariate Brain-Behavior Correlations

As CL-S was behaviorally distinguished from the other conditions on the basis of the mean and slope of its RT distribution, with the latter being unique to the CL-S versus CL-G difference, we sought to assess whether activation in PFC and striatum related to these behavioral signatures. Importantly, as these are statistics computed only at the whole participant level, we tested whether the mean and slope of RT would relate to individual differences in the recruitment of this area. Specifically, mean PSC was extracted from the PrePMd, as well as the adjacent IFS and PMd. Given the good agreement between the current foci and those identified in prior work, these ROIs were defined on the basis of this prior work(Badre and D'Esposito, 2007) as well as the homologous regions on the right hemisphere (though, we note that similar effects were observed using functionally defined ROIs).

The most robust correlation to emerge from this analysis was between mean RT in the CL-S condition and mean PSC in the left and right PrePMd during the selective contexts when presented last (i.e., CL-S events; R=-.46, p=.03, R=-.52, p=.01; Fig. 4c-d).

The observed bilateral correlation was highly specific to this condition, measure, and region (as determined through semipartial correlations; see also Experimental Procedures). (a) The relationship of PrePMd recruitment to mean RT during the CL-S condition was independent of the analogous measures in the other conditions (R=-.43, p<.05; see Fig. 4e). (b) The correlation was also independent of the recruitment of both the more caudal PMd and the more rostral IFS in the same condition (R=-.55, p<.01 and R=-.54, p<.01 respectively), both of which failed to correlate with CL-S mean RT in either hemisphere (all p's>.5), with PMd significantly less correlated than PrePMd (Fisher's z=1.93, p=.05). (c) The correlation was independent of the other behaviorally-correlated neural measures we report below (partial R=-.58, p<.01), which themselves failed to correlate with mean RT during the CL-S condition (p's>.14). Given this specificity of the link between mean RT during the CL-S condition and PrePMd recruitment to CL-S contexts, this brain-behavior correlation is unlikely to reflect generalized factors such as differences in sensorimotor abilities between individuals, or differences in interference within WM caused by either WM load or simply by presenting context last.

Next, given our a priori focus on frontostriatal systems, we tested an anatomically-defined ROI in the dorsal striatum (specifically, caudate; Fig. S5). We reasoned that if the caudate is involved in output gating, then activity in this region should reduce the RT cost associated with these demands. Indeed caudate activation correlated with reductions in CL-S RT slope, once again bilaterally (left: R=-.50, p<.02; right: R=-.51, p=.01; Fig. 4f&g).

As with the correlations described above, caudate's correlation with RT Slope was also highly specific. (a) The relationship of RT Slope in the CL-S condition to the effect of CL-S context events on caudate recruitment was independent of the corresponding brain and behavioral measures from all other conditions at a trend level (R=-.354, p<.11; Figure 4h), reaching significance when bilateral caudate recruitment during CL-G was not partialled from that during CL-S (R=-.49, p<.02), or when bilateral caudate recruitment was averaged across both CL conditions (R=-.54, p=.01). Thus, CL-S RT slope is correlated with variance in bilateral caudate recruitment shared across conditions where context appears last, and thus demand selection from WM. (b) The relationship of RT Slope in the CL-S condition to the effect of CL-S context events on caudate recruitment was also independent of the other behaviorally-correlated neural measures we report (R=-.49, p<.03). Thus, these observations again argue against an interpretation in terms of differences in general sensorimotor abilities or WM interference across subjects or conditions.

Corticostriatal Interactions and Correlations with Behavior

To test whether the CL conditions might drive differential corticostriatal interactions than the CF conditions, we conducted a whole-brain search for voxels differentially correlated with our caudate ROI's timeseries during the CL than CF conditions (i.e., a psychophysiological interaction; see Experimental Procedures). Caudate increased its coupling with left lateral PFC, including a subset of the PrePMd, when context appeared last relative to first (Fig. 5a and b). These results indicate that corticostriatal interactions are pronounced in the CL relative to CF conditions (when putative demands on output gating are high).

Figure 5. Coupling-behavior correlations.

(a) Left lateral PFC increased its coupling with caudate during the CL conditions relative to CF conditions, and this again was within the vicinity of the PrePMd identified in prior work (black outline in b). The PSC associated with this change in coupling was extracted from this a priori ROI (c) and its right hemisphere counterpart (d) for the CL-S condition. These measures robustly correlated with the slope of the RT distribution (see Fig. 1d). (e) This correlation was independent of PrePMd's coupling with caudate in the other conditions, and of the slope in RT observed in other conditions, as demonstrated through partial correlation.

We next assessed whether these corticostriatal interactions might predict performance during the CL-S condition. Coupling of caudate with our a priori PrePMd ROI during the context events of the CL-S condition predicted the slope of the RT distribution in this same condition, bilaterally (R=.45, p<.05, R=.56, p<.01 for left and right Pre-PMd, respectively; Fig. 5c&d). Once again this correlation was highly specific. It remained significant when controlling for the slope of the RT distribution in other experimental conditions, PrePMd-caudate coupling after the during context in other experimental conditions, and PMd-caudate coupling during context in the CL-S condition (R's >=.45, p's<.05; Fig. 5e). In addition, this correlation was significant even when partialling univariate PrePMd and caudate recruitment to the same events (R=.54, p<.02). Together these effects once again rule out general factors like sensorimotor abilities or WM interference, e.g., due to differences in WM load or the presentation of context last.

Discussion

We report that selection from within WM is supported by corticostriatal dynamics analogous to those involved in selective control over the input to WM, with lateral frontal and striatal components of this circuit relating to separable individual differences. As detailed below, our results support the output gating hypothesis by suggesting that lateral frontal cortex may maintain contextual representations which influence striatum to gate which subset of WM representations can influence behavior.

Our task was designed to isolate the effects of selective output gating, independent of those due to input gating or overall WM maintenance demands. Our behavioral results confirmed that these manipulations were effective. RT was decreased when a selective context was presented first (CF-S), relative to when context was presented last (CL-G and CL-S), suggesting that subjects took advantage of the input gating strategy available to them on CF-S trials. Moreover, CL-S trials were associated with additional behavioral costs than CL-G trials. As WM load is equated between these conditions, these additional costs on CL-S trials are directly attributable to the demand to select from within WM.

Following from the above logic, comparing CL vs. CF conditions targeted selection from WM independent of WM load. This comparison yielded two key results consistent with the output gating hypothesis: (a) Activation was greater for context presented last relative to first in lateral frontal and parietal cortex, peaking in PrePMd. (b) Connectivity between dorsal striatum and lateral frontal cortex increased selectively during context presented last relative to first. This association of output gating with the CL conditions, and with the CL-S condition in particular, is further supported by the brain-behavior correlations unique to the CL-S condition – a condition whose behavioral costs uniquely predicted its neural correlates (as assessed in a separate session) within the pre-PMd, the dorsal striatum, and the coupling between these regions. Together, these observations are consistent with the hypothesis that frontostriatal gating dynamics support selection from WM, like those previously documented for input gating. Indeed, contexts used to support input gating (i.e., contexts presented first) recruited overlapping areas with those used to support output gating (i.e., contexts presented last), albeit with a difference in degree (see Fig. 3). This overlap is consistent with the hypothesis that co-local corticostriatal circuits support both types of gating, and perhaps with the notion of spatially-interdigitated “stripes” supporting them (e.g., Hazy et al., 2007).

A role for output gating during selection from WM could extend from the more well-established role of these mechanisms in action selection. Specifically, corticothalamic loops are separably modulated by “Go” and “NoGo” striatal pathways, with activity in Go pathways acting to disinhibit each loop, and activity in NoGo pathways acting to inhibit that same loop (Mink et al., 1996; Graybiel, 1998; Frank et al., 2001; Hazy et al. 2007). Classically, this opponent process is thought to modulate excitability in effector-specific thalamo-M1 loops and thereby underlie the role of Go and NoGo pathways in the execution and inhibition of specific motor responses (respectively). In addition, premotor representations are thought to influence which of several candidate motor responses constitutes the effective output of M1 (i.e., which representation is output gated) by biasing the Go and NoGo pathways modulating those thalamo-M1 loops. Here, by what might fundamentally be a similar mechanism, pre-premotor representations (e.g., representations of context in PrePMd) may influence which of several candidate premotor representations (e.g., candidate WM items, here letters and wingdings) constitutes the most effective output of premotor cortex, by biasing the Go and NoGo pathways modulating those thalamo-PMd loops. More concretely, a top-down contextual signal from PrePMd to Go cells might amplify the representation of letter over wingding, such that the maintained letter becomes the most effective output of the PMd (see Fig. S3 for evidence of PMd involvement in the current task). In this way our results support the output gating mechanisms of many models (Hazy et al., 2007; Kriete and Noelle, 2011; Chatham et al., 2011; Collins and Frank, 2013; Kriete, et al., 2013; Huang et al., 2013), including models of hierarchical cognitive control in particular (Frank and Badre, 2012).

Given the analogy of the output gating WM mechanism tested here to the circuit responsible for motor selection, it is important to note that the present imaging results cannot be accounted for by motor preparation. This crucial alternative was addressed in two ways in the fMRI design. First, the response phase was separated in time from the presentation of the final stimulus event, including during CL conditions. This design feature ensured that we measured changes in BOLD related to processing of the contextual cue in CL conditions rather than any additional demands related to selection and execution of a motor response. Second, the response mappings themselves changed unpredictably on each trial. Thus, it was not possible for the participant to prepare a response prior to the delayed response cue and so confound our effects. For these reasons, the corticostriatal correlates identified here are entirely attributable to the demands on selection from within WM rather than motor selection.

Though we observed increased frontostriatal coupling during CL conditions, we did not find greater univariate activation in the caudate in this condition relative to controls. Notably, prior work has shown a high degree of variability in the observation of striatal recruitment across different WM manipulations; including maintenance (e.g., Postle and D'Esposito, 1999, 2003; Marklund et al., 2007a and b; Narayanan et al., 2005; Olsen et al., 2013), encoding/updating (e.g., Narayanan et al., 2005; McNab and Klingberg, 2008; Dahlin et al., 2008; Nee and Brown, 2013), manipulation (e.g., Lewis et al. 2004; Dodds et al., 2009), and selection (Tamber-Rosenau et al., 2011). This inconsistency might reflect a dependence of the striatal BOLD response on the relative balance of Go and NoGo striatal cells recruited by each task. Perhaps parametric effects coding for both the direction and magnitude of effects, like those reported here with respect to RT slope – or by prediction errors derived from reinforcement learning (RL) models in much recent work – may be better suited to locate clear striatal BOLD effects than simple aggregate mean contrasts across conditions. However, simple RL dynamics do not seem to account for the effects we report here (c.f. Chatham and Badre, 2013).

A second key observation from this study is that though frontal cortex and striatum participate in a common circuit for output gating, they correlated with separable behaviors during selection from within WM. Specifically, analysis of the RT distribution across experimental conditions revealed a mean shift in RT when context was presented last, but WM load was matched (e.g., CL-G > CF-G), reflecting a general cost of selection from WM. This effect was relatively constant across deciles of the distribution, as expected of a general factor influencing the efficiency of stimulus-response mappings. However, this cost was dissociable from a change in the shape of the RT distribution quantified by RT Slope, which accompanied demands on selective versus global output gating (CL-S condition). In brain-behavior correlations, individual differences in the activation of PrePMd were related to mean RT during the condition involving selective output gating (CL-S). By contrast, individual differences in striatal activation and striatal coupling with PFC correlated with the shape of the RT distribution during the CL-S condition, as expected of mechanisms involved in gating.

This dissociation is potentially consistent with a frontostriatal division of labor between maintenance and gating (Frank et al., 2001; Gruber et al., 2006; Frank and O'Reilly, 2006; Cools et al 2007). More specifically, lateral frontal cortex (PrePMd in this case) is hypothesized to maintain the context that influences what lower order information should be gated by the striatum. Consequently, PrePMd sets the appropriate policy and so determines how rapidly selection can begin on average, thereby affecting the whole distribution equivalently (rather than its shape). Also consistent with this interpretation, PrePMd was generally more activated by context stimuli (i.e., numbers) than load- and order-matched item stimuli (i.e., letters and wingdings). This context-preference was enhanced when the context occurred last and could therefore drive output gating during the selection from WM. Such effects can be broadly understood as reflecting the efficiency of a context-mediated mapping between stimulus and response, analogous to the directness of a route.

Importantly, outflowing information in an output-gated system must wait until the gate is opened before it can influence behavior. Consequently, the selection process itself exerts further “waiting times” on responses beyond those associated with the policy's efficiency itself. These wait times can only make RT longer and so are more likely to contribute to longer than shorter RTs. As in traffic, such wait times will elongate the distribution in a way that can be quantified by RT slope (as well as alternative nonparametric measures of spread; see Table S2). Though such changes in distribution shape have not been previously demonstrated for output gating or selection from WM specifically, simulation studies have demonstrated similar effects on RT arising from input gating failures (Reynolds et al., 2006). Thus, the correlation of individual differences in striatal activation and frontostriatal connectivity with the RT Slope could reflect the efficiency with which a given policy (established by PFC) is implemented by striatally-mediated gating mechanisms (see also Supplemental Fig. S4 for other correlates of RT Slope, including trial-to-trial variability in striatal BOLD). Though this view of the brain-behavior dissociation is consistent with current output gating models of selection from WM, further work is required to confirm this conclusion and distinguish it from alternatives.

Future models of output gating may be importantly constrained by our results. The non-constant magnitude of the CL-S vs. CL-G difference across deciles of each distribution implies a greater influence of gating in CL-S – in the same way that greater increases in transit time by rank order imply a greater number of traffic lights along a route. Yet this effect is counterintuitive from a classical account of WM selection: more items must be selected from WM in CL-G than CL-S. One possibility is incongruency: all WM representations should influence behavior in the CL-G condition (a congruent “open gates” policy across items in WM) whereas in CL-S only one item should influence behavior (an incongruent “one open, one closed” gating policy). Such congruency effects would mirror those seen in the motor domain. Alternatively, if an output gate is closed for just one item via striatal NoGo mechanisms, perhaps this NoGo pathway exerts behavioral slowing during selection from WM, as it does in action selection (e.g., Kravitz et al., 2010). These possibilities are not mutually exclusive, but both imply analogous roles for output gating during selection from WM and action selection.

One outstanding issue is how precisely output gating reshapes RT distributions. Here non-parametric methods quantify changes in shape. While waiting times can be modeled with gamma distributions (e.g., Chatham et al., 2012), those due to output gating are unlikely to arise from the gamma's most widely-used special case: the exponential distribution (Balota and Yap, 2011). The exponential may be poorly suited to model output gating, which may involve information accrual, contrasting with the “memoryless” nature of the exponential, and a modal influence on RT (Wiecki and Frank, 2013), contrasting with the mode of zero on the exponential. The centrality of parametric assumptions for modeling the latent component of RT (Jones and Dzhafarov, in press) further motivates such caution.

Though here we show selection from WM may rely on corticostriatal output gating mechanisms, such mechanisms may play a more general role in selection of internal representations. For example, striatal involvement in object switching has been reported (Tamber-Rosenau et al., 2011), but it is unknown whether frontostriatal coupling also accompanies such shifts. It is also unknown whether either striatal recruitment or coupling correlate with either the mean or shape of the distribution of object switch costs. Likewise, striatal recruitment during selective retrieval from long-term memory (Scimeca and Badre, 2012) could reflect output gating of hippocampus itself, of cortical regions responsible providing retrieval cues to hippocampus, or of cortical regions responsible for post-retrieval selection (e.g., Badre et al., 2005; Badre and Wagner, 2007; Snyder et al., 2010; 2011). This array of predictions, if confirmed, would suggest substantial generality to the current account of corticostriatal output gating interactions for selection in WM.

Finally, we demonstrate these effects in the domain of hierarchical control, where output gating confers computational advantages (Frank and Badre, 2012; Kriete and Noelle, 2011). Together our results suggest a critical role for more rostral PFC, and most consistently PrePMd, for output gating in second-order hierarchical control. In this way, both current and prior results associate PrePMd with second-order control across a variety of hierarchical control tasks (Koechlin et al., 2003; Badre and D'Esposito, 2007; Badre et al., 2009; 2010; Badre and Frank, 2012). However, our effects are not perfectly unique to PrePMd; posterior parietal cortex is also more strongly recruited for CL than CF conditions (e.g., Fig. 4b). Similarly, while effects in PrePMd are consistently independent of the more caudal PMd and the most rostral RLPFC, PrePMd only differed from the adjacent IFS in its correlation with mean RT. Thus, while our results strongly support a role for PrePMd in second-order control over output gating, they should not be taken to show that PrePMd alone is responsible for performing the second order rule.

This observation informs active debates on frontal organization, where two recent studies report unexpectedly rostral frontal BOLD for second-order control tasks (Reynolds et al., 2012; Crittenden and Duncan, in press). We note this is also true both in this study (Fig. 3) and prior studies from our lab (Badre et al., 2010). In contrast, selective effects along the rostro-caudal axis emerged only from parametric manipulations of the number of mappings (e.g., stimulus-response mappings in first-order control) resolved at each level of control (Koechlin et al., 2003; Badre and D'Esposito, 2007). Indeed, Crittenden and Duncan (in press) replicated this effect, reporting selectively increased recruitment in caudal PMd when lower-order response competition was manipulated as in prior work. Thus, the question for theory becomes why a parametric manipulation of choice at each level of control might expose these rostrocaudal differences whereas certain other manipulations yield less consistent and functionally-selective results. Previous attempts to reconcile these results focus on demands on sustained context maintenance and differential encoding vs. retrieval demands, though our results do not clearly support these accounts either (Fig. S3). One possibility is that parametric manipulations better control demands on the gating processes that govern directed influences across frontal subregions – a possibility that motivates tight control over gating in future studies of rostrocaudal frontal gradients.

In support of this possibility, Nee and Brown (2013) reported results more compatible with prior accounts of rostrocaudal gradients when gating demands were more closely controlled. In this study (and also in Nee and Brown, 2012), rostral prefrontal areas (albeit in regions rostral to PrePMd during a 3^rd order control task) showed stronger recruitment by demands to input gate a higher-level context (vs. retaining the prior higher-level context), whereas the more caudal PMd was more strongly recruited by the demand to input gate a lower-level context (vs. retaining the prior lower-level context). Nee and Brown (2013) failed to observe corticostriatal connectivity during input gating of the lower-level context; we likewise failed to obtain significant increases in corticostriatal coupling during the input gating of lower-level items. Instead corticostriatal coupling increased during the output gating of lower-level items. This difference is consistent with our conclusion that hierarchical corticostriatal interactions may be particularly crucial during output gating.

Experimental Procedures

Subjects

Twenty-two right-handed adults (aged 18-35; 8 female) with normal or corrected-to-normal vision completed the experiment. All spoke English natively, were screened for both neurological medications/conditions and MRI contraindications, and provided informed consent in accordance with the Research Protections Office at Brown University. All subjects first performed a behavioral version of the task, and then a modified version of the task while undergoing fMRI.

Behavioral Procedure

Three tokens were presented on each trial. Each token was of a distinct type: digits, letters, and symbols. The tokens appeared in random order, but digits acted as a context for a hierarchical rule. 1 and 2 specified “Selective” contexts. For example, 1 might specify that the symbol observed on that trial would be relevant for a response, and 2 would specify that the letter from that trial would be relevant. The 3 always acted as “Global” context, indicating that both the letter and the symbol would be relevant for the participant's response. These three contexts were equiprobable, as was the likelihood of a digit at stimulus position in a trial. These trial characteristics were randomized in the behavioral experiment but optimized for power in the fMRI experiment.

Response mappings were presented at the bottom left and right of the screen as the final stimulus event for each trial. Only one mapping contained the relevant item or items for that trial, as determined by the hierarchical rule. Subjects indicated which side (left vs. right) contained the relevant item(s) using a button press (Fig. 1a). For reasons described below, during the behavioral experiment, the mappings were presented simultaneously with the last token, whereas during fMRI scanning, the response mappings were separated in time from the last item by a jittered delay.

Half of all trials included a response lure, where the incorrect response option contained one item that had been seen on that trial. Subjects were instructed that lure trials would occur and that every trial would therefore require them to proactively attend to all items specified as relevant by the context (or the lack thereof). Analysis of lure trials confirmed that subjects maintained both items, since lure accuracy was well above that expected if subjects randomly-attended to only one item, both in the behavioral (CF-G: 93.6%, t(21)=10.85, p<.001; CL-G: 90.9%, t(21)=6.26, p<.001) and scanned experiments (CF-G: 93.2%, t(21)=8.44, p<.001; CL-G: 91.4%, t(21)=8.66, p<.001).

Our task's logic is that context can drive input gating only when it appears first (Context First [CF]); by contrast, context can only be used to drive output gating when it appears last (Context Last [CL]). One complication is that when context appears last, subjects will experience a larger WM load, relative to when a Selective context appears first. The inclusion of the global context remedies this problem. Regardless of whether the global context appears first or last, all items from that trial are behaviorally relevant. These global trials were thus matched in terms of WM load with the Context Last-Selective (CL-S) condition.

In the fMRI version of the task, between-position contrasts were also used to match load. For example, the BOLD responses to context last were compared to the BOLD responses to items appearing last in the global context. In both cases, two items from the trial are being maintained; these “global item last” trials thus constitute the appropriate load- and order-matched control for Context Last.

Following detailed instructions (see Supplemental Experimental Procedures S1), subjects underwent training and practice followed by 180 trials of behavioral testing, and finally scanning.

MRI procedure

Whole-brain imaging was performed on a Siemens 3T TIM Trio MRI system with a 32-channel head coil. A high resolution T1 multi-echo MPRAGE was collected from each participant. Four functional runs each consisting of 303 volumes used a fat-saturated gradient-echo echo-planar sequence (TR = 2s, TE=28ms, flip angle = 90°, 38 interleaved axial slices, 192mm FOV with voxel size of 3mm³). Head motion was restricted with padding, visual stimuli were rear-projected and viewed with a mirror attached to the head coil. Subjects responded using an MRI-compatible button box.

Data processing

Behavioral data

The first 10 trials of the behavioral experiment were excluded from analysis, as were incorrect trials and trials with RTs ≥5 standard deviations from a participant's mean RT.

Only results from one of two qualitatively distinct subtypes of the CF-S condition are presented here. The presented subtype consists of those trials where the contextually-irrelevant item appears as the second event in the trial (the other subtype consists of those where the contextually-irrelevant item appears last). This matches all conditions for the demand to attend to the central portion of the final stimulus display.

Imaging data

Data were processed using a combination of SPM and FSL. First, SPM8 tools (artglobal and tsdiffana) were used for artifact detection, and slice timing correction was then performed. The first six volumes of each run were discarded to allow the scanner to reach steady state. Data were motion-corrected using rigid transformations in MCFLIRT to the middle acquisition of each run. Runs with movement of more than 2mm were excluded from analysis (N=1). Grand-mean intensity normalization of the entire 4D dataset was performed with a single multiplicative factor, and the data were subjected to a temporal highpass filter (Gaussian-weighted least-squares straight line fitting, with sigma=32.5s), and the data were smoothed at 8mm FWHM. The middle acquisition of each run was then registered to each participant's brain-extracted MPRAGE using a linear 7DOF transform, and the MPRAGE was registered to the MNI standard brain using a linear 12DOF transform.

Statistical Analysis

Our GLM was estimated using FEAT (FMRI Expert Analysis Tool) version 5.98 (FMRIB's Software Library, www.fmrib.ox.ac.uk/fsl), on the basis of explanatory variables (EVs) coding for the following event types: Digits 1 or 2 (i.e., selective contexts) or 3 (i.e., the global context) appearing in the first position, middle position, or last position; and items appearing in the first or middle position. Separate EVs were used to capture items appearing in the last position under the Selective context and Global contexts. The duration of each event in all these EVs was set to the actual stimulus duration in the scanner (500ms). Separate EVs were used to capture responses; the duration of each event in these EVs was set to the observed RT on each trial. Additional EVs of no interest were also included in the GLM, including those corresponding to incorrect trials and to 6 degrees of motion. All EVs except those corresponding to motion were convolved with a standard hemodynamic response function (HRF), high-pass filtered in the same way as the functional data, and then used as regressors (including temporal derivatives) in the GLM.

The following linear contrasts were constructed: the general effect of context appearing first, context appearing last, and the difference between these; the analogous effects of items appearing first, items appearing last (in the global context, as noted above), and the difference between these. A higher order contrast compared the effect of order on context (first vs. last) with the effect of order on items that were matched for WM load (items first vs. items in the global context last condition, as discussed above). Alternative ways of addressing potential WM load differences between context and other items in the last position (e.g., involving a parametric WM load regressor) yielded qualitatively identical effects. The location and extent of clusters of activation for critical contrasts in this design can be found in Table S1.

For the generalized PPI analyses, the anatomically-defined caudate of the Harvard-Oxford subcortical atlas was warped to subject space, thresholded at 50% probability, and then used as a mask for extracting the timeseries of this region from the filtered functional data. These values were used as a regressor in the design matrix without convolution or temporal derivatives, and allowed to interact with each of the other regressors estimated above. This procedure effectively removes variance due to other experimental factors, and demonstrably improves PPI model fit (McLaren et al., 2012). Interaction regressors for contexts appearing last and first were then subject to a linear contrast (contexts appearing last > contexts appearing first). Qualitatively similar increases in coupling during context last relative to first are observed using the non-generalized form of PPI, though such “non-generalized” PPIs can suffer from unstable estimation (McLaren et al., 2012). Such consistency suggests robustness to this key result.

For both the univariate and gPPI analyses, a second-level fixed effects analysis combined the results of these first level models across runs, and a third-level mixed effects analysis combined these results across subjects (using FSL's local analysis of mixed effects; FLAME1). For all whole-brain analyses, the resulting voxelwise z-statistics were initially thresholded at z=2.3, and cluster corrected to p<.05 using Gaussian Random Field theory.

To assess brain-behavior correlations, peak PSC from a 10mm radius sphere centered on the IFS, PrePMd and PMd ROIs of Badre and D'Esposito (2007; MNI: -50, 24, 24; -36, 8, 34; and -30, -12, 66) and their right hemisphere homologues were extracted using the featquery tool from the conditions of interest and correlated with the behavioral measures. These ROIs are illustrated in Figure S5. For partial correlations, PSC was first averaged across the ROIs of each hemisphere. Average PSC from the other conditions was then used to predict PSC during the CL-S condition via linear regression; these standardized residuals were then correlated with the standardized residuals of the analogous regression predicting behavior in the CL-S condition from that in the other conditions. Such semipartial correlations reveal whether there is a relationship between the PrePMd PSC that is unique to the CL-S condition and the behavior that is unique to the CL-S condition (see also Supplemental Experimental Procedures S1).

Sustained BOLD following contexts was modeled with regressors beginning at the onset of the digits lasting until the response. After convolution with the HRF and its derivative, Betas for these regressors were estimated by a flexible model of event-related transients (Woolrich et al., 2004). Similar results obtain if sustained regressors are estimated with canonical HRFs.

All p-values are two-tailed, and error bars indicate standard errors of the mean. Correlations are uncorrected for attenuation due to reliability or restricted range.