Multiple gates on working memory

Abstract

The contexts for action may be only transiently visible, accessible, and relevant. The corticobasal ganglia (BG) circuit addresses these demands by allowing the right motor plans to drive action at the right times, via a BG-mediated gate on motor representations. A long-standing hypothesis posits these same circuits are replicated in more rostral brain regions to support gating of cognitive representations. Key evidence now supports the prediction that BG can act as a gate on the input to working memory, as a gate on its output, and as a means of reallocating working memory representations rendered irrelevant by recent events. These discoveries validate key tenets of many computational models, circumscribe motor and cognitive models of recurrent cortical dynamics alone, and identify novel directions for research on the mechanisms of higher-level cognition.

Introduction

The world is rich with information, much of it only transiently available to the senses. And yet, an animal must leverage a small, but crucial, fraction of this input in order to provide a context for its behavior. Working memory is a central adaptation to confront this problem, selecting behaviorally relevant information, maintaining it in time, and referencing it when appropriate in order to make decisions about how to act in the world. Indeed, the elaborated working memory system of higher primates partly underlies their distinguishing intelligence and flexible behavior.

Working memory is capacity limited. Measures of capacity predict individual differences in cognitive ability, including scholastic aptitude, intelligence, and aging-related cognitive change [1,2]. Moreover, changes in working memory capacity accompany neurological and psychiatric disease [3] and may underlie behavioral and cognitive deficits associated with these disorders [4]. However, just as the world is dynamic, so is the working memory system adapted to address these dynamics. Thus, control processes are required in order to rapidly and selectively store information in memory (input control), to rapidly and selectively deploy subsets of that information for use in behavior (output control), and to selectively eliminate an obsolete representation from memory when its predicted utility declines (reallocation). Such control functions would seem to be crucial for strategically making use of capacity-limited working memory. And indeed, though less understood, individual differences in these control processes could be equally or even more important than the size of a static capacity for intellectual ability.

Though still in its early stages, the last few years have yielded rapid advances in our understanding of how the brain solves the input, output, and allocation control problems facing working memory. These experiments have associated all three functions with interactions between frontal and basal ganglia systems. Below, we review this work to outline an account of how the brain manages working memory.

From motor control to cognitive control

There is a clear parallel between the problems addressed by working memory control processes and the fundamental challenges faced by an animal’s motor system. Consider the task of hunting for dinner. For example, a predator must program motor actions on the basis of transiently observed information about prey (input control); maintain these programs until the time is right, enacting only the most appropriate motor program at that time (output control); and finally, refrain from perseveratively considering outdated motor programs, should the prey escape (reallocation; Figure 1a). Thus, demands on selective encoding, maintenance, utilization, and clearing of information face a variety of species.

Figure 1.

Theoretical overview. (a) All behaving animals must be capable of selecting useful motor actions at the right times. A long-standing hypothesis [11] holds that the same frontostriatal mechanisms supporting this kind of action selection might also support higher-order cognitive functions. (b) Frontostriatal mechanisms can implement a gate to select useful but transient information for rapid storage in working memory, as well as a gate to select of information from working memory to inform motor planning [6,10,13]. (c) Models involving rostral to caudal nesting of corticostriatal input and output gating loops have been shown to solve abstract, multiply contingent action problems [18] as well as forms of Bayesian inference [22^••] and symbolic referencing [23^••]. A key feature of these models is the presence of a ‘diagonal’ rostrocaudal projection (red arrows) allowing rostral areas to modulate the striatal input to more caudal basal ganglia; one implemented model is shown here. (d) Multiple such frontostriatal circuits are thought to exist, each modulated in a top-down manner by more rostral circuits (PMd by pre-PMd; pre-PMd by the inferior frontal sulcus [IFS]; and IFS by the rostrolateral prefrontal cortex [RLPFC]). The diagonal rostrocaudal projections are thought to be particularly important for modulating output gating mechanisms (‘BG out’) as opposed to input gating mechanisms (‘BG in’).

This similarity motivates the search for neural solutions that might also be shared across species. Indeed, recent phylogenetic analyses show that the basal ganglia (BG) has been highly conserved evolutionarily — all its major structures preserved since their debut in an unknown ancestor common to all vertebrates [5]. This conservation of structure may attest to the BG’s efficacy in solving the action selection problems faced by many species.

One way to describe the dynamics of this selection function is as a gate that regulates the passage of information from one neural circuit to another [6], such as in the case of motor selection, between thalamus and motor cortex. Theoretical models posit that motor gating occurs via the opposing circuit-level effects of the two classes of medium spiny neurons of the striatum: Go and NoGo cells. The net effect of D1-receptor - expressing Go cells is to ‘open the gate’ by facilitating recurrent thalamocortical information flow, whereas D2-receptor-expressing NoGo cells ‘close the gate’ by blocking thalamocortical information flow. By this scheme, a planned motor action represented cortically might trigger the activation of Go cells via a corticostriatal projection, in turn facilitating a projection from thalamus to the primary motor neurons responsible for enacting specific movements. At the same time, alternative action plans would trigger NoGo cells and so would have negligible thalamocortical influence.

A variety of recent evidence has offered novel support for this framework. Go and NoGo cells are coactive when animals are motorically active, but not quiescent [7], in particular when action sequences are being initiated [8] — all consistent with a role for these cells in gating for action selection as opposed to a more general pro-kinetic vs. anti-kinetic dichotomy between Go and NoGo cells. Further evidence for this framework has recently been provided by optogenetic techniques [9^••]. Transgenic mice expressing light-activated ion channels in putative Go and NoGo cells chose between one of the two ports after the onset of a cue. Light-induced firing of Go cells led to an increase in contralateral movements, whereas light-induced firing of NoGo cells led to an decrease in contralateral movements. The effect of stimulation was greatest when the value of the two potential actions was closely matched (as estimated by a computational model), suggesting stimulation was capable of mimicking a small shift in their relative value. Moreover, this stimulation was effective only when delivered simultaneously with the cue, consistent with a particular influence of action value during action selection.

As discussed below, these BG-mediated gating mechanisms may extend beyond the selection of motor actions and into the more abstract domains of working memory [10] (Figure 1b) and cognitive control (Figure 1c); where they can be used to solve analogous problems of selection and updating. Indeed, the known anatomy of parallel motor, frontal, and prefrontal basal ganglia-thalamocortical circuits hints at analogous computation (Figure 1d) [11]. And, a variety of computational models have demonstrated the feasibility of such an architecture for solving complex working memory control problems [6,10, 12–21,22^••,23^••]. However, only recently have animal and human behavioral, neuropsychological, pharmacological, PET and fMRI studies provided direct functional evidence for multiple BG gating dynamics in WM and their importance for higher thought and action.

Input gating of working memory

Gating dynamics provide a powerful solution to the input control problem for working memory [6,10,12]. When useful information becomes available in the environment, the gate is open and working memory is updated with this useful information. Otherwise, the gate is closed and irrelevant information is kept from needlessly occupying capacity.

Several computational models of working memory have achieved this gating dynamic using corticostriatal mechanisms analogous to those described for the motor system. Just as a cortically represented motor action could cause Go cells to fire via corticostriatal projections, thereby facilitating thalamic-motoneuron information flow for movement programming (as described above), a cortically represented stimulus could also cause Go cells to fire, again via corticostriatal projections, and thereby facilitate thalamic-prefrontal information flow for working memory updating. By contrast, distracting sensory representations would trigger NoGo cells and so would have negligible thalamoprefrontal influence. By this scheme, updating is favored (and stable maintenance prevented) by input to Go cells, whereas updating is prevented (and stable maintenance favored) by input to NoGo cells. Thus, the Go/NoGo system is a potent means of circumventing stability/flexibility tradeoffs that plague single-component systems.

Several features of this and related striatal input gating models are supported by human neuroscience evidence. First, there is evidence that D1-expressing Go cells support the rapid updating of information in working memory. Striatal activation in fMRI, thought to be driven primarily by D1 receptor activation [24] is a common observation during working memory tasks that require updating (Figure 2a). Training of updating transfers to other tasks involving overlapping striatal BOLD responses [25]; this transfer is accompanied by alterations in the striatal hemodynamic response to updating challenges [26] and results in increased striatal dopamine receptor binding [27] (Figure 2b) as assessed via PET. Shifting the striatal balance toward Go firing (via blockade of D2 receptors with haloperidol) also enhances working memory updating [28]. Second, there is evidence that D2-expressing NoGo cells act to limit the rapid updating of information in working memory. For example, the ‘attentional blink’ is more pronounced among individuals with enhanced D2/D3 receptor binding in the BG [29^•] (Figure 2c). Likewise, the depletion of central dopamine due to Parkinson’s disease counterintuitively enhances resistance to distraction in these patients, while producing deficits in the updating of working memory [30]. In summary, a variety of recent evidence strongly implicates BG-mediated input gating in working memory updating.

Figure 2.

Basal ganglia (BG) contributions to working memory input control. (a) A meta-analysis of over 8000 studies, carried out with Neurosynth’s python package [53], reveals that studies with abstracts including the term ‘updating’ are significantly more likely to report a BOLD response in the bilateral BG (Z = 2.58–7.03, FDR to p < .05; among other regions, not shown). (b) Binding of the competitive dopamine agonist raclopride within the BG is decreased during an updating task (letter memory), relative to a control task (Stroop). (c) Individual differences in D2 binding affinity within the bilateral BG predict individual differences in the rapid updating of working memory (as assessed by the attentional blink), uniquely throughout the brain (even at a liberal threshold of p < .01). (b,c) adapted from [27,29^•] respectively.

It is important to note that BG-mediated gating is unlikely to be the only mechanism by which working memory is updated. For example, dopaminergic projections might directly ‘toggle’ prefrontal ensembles from a labile state to a more stable one, and hence act as a second kind of gating mechanism [21]. Indeed, high-resolution fMRI reveals the dopaminergic midbrain reliably responds to demands on working memory updating [31^•] even after the requisite gating policies have been acquired — a finding not required by BG-mediated gating models, and broadly consistent with direct dopaminergic gating of PFC. However, there remain some challenges for this kind of account, most notably in cases where updating would be selective. Dopaminergic projections into PFC are diffuse and may not have the necessary spatial specificity for selective updating of distinct representations [32]. Selective updating by dopaminergic input might occur temporally instead (e.g. via phase-tuned or frequency-tuned signals), but the prefrontal dopamine response may also lack the temporal resolution required by this scheme [33] (unlike BG output to thalamus [34^••]). Thus, while dopamine clearly has effects in PFC (perhaps largely via effects on the gain of neuronal ensembles), the spatial-coarseness and temporal-coarseness of prefrontal dopaminergic afferents might render those projections ineffective for selective working memory updating. Nonetheless, people are capable of simultaneously updating the entirety of working memory [35]; diffuse dopaminergic neuromodulation might be well adapted for such ‘global updates’ (but see [36,37]).

Output gating of working memory

According to the prevailing top-down ‘biased competition’ model of prefrontal function, information residing in working memory actively biases behavior. However, not all information in working memory needs to be relevant at the same time, and indeed might cross-talk or mutually interfere if mere maintenance yielded an obligatory biasing influence. Clearly, the capacity to ‘single out’ or select relevant representations stored within working memory is adaptive [38]. Behavioral evidence indicates that humans are capable of selecting information from within working memory [39].

One possibility is that BG-mediated gating mechanisms for selecting actions might also be extended for selecting the outputs of working memory. In fact, the analogy between the BG’s role in action selection and its potential role in selecting working memory output is straightforward. Premotor areas gating the output of primary motor neurons requires similar rostral-to-caudal frontostriatal projections as required for more abstract representations in working memory to influence premotor planning. In other words, higher-order plans can select motor plans via rostral corticostriatal circuits, just as motor plans can select individual movements via caudal ones.

Hierarchical, rostrocaudal neural architectures have recently been argued to support the performance of complex tasks involving conditional rules [40,41,42^••,43–45,46^•]. A priori, output gating is an advantageous scheme in frontostriatal hierarchies of this kind. Unlike hierarchical input gating, hierarchical output gating allows subordinate regions to proceed with their own input and reallocation policies until (or unless) higher-order regions identify an important context or conditionality. Only at that point would higher-order regions impinge on the function of lower-order regions by biasing their output toward a contextually appropriate subset of candidates. Computational models have demonstrated the feasibility of this corticostriatal output-gating architecture for solving hierarchical tasks [18,22^••,42^••], and at least one such model has been supported by data from fMRI [42^••]. Moreover, human diffusion tractography confirms a prediction motivated by this model — namely, that any given area of striatum is more likely to also receive projections from frontal areas more rostral, rather than caudal, to its primary input source [47].

Though a variety of computational modeling thus indicates that corticostriatal circuits can support output gating, empirical studies have only begun to test the function of this hypothesized system. We recently confirmed the differential importance of output gating in hierarchical control [48^••]. Our task used three sequentially presented and completely reorderable stimuli: two ‘item’ stimuli and a ‘context’ stimulus that specified which of the two items would be relevant for responses. The core logic was straightforward: when the context appears first, it can be used to drive selective input gating of only the relevant subsequent item into working memory; however, when context appeared last, it could only be used for selectively output gating the relevant item out of all those seen. All trials showed sustained recruitment of a relatively caudal sector of frontal cortex (the dorsal pre-motor cortex, or PMd), but a somewhat more rostral area (the pre-PMd) transiently increased its recruitment specifically when context was provided last, and was therefore implicated output gating (Figure 3a). An overlapping region of the pre-PMd also increased its coupling with the BG in the same conditions (Figure 3b). These two dynamics in pre-PMd each predicted a distinct kind of individual difference during selective output gating alone: whereas bilateral prePMd recruitment predicted the mean efficiency of responses during selective output gating, its bilateral coupling with BG predicted response variability, as expected of a stochastic BG-mediated output gate.

Figure 3.

Output gating and reallocation. (a) A transient BOLD response is elicited in the dorsal pre-premotor cortex (pre-PMd) by demands on selective output gating. Individual differences in the recruitment of this area and its right hemisphere homologue uniquely predict the mean efficiency of selective output gating, as assessed in behavior. (b) A partially overlapping region also in the vicinity of the pre-PMd shows a differential increase in coupling with the BG during output gating. Individual differences in this coupling uniquely predict behavioral estimates of stochastic variability during selective output gating. (c) BOLD in a more caudal sector of frontal cortex, the PMd, tracks trial to trial changes in the predicted utility of information (as estimated from a reinforcement learning model), but only when it is specified as relevant by a higher-order contextual stimulus. (d) By contrast, when contexts specify information in working memory as irrelevant, predicted utility is differentially tracked by the BOLD response in the bilateral ventral BG.

(a,b) Adapted from [48^••]; (c,d) adapted from [50^••].

Working memory content control: the case of reallocation

The rapidly developing literature on working memory input and output control has been strongly guided by the numerous models to posit that BG-mediated gating processes may address these problems. Unfortunately, computational models differ widely in how they treat a third kind of control problem. How is working memory reallocated when already-stored information is later revealed to be irrelevant? By some accounts, an active removal process is necessary; by others, passive decay could be sufficient [49]. Finally, a third class of models posit that irrelevant representations will tend to linger until (or unless) they are overwritten with new information, such as by input gating mechanisms [6,10,15,23^••]. All such accounts lead to the prediction that the utility of information in WM for future behavior is tracked in some way. Given its established role in action value coding, the BG is again an a priori candidate for this function.

We recently found evidence consistent with this hypothesis [50^••]. We analyzed trials of our reorderable working memory task where context appeared in the middle position, between the presentation of the two lower-level items. When this ‘context middle’ stimulus rendered the preceding lower-level item irrelevant, we observed a large benefit to behavioral performance when sufficient time followed presentation of the context. This benefit was much larger than that seen in any other condition — as though subjects required time to reallocate working memory capacity occupied by the irrelevant item. This result parallels others (see [50^••]) demonstrating a sluggish time course for WM reallocation, with irrelevant information impacting behavior even 1.5 s later.

We predicted that this slowing could occur because to-be-removed items were nonetheless predicted to have utility, even though they were specified as irrelevant by the contextual stimulus. To test this counterintuitive prediction, we adapted a simple reinforcement learning model to track the likelihood that each item, regardless of the context in which it was presented, would in fact be associated with the correct answer. Learning rates in this model were fit to reaction times in our behavioral task, and from this, we predicted a function of trial-to-trial predicted utility of irrelevant items. This timecourse correlated with activation in ventral striatum in a separate fMRI experiment. By contrast, the model-based estimates of the utility of relevant items were tracked by recruitment in frontal, not striatal regions (Figure 3c,d).

These results motivate the inclusion of BG-mediated mechanisms in models of WM reallocation [51] and other WM control processes. They also reaffirm the dichotomous stability vs. flexibility functions sometimes ascribed to frontal vs. striatal regions in the service of working memory, as well as the opposing actions of dopamine on these two areas. One intriguing possibility consistent with these results is that BG-mediated gating mechanisms might be capable of ‘vetoing’ the clearance of information from working memory, analogous to the motoric preservation induced by stimulation of the ventral striatum [52].

Conclusions

Working memory contends with the complexity of the real world via a set of control processes that select what items to maintain, which maintained items to use, and the priority of items within memory. Many of these demands are analogous to those faced in movement selection by the motor system. Accordingly, frontostriatal mechanisms for motor selection might be elaborated in more rostral frontostriatal circuits and used for more abstract working memory operations. This long-held hypothesis has now been subjected to empirical tests. Abundant evidence supports a role for BG-mediated input gating mechanisms during working memory updating. In addition, there is now emerging evidence for BG-mediated mechanisms during selection from working memory and in tracking the predicted utility of items within working memory. Both of these latter functions may be crucial in supporting more sophisticated forms of planning and thought. And though many unanswered questions remain (Box 1), these new discoveries represent a major success story for the use of neurocomputational modeling to inform the cognitive neuroscience of how working memory might actually work, in the brain.

Box 1. Open questions.

• How do gating dynamics develop across the lifespan [54^•,55], and could they underpin age-related shifts in modes of cognitive control [56,57]?

• What is the pharmacology and neurogenetics of working memory output control and reallocation?

• Might BG-mediated gating enable frontal cortex to implement Bayesian inference [22^••] and symbolic referencing [23^••] (see also [58])?

• How do BG contribute to the time-varying, high-dimensional cortical representations highlighted in the cortex-centric recurrent network models of motor [59] and cognitive [60^•,61^•] control?

• Are motor and cognitive corticostriatal circuits distinct truly isomorphic save their rostrocaudal locus, and evolutionary history (for at least one exception, see [62])?

• How might BG-mediated gating dynamics illuminate classic cognitive phenomena like the psychological refractory period, the focus of attention, and object-based encoding in visual working memory [63^••,64,65^••], independent of expectancy violations [65^••]?