Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2013 Aug 28;33(35):14040–14047. doi: 10.1523/JNEUROSCI.1176-13.2013

Working Memory and Anticipatory Set Modulate Midbrain and Putamen Activity

Yen Yu 1,✉, Thomas HB FitzGerald 1, Karl J Friston 1
PMCID: PMC3756752  PMID: 23986240

Abstract

To behave adaptively, an organism must balance the accurate maintenance of information stored in working memory with the ability to update that information when the context changes. This trade-off between fidelity and flexibility may depend upon the anticipated likelihood that updating will be necessary. To address the neurobiological basis of anticipatory optimization, we acquired functional magnetic resonance imaging data, while healthy human subjects performed a modified delayed-response task. This task used cues that predicted memory updating, with high or low probability, followed by a contingent updating or maintenance event. This enabled us to compare behavior and neuronal activity during conditions in which updating was anticipated with high and low probability, and measure responses to expected and unexpected memory updating. Based on the known role of dopamine in cognitive flexibility and working memory updating, we hypothesized that differences in anticipatory set would be manifest in the dopaminergic midbrain and striatum. Consistent with our predictions, we identified sustained activation in the dopaminergic midbrain and the striatum, associated with anticipations of high versus low updating probability. We also found that this anticipatory factor affected neural responses to subsequent updating processes, which suppressed, rather than elevated, midbrain and striatal activity. Our study addresses for the first time an important and hitherto understudied aspect of working memory.

Introduction

Working memory (WM) involves actively maintaining and manipulating mental representations in the absence of external stimuli (Baddeley, 1992, 2012). Balancing maintenance and manipulation involves trading off flexibility against the robustness of representations, but little is known about how this is achieved.

The striatum is generally thought to be important for the manipulation of WM representations such as WM updating (Veltman et al., 2003; Marklund et al., 2009), while the dorsolateral prefrontal cortex (DLPFC), which is an integral part of prefrontostriatal functioning, has the neuronal architecture necessary for WM maintenance (Goldman-Rakic, 1995). Dopamine exerts antagonistic influences in the two systems (Crofts et al., 2001) by modulating neuronal excitability through distinct distributions of D1/D2 receptors (Camps et al., 1989), and is thus a promising candidate for balancing the stability and flexibility of WM representations. Tonic dopamine tends to stimulate high-affinity D2 receptors, whereas phasic dopamine generally increases low-affinity D1 stimulation level (Goto et al., 2007; Dreyer et al., 2010). Phasic dopamine is thought to be important for updating (Frank et al., 2001; O'Reilly and Frank, 2006), but we do not know how tonic dopamine modulates the updating of WM representations.

Existing neuroimaging studies of memory updating generally focus on the comparison between non-updating (maintenance) and (selective/total) updating (Lenartowicz et al., 2010; Podell et al., 2012). The findings generally concur with theoretical predictions (Frank et al., 2001; O'Reilly and Frank, 2006; Hazy et al., 2007), in which the frontostriatal network controls access to WM (McNab and Klingberg, 2008), with phasic dopamine acting as a gating signal (Murty et al., 2011; D'Ardenne et al., 2012). Although this provides a compelling account of updating, it does not address a crucial aspect of adaptive behavior and brain function: how the brain balances the maintenance of beliefs about the world with the assimilation of new information (Rao and Ballard, 1999; Friston and Stephan, 2007), a balance that is likely to depend upon the anticipated changeability or volatility of environmental cues (Behrens et al., 2007). Manipulating the anticipated likelihood of updating may thus provide a new insight into the functional anatomy of memory updating. Tonic dopamine has been associated with uncertainty on both empirical (Fiorillo et al., 2003) and theoretical (Friston et al., 2012) grounds, suggesting a possible augmentation of the phasic updating model to include a role for tonic dopamine in encoding the precision of, or confidence in, the task-relevance of current representations.

To characterize the functional anatomy of updating in WM, we used predictive cues to manipulate subjects' anticipatory set or beliefs about the probability that WM updating would be called upon. Our principal hypothesis was that anticipations about imminent updating would increase cognitive flexibility via modulations of tonic activity in the dopaminergic system and would thereby interact with the subsequent updating per se.

Materials and Methods

Subjects.

Seventeen subjects (eight females; mean ± SD age, 28.0 ± 4.4 years; range, 21–36 years) were recruited via the University College London Psychology Subject Pool. Subjects were right-handed, had unimpaired or correct-to-normal visual acuity, and normal color vision, with no history of psychiatric or neurological illness. This study was approved by the Institute of Neurology (University College London) Ethics Committee. All subjects provided informed consent before the study.

Task and stimuli.

The stimuli comprised predictive cues and memory arrays (Fig. 1a). Predictive cues were a green triangle pointing upward [updating cue (UC)], which predicted a high (80%) likelihood of updating subsequently remembered items, and a red triangle pointing downward [maintenance cue (MC)], predicting a low (20%) likelihood of updating (and therefore a high likelihood of maintenance).

Figure 1.

Figure 1.

Open in a new tab

Stimuli and task design. a, Each row illustrates an example of a predictive cue and subsequent memory arrays, under one of the four conditions: MCM, MCU, UCM, and UCU. The cues reported the update predictability, in which the probability of updating is 80% given a green cue and 20% for a red cue (equivalent to 80% maintenance probability). The shaded rows represent valid cue–outcome associations. The examples shown use a consistent probe array, in which subjects should give positive (“true”) responses. b, Events and durations within a single trial shown as stimulus functions. Activity associated with anticipatory set (updating set and maintenance set) was modeled with a boxcar function of 6 s. Action (updating or maintenance) was modeled for unsurprising and surprising trials. Surprises (dashed spike) were modeled separately for MCU and UCM conditions. Nonspecific task effects (NS) were treated as nuisance effects. These included the visual onsets of cue, encoding, and probe events. Sets, actions, and surprises were not modeled for error trials. They were modeled as NS (the dashed line on NS correspondence to action on set). Although the anticipatory set should be disengaged upon the display of action array, additional set-switching effects were modeled at the cue onset.

Following the predictive cue, each trial had three sequential components, following the predictive cue: encoding, action, and probe (Fig. 1b). Note that “action” refers to the cognitive operation (updating or maintenance) predicted by the cues, not to the motor output per se. Each component was cued by an array containing items to be remembered or matched. Each array consisted of a 1 × 5 grid presented in the same color as the preceding updating or maintenance cue. Depending on the number of items, the arrays could be full, partial, or empty. The encoding and probe arrays were full arrays, in which the elements were five nonrepeating capital letters sampled randomly from 19 English consonants (excluding W and Y) to ensure phonologically distinct combinations. In addition, arrays were excluded if the letter sequence or its neighbor formed a common acronym or word. An action array could be a partial array or an empty array, cueing updating and maintenance events, respectively. A partial action array contained a single letter in a random position: this update letter was generated from the same set of consonants but excluding the five used in the preceding encoding array. On update trials, subjects were required to update their memory and replace the encoding letter with the update letter at the corresponding position. Note that the need to update was only explicit after the update array appeared, although the predictive cue could establish an appropriate cognitive (anticipatory) set, depending on whether updating or maintenance was, a priori, more likely.

The probe array either matched the array that subjects had in memory (including the update if it had occurred), or differed by a single letter, with equal probability in each of the five locations. Subjects responded by pressing a “true” key when they thought that the probe array matched the array they had in memory, or a “false” key otherwise. Subjects were told that there would be only one inconsistent letter if the probe did not match the remembered items. A fixation cross was presented during intertrial intervals and during the retention between the offset of the encoding array and the onset of the updating array. The stimuli were presented using Cogent 2000 and Matlab (MathWorks).

We defined “valid” outcomes as trials where subjects had to update after an updating cue, or they had to maintain after a maintenance cue. Violations of the cue–action associations were termed invalid or “surprise” trials. The subjects were informed that cue contingencies were veridical and were encouraged to rely on the cue to guide the task performance.

Our task therefore conforms to a 2 × 2 factorial design, with the two factors comprising anticipatory set (high vs low probability of updating) and updating (update vs maintenance). This provided four conditions with regard to the valid and invalid (surprising) cue–outcome pairings: maintenance cue–maintenance (MCM), maintenance cue–updating (MCU), updating cue–maintenance (UCM), and updating cue–updating (UCU).

The predictive cue was presented for 1500 ms, followed by a fixation cross for 500 ms. Then, the encoding array appeared for 2000 ms, followed by a fixation cross for 2000 ms, while the subjects maintained the items in the encoding array. The action array then appeared for 1000 ms, followed by a blank screen for 1000 ms. Finally, the probe array was presented for 2000 ms. Subjects were required to respond as fast as possible on the appearance of the probe array. No feedback was provided during the experiment. Reaction times (RTs) were measured from the onset of the probe array. The total duration of a single trial was 12 s, with an intertrial interval of 2000 ms.

The task consisted of a single session of 100 trials. The maintenance cue and updating cue trials alternated every trial. There were equal numbers of true and false trials, counterbalanced across MCM and UCU conditions, as well as MCU and UCM conditions. Each session lasted 1200 s. Subjects responded with their index and middle fingers of their right hand using an MRI-compatible keypad. In half of the subjects, the answer “true” was mapped to the index finger and “false” to the middle finger; in the other half, the converse was the case. To minimize nonspecific processing demands, the words “True” and “False” were visible on the lower third of each probe display, on the side of the corresponding response finger.

Immediately before the fMRI experiment, each subject underwent a 1 h instructed training session. Then, a 10-trial version of the task was administered with feedback to confirm the subject had understood the task. Each subject was required to achieve 100% accuracy to enter the second part of the training, which comprised 100 trials without feedback to prepare the subject for the fMRI experiment.

WM capacity.

It has been proposed that individual differences in WM span may contribute to performance in WM updating (Ecker et al., 2010). Thus, we administered a task before the training to measure WM capacity (WMC) for each subject. This allowed us to adjust for any effect of WMC on updating. This task required the subjects to recall a letter sequence in order. The sequence was randomly generated using the same letter set used in the updating task. The subjects viewed the letters against a black background at the rate of 1 per second. The list began with four letters, increasing by one letter with every two successful trials until the subject committed an error. During responding, the subjects were allowed to type and to make corrections before they submitted their answers. The highest span size that was achieved twice was recorded for each subject.

Image acquisition.

Structural and functional images were acquired on a 3 tesla Magnetom Trio MRI system (Siemens). Functional images were acquired with a 32-channel head coil, using a single-shot echo-planar imaging sequence (slice repetition time, 70 ms; echo time, 30 ms; ascending slice acquisition order; voxel size, 3 × 3 × 3 mm3; field of view, 192 mm). During functional acquisition, a respiratory belt and pulse oximeter were used to monitor peripheral physiological variations. Field inhomogeneity was sampled by means of field mapping (short echo time, 10 ms; long echo time, 12.46 ms; total EPI readout time, 37 ms). Multiparameter images, including T1-weighted, proton density, and magnetization transfer contrasts, were acquired for structural information using 3D FLASH (fast low-angle shot) sequences.

Behavioral analysis.

The RT data were taken as the dependent variables and analyzed with a 2 × 2 repeated-measure ANCOVA (Fig. 2). Harmonic means of the RTs were calculated for each condition in each subject to better characterize their central tendency (Ratcliff, 1993). The RTs of error trials were excluded from the analysis. The ANCOVA included the effect of individual differences in WMC. Response accuracy (proportion correct) was calculated within each subject for each condition.

Figure 2.

Figure 2.

Open in a new tab

RT results indicated a significant interaction between surprising (invalid) and unsurprising (valid) conditions (ANCOVA controlled for individual differences in WM capacity; F(1,15) = 9.43, p = 0.008), suggesting that the subjects were able to discriminate the cues behaviorally. The MAI-set predicts the MAI event, whereas the UPD-set predicts the UPD event, both at 80% probability.

fMRI data acquisition and preprocessing.

Imaging data were analyzed using SPM12 (Statistical Parametric Mapping; Wellcome Trust Centre for Neuroimaging, London, UK). Preprocessing of functional images included correction for geometric distortion using field maps (Jezzard and Balaban, 1995; Hutton et al., 2002), realignment via affine registration to correct for head movement, slice timing correction, coregistration with respect to anatomical images, normalization to MNI space based on the anatomical normalization parameters, interpolation to voxel size of 2 × 2 × 2 mm3, and smoothing with a Gaussian kernel of 4 mm FWHM (full-width at half-maximum).

First (within subject) level analysis included eight task-related regressors in a general linear model (GLM): maintenance set (MAI-set), updating set (UPD-set), updating (UPD), maintenance (MAI), surprising updating (sU), surprising maintenance (sM), nonspecific task effects (NS), and set-switching. MAI-set and UPD-set were models with 6 s boxcar functions, extending from the onset of cue stimuli to the offset of the retention period. These regressors modeled the sustained cue-specific anticipatory set-related activity, during which subjects prepared for the forthcoming action array. The set-switching regressor modeled transient responses at the cue onset, which can be associated with the effect of trial transition. UPD and MAI entered the GLM for all events, whether they were surprising or unsurprising. sM and sU modeled the interaction between anticipatory set and action (i.e., UPD-set–MAI and MAI-set–UPD), where invalid outcomes violated anticipatory expectations. Surprises were modeled as transient responses at the onsets of action arrays under the MCU or the UCM condition. Using nonspecific UPD and MAI regressors, together with regressors encoding surprising UPD and MAI, is equivalent to comparing valid/invalid trials. Finally, responses to encoding, probe, and all cues from error trials were modeled by a NS regressor as a nuisance effect. The eight stimulus functions were convolved with a canonical hemodynamic response function to produce hemodynamic regressors for the GLM. Other effects of no interest, including head motion and low-level physiological variations, were modeled with an additional 20 regressors. Head motion was summarized using three translations (x, y, and z directions) and three rotations (pitch, roll, and yaw) derived from the realignment procedure. The physiological nuisance effects comprised six cardiac regressors, six respiratory regressors, and two regressors for heart rate change and change in respiratory volume (Hutton et al., 2011).

Region of interest analysis.

Empirical and theoretical accounts of the “gating” hypothesis implicate the dopaminergic midbrain and the striatum in memory updating (Frank et al., 2001; O'Reilly, 2006; Murty et al., 2011; D'Ardenne et al., 2012). We hypothesized that activity in these regions would also be modulated by anticipatory set. We therefore defined regions of interest (ROIs) in the substantia nigra/ventral tegmental area (SN/VTA), the striatum, and the DLPFC and analyzed responses within these regions across each level of anticipatory set and action.

Anatomically informed functional ROIs were created for the midbrain and the striatum in two steps: (1) after the SN/VTA region was identified in the mean normalized magnetization transfer image averaged across subjects (Helms et al., 2009; Fitzgerald et al., 2012), we manually traced the SN/VTA to create a (preliminary) anatomical ROI; (2) the anatomical ROI was then masked with the thresholded activation map of set (main effect of UPD-set and MAI-set, uncorrected p = 0.005). A similar procedure was adopted for action. A small-volume correction was performed on tests for responses within the ROI search volume. The main effect of set (or action) was specified with appropriate contrasts averaging over UPD-set and MAI-set (or for UPD and MAI in the case of action). The resulting contrast images were then entered into a second (between-subject) level analysis using a one-sample t test. Importantly, these localizing (ROI defining) contrasts are orthogonal to the differential effects of maintenance and updating (within set or action) that were tested using one-sample t tests.

The functional SN/VTA ROI for set consisted of 240 voxels [p = 0.001, cluster false discovery rate (FDR); Fig. 3a]. This ROI was used to summarize UPD-set and MAI-set effects in terms of their principal eigenvariates. A one-sample t test was performed to test for updating versus maintenance effects of anticipatory set on these summary statistics.

Figure 3.

Figure 3.

Open in a new tab

SN/VTA BOLD responses of set and action phases: regional responses to each level of set and action were extracted using a functional ROI defined with an orthogonal contrast. a, The functional ROI for set was defined by the main effect of set over both its levels (240 voxels, p = 0.001, cluster FDR). b, The functional ROI for action was defined by the main effect of action over both its levels (186 voxels, p = 0.002, cluster FDR). Voxels within these functional ROIs were activated, as determined by small-volume correction using a predefined anatomical ROI based on mean normalized magnetization transfer across subjects. c, ROI analysis for the SN/VTA region across experimental phases. The SN/VTA activity was significantly larger when expecting an updating event (left bar, **p = 0.008), whereas the SN/VTA was slightly decreased on updating per se compared with maintenance (right bar, p = 0.127). An interaction between set and action in the SN/VTA was also evident (*p = 0.040). *p < 0.05; **p < 0.01.

The functional SN/VTA ROI for action consisted of 186 voxels (p = 0.002, cluster FDR; Fig. 3b). Principal eigenvariates were then extracted to summarize UPD and MAI effects. The effect of updating versus maintenance was then tested with a one-sample t test.

Striatal set activation was only observed in the left putamen. For the sake of consistency, ROI analysis of the action phase was reported in the same region. We referred to the Automatic Anatomical Labeling atlas (Tzourio-Mazoyer et al., 2002) for the anatomical ROI of the left putamen. This anatomical ROI was masked with the thresholded activation map of the main effect of set and action separately (both used uncorrected p = 0.001). This yielded two functional ROIs: the putamen-set ROI consisted of 205 voxels (p < 0.001, cluster FDR; Fig. 4a); the putamen-action ROI consisted of 460 voxels (p < 0.001, cluster FDR; Fig. 4b). The same contrasts, UPD-set > MAI-set and UPD > MAI, were tested with one-sample t tests after extracting the principal eigenvariates for each condition.

Figure 4.

Figure 4.

Open in a new tab

Striatal BOLD responses of set and action: regional responses to each level of set and action were extracted using a functional ROI defined with an orthogonal contrast. a, The functional ROI for set was defined by the main effect of set over both levels (205 voxels, p < 0.001, cluster FDR). b, The functional ROI for action was defined by the main effect of action over both levels (460 voxels, p < 0.001, cluster FDR). Voxels within these functional ROIs were activated, as determined by small-volume correction using the left putamen mask from the Automatic Anatomical Labeling atlas. c, ROI analysis for the left putamen across experimental phases. Anticipatory activity in the left putamen was significantly larger with high update probability, compared with low update probability (left bar, *p = 0.012). There was no difference in the striatal activity between updating and maintenance (right bar, not significant p = 0.652). *p < 0.05; **p < 0.01.

Activation in the DLPFC was identified in the right hemisphere during action. Given no a priori anatomical constraint, the ROI specification was based on an isolated cluster in the right middle frontal gyrus [peak (44, 30, 24), p < 0.001, cluster FDR, 203 voxels; Fig. 5a] on the main effect of action. UPD and MAI parameter estimates were extracted accordingly, followed by testing the contrast UPD > MAI using a one-sample t test.

Figure 5.

Figure 5.

Open in a new tab

DLPFC BOLD responses during action phase. a, The functional ROI for extracting UPD-specific and MAI-specific DLPFC responses was defined by the main effect of action over both levels, where an isolated cluster was identified in the right hemisphere (arrow; 203 voxels, p < 0.001, cluster FDR). b, ROI analysis revealed a significant UPD > MAI contrast in the right DLPFC, showing a larger response to updating events (**p = 0.009). *p < 0.05; **p < 0.01.

Results

Behavioral results

An ANCOVA, including individual WMC measures as a covariate, demonstrated a significant crossover interaction between cue (high or low updating probability) and action on the RTs (Fig. 2; F(1,15) = 9.43, p = 0.008; mean RTs ± SD in seconds: MCM, 1.15 ± 0.17; MCU, 1.22 ± 0.20; UCM, 1.18 ± 0.18; UCU, 1.17 ± 0.20). There was no main effect for cue (F(1,15) = 1.08, p = 0.315) or action (F(1,15) = 0.005, p = 0.945). On average, the subjects performed the task to an accuracy level of 80.71% (SD, 15.72%) during the scanning session. Statistical tests revealed no significant main effect or interaction for response accuracy. The average measure for the subjects' WMC was 6.2 letters. Performance on the task, as measured by Cowan's capacity index (Cowan, 2005), significantly predicted the WMC measure in a linear regression (R2 = 0.418, β = 0.646, p = 0.005).

Neuroimaging results

SN/VTA responses during set and action

Set activity for UPD-set was significantly larger than for MAI-set (Fig. 3c, left; t(16) = 3.003, p = 0.008) in the SN/VTA. Then, we tested UPD > MAI using a one-sample t test and showed a slight trend decrease in SN/VTA activity for UPD (Fig. 3c, right), albeit insignificant (t(16) = 1.609, p = 0.127). A significant interaction was observed between the effect of set and action (Fig. 3; t(16) = 2.237, p = 0.040).

The main effect of set revealed a common response over the UPD-set and MAI-set in the SN/VTA, the left putamen, the left premotor cortex, the SMA, the left posterior parietal cortex, and the bilateral visual cortices (Table 1). No activation in the DLPFC was detected. Widespread activation under the main effect of action was observed primarily in the occipital cortices, extending into the superior parietal cortices and the bilateral frontal cortices (Table 2). Subcortical activation included bilateral striatum, the SN/VTA, and the thalamus. No activation was detected either in our ROIs or whole-brain correction for either the sU or sM contrasts.

Table 1.

Localization of set-related activation

Anatomical regions Hemisphere BA Size Local maxima
 t value
x y z
Main effect of seta
    Calcarine cortex Right 18 415 18 −96 0 11.21
    SMA Bilateral 6 546 −4 0 66 9.59
    Inferior occipital cortex Left 18 553 −16 −94 −8 9.52
    Premotor cortex Left 6 423 −48 −2 40 8.65
    Cerebellum Right — 170 26 −66 −24 7.63
    Putamen Left — 205 −18 2 10 7.21
    Thalamus Left — 422 −6 −26 −6 6.61
    Inferior parietal cortex Left 40 116 −34 −46 40 6.58
    Midbrain: SN/VTA Bilateral — 64 10 −12 −12 6.16
    Premotor cortex Left 6 62 −28 −8 52 5.88
    Middle occipital cortex Left 18 100 −30 −90 12 5.84
    Superior parietal cortex Left 7 132 −22 −66 42 5.00
Contrast UPD-set > MAI-setb
    Calcarine cortex Right 17 2046 8 −74 16 5.54
    Middle occipital cortex Left 18 495 −34 −92 −2 5.50
    Inferior parietal cortex Left 40 254 −38 −44 38 5.08
    Premotor cortex Right 6 342 28 −8 52 5.06
Open in a new tab

aCluster-wise significance at 0.05 FDR, using a cluster-defining threshold of p = 0.001; critical cluster size was 62.

bCluster-wise significance at 0.05 FDR, using a cluster-defining threshold of p = 0.01; critical cluster size was 254.

Table 2.

Localization of action-related activation

Anatomical regions Hemisphere BA Size Local maxima
 t value
x y z
Main effect of actiona
    Thalamus Left — 11022 −16 −22 10 13.62
    Lingual gyrus Right 18 13752 16 −86 −8 12.75
    Parietal operculum cortex Right 2 228 54 −22 22 9.80
    Middle frontal gyrus Right 45 203 44 30 24 7.09
    Cingulate cortex Bilateral 24 298 −4 2 40 6.85
    Precuneus cortex Right 7 240 8 −76 52 6.73
    Inferior orbital frontal cortex Right 47 211 42 48 −8 5.95
    Precuneus cortex Left 7 168 −10 −74 40 5.64
    Cingulate cortex Right 32 205 14 24 36 5.63
    Cingulate cortex Bilateral 23 162 2 −26 26 5.39
    Inferior parietal cortex Right 40 364 46 −50 44 5.16
Contrast UPD > MAIb
    Superior parietal lobule Left 7 1041 −28 −60 44 10.58
    Middle frontal gyrus Right 45 141 38 30 30 8.67
    Superior parietal lobule Right 7 1104 34 −60 46 8.32
    Superior parietal lobule Right 7 140 14 −74 58 6.38
    Premotor cortex Left 6 84 −42 2 32 5.39
    Premotor cortex Left 6 73 −28 −8 50 5.18
Open in a new tab

aCluster-wise significance at 0.05 FDR, using a cluster-defining threshold of p = 0.001; critical cluster size was 158.

bCluster-wise significance at 0.05 FDR, using a cluster-defining threshold of p = 0.001; critical cluster size was 73.

Striatal responses during set and action

Striatal activation was only observed in the left putamen for the main effect of set, while responses in bilateral basal ganglia were observed for the main effect of action. For consistency, we report set and action effects for the left putamen. Comparing activations in the putamen revealed that the UPD-set elicited a significantly larger response than the MAI-set (Fig. 4c, left; t(16) = 2.832, p = 0.012). Comparing the principal eigenvariates extracted from UPD and MAI using the functional mask showed no significant difference (Fig. 4c, right; t(16) = −0.460, p = 0.652).

DLPFC responses during action

We specified a cluster in the right DLPFC as a functional mask. Activity in the DLPFC during updating was significantly larger than during non-updating (Fig. 5b; one-sample t test; t(16) = 2.993, p = 0.009).

Neurobehavioral correlations

In neural terms, exploiting cognitive set usually speaks to optimal gain and efficiency in the presence of limited resources, thereby favoring behavioral outcomes in the absence of surprising outcomes (Fuster, 2008; Gazzaley and Nobre, 2012). We therefore tested for correlations between set activity and behavioral responses. Specifically, nonparametric correlations were performed to discover whether a greater neurophysiological set activity improved response accuracy. The response accuracy in the UCU condition was positively correlated with the UPD-set activity in the SN/VTA (Spearman's ρ = 0.546, p = 0.023). Similarly, the MAI-set activity in the SN/VTA was positively correlated with the response accuracy in the MCM condition (SN/VTA: ρ = 0.483, p = 0.049). No correlations were observed for invalid trials: UCM accuracy and UPD-set activity (ρ = 0.415, p = 0.098); MCU accuracy and MAI-set activity (ρ = 0.188, p = 0.469). These significant correlations between measures of neuronal responses and behavior lend a further validity to the physiological effects reported above.

Discussion

We tested the hypothesis that anticipating a WM update is accompanied by activation in the dopaminergic midbrain. Consistent with our hypothesis, we found that updating-related anticipation induced sustained activity in the midbrain and striatum, suggesting a key role for tonic dopamine in the maintenance of anticipatory set (rather than for maintaining memory). In addition, the amplitude of set-related activity in the midbrain and striatum was positively correlated with response accuracy in valid trials (UCU and MCM).

The connection between the SN/VTA BOLD response and dopamine neuron firing speaks to several plausible cellular mechanisms (Düzel et al., 2009). Fiorillo et al. (2003) have demonstrated that the level of tonic dopamine firing varies with the uncertainty about future events. Predictive cues may entail uncertainties about updating, thereby inducing changes in tonic dopamine release. In addition, by using mixed task regressors with different temporal profiles (Donaldson, 2004), we were able to distinguish set-related processing from transient responses. It therefore seems plausible to associate the sustained activation we observed with a tonic mode of dopamine release. We hope to test this assumption in future work using a pharmacological intervention.

How might tonic dopamine contribute to updating? A plausible model of dopaminergic activity in this context can be considered in terms of attractor dynamics for WM. It has been argued that the prefrontal cortex maintains WM representations (Goldman-Rakic, 1995; Cohen et al., 1997; Courtney et al., 1997) in multiple attractor states. To enable flexible representation, a relatively flat energy landscape is required, such that the system can move easily from one metastable attractor state to another (Durstewitz et al., 2000). The tonic level of dopamine discharges may play a role in modulating this transition by activating D2 receptors (Schultz, 2007; Rice and Cragg, 2008; Dreyer et al., 2010), but not the D1 receptors that have lower dopamine affinity. In effect, a prefrontal “D2 state” would reduce the stability of attractors and facilitate updating or transitions (Durstewitz and Seamans, 2008), thus rendering the system more responsive to afferent inputs. Tonic dopamine release might have a greater influence in the striatum due to higher D2 prevalence, relative to the prefrontal cortex, where D1 receptors are more abundant (Camps et al., 1989; Goldman-Rakic et al., 1992). In this setting, flexible updating may come at the cost of lowered precision or signal-to-noise ratio. In other words, the neuronal instantiation of anticipatory set for updating may be accompanied by changes in the precision or confidence afforded to cues.

We observed elevated BOLD responses in the left putamen when updating was expected, which might reflect D2 stimulation on the striatal spiny neurons or on the prefrontal afferents in layer V (Cools and D'Esposito, 2011). A possible consequence would be inhibiting the default “NoGo” indirect pathway that, in turn, disinhibits thalamocortical connections (O'Reilly, 2006).

The between-subject correlations revealed that the midbrain responses to anticipatory set predicts performance in valid, but not invalid, trials, which is consistent with the idea that the brain optimizes performance according to anticipated outcomes (Posner, 1980; Garrido et al., 2011). The correlations may, nevertheless, seem somewhat counterintuitive. Given that the midbrain was more active in the updating set than in maintenance set, one might expect that dopamine release is essential for updating but not for maintenance. However, this is not necessarily the case. It is likely that dopamine release is optimized for specific contexts, and that for each anticipatory set there is an optimal range of dopamine levels. Cognitive performance may then have an “inverted-U” dependency on dopamine levels (Cools and D'Esposito, 2011), leading to positive correlations between performance and tonic dopamine in both maintenance and updating sets (Fig. 6).

Figure 6.

Figure 6.

Open in a new tab

A schematic illustrating “dose–performance” functions under different anticipatory sets: the two curves indicate that the relationship between baseline dopamine level and behavioral performance may have an inverted-U shape. They also show that there is an optimal range in which an increase of dopamine level would improve performance. Crucially, the position of the curve may vary depending on the cognitive operation (e.g., maintenance or updating, as red and green curve, respectively). The anticipatory optimization (i.e., set) may help defining the range of baseline dopamine (represented as shaded areas), thereby optimizing performance for anticipated outcomes. This nonlinear relationship may partly account for our observation of positive correlations (between set activity and accuracy) and differential set-related activation in the midbrain. Here, we depict a possible mechanism that appeals to the notion that different behaviors (e.g., updating and maintenance) implicate different brain systems, in which optimal dopamine range is system-dependent. If set-related activity during maintenance and updating fell under the rising parts of the curves (as illustrated by the red and green box), one would expect a positive correlation, as reported in the main text. At the between-subject level, set-related modulation of dopamine could change performance in a way that depends upon subject-specific baseline dopamine levels (compare the “law of initial value”; for review, see Cools et al., 2011).

Dopamine has long been implicated as an integral component of WM function, both in terms of the stability (maintaining) and flexibility (adaptive updating) of active representations (Miller and Cohen, 2001). In particular, neurocomputational models have proposed biologically realistic mechanisms by which dopamine can contribute to WM (Frank et al., 2001; Gruber et al., 2006; O'Reilly and Frank, 2006; Hazy et al., 2007). In these accounts, phasic bursts of dopamine are associated with selective updating of WM representations through the frontostriatal circuitry ( Frank et al., 2001). Recent empirical findings in human functional imaging have demonstrated midbrain activation when updating WM of visual stimuli or contexts (Murty et al., 2011; D'Ardenne et al., 2012). Murty et al. (2011) concluded that selectively updating WM contents, as opposed to maintaining or overwriting WM contents, activated the SN/VTA region. D'Ardenne et al. (2012) reported a phasic BOLD increase in the SN/VTA during updating compared with non-updating. In general, midbrain responses are potentially attributable to updating. However, our results showed the midbrain was less active during updating, relative to maintenance (Fig. 3, right). One possible explanation for this may lie in our experimental design: in our study we manipulated anticipation about updating, which affected (putative) dopamine activity before updating. This may be relevant if the same (dopaminergic) systems have been implicated in updating per se. In other words, the anticipatory set interacts with the effect of updating.

Interactions between anticipatory set and update phases may be explained with reference to the hypothesis of tonic–phasic homeostasis (Grace, 1991; Bilder et al., 2004). This hypothesis states that the level of extracellular dopamine, as determined by tonic release, regulates the magnitude of phasic discharge. In principle, this is based on the D2 autoreceptor stimulation located in the dopamine terminals. Only tonic dopamine release is proposed to be capable of stimulating these D2 autoreceptors, which consequently restrict the neuronal responsiveness and hence the depressed spike-dependent (phasic) discharge. According to this hypothesis, the tonic release of dopamine is likely to be elicited by the presynaptic glutamatergic afferents of the prefrontal cortex. The relationship between prefrontal cortical activity and the midbrain may be an important determinant of sustained BOLD responses in the midbrain (Düzel et al., 2009).

We noted that our striatal responses (Fig. 4c, right) were inconsistent with findings from previous studies (Bledowski et al., 2009; Bäckman et al., 2011; Murty et al., 2011; Kuhl et al., 2012; Nee and Brown, 2012; Podell et al., 2012; Kühn et al., 2013). These studies suggested stronger striatal recruitment with WM updates, whereas we found no differential activation between updating and maintenance. Our finding may be sensible when viewed from a predictive coding perspective. Several studies have suggested that the striatum mediates unexpected events. That is, the response in the striatum can be related to prediction error (O'Doherty et al., 2003, 2004; den Ouden et al., 2010). Failing to observe significant updating-specific striatal recruitment may reflect the fact that an update, once predicted, is less surprising. This argument could be further extended to cover updating-specific striatal activation. That is, without manipulating anticipation, WM has the propensity to occupy a stable state (that supports robust maintenance, or “D1-state”; O'Reilly, 2006). Sudden but infrequent updating may incur greater prediction error through striatal activation. Whereas, for frequent and expected updating, a low degree of metastability could be suboptimal; instead, migrating to a regime of higher metastability may reduce the average surprise over time (cf. Friston, 2009). In other words, expected surprise is not really surprising and may not be accompanied by the precise prediction error responses that are seen when the updates are unexpected.

The DLPFC is strongly implicated in various WM tasks. Studies fractionating WM subprocesses have suggested that the DLPFC is responsible for encoding, maintenance, and manipulation of representations (D'Esposito et al., 2000). It is therefore difficult to disentangle the actual role of updating-related DLPFC activation in the current study. In particular, the functional implication of right-lateralized activation is unclear. However, transcranial magnetic stimulation (TMS)-induced disruption in the WM network may shed light on processing, where the right DLPFC appears to be critical at an early phase of updating (Linden, 2007). More recently, D'Ardenne et al. (2012) showed that time-locked, TMS-induced disruption was only effective on the right DLPFC, after the onset of updating information.

In summary, our data suggest that anticipating a WM update activates the dopaminergic midbrain and striatum, which speaks to a key role for tonic dopaminergic activity in modulating the flexibility of representations to reflect the volatility of environmental cues. These anticipations interacted with subsequent updating processes in the same regions to suppress transient responses in the midbrain and the striatum, which otherwise respond strongly to updating events. While these latter findings are prima facie inconsistent with previous findings (Murty et al., 2011; D'Ardenne et al., 2012), they can be easily accounted for from the perspective of predictive coding (Rao and Ballard, 1999; Friston and Stephan, 2007) in that expected updates are not inherently surprising. In general, our data speak to a role for dopamine in modulating the precision, or gain, on sensory information during WM processing (Frank and Badre, 2012; Friston et al., 2012). Our findings thus represent a step toward understanding both how WM flexibility is modulated in response to the demands of environment, and the likely role of tonic dopamine in WM function.

Footnotes

This work was supported by The Wellcome Trust (Grant Code WT088130/Z/09/Z). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z). We thank the Cogent 2000 team at the Functional Imaging Laboratory and the Institute of Cognitive Neuroscience, and John Romaya, who developed Cogent Graphics at the Laboratory of Neurobiology at the Wellcome Department of Imaging Neuroscience.

The authors declare no competing financial interests.

References

  1. Bäckman L, Nyberg L, Soveri A, Johansson J, Andersson M, Dahlin E, Neely AS, Virta J, Laine M, Rinne JO. Effects of working-memory training on striatal dopamine release. Science. 2011;333:718. doi: 10.1126/science.1204978. [DOI] [PubMed] [Google Scholar]
  2. Baddeley A. Working memory. Science. 1992;255:556–559. doi: 10.1126/science.1736359. [DOI] [PubMed] [Google Scholar]
  3. Baddeley A. Working memory: theories, models, and controversies. Annu Rev Psychol. 2012;63:1–29. doi: 10.1146/annurev-psych-120710-100422. [DOI] [PubMed] [Google Scholar]
  4. Behrens TE, Woolrich MW, Walton ME, Rushworth MF. Learning the value of information in an uncertain world. Nat Neurosci. 2007;10:1214–1221. doi: 10.1038/nn1954. [DOI] [PubMed] [Google Scholar]
  5. Bilder RM, Volavka J, Lachman HM, Grace AA. The catechol-O-methyltransferase polymorphism: relations to the tonic-phasic dopamine hypothesis and neuropsychiatric phenotypes. Neuropsychopharmacology. 2004;29:1943–1961. doi: 10.1038/sj.npp.1300542. [DOI] [PubMed] [Google Scholar]
  6. Bledowski C, Rahm B, Rowe JB. What “works” in working memory? Separate systems for selection and updating of critical information. J Neurosci. 2009;29:13735–13741. doi: 10.1523/JNEUROSCI.2547-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Camps M, Cortés R, Gueye B, Probst A, Palacios JM. Dopamine receptors in human brain: autoradiographic distribution of D2 sites. Neuroscience. 1989;28:275–290. doi: 10.1016/0306-4522(89)90179-6. [DOI] [PubMed] [Google Scholar]
  8. Cohen JD, Perlstein WM, Braver TS, Nystrom LE, Noll DC, Jonides J, Smith EE. Temporal dynamics of brain activation during a working memory task. Nature. 1997;386:604–608. doi: 10.1038/386604a0. [DOI] [PubMed] [Google Scholar]
  9. Cools R, D'Esposito M. Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol Psychiatry. 2011;69:e113–e125. doi: 10.1016/j.biopsych.2011.03.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Courtney SM, Ungerleider LG, Keil K, Haxby JV. Transient and sustained activity in a distributed neural system for human working memory. Nature. 1997;386:608–611. doi: 10.1038/386608a0. [DOI] [PubMed] [Google Scholar]
  11. Cowan N. Working memory capacity. New York: Psychology; 2005. Capacity limits for unstructured materials; pp. 105–137. [Google Scholar]
  12. Crofts HS, Dalley JW, Collins P, Van Denderen JC, Everitt BJ, Robbins TW, Roberts AC. Differential effects of 6-OHDA lesions of the frontal cortex and caudate nucleus on the ability to acquire an attentional set. Cereb Cortex. 2001;11:1015–1026. doi: 10.1093/cercor/11.11.1015. [DOI] [PubMed] [Google Scholar]
  13. D'Ardenne K, Eshel N, Luka J, Lenartowicz A, Nystrom LE, Cohen JD. Role of prefrontal cortex and the midbrain dopamine system in working memory updating. Proc Natl Acad Sci U S A. 2012;109:19900–19909. doi: 10.1073/pnas.1116727109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. den Ouden HE, Daunizeau J, Roiser J, Friston KJ, Stephan KE. Striatal prediction error modulates cortical coupling. J Neurosci. 2010;30:3210–3219. doi: 10.1523/JNEUROSCI.4458-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. D'Esposito M, Postle BR, Rypma B. Prefrontal cortical contributions to working memory: evidence from event-related fMRI studies. Exp Brain Res. 2000;133:3–11. doi: 10.1007/s002210000395. [DOI] [PubMed] [Google Scholar]
  16. Donaldson DI. Parsing brain activity with fMRI and mixed designs: what kind of a state is neuroimaging in? Trends Neurosci. 2004;27:442–444. doi: 10.1016/j.tins.2004.06.001. [DOI] [PubMed] [Google Scholar]
  17. Dreyer JK, Herrik KF, Berg RW, Hounsgaard JD. Influence of phasic and tonic dopamine release on receptor activation. J Neurosci. 2010;30:14273–14283. doi: 10.1523/JNEUROSCI.1894-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Durstewitz D, Seamans JK. The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-O-methyltransferase genotypes and schizophrenia. Biol Psychiatry. 2008;64:739–749. doi: 10.1016/j.biopsych.2008.05.015. [DOI] [PubMed] [Google Scholar]
  19. Durstewitz D, Seamans JK, Sejnowski TJ. Neurocomputational models of working memory. Nat Neurosci [Suppl] 2000;3:1184–1191. doi: 10.1038/81460. [DOI] [PubMed] [Google Scholar]
  20. Düzel E, Bunzeck N, Guitart-Masip M, Wittmann B, Schott BH, Tobler PN. Functional imaging of the human dopaminergic midbrain. Trends Neurosci. 2009;32:321–328. doi: 10.1016/j.tins.2009.02.005. [DOI] [PubMed] [Google Scholar]
  21. Ecker UK, Lewandowsky S, Oberauer K, Chee AE. The components of working memory updating: an experimental decomposition and individual differences. J Exp Psychol Learn Mem Cogn. 2010;36:170–189. doi: 10.1037/a0017891. [DOI] [PubMed] [Google Scholar]
  22. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  23. FitzGerald TH, Friston KJ, Dolan RJ. Action-specific value signals in reward-related regions of the human brain. J Neurosci. 2012;32:16417–16423a. doi: 10.1523/JNEUROSCI.3254-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Frank MJ, Badre D. Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. Cereb Cortex. 2012;22:509–526. doi: 10.1093/cercor/bhr114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Frank MJ, Loughry B, O'Reilly RC. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn Affect Behav Neurosci. 2001;1:137–160. doi: 10.3758/CABN.1.2.137. [DOI] [PubMed] [Google Scholar]
  26. Friston K. The free-energy principle: a rough guide to the brain? Trends Cogn Sci. 2009;13:293–301. doi: 10.1016/j.tics.2009.04.005. [DOI] [PubMed] [Google Scholar]
  27. Friston KJ, Stephan KE. Free-energy and the brain. Synthese. 2007;159:417–458. doi: 10.1007/s11229-007-9237-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Friston KJ, Shiner T, FitzGerald T, Galea JM, Adams R, Brown H, Dolan RJ, Moran R, Stephan KE, Bestmann S. Dopamine, affordance and active inference. PLoS Comput Biol. 2012;8:e1002327. doi: 10.1371/journal.pcbi.1002327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Fuster J. The prefrontal cortex. London: Academic; 2008. [Google Scholar]
  30. Garrido MI, Dolan RJ, Sahani M. Surprise leads to noisier perceptual decisions. i-Perception. 2011;2:112–120. doi: 10.1068/i0411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Gazzaley A, Nobre AC. Top-down modulation: bridging selective attention and working memory. Trends Cogn Sci. 2012;16:129–135. doi: 10.1016/j.tics.2011.11.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Goldman-Rakic PS. Cellular basis of working memory. Neuron. 1995;14:477–485. doi: 10.1016/0896-6273(95)90304-6. [DOI] [PubMed] [Google Scholar]
  33. Goldman-Rakic PS, Lidow MS, Smiley JF, Williams MS. The anatomy of dopamine in monkey and human prefrontal cortex. J Neural Transm Suppl. 1992;36:163–177. doi: 10.1007/978-3-7091-9211-5_8. [DOI] [PubMed] [Google Scholar]
  34. Goto Y, Otani S, Grace AA. The yin and yang of dopamine release: a new perspective. Neuropharmacology. 2007;53:583–587. doi: 10.1016/j.neuropharm.2007.07.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Grace AA. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience. 1991;41:1–24. doi: 10.1016/0306-4522(91)90196-U. [DOI] [PubMed] [Google Scholar]
  36. Gruber AJ, Dayan P, Gutkin BS, Solla SA. Dopamine modulation in the basal ganglia locks the gate to working memory. J Comput Neurosci. 2006;20:153–166. doi: 10.1007/s10827-005-5705-x. [DOI] [PubMed] [Google Scholar]
  37. Hazy TE, Frank MJ, O'Reilly RC. Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system. Philos Trans R Soc B Lond B Sci. 2007;362:1601–1613. doi: 10.1098/rstb.2007.2055. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Helms G, Draganski B, Frackowiak R, Ashburner J, Weiskopf N. Improved segmentation of deep brain grey matter structures using magnetization transfer (MT) parameter maps. Neuroimage. 2009;47:194–198. doi: 10.1016/j.neuroimage.2009.03.053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Hutton C, Bork A, Josephs O, Deichmann R, Ashburner J, Turner R. Image distortion correction in fMRI: a quantitative evaluation. Neuroimage. 2002;16:217–240. doi: 10.1006/nimg.2001.1054. [DOI] [PubMed] [Google Scholar]
  40. Hutton C, Josephs O, Stadler J, Featherstone E, Reid A, Speck O, Bernarding J, Weiskopf N. The impact of physiological noise correction on fMRI at 7 T. Neuroimage. 2011;57:101–112. doi: 10.1016/j.neuroimage.2011.04.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Jezzard P, Balaban RS. Correction for geometric distortion in echo planar images from B0 field variations. Magn Reson Med. 1995;34:65–73. doi: 10.1002/mrm.1910340111. [DOI] [PubMed] [Google Scholar]
  42. Kuhl BA, Bainbridge WA, Chun MM. Neural reactivation reveals mechanisms for updating memory. J Neurosci. 2012;32:3453–3461. doi: 10.1523/JNEUROSCI.5846-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Kühn S, Schmiedek F, Noack H, Wenger E, Bodammer NC, Lindenberger U, Lövden M. The dynamics of change in striatal activity following updating training. Hum Brain Mapp. 2013;34:1530–1541. doi: 10.1002/hbm.22007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Lenartowicz A, Escobedo-Quiroz R, Cohen JD. Updating of context in working memory: an event-related potential study. Cogn Affect Behav Neurosci. 2010;10:298–315. doi: 10.3758/CABN.10.2.298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Linden DE. The working memory networks of the human brain. Neuroscientist. 2007;13:257–267. doi: 10.1177/1073858406298480. [DOI] [PubMed] [Google Scholar]
  46. Marklund P, Larsson A, Elgh E, Linder J, Riklund KA, Forsgren L, Nyberg L. Temporal dynamics of basal ganglia under-recruitment in Parkinson's disease: transient caudate abnormalities during updating of working memory. Brain. 2009;132:336–346. doi: 10.1093/brain/awn309. [DOI] [PubMed] [Google Scholar]
  47. McNab F, Klingberg T. Prefrontal cortex and basal ganglia control access to working memory. Nat Neurosci. 2008;11:103–107. doi: 10.1038/nn2024. [DOI] [PubMed] [Google Scholar]
  48. Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Annu Rev Neurosci. 2001;24:167–202. doi: 10.1146/annurev.neuro.24.1.167. [DOI] [PubMed] [Google Scholar]
  49. Murty VP, Sambataro F, Radulescu E, Altamura M, Iudicello J, Zoltick B, Weinberger DR, Goldberg TE, Mattay VS. Selective updating of working memory content modulates meso-cortico-striatal activity. Neuroimage. 2011;57:1264–1272. doi: 10.1016/j.neuroimage.2011.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Nee DE, Brown JW. Dissociable frontal-striatal and frontal-parietal networks involved in updating hierarchical contexts in working memory. Cereb Cortex. 2012 doi: 10.1093/cercor/bhs194. doi: 10.1093/cercor/bhs194. Advance online publication. Retrieved February 12, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ. Temporal difference models and reward-related learning in the human brain. Neuron. 2003;38:329–337. doi: 10.1016/S0896-6273(03)00169-7. [DOI] [PubMed] [Google Scholar]
  52. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–454. doi: 10.1126/science.1094285. [DOI] [PubMed] [Google Scholar]
  53. O'Reilly RC. Biologically based computational models of high-level cognition. Science. 2006;314:91–94. doi: 10.1126/science.1127242. [DOI] [PubMed] [Google Scholar]
  54. O'Reilly RC, Frank MJ. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 2006;18:283–328. doi: 10.1162/089976606775093909. [DOI] [PubMed] [Google Scholar]
  55. Podell JE, Sambataro F, Murty VP, Emery MR, Tong Y, Das S, Goldberg TE, Weinberger DR, Mattay VS. Neurophysiological correlates of age-related changes in working memory updating. Neuroimage. 2012;62:2151–2160. doi: 10.1016/j.neuroimage.2012.05.066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Posner MI. Orienting of attention. Q J Exp Psych. 1980;32:3–25. doi: 10.1080/00335558008248231. [DOI] [PubMed] [Google Scholar]
  57. Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci. 1999;2:79–87. doi: 10.1038/4580. [DOI] [PubMed] [Google Scholar]
  58. Ratcliff R. Methods for dealing with reaction time outliers. Psychol Bull. 1993;114:510–532. doi: 10.1037/0033-2909.114.3.510. [DOI] [PubMed] [Google Scholar]
  59. Rice ME, Cragg SJ. Dopamine spillover after quantal release: rethinking dopamine transmission in the nigrostriatal pathway. Brain Res Rev. 2008;58:303–313. doi: 10.1016/j.brainresrev.2008.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Schultz W. Multiple dopamine functions at different time courses. Annu Rev Neurosci. 2007;30:259–288. doi: 10.1146/annurev.neuro.28.061604.135722. [DOI] [PubMed] [Google Scholar]
  61. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage. 2002;15:273–289. doi: 10.1006/nimg.2001.0978. [DOI] [PubMed] [Google Scholar]
  62. Veltman DJ, Rombouts SA, Dolan RJ. Maintenance versus manipulation in verbal working memory revisited: an fMRI study. Neuroimage. 2003;18:247–256. doi: 10.1016/S1053-8119(02)00049-6. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES