A unidirectional but not uniform striatal landscape of dopamine signaling for motivational stimuli

Significance

Although it is undisputed that striatal dopamine plays a prominent role in motivated behavior and learning, the precise information conveyed by dopamine signals as such is under active debate. For a long time, the idea dominated that dopamine encodes a reward prediction error and that this signal is broadcast uniformly throughout the brain. However, here, we capture dopamine dynamics across many striatal regions and demonstrate that dopamine release is, regionally, extremely heterogeneous and that a reward prediction error–like signal is predominantly found in the relatively small limbic domain of the striatum. Another striking organizing principle is that stimulus valence directs dopamine concentration homogeneously across all regions (i.e., appetitive stimuli increase dopamine and aversive stimuli decrease dopamine).

Abstract

Dopamine signals in the striatum are critical for motivated behavior. However, their regional specificity and precise information content are actively debated. Dopaminergic projections to the striatum are topographically organized. Thus, we quantified dopamine release in response to motivational stimuli and associated predictive cues in six principal striatal regions of unrestrained, behaving rats. Absolute signal size and its modulation by stimulus value and by subjective state of the animal were interregionally heterogeneous on a medial to lateral gradient. In contrast, dopamine-concentration direction of change was homogeneous across all regions: appetitive stimuli increased and aversive stimuli decreased dopamine concentration. Although cues predictive of such motivational stimuli acquired the same influence over dopamine homogeneously across all regions, dopamine-mediated prediction-error signals were restricted to the ventromedial, limbic striatum. Together, our findings demonstrate a nuanced striatal landscape of unidirectional but not uniform dopamine signals, topographically encoding distinct aspects of motivational stimuli and their prediction.

Dopamine neurotransmission is pivotal for neuronal processing of, and behavioral responding to, appetitive and aversive stimuli (1–3). The largest releasable pool of dopamine is found in the striatum, a large brain nucleus that is the main input structure of the basal ganglia and the primary projection target of midbrain dopamine neurons (4, 5). The striatum crucially mediates dopamine’s role in motivated behavior (6, 7) and, vice versa, the function of the striatum is critically dependent on dopamine innervation (8, 9). Delineated by topographically organized afferents from cortex, thalamus, and the dopaminergic midbrain (substantia nigra pars compacta and ventral tegmental area), the striatum can be divided into smaller, functionally distinct domains: the ventromedial striatum (VMS; which consists of the nucleus accumbens core [NAC] and nucleus accumbens shell [NAS]), the dorsomedial striatum (DMS), the dorsolateral striatum (DLS), the ventrolateral striatum (VLS), and the tail of striatum (TS). Inputs from cortex, thalamus, and midbrain distribute across the entire striatum, whereas afferents from amygdala and hippocampus are more regionally restricted (10–15) (Fig. 1A). The striatal regions researched most over the past decades are the NAC and NAS, followed by the DMS and DLS; both of these regional pairs are assumed to support dichotomous or even antagonistic functions (16, 17). Other regions, such as the VLS and the TS, have moved into focus more recently (18–20).

Fig. 1.

Targeting dopamine release in six topographically defined domains of the striatum. (A) Schematic depicting the rodent striatum and its main topographic afferents from the cortex on a series of coronal sections. Dopaminergic inputs to the striatum are indicated in color, delineating the six target regions of the striatum in which dopamine signals were recorded. (B) Representative electrolytic lesions circled on Cresyl violet–stained coronal brain sections for histologic verification of electrode-tip placement. From right to left: NAS (purple circles), NAC (blue circles), DLS (red circles), VLS (orange circles), DMS (green circles), and TS (pink circles), and corresponding electrode tracks (arrows). (C) Summary schematic of all dopamine recording sites in NAS (purple circles), NAC (blue circles), DLS (red circles), VLS (orange circles), DMS (green circles), and TS (pink circles). Coronal sections range from +2.04 mm to 0.0 mm, and −1.80 to −2.40 mm anterior to bregma (88).

Although it is undisputed that striatal dopamine plays a prominent role in motivated behavior and reward learning, the precise information conveyed by dopamine signals is under active debate (21). For example, dopamine-neuron activity has been shown to relate to movement (22–29), incentive salience (6), reward value (21, 30), reward prediction, and a so-called temporal-difference reward-prediction error (RPE) (31–36). An important question is how the dopamine system integrates the aforementioned (and other) functions in its output, or whether any of these functions dominate dopamine-release dynamics. Relatedly, given the existence of the striatal regions we just described, another prominent question is whether dopamine signals broadcasted to these striatal subregions are uniform or diverse; in other words, does each region receive the same dopamine signals or are there region-specific differences? For instance, for a long time, the idea dominated that the RPE encoded in midbrain dopamine-neuron firing [signaling the difference between expected and obtained reward (31–35)] is broadcast uniformly throughout the striatum (29, 37–39). However, since more recent findings demonstrate that dopamine release is regionally heterogeneous (19, 22, 40–50), one emerging view is that different striatal regions receive distinct dopamine signals, and that discrepant findings and the varying functions assigned to dopamine may be explained by this variance of signal location within the striatum (45). The answers to these two questions would help explain the breadth and regional specificity of neural computations performed in the striatum and basal-ganglia function as a whole.

An additional question is how the dopamine system is implicated in the processing of and responding to aversive stimuli. Fewer studies have been conducted on this topic, and the outcomes are less consistent than for appetitive stimuli. For example, whether dopamine activity increases or decreases in response to aversive stimuli is still debated and may depend on several factors, such as the striatal subregion sampled, the type of stimulus, and the subject’s ability to escape or avoid it (19, 20, 46, 50–59). Reports that the dopamine system also encodes an aversive prediction error exist, but results vary widely, ranging from no APE encoding (60), to partial encoding (59), to full encoding (3).

Thus, we set out to systematically explore which information is contained in dopamine signals related to prediction error–based reinforcement learning (i.e., stimulus valence [reward and aversion] and value [magnitude], motivational state, and outcome prediction), and how this maps onto the striatum topographically. Most previous studies have sampled from a single mesostriatal connected network and, thus, comparisons across literature are often challenged by inconsistent experimental conditions (e.g., species, stimulus identity, study design, behavioral paradigm, anatomic targeting, dopamine-sensing method). To warrant consistent conditions throughout, we characterized dopamine dynamics in all six of the aforementioned striatal domains (i.e., VMS consisting of NAS and NAC, DMS, DLS, VLS, and TS) of unrestricted, behaving rats using chronically implanted microelectrodes for fast-scan cyclic voltammetry (FSCV) with subsecond detection of dopamine. Using FSCV to monitor terminal release of dopamine provides several advantages over inferring dopamine release based on dopamine-neuron activity for this type of topographical study: First, dopamine cell bodies in the midbrain are tightly packed and their identity and projection targets require additional verification. Second, some midbrain dopamine neurons co-release other neurotransmitters, rendering the interpretation of effects of dopamine cell-body activity more difficult. Third, activity at the level of dopamine-neuron cell bodies, and even on the axonal level, does not necessarily translate to dopamine release in their projection targets (61), as axonal-terminal release undergoes local modulation in the striatum (62). Fourth, the effects of interindividual differences can be minimized by monitoring multiple regions simultaneously in the same animal. Fifth, FSCV electrodes are very small and, thus, enable precise anatomic targeting combined with minimal disruption of brain tissue (63).

Since the sampled striatal regions have been assigned different behavioral functions, we hypothesized that local dopamine signals must exhibit regional differences. Our findings provide substantiation for similarities as well as differences in striatal dopamine signaling across the VMS, DMS, DLS, VLS, and TS, where the anatomic organization of striatal dopaminergic innervation determines a variety of dopamine-signal properties, including absolute size, value and subjective-state coding, valence prediction and valence-prediction error, while the animals’ movement was not encoded in any region.

Results

We characterized the regional release of dopamine throughout rat striatum in response to appetitive and aversive stimuli and their predictors, using chronically implanted FSCV microelectrodes to record dopamine concentration with subsecond resolution in NAS, NAC, DMS, DLS, VLS, and TS. FSCV-electrode placement was verified postmortem via electrolytic lesion (Fig. 1 B and C). As a control for recording quality, delivery of an unpredicted food pellet prior to the start of the behavioral experiment provoked reliable and stable increases of extracellular dopamine concentration in all sampled striatal regions (SI Appendix, Fig. S1A) and, conversely, unpredicted exposure to aversive white noise (WN) reliably decreased extracellular concentrations of dopamine in all regions (SI Appendix, Fig. S1B). We explored dissimilarities in dopamine signaling between striatal regions utilizing several behavioral paradigms designed to identify how motivational stimuli and their predictors are reflected across them.

Dopamine Encodes Subjective Reward Value Region Specifically: Dose–Response Experiment.

Using a probabilistic reward dose–response paradigm (Fig. 2A) in which food pellet rewards of varying magnitude (one, three, or nine pellets) were delivered at random time intervals (unpredicted) to freely moving rats (n = 39), we characterized real-time dopamine responses to determine the precise reward-related information conveyed by striatal dopamine signals and their regional specificity. Absolute changes in extracellular dopamine concentration between regions (Fig. 2B) as well as relative changes within regions (Fig. 2C) varied substantially: Reward itself is reflected homogeneously throughout striatum, as food-reward delivery significantly increased dopamine release (compared with respective pre-reward baseline dopamine concentrations) in all striatal regions (NAS: n = 10, Z = −2.803, P = 0.005; NAC: n = 7, Z = −2.366, P = 0.018; DMS: n = 14, Z = −3.296, P < 0.001; DLS: n = 10, Z = −2.803, P = 0.005; VLS: n = 10, Z = −2.803, P = 0.005; TS: n = 9, Z = −2.666, P = 0.008; Fig. 2D). Reward magnitude, however, was reflected in NAS, NAC, DMS, and DLS, but not in VLS and TS (main effect of reward magnitude: NAS, F(1.083, 9.743) = 12.85, P = 0.005; NAC, F(1.414, 8.485) = 18.98, P = 0.001; DMS, F(1.304, 16.95) = 42.55, P < 0.001; DLS, F(1.933, 17.39) = 3.825, P = 0.043; VLS, F(1.342, 12.08) = 0.6611, P = 0.476; TS, F(1.096, 8.769) = 0.931, P = 0.370; Fig. 2D); the subjective values of one, three, and nine pellets were significantly discernible in NAS, NAC, and DMS, but not in DLS, VMS, and TS (post hoc analysis: NAS: 1 vs. 3, P = 0.035; 1 vs. 9, P = 0.006, 3 vs. 9: P = 0.027. NAC: 1 vs. 3, P = 0.034; 1 vs. 9, P = 0.008; 3 vs. 9, P = 0.015. DMS: 1 vs. 3, P < 0.001; 1 vs. 9, P < 0.001; 3 vs. 9, P < 0.001. DLS: 1 vs. 3, P = 0.108; 1 vs. 9, P = 0.108; 3 vs. 9, P = 0.754. VLS: 1 vs. 3, P = 0.842; 1 vs. 9, P = 0.842; 3 vs. 9, P = 0.265. TS: 1 vs. 3, P = 0.589; 1 vs. 9, P = 0.589; 3 vs. 9, P = 0.589; Fig. 2D). The DMS is frequently subdivided into the anterior DMS (aDMS) and posterior (pDMS) subdivisions (SI Appendix, Fig. S2A) that are thought to serve differing behavioral functions, but dopamine-encoding of reward value in aDMS and pDMS did not differ significantly in our paradigm (one pellet: U(n_aDMS = 6; n_pDMS = 8) = 10, Z = −1.807, P = 0.243; three pellets: U(n_aDMS = 6; n_pDMS = 8) = 13, Z = −1.420, P = 0.181; nine pellets: U(n_aDMS = 6; n_pDMS = 8) = 20, Z = −0.516, P = 0.662; SI Appendix, Fig. S2B). Together, these data show that regional dopamine signals encode reward uniformly throughout the striatum, but that the overall size of dopamine signals and the encoding of reward magnitude are region specific.

Fig. 2.

Varying reward magnitudes elicit distinct regional dopamine (DA)-signal profiles. (A) Food-pellet rewards of different magnitudes were delivered into the food magazine (Top) in semirandom order on a variable ITI of 30 s. Rats received a reward of either nine food pellets (15% of trials), three food pellets (15% of trials), or one food pellet (70% of trials) (Bottom). (B) Changes in extracellular dopamine concentration in NAS, NAC, DMS, DLS, VLS, and TS in response to the unpredicted delivery of one, three, or nine food pellets (traces are aligned to reward delivery). (C) Data from B displayed as a percentage of maximum (average) dopamine release. (D) Dopamine release in response to reward was increased in all striatal regions when compared with baseline (B). Dopamine release in NAS, NAC, DMS, and DLS scaled with reward magnitude, but not in VLS and TS. Bar graphs depict (positive) peak concentrations of dopamine averaged across rats for baseline (−2 to 0 s) and reward (0 to 10 s) epochs. +, a main effect of reward magnitude, but without post hoc difference. (B and D) Data are reported as mean + SEM. Comparison to baseline (B): #P < 0.05, ##P < 0.01, ###P < 0.001; Comparison 1 vs. 3 vs. 9: *P < 0.05, **P < 0.01, ***P < 0.001.

Dopamine Reflects Subjective State Region Specifically: Satiety Experiment.

How dopamine signaling is affected by subjective state is debated (64). Thus, we tested whether changes in subjective state (e.g., hunger) alter dopamine-mediated valuation of reward immediately or only after additional experience with the reward. To this end, we evaluated the effects of satiety and hunger as states on (unpredicted) food-induced release of regional dopamine, comparing within-subject dopamine responses in rats between food-restricted and sated states (where rats were given access to a large amount of food pellets immediately prior to the recording session; Fig. 3A). As in the previous experiment (Fig. 2), absolute changes between regions (Fig. 3B) as well as relative changes within regions (Fig. 3C) in extracellular dopamine concentration varied substantially. Food-reward delivery significantly increased dopamine release in all striatal regions compared with respective pre-reward baseline dopamine concentrations (NAS: n = 11, Z = −2.934, P = 0.003; NAC: n = 17, Z = −3.621, P < 0.001; DMS: n = 14, Z = −3.296, P < 0.001; DLS: n = 18, Z = −3.724, P < 0.001; VLS: n = 9, Z = −2.666, P = 0.008; TS: n = 10, Z = −2.395, P = 0.017; Fig. 3D), corroborating our results described in Fig. 2D. Satiety state, however, is only reflected in medial regions of the striatum: dopamine responses in NAS, NAC, and DMS differed significantly between hungry and sated sessions, whereas those in in DLS, VLS, and TS did not (NAS: n = 11, Z = −2.845, P = 0.004; NAC: n = 17, Z = −3.479, P < 0.001; DMS: n = 14, Z = −2.040, P = 0.041; DLS: n = 18, Z = −1.198, P = 0.231; VLS: n = 9, Z = −0.296, P = 0.767; TS: n = 10, Z = −0.663, P = 0.508; Fig. 3D). Again, no significant differences were found between DMS subdivisions (post hoc analysis: hungry: U(n_aDMS = 6; n_pDMS = 8) = 29, Z = 0.645, P = 1.000; sated: U(n_aDMS = 6; n_pDMS = 8) = 26, Z = 0.258, P = 0.852; SI Appendix, Fig. S2C). Rats were given two single-food-pellet trials in each session to compare dopamine responses between hungry and sated states. Dopamine responses to the first and the second pellet did not differ, except for a minor difference in the NAC (sated state), but hungry and sated responses vastly differed from one another (SI Appendix, Fig. S3). In summary, medial but not lateral regions of the striatum exhibit immediate changes in subjective state.

Fig. 3.

Satiety state–dependent regional differences in reward-induced dopamine (DA) release. (A) On two days, rats received unpredicted food pellets separated by a variable ITI. On 1 d, rats were hungry (H; food restricted to 85% of their free-feeding weight). On the other day, rats were sated (S; prefed via 1-h ad libitum access to food pellets immediately prior to testing). (B) Changes in dopamine concentration in NAS, NAC, DMS, DLS, VLS, and TS in response to an unpredicted food pellet in hungry and sated rats (traces are aligned to food-pellet delivery). (C) Data from B are displayed as a percentage of maximum (average) dopamine release. (D) Dopamine release in response to reward was increased in all striatal regions when compared with baseline (B). Dopamine release in NAS, NAC, and DMS was lower in sated than in hungry rats, but not in DLS, VLS, and TS. Bar graphs depict (positive) peak concentrations of dopamine averaged across rats for baseline (−2 to 0 s) and reward (0 to 5 s) epochs. (B and D) Data are reported as mean + SEM. Comparison to baseline (B): #P < 0.05, ##P < 0.01, ###P < 0.001; Comparison H vs S: *P < 0.05, **P < 0.01, ***P < 0.001.

Appetitive and Aversive Pavlovian-Cue Conditioning Produces Region-Specific Dopamine Signatures.

Although dopamine is implicated in signaling aversion (3, 19, 20, 46, 51–59), the information content and regional specificity of dopamine signals as they relate to aversive stimuli are even less well understood than those of rewards. Thus, we explored regionally specific encoding of appetitive and aversive outcomes (unconditioned stimuli [US]: respectively food pellets and aversive WN), as well as their discriminative cues (conditioned stimuli [CS]: a light and a tone, predicting delivery of, respectively, the appetitive and aversive US), while recording dopamine transients during probabilistic Pavlovian conditioning (Fig. 4 A and B).

Fig. 4.

Dopamine (DA) encoding of appetitive and aversive Pavlovian conditioning is region specific and unrelated to locomotor behavior. (A) Schematic of the Pavlovian task with appetitive and aversive stimuli. After two conditioning sessions on previous days, rats received 30 pairings of cue 1 with food-pellet delivery and 30 pairings of cue 2 with 3-s WN presentation. (B) Trial structure of the probabilistic Pavlovian conditioning task. (C) Locomotion speed during appetitive trials (Top) was decreased following food-pellet delivery or its prediction (cue 1). During aversive trials (Bottom), speed was increased during the presentation WN and its prediction (cue 2). The bar graphs depict average speed for baseline (B −5 to 0 s), 5-s cue (C; 0 to 5 s), or the same period for no-cue conditions, food pellet (5 to 10 s), and 3-s WN (5 to 8 s) epochs. Traces are aligned to food-pellet delivery or WN presentation, or to cue offset when outcome was omitted. Data are reported as mean ± SEM. (D) Regional dopamine release in response to cue-1 presentation (predicting food-pellet delivery) was increased compared with baseline. The bar graph insets depict averaged dopamine concentrations for baseline (−5 to 0 s) and cue (0 to 5 s) epochs. Different proportions of temporal shift from US (pellet) to CS (cue 1) across regions are indicated by black arrows. (E) Dopamine release in NAS, NAC, DLS, VLS, and TS in response to a predicted food pellet was diminished compared with an unpredicted food pellet, and no differences were observed in DMS. The bar graph insets depict averaged dopamine concentrations for unpredicted food-pellet (U; 5 to 20 s) and predicted food-pellet (P; 5 to 20 s) epochs. (F) Dopamine release in NAS and NAC in response to an omitted food pellet was decreased compared with a predicted food pellet, and no differences were observed in DMS, DLS, VLS, and TS. The bar graph insets depict averaged dopamine concentrations for predicted food-pellet (5 to 20 s) and omitted food-pellet (O; 5 to 20 s) epochs. (G) Dopamine release in NAS, NAC, DMS, DLS, and VLS in response to cue-2 presentation (predicting WN) was decreased compared with baseline, but no difference was observed in TS, although WN, itself, did decrease TS dopamine (extra bar graph inset). Bar graph insets depict averaged dopamine concentrations for baseline (−5 to 0 s), cue (0 to 5 s), and WN (5 to 12 s) epochs. Different proportions of temporal shift from US (WN) to CS (cue 2) across regions are indicated by black arrows. (H) Dopamine release in DMS in response to predicted WN was diminished compared with unpredicted WN, and no differences were observed in NAS, NAC, DLS, VLS, and TS. The bar graph insets depict averaged dopamine concentrations for unpredicted WN (5 to 20 s) and predicted WN (5 to 20 s) epochs. (I) Dopamine release in NAS, NAC, and DMS in response to omitted WN was less decreased compared with predicted WN, and no differences were observed in DLS, VLS, and TS. The bar graph insets depict averaged dopamine concentrations for predicted WN (5 to 20 s) and omitted WN (5 to 20 s) epochs. (J) Locomotion speed (black) superimposed on dopamine traces during appetitive trials (Left) and table of correlations between locomotion speed and dopamine concentration during cue-1 presentation and food-pellet delivery during these trials (Right). No correlations were significant. (K) Locomotion speed (black) superimposed on dopamine traces during aversive trials (Left) and table of correlations between locomotion speed and dopamine concentration during cue-2 presentation and WN presentation during these trials (Right). No correlations were significant. (D–I) Data are reported as mean + SEM. (J and K) Speed data are reported as mean + SEM. *P < 0.05, **P < 0.01, ***P < 0.001.

Appetitive and aversive US provoke opposing behavioral responses in rat.

Rat behavioral response (derived from DeepLabCut video analysis) to presentation of the appetitive (food pellets) and aversive (WN) US was dichotomous: In appetitive trials, we observed a significant, transient decrease in locomotion speed, whereas in aversive trials locomotion speed was transiently increased [unpredicted pellet: χ² (2, 48) = 32.042, P < 0.001; unpredicted WN: χ² (2, 48) = 52.792, P < 0.001; Fig. 4 C, Left Column]. Pavlovian conditioning was successfully induced, as this effect extended to the CS associated with the US [predicted pellet: χ² (2, 48) = 74.667, P < 0.001; omitted pellet: χ² (2, 48) = 66.500, P < 0.001; predicted WN: χ² (2, 48) = 90.125, P < 0.001; omitted WN: χ² (2, 48) = 24.125, P < 0.001; Fig. 4 C, Middle and Right Columns].

In appetitive conditioning trials, animal speed always decreased during US delivery (post hoc analysis; Fig. 4 C, Top Left: baseline vs. reward, P < 0.001; Fig. 4 C, Top Middle: baseline vs. reward, P < 0.001), and animal speed always decreased following (but not during) appetitive CS presentation regardless of whether US was delivered (post hoc analysis; Fig. 4 C, Top Middle: baseline vs. cue, P = 0.205; baseline vs. reward, P < 0.001) or omitted (post hoc analysis; Fig. 4 C, Top Right: baseline vs. cue, P = 1.000; baseline vs. no reward, P < 0.001), suggesting that the cue prompts animals to slow down and inspect the food magazine for delivered pellets, which are consumed in case of delivery. In contrast, during aversive conditioning trials, US presentation increased animal locomotion speed (post hoc analysis; Fig. 4 C, Bottom Left: baseline vs. WN, P < 0.001; Fig. 4 C, Bottom Middle: baseline vs. WN, P < 0.001), and animal speed always increased both during and following CS presentation regardless of whether US was presented (post hoc analysis; Fig. 4 C, Bottom Middle: baseline vs. cue, P < 0.001; baseline vs. WN, P < 0.001) or omitted (post hoc analysis; Fig. 4 C, Bottom Right: baseline vs. cue, P < 0.001; baseline vs. no WN, P = 0.012), corroborating our previously reported observation that WN induces mild behavioral activation in rats, which may reflect an increased motivation to escape the operant box (59).

Dopamine transients reflect CS prediction of appetitive and aversive US (predicted US).

We mapped out regional differences in outcome-prediction encoding across the striatum by quantifying regional dopamine release in response to cues predicting appetitive and aversive outcomes. Dopamine release significantly increased in all striatal regions in response to the appetitive CS (cue 1: NAS: n = 10, Z = −2.803, P = 0.003; NAC: n = 13, Z = −3.180, P < 0.001; DMS: n = 14, Z = −3.296, P < 0.001; DLS: n = 19, Z = −3.219, P < 0.001; VLS: n = 10, Z = −2.803, P = 0.003; TS: n = 10, Z = −1.988, P = 0.023; Fig. 4D) and significantly decreased in response to the aversive CS (cue 2) in all regions except TS (NAS: n = 10, Z = −2.803, P = 0.003; NAC: n = 13, Z = −3.180, P < 0.001; DMS: n = 14, Z = −3.296, P < 0.001; DLS: n = 21, Z = −3.980, P < 0.001; VLS: n = 12, Z = −2.275, P = 0.011; TS: n = 11, Z = −0.533, P = 0.297; Fig. 4G); TS dopamine did, however, significantly decrease during WN presentation (TS: n = 11, Z = −2.667, P = 0.004; Fig. 4G). Thus, NAS, NAC, DMS, DLS, and VLS dopamine release predict both appetitive and aversive stimuli, whereas in TS, it predicts only appetitive stimuli. By definition, if dopamine acts as an RPE signal, its moment of release should gradually shift from the US to the CS; an incomplete shift implies dopamine signals reward prediction but not its error. We quantified this by calculating the share of overall dopamine increase (CS + US) attributable to the appetitive CS throughout different striatal regions. The CS acquired greatest weight in NAC, VLS, and TS (proportion of dopamine released during cue presentation: NAS, 42%; NAC, 80%; DMS, 59%; DLS, 47%; VLS, 80%; and TS, 89%; Fig. 4 D, see arrows). In contrast, the share of aversive CS was significantly smaller compared with the appetitive CS (n = 6, Z = −2.201, P = 0.028) and more homogenous between regions, acquiring its smallest weight in TS (proportion of dopamine released during cue presentation: NAS, 19%; NAC, 21%; DMS, 18%; DLS, 15%; VLS, 19%; and TS, 6%; Fig. 4 G, arrows). Taken together, the appetitive CS evokes reward-prediction dopamine signals throughout the striatum, but only in a subset of regions is this shift from US to CS (a requirement of RPE signals) substantial. The aversive CS receives only a small percentage of the total dopamine signal, indicating the absence of APE-like signaling throughout the striatum.

Appetitive and aversive CS evoke heterogenous dopamine responses to US prediction (predicted vs. unpredicted US).

By definition of a prediction-error signal, the size of the dopamine signal in response to a US should decrease when predicted by a CS. We evaluated to what extent the CS shaped dopamine responses throughout the striatum by comparing predicted outcomes (CS + US) with unpredicted outcomes (US only, in 8% of trials: 4% appetitive, 4% aversive; Fig. 4B). The appetitive CS reduced US-triggered dopamine-release in NAS, NAC, DLS, VLS, and TS, but not DMS (NAS: n = 10, Z = −2.293, P = 0.011; NAC: n = 13, Z = −3.110, P = 0.002; DMS: n = 14, Z = −0.534, P = 0.297; DLS: n = 19, Z = −2.133, P = 0.033; VLS: n = 10, Z = −2.803, P = 0.005; TS: n = 10, Z = −2.803, P = 0.005; Fig. 4E). In contrast, the aversive CS did not affect dopamine release, except in DMS, where it increased the size of the dopamine response to the US (i.e., the opposite effect of a prediction-error response; NAS: n = 10, Z = −0.866, P = 0.386; NAC: n = 13, Z = −0.314, P = 0.753; DMS: n = 14, Z = −2.354, P = 0.019; DLS: n = 21, Z = −2.033, P = 0.084; VLS: n = 12, Z = 0.000, P = 1.000; TS: n = 11, Z = −1.423, P = 0.155; Fig. 4H). Thus, prediction of appetitive and aversive stimuli impacts dopamine release heterogeneously throughout striatum, where DMS dopamine is the only US signal both not impacted by prediction of reward and, vice versa, the only US signal that is (mildly) impacted by prediction during aversive trials.

Appetitive and aversive CS evoke heterogenous dopamine responses to US prediction error across striatal regions (predicted vs. omitted US).

By definition, prediction-error signals are learning signals generated by a discrepancy between actual and predicted aversive outcomes and come in two varieties: positive and negative. Positive and negative refer to the direction of the discrepancy, not the value of the outcome (65). There is no consensus on the manner in which prediction errors are represented by dopamine transients. We compared dopamine responses with predicted outcomes (CS + US) with those that were omitted (CS only, in 8% of trials: 4% appetitive, 4% aversive; Fig. 4B) throughout the striatum. Dopamine release decreased significantly when a predicted food pellet was omitted (i.e., a worse-than-expected outcome, in NAS and NAC, but not in DMS, DLS, VLS, and TS; NAS: n = 10, Z = 2.497, P = 0.013; NAC: n = 13, Z = 2.551, P = 0.006; DMS: n = 14, Z = −1.601, P = 0.109; DLS: n = 19, Z = 1.006, P = 0.157; VLS: n = 10, Z = 0.357, P = 0.361; TS: n = 10, Z = −0.051, P = 0.480; Fig. 4F). In contrast, dopamine release increased significantly when predicted WN was omitted (i.e., a better-than-expected outcome, in NAS, NAC, and DMS, but not in DLS, VMS, and TS; NAS: n = 10, Z = −2.701, P = 0.007; NAC: n = 13, Z = −2.062, P = 0.039; DMS: n = 14, Z = −2.417, P = 0.016; DLS: n = 21, Z = 0.226, P = 0.411; VLS: n = 12, Z = −1.334, P = 0.182; TS: n = 11, Z = −1.423, P = 0.155; Fig. 4I). These data demonstrate that negative prediction-error coding of dopamine is region specific rather than a striatum-wide uniform signal. Only in NAC and NAS did dopamine release resemble a negative prediction-error signal for both appetitive and aversive stimuli, whereas dopamine in VLS, DLS, and TS did not show any negative prediction-error coding, neither for appetitive nor aversive stimuli (DMS showed a negative APE response). This conclusion was further supported by a post hoc analysis for VLS and TS, intended to increase sensitivity for the fast dopamine fluctuations observed in Fig. 4 F and I by comparing dopamine-response between predicted and omitted stimuli in the 15-s postcue epoch using a second-by-second statistical analysis (SI Appendix, Fig. S4).

A second post hoc analysis revealed no significant differences between aDMS and pDMS dopamine responses [appetitive CS: U(n_aDMS = 6; n_pDMS = 8) = 12, Z = −1.549, P = 0.142; unpredicted pellet-delivery: U(n_aDMS = 6; n_pDMS = 8) = 15, Z = −1.162, P = 0.282; omitted pellet-delivery: U(n_aDMS = 6; n_pDMS = 8) = 11, Z = −1.678, P = 0.108; aversive CS: U(n_aDMS = 6; n_pDMS = 8) = 34, Z = 1.291, P = 0.228; unpredicted WN presentation: U(n_aDMS = 6; n_pDMS = 8) = 20, Z = −0.516, P = 0.662; omitted WN presentation: U(n_aDMS = 6; n_pDMS = 8) = 22, Z = −0.258, P = 0.852; SI Appendix, Fig. S2 D and E], indicating that the reported DMS results apply to both aDMS and pDMS.

Striatal input from the dopaminergic midbrain is topographically organized along the anterior–posterior (A-P) axis and thus, variability in anatomic placement of our electrodes along the A-P axis might possibly introduce sufficient variance in our dopamine recordings to conceal potential prediction errors, such as (most notably) in VLS (Fig. 4I) and TS (Fig. 4 H and I). Thus, we performed a third post hoc analysis to investigate whether variance in these dopamine responses could be explained by electrode placement (SI Appendix, Fig. S5). Exclusion of the most caudal electrodes improved VLS-signal variance, but not for the TS. This indicates that in the TS, dopamine-response variance is not organized along the A-P axis, whereas in the VLS, there is an A-P gradient in representation of aversive stimuli. In either case, although negative data are more difficult to interpret, the variance of VLS and TS data is, in itself, valuable, implying that these regions are less involved in prediction-error signaling than is the VMS.

Locomotion speed does not influence dopamine transients reflecting appetitive and aversive stimuli.

As dopamine-neuron activity often relates to movement, we controlled for the influence of locomotion on CS- and US-triggered dopamine release by correlating speed of locomotion with dopamine concentrations during epochs of cue presentation (0 to 5 s) and outcome delivery (5 to 10 s) during appetitive (SI Appendix, Fig. S6A) and aversive (SI Appendix, Fig. S6B) trials. We found no correlation between dopamine release and animal locomotion speed in any striatal region regardless of trial type (appetitive or aversive) during cue presentation or outcome delivery (Fig. 4 J and K); thus, we may rule out locomotion as a substantial linear contributor to the magnitude and direction of dopamine transients in this study.

Discussion

To reveal behaviorally relevant organizational principles of dopamine neurotransmission in its primary projection target, the striatum, we conducted an investigation that spans its different functional domains. Our findings demonstrate several general organizational principles, some of which are expressed homogeneously throughout the striatum, and others subject to regional heterogeneity (summarized in Tables 1 and 2). The most striking principle is that stimulus valence directs dopamine concentration homogeneously across all regions (i.e., reward increases dopamine and aversive stimuli decrease dopamine). CS, predictive of these US, acquired this effect on dopamine release homogeneously throughout the striatum. Furthermore, dopamine signals are homogeneously unaffected by the animals’ movement; in other words, striatal dopamine-population signaling (volume transmission) is not linked to general locomotion, in contrast to individual dopamine neurons that are known to participate in locomotor function or movement (23–25, 66).

Table 1.

Summary of regional functional features of striatal dopamine: Reward

	Reward	Reward magnitude	DA signal size	Subjective state (satiety)	RP during CS	Impact of CS prediction on US (reward) signal size	Signal transfer from US to CS, %	Positive RPE (bigger reward)	Negative RPE (reward omission)	Encoding summary
NAS	↑	X	••••	X	X	↓	42	X	X	Partial RPE
NAC	↑	X	••••	X	X	↓	80	X	X	Full RPE
DMS	↑	X	••	X	X		59	X		Quantitative RP + reward
DLS	↑	(x)	•		X	↓	47			RP + reward
VLS	↑		••		X	↓	80			RP
TS	↑		•		X	↓	89			RP

Dopamine signals across the six sampled striatal domains encode information related to rewarding stimuli in a multifaceted, region-specific manner, where signals differ qualitatively and quantitatively, yet adhere to unifying organizing principles. Dopamine dynamics in all striatal domains responded to US presentation (food pellet) in a consistent manner, where dopamine increased to the reward, in some regions even relative to reward magnitude and subjective internal state. Dopamine responded to the CS (predictive of the US; reward prediction) in almost all regions, but the impact of and signal transfer to CS (from US) differed substantially between regions, and only some regions exhibited dopamine dynamics that resembled an RPE. DA, dopamine; RP, reward prediction; RPE, reward-prediction error.

Table 2.

Summary of regional functional features of striatal dopamine: Aversive stimulation

	Aversive stimulus	DA signal size	AP during CS	Impact of CS prediction on US (aversive) signal size	Signal transfer from US to CS, %	Negative APE (WN omission)	Encoding summary
NAS	↓	••	X		19	X	AP + aversive
NAC	↓	••	X		21	X	AP + aversive
DMS	↓	••	X	↑	18	X	AP + aversive
DLS	↓	•	X		15		AP + aversive
VLS	↓	•	X		19		AP + aversive
TS	↓	•			6		Aversive

Dopamine signals across the six sampled striatal domains encode information related to aversive stimuli in a relatively consistent manner, where dopamine concentration decreased in response to US presentation (aversive WN). The CS (predictive of the US; aversive-stimulus prediction) decreased dopamine in almost all regions, but with little impact on US-induced dopamine and only limited signal transfer from US to CS. Only some regions responded to WN omission. DA, dopamine; AP, aversive-stimulus prediction; APE, aversive-stimulus prediction error.

Heterogeneous features are topographically expressed in a gradient from medial to lateral striatum, with more pronounced encoding of subjective state and reward magnitude in medial regions. VMS exhibits the greatest dopamine release and scales most linearly to reward magnitude (interval scale), DMS displays qualitatively scaled dopamine (ordinal scale), and the three lateral regions show dopamine responses to reward but are insensitive to reward magnitude (nominal scale). The absolute size of dopamine signals was significantly heterogeneous across the striatum. More specifically, absolute signal magnitude was consistently greatest in VMS, where dopamine concentration increased 2 to 4 times more than in other regions (in response to reward). This is remarkable because the dopamine tissue content does not differ substantially between striatal regions; if at all, slightly less content is found in the VMS (67, 68). Smaller dopamine signals may be due to recruitment of fewer dopamine-releasing axons. Consistently, reward-related signaling has been found more frequently in dopaminergic axons projecting to the ventral striatum than in axons projecting to the dorsal striatum (24). However, our recordings also reveal a marked difference in signal size between VMS and VLS, indicating that reward-related signal size is also stratified on a medial to lateral gradient (in addition to a ventral-to-dorsal axis). Taken together, organizational principles of dopamine signaling are governed by binary valence-encoding and independence from locomotion throughout the striatum, where signal size underlies a “VMS-centric” profile.

We contrasted our investigation of reward processing by the dopamine system with aversive-stimulus processing. We used WN, a well-controllable, aversive, auditory stimulus (69, 70) that provokes mild behavioral activation and depresses NAC dopamine release, as validated in our previous work (59). Consistent with previous studies aimed at the NAC (46, 51, 57, 58), we found that striatal dopamine release encodes rewarding and aversive events with opposing directionality across striatal regions. The degree of aversive-stimulus–induced dopamine decrease manifested similarly to rewards (medial regions show greater signals [dips] compared with lateral regions), albeit with less pronounced regional disparity in signal magnitude. However, despite similarities, dopamine in all regions encodes negative-valence stimuli qualitatively different from positive-valence ones, as we demonstrated previously for the NAC (59), a finding that is inconsistent with the framework of APE signaling. First, aversive-related changes in extracellular dopamine concentration are substantially smaller compared with appetitive-related ones, potentially due to hard-wired differences between dopamine release (signal induction) and dopamine-signal termination: Positive signals have a bigger dynamic range as the number of neurons firing and their firing frequency can be increased substantially (71) compared with negative signals that are limited to neurons ceasing to fire, tied to relatively slow dopamine reuptake and diffusion (72). Second, there is little relative temporal shift in dopamine signal from the aversive US to its predicting CS, a prominent feature of dopamine signals to appetitive stimuli. Third, consistently, signal magnitude is not decreased by stimulus prediction. Fourth, decreased dopamine concentration in response to an aversive stimulus does not immediately return to baseline after stimulus termination, evoking temporally imprecise stimulus encoding. Fifth, besides aversion-induced dopamine dips, we also observed dips below prestimulus baseline subsequent to a reward-induced increase, as reported previously (6, 73), whereby such dips are bigger (relative to positive signal size) and occur more frequently in dorsal and lateral regions of the striatum (VLS, DMS, DLS, TS). Such bipolar responses may accentuate the termination of a reward signal, signify the setting of a new baseline for ambient dopamine concentration (30), or reflect the ceasing of an appetitive event; as our experimental design does not evaluate these postreward epochs, we can merely speculate about the nature of these dips. Taken together, we report that aversive stimuli induce negative dopamine signals that vary in size regionally and mirror positive-reward dopamine signals but encode qualitatively different information than reward signals and do not encode APEs in any of the striatal regions, inconsistent with findings that support the idea that TS dopamine functions as an APE (19, 20).

Dopamine neuron firing (32), dopamine axon activity (39), and dopamine release in the striatum (74, 75) have been found to reflect RPE signals. To qualify as an RPE signal, several criteria need to be satisfied (75, 76), some of which we investigated. Specifically, we tested whether dopamine release encoded reward, reward value, reward prediction, and positive or negative deviation from expected reward value. A strikingly heterogeneous feature was that, although all sampled regions encoded reward and reward prediction and half the regions encoded reward value, pure and robust encoding of a canonical RPE was exclusive to the VMS. Importantly, this VMS RPE signal was shaped by the subjective state of the animal (i.e., prefeeding-induced, sensory-specific satiety [toward the food reward]) and had an immediate effect on dopamine release in the experimental context (i.e., dopamine signals were affected already during the first pellet delivery after prefeeding), instead of a slower, experience-based adjustment (64, 77, 78). Within the VMS, the NAC qualified more strictly as an RPE, as reported by others (79), because more dopamine release transferred from the US to the CS than in NAS; however, behavioral performance had already plateaued before our dopamine measurement, consistent with previous studies that used comparable training durations (6, 80). Notably, a recent study (78) elegantly demonstrated how task uncertainty can alter the shape of NAC-dopamine RPE signals, including residual dopamine signals in response to the reward US (81, 82). However, due to a relatively short CS of 5 s in our task, that was presented with maximal contingency and contiguity during extensive training, we believe our animals experienced, at most, very limited uncertainty (evidenced by the fact that NAC did not exhibit such residual US dopamine signals). In summary, the dopaminergic RPE signal, previously assumed ubiquitous and global, was most prominent in the NAC, underlining the exceptional functional status of this limbic brain hub, thought to translate motivation to action (83) and, thus, revealing substantial regional differences in striatal dopamine signaling.

The directionality of dopamine concentration changes in NAS, DLS, and TS were consistent with one another and with the other regions in that appetitive and aversive stimuli increased and decreased dopamine, respectively, in contrast to reports demonstrating increased dopamine activity after aversive stimuli in these regions (19, 46, 52). Notably, this discrepancy is probably not due to sampling of functionally different striatal domains, since anatomic targeting for these regions largely overlaps between studies (19, 46, 52). Specifically, TS dopamine was decreased by the aversive stimulus but not its prediction (which was not due to insufficient behavioral conditioning, since other regions did encode the cue association), and NAS and DLS dopamine decreased strongly to both the aversive stimulus and its prediction. Similarly, we found categorical regional homogeneity between NAS and NAC, as well as DMS and DLS, where others have reported opposite dopamine signals (48, 52). Perhaps, these DMS and DLS differences are attributable to biased or specialized RPEs (39), as DMS dopamine displays the only regional US signal not impacted by reward prediction and DLS dopamine is not modulated by subjective state. Overall, some of the discrepancies we have described between studies could be explained by differences in animal species, stimulus identity, study design, or dopamine-measuring technique. Alternatively, discrepant findings may be explained by whether experimental conditioning was instrumental or Pavlovian, since recent evidence indicates differential response profiles for the dopamine system to each (84, 85). Furthermore, measuring dopamine axon-terminal activity with calcium indicators versus dopamine release from terminals may capture physiological processes that are not identical, as dopamine-terminal activity is thought to be modulated locally in the striatum (61, 62). Together, discrepant with recent reports, our data points at striatum-wide unidirectionality of dopamine signals, where appetitive and aversive stimuli induce increased and decreased activity of the local dopamine projections, respectively.

Dopamine signals we measured in DMS, a region strongly associated with goal-directed, outcome-dependent behavior, displayed properties more fitting with a region associated with stimulus-driven, outcome-independent behavior (e.g., DLS): The DMS dopamine signal was unaffected by outcome (i.e., reward omission, predicted reward, and unpredicted reward all evoked DMS signals of equal size). This supports our previous attribution of an unsuspected role of DMS dopamine in habitual behavior (86). Intriguingly, DMS dopamine was sensitive to reward value and subjective state, placing it in a transitional position between VMS and the lateral striatal regions. Thus, these findings update actor-critic reinforcement-learning models, where the critic (influenced by motivational state, e.g., hunger) is the ventral striatum (composed of VMS and VLS), and the actor is the dorsal striatum (87), Based on our findings, we may speculate that VMS dopamine signaling functions as critic but that VLS dopamine does not. Thus, we move the line that divides actor and critic functions into ventral and dorsal striatum, instead into VMS and DMS on one hand, and VMS and VLS on the other.

In summary, we demonstrate that dopamine signals across the striatum homogeneously encode stimulus valence and prediction of motivational stimuli (of both positive and negative valence) and are homogeneously unrelated to locomotion and APE. This is in contrast to heterogeneous encoding of reward magnitude, subjective state, and RPE, all of which are VMS centric, where NAC most accurately reflects a formal temporal-difference RPE signal. Thus, considering the anatomic size of the striatum, only a relatively small limbic portion, tightly associated with motivation, is exposed to a dopaminergic, RPE-like, volume-transmission signal (pooling signals of hundreds of different dopamine neurons), whereas dopamine in the remaining regions predominantly tracks both the presence of appetitive and aversive stimuli and their predictors. Some of these findings are strongly inconsistent with the literature on recordings from individual dopamine neurons (32, 42), which requires further investigation; however, a mismatch between activity of individual dopamine neurons and the striatal-dopamine population signal (which is based on release from striatal terminals) may be due to dopamine-terminal modulation that divorces dopamine release from cell-body activity. Regardless, the dopamine population signal, which is what modulates the activity of postsynaptic neurons in the striatum, acts substantially less as an RPE than individual dopamine neurons do. Together, our findings contribute to unraveling the long-standing question of how regional dopamine in the striatum realizes its many functions.

Materials and Methods

Animals.

Adult male Long Evans rats (300 to 380 g; Janvier Labs) were individually housed and kept on a reversed 12-h light/dark cycle (lights off from 08:00 to 20:00) with controlled temperature and humidity. Rats were food restricted to 85% of their free-feeding body weight, and water was provided ad libitum. Rats were fed regular laboratory chow in their home cage 2 h after the end of the daily behavioral training, to supplement food intake during training. All cohorts of rats were trained consistently at the same time of day, between 08:00 and 17:00, during the dark phase. All animal procedures were conducted in accordance with Dutch and European laws and were approved by the Animal Experimentation Committee of the Royal Netherlands Academy of Arts and Sciences. A total of 58 rats had at least one functional and histologically verified recording electrode and were used for dopamine recordings. Subgroups of this total number of rats were used in the different experiments described later in this section.

Stereotactic Surgery.

Stereotaxic surgery was performed as described previously (43, 44). Rats were anesthetized with 1 to 3% isoflurane and placed in a stereotactic frame, analgesic care was delivered as subcutaneous injection of Metacam (0.2 mg meloxicam/100 g). Body temperature was monitored and maintained with a heating pad. The scalp was shaved and disinfected with 70% alcohol, and an incision (treated with lidocaine, 100 mg/mL) exposed the cranium at the midline of the scalp. Subsequently, holes were drilled for three anchor surgical screws, an Ag/AgcCl reference electrode targeting the forebrain, and two custom-made carbon-fiber microelectrodes (63) unilaterally targeting two of the following regions: NAS [1.2 mm rostral, 1.0 mm lateral, and −7.7 mm ventral from bregma (88)], NAC (1.2 mm rostral, 1.5 mm lateral, and −7.1 mm ventral from bregma), DMS (−0.2 mm rostral, 2.5 mm lateral, and −4.5 mm ventral from bregma), DLS (1.2 mm rostral, 3.6 mm lateral, and −4.5 mm ventral from bregma), VLS (0.36 mm rostral, 3.8 mm lateral, and −6.8 mm ventral from bregma), or TS (−2.1 mm rostral, 4.8 mm lateral, and −5.8 mm ventral from bregma). Electrodes were secured and anchored to the surgical screws with dental acrylic cement. Dopamine release was quantified by 11 NAS electrodes, 17 NAC electrodes, 14 DMS electrodes, 21 DLS electrodes, 12 VLS electrodes, and 11 TS electrodes.

Behavioral Procedures.

All behavioral experiments were conducted in modified modular operant chambers (32 × 30 × 29 cm; Med Associates Inc.) equipped with a food magazine with an integrated cue light (connected to an automated food-pellet dispenser), a house light, a tone generator, a WN generator, and a metal grid floor. Each operant chamber was surveilled with a video camera. The boxes were housed in metal Faraday cages that were insulated with sound-absorbing polyurethane foam and ventilated by a fan.

Experiment 1: Dopamine recording during food-pellet dose response.

Rats (n = 39) were placed in an operant chamber (Fig. 2) and tethered to the FSCV recording setup to record fluctuations in extracellular dopamine concentration in response to different magnitudes of (unpredicted) food-pellet rewards (“purified” pellets; Bio.Serv Inc.). During the first three trials of the session, a single food pellet was delivered per trial. Then, rats received a food reward consisting of either one, three, or nine pellets per trial in a semirandom order. The probability of receiving one pellet was 0.7, and for three or nine pellets, the probability was 0.15. Trials were separated by a variable intertrial interval (ITI) of 30 s (range: 20 to 40 s). Pellets were delivered sequentially, one by one, with a time interval of 0.4 s between deliveries. Thus, it took approximately 8 × 0.4 s = 3.2 s in total for nine pellets to be delivered.

To determine the effect of a single food pellet on dopamine, the first three trials were analyzed. For all regions, we compared the (positive) peak dopamine concentration in the first 10 s after pellet delivery (approximate duration of the observed phasic signal) among trials of one, three, and nine pellets. To test whether dopamine increased in response to a reward, the peak concentration in the first 10 s after pellet delivery was compared with the (positive) peak concentration in the 2 s before pellet delivery (baseline).

Experiment 2: Dopamine recording during food-pellet delivery at different subjective states.

On two separate days, rats (n = 51) were placed in an operant chamber and tethered to the FSCV recording setup (Fig. 3) to determine the effect of satiety (subjective state) on regional dopamine release. On 1 d, rats were tested in a food-restricted state. On the other day, rats were given 1-h access to 20 g of food pellets (approximately 450 pellets; ad libitum) prior to the start of the session (pellets were placed in a custom-made metal cup in the operant chamber). Sessions on both days consisted of two trials; during each, a single unpredicted pellet was delivered. Trials were separated by a variable ITI of 30 s (range: 20 to 40 s).

The positive peak concentration of dopamine over 5 s following food-reward delivery was compared between hungry and sated conditions. To determine a regional reward response, the average across conditions was compared with the positive peak concentration over 2 s before the presentation of the food reward (baseline) that was also averaged across both conditions.

Experiment 3: Dopamine recording during a Pavlovian task with appetitive and aversive stimuli.

Rats (n = 50) were conditioned in two sessions to the delivery of food pellets (appetitive outcome) and the presentation of loud WN [90 dB, an intensity we previously showed to be aversive to rats (59); aversive outcome] (Fig. 4). Sessions consisted of 30 pairings (trials) of 5 s of predictive cue-light illumination followed by a food-pellet delivery (delivered immediately after cue offset) and 30 pairings (trials) of 5 s of predictive cue tone (1.5 kHz, 75 dB) followed by a 3-s WN presentation (presented immediately after cue offset). Trials were presented in a semirandom order and separated by a variable ITI of 30 s (range: 20 to 40 s).

In the third session, rats were placed in the operant chamber and tethered to the FSCV recording equipment. The probability for each trial type (appetitive or aversive) was 50%. During most appetitive trials, food-pellet delivery was preceded by a predictive 5-s CS (cue 1; 42% of trials); in probe trials, the predictive 5-s cue 1 was presented alone (4% of trials) or the food pellet was delivered alone ( 4% of trials). During most aversive trials, 3-s WN was preceded by predictive 5-s CS (cue 2; 42% of trials); in probe trials, the predictive 5-s cue 2 was presented alone (4% of trials) or the 3-s WN was presented alone (4% of trials). Probe trials helped identify whether dopamine release was related to reward outcomes (R), reward prediction (RP), or RPE, as well as aversive outcomes (A), aversive prediction (AP), or APE. All trials were presented in a semirandom order and separated by a variable ITI of 30 s (range: 20 to 40 s).

We compared the average dopamine concentration during cue presentation (0 to 5 s; after cue onset) to baseline concentration (−5 to 0 s; before cue onset); for TS, we additionally compared dopamine during the WN epoch (5 to 12 s after cue onset) to baseline. In trials with different contingencies (unpredicted or omitted outcome), we compared dopamine transients during relevant epochs (5 to 20 s after cue onset). To assess regional differences in the temporal dopamine-signal shift toward the earliest predictor of the respective appetitive and aversive stimulus, we determined the relative contribution of the CS to the combined positive deviation from baseline for both CS and US epochs by dividing the positive values of the area under the curve during CS epoch (0 to 5 s) by the positive values of the area under the curve during both CS and US epochs (0 to 20 s; CS plus US combined).

FSCV Measurements and Analysis.

We used FSCV incorporating chronically implanted carbon-fiber microelectrodes (63) to record rapid changes in extracellular dopamine concentration in different striatal domains. Before the start of a recording session, microelectrodes were manually connected to the head-mounted voltammetric amplifier. The amplifier was interfaced through an electrical swivel (Med Associates) with a personal computer (PC)–driven data-acquisition and analysis system (National Instruments) (63).

The voltammetric scans were repeated every 100 ms (sampling rate of 10 Hz). The alternating potential at the carbon-fiber electrode tip was ramped linearly during each voltammetric scan from −0.4 V versus Ag/AgCl to +1.3 V (anodic sweep) and back to −0.4 V (cathodic sweep) at 400 V/s (total scan time of 8.5 ms) and held at −0.4 V between scans. Waveform generation and data acquisition and analysis were carried out on a PC-based system using two peripheral component interconnect multifunction data acquisition cards and software written in LabVIEW (National Instruments). When present at the carbon-fiber surface, dopamine is oxidized to form dopamine-o-quinone during the anodic sweep (peak reaction at approximately +0.7 V), which is reduced back again to dopamine during the cathodic sweep (peak reaction at approximately −0.3 V). The ensuing flux of electrons is measured as current and is directly proportional to the number of molecules that undergo electrolysis. For each scan, the obtained background-subtracted, time-resolved current provides a chemical signature characteristic of the analyte, allowing resolution of dopamine from other substances (89). Dopamine traces were isolated from the voltammetric signal by chemometric analysis using a standard training set, based on electrically stimulated dopamine release detected with chronically implanted electrodes; resulting dopamine concentration was estimated on the basis of the average postimplantation sensitivity of electrodes (63). Individual dopamine traces were smoothed with a moving 10-point median filter prior to analysis of average concentration.

Analysis of Operant-Box Behavior.

DeepLabCut software (90) was used to track rat position in the operant chamber using video data recorded during FSCV measurements. These tracking data were analyzed in MATLAB (version 2019a; The Mathworks, Inc.) to determine speed of movement (cm/s). Analyses were performed using the average traveled distance or locomotion speed during the cue (5 s) or WN (3 s) epochs.

Statistical Analysis.

Data collection and analysis were not performed blind to the conditions of the experiments. Statistical analysis was carried out using Prism (GraphPad Software), SPSS (version 25; IBM Corp.) and MATLAB R2018b (The Mathworks, Inc.). Individual electrochemical signals were averaged across trials within a session (within animals) and then across animals. Statistical significance was set to P < 0.05. P values were adjusted according to the Holm-Bonferroni correction method when multiple comparisons were carried out. FSCV and behavioral data were analyzed using two-tailed paired or unpaired Student’s t tests. Equivalent nonparametric tests were applied when data were not normally distributed. Graphical representations were made using Prism and Matlab. All data are presented as mean plus SEM.

Data Availability

Data and code have been deposited in the Open Science Framework (OSF) https://osf.io/zj9rk/ (91).

Change History

May 25, 2022: The SI Appendix has been updated.