Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli

Abstract

Midbrain dopamine neurons are well known for their role in reward-based reinforcement learning. We found that the activity of dopamine axons in the posterior tail of the striatum (TS) scales with the novelty and intensity of external stimuli, but does not encode reward value. We demonstrated that the ablation of TS-projecting dopamine neurons specifically inhibited avoidance of novel or high intensity stimuli without affecting animals’ initial avoidance responses, suggesting a role in reinforcement rather than simply in avoidance itself. Furthermore, we found that animals avoid optogenetic activation of dopamine axons in TS during a choice task and that this stimulation can partially reinstate avoidance of a familiar object. These results suggest that TS-projecting dopamine neurons reinforce avoidance of threatening stimuli. More generally, our results indicate that there are at least two axes of reinforcement learning using dopamine in the striatum: one based on value and one based on external threat.

Introduction

Early electrophysiological recordings in monkeys and rodents revealed that many dopamine neurons are excited by unpredicted rewards or reward-predicting stimuli. Conversely, these neurons are inhibited by the omission of expected reward^1,2 and by aversive events^3,4. Transient activation of dopamine neurons can substitute for positive reward^5,6, whereas transient suppression of dopamine neurons can mimic negative outcomes⁷. These results led to the proposal that dopamine acts as a bi-directional reinforcement signal used by the brain to maximize the value of future outcomes^1,8. However, multiple studies have found that at least some dopamine neurons are activated by non-rewarding events^9,10. For instance, some dopamine neurons in the substantia nigra pars compacta (SNC) are activated by both rewarding and aversive stimuli⁹. This led to the proposal that these dopamine neurons signal “motivational salience” (the absolute value of value) and facilitate a behavioral reaction when an important stimulus (whether it is good or bad) is detected⁹. However, the function of these dopamine neurons remains unknown. Additionally, multiple studies have reported that some dopamine neurons have larger responses to novel stimuli than to familiar stimuli¹¹. This has been interpreted as a “novelty bonus” because novelty may be rewarding itself or signal potential reward¹¹.

To more clearly understand the diversity of dopamine signals, recent studies have focused on the projection targets of dopamine neurons and have shown that different regions of the striatum receive distinct dopamine signals^12,10,13,14. Particularly, whereas dopamine neurons projecting to the ventral striatum (VS) display patterns of activity consistent with the value prediction error seen in canonical dopamine neurons^10,3,13, those projecting to the posterior tail of the striatum (TS) are activated by aversive and neutral stimuli¹⁰. A previous study demonstrated that dopamine responses to novel stimuli are localized in TS dopamine axons, and do not coincide with value-related dopamine signals in VS¹⁰, suggesting that these signals are unlikely to function as a “novelty bonus” for value learning and instead could have a different function. In this study, we investigate the functional significance of the responses to non-rewarding stimuli in dopamine axons in TS.

Results

Dopamine axons in TS encode external stimulus intensity but not value

Previous studies of projection-specified populations have not examined the covariation of dopamine activity with value or salience, which are fundamental characteristics for evaluating the function of each population. To better understand the nature of dopamine signals in the striatum, we first characterized the activity of dopamine axons by presenting an array of stimuli to head-fixed mice. We monitored the activity of dopamine axons at their projection targets using fiber fluorometry/photometry (Figure 1, Supplemental Figure 1, Supplemental Figure 2). To mask any potential responses related to the auditory detection of non-auditory stimuli (e.g. water delivery), training and experiments were performed with constant background noise (see Methods).

Figure 1. Signals from dopamine axons in TS scale with external stimulus intensity but not with reward size.

Fluorescence was collected from GCaMP-expressing dopamine axons and cell bodies (green) using optic fibers while animals received differed sized rewards (blue) or tones (orange). (a) Signals from dopamine axons in VS (mean ± s.e.m. across n = 11 animals) in response to different size reward (f(10) = 3.91, p = 0.0072, 1-way ANOVA) or tones (f(10) = 0.21, p = 0.929, 1-way ANOVA). (b) Signals from dopamine axons in TS (mean ± s.e.m. across n = 9 animals) in response to different size reward (f(8) = 2.05, p = 0.1057, 1-way ANOVA) or tones (f(8) = 3.05, p = 0.0275, 1-way ANOVA). (c) Signals from dopamine neurons in lateral SN (mean ± s.e.m. across n = 9 animals) in response to different size reward (f(8) = 0.68, p = 0.61, 1-way ANOVA) or tones (f(8) = 3.93, p = 0.0088, 1-way ANOVA). (d) Signals from TS-projecting dopamine neurons in lateral SN (mean ± s.e.m. across n = 6 animals) in response to different size reward (f(5) = 0.35, p = 0.84, 1-way ANOVA) or tones (f(5) = 5.13, p = 0.0037, 1-way ANOVA). (e) TS dopamine axon responses to a variety of external stimuli (left, n = 9 animals, solid line is mean and transparent area is s.e.m.). Responses to unexpected air puff (magenta), unexpected bitter water (green), and omission of expected water (blue) in dopamine axons in VS (middle, water omission: t = −4.98, p = 0.00076, n = 10 animals; air puff: t = −2.95, p = 0.015, n = 10 animals; quinine: t = −11.30, p = 0.000028, n = 7 animals; paired t-test) and TS (right, water omission: t = 0.83, p = 0.43, n = 11 animals; air puff: t = 4.93, p = 0.00081, n = 10 animals; quinine: t = 0.90, p = 0.40, n = 8 animals; paired t-test). Solid line is mean and and transparent area is s.e.m. * P < 0.05, ANOVA with post hoc two-sided t-test, Tukey correction for multiple comparisons. ** P < 0.01, two-sided t-test (peak signal in 1 sec following stimulus × peak signal during 1 sec within the inter-trial interval).

We first examined the co-variation of dopamine axon activity in VS or TS with outcome value. Consistent with previous results^2,15, dopamine axons in VS responded strongly to water but only weakly to a neutral tone, whereas dopamine axons in TS responded strongly to tones and air puffs but only weakly to water. The activity of dopamine axons in VS scaled with the size of the reward (Figure 1a), consistent with the idea that dopamine signals encode reward value in this area. Surprisingly, responses of dopamine axons in TS to water delivery were not significantly modulated by reward size (Figure 1b), indicating that the small responses to water observed here and in previous studies are potentially caused by primitive sensory information such as the water valve click rather than reward value. Unlike dopamine axons in VS, the responses in TS scaled with the intensity of the tone (Figure 1b) or air puff (Supplemental Figure 3).

Because high intensity external stimuli (such as tones or air puffs) are potentially aversive, we next examined whether dopamine axons in TS respond to all events with negative value. We found that, whereas dopamine axon signals in VS were inhibited by every negative stimulus that we tested (Figure 1e), dopamine axons in TS did not respond to some types of negative stimuli such as bitter taste (quinine) or the omission of expected water (Figure 1e, Supplemental Figure 4). Instead, dopamine axons in TS were strongly activated by high intensity stimuli of multiple modalities including somatosensory, auditory, visual, and olfactory stimuli (Figure 1e, Supplemental Figure 4). Further, consistent with a previous study¹⁰, dopamine axon responses in TS scaled with the novelty of each stimulus, and the signals decayed differently depending on the stimulus intensity and type (Supplemental Figure 3). Together, these data demonstrate that dopamine axons in TS do not respond to all forms of reward and/or punishment, indicating that they do not simply encode positive and/or negative value⁹. Instead, these data indicate that dopamine axons in TS respond specifically to novel or high intensity external stimuli.

Because dopamine signals are potentially modulated at axon terminals¹⁶, we also recorded activity at the cell bodies of midbrain dopamine neurons. To target TS-projecting dopamine neurons, we first examined their distribution within the midbrain by retrogradely infecting their axons in TS with rabies virus expressing GFP (Figure 2, Supplemental Figure 5). We found that TS-projecting dopamine neurons are concentrated in the lateral part of substantia nigra (SN) especially the most lateral part, called substantia nigra pars lateralis (SNL). We found that most of their axons were located within TS, with no other region containing fluorescence that differed significantly from baseline (Figure 2b, e), indicating that TS-projecting dopamine neurons send very few collaterals to other regions. We recorded the population activity of dopamine neurons in lateral SN, which is the primary location of TS-projecting dopamine neurons (Figure 2, Supplemental Figure 5) using fiber photometry. Like dopamine axon signals in TS, signals from dopamine neurons in lateral SN covaried with tone intensity (Figure 1c), and their small response to reward was not significantly modulated by reward size (Figure 1c).

Figure 2. TS-projecting dopamine neurons are mainly localized in lateral substantia nigra and do not send substantial collaterals to other regions.

(a) Midbrain dopamine neurons labelled with anti-TH antibody (magenta) and TS-projecting dopamine neurons labelled with GFP (green). (b) Forebrain dopamine axons (magenta) and the axons of TS-projecting dopamine neurons (green). (c) Schematic for labelling TS-projecting dopamine neurons and their axons. (d) Distribution of GFP-labelled (green) cell bodies and TH-labelled cell bodies in the midbrain (mean ± s.e.m. across n = 6 animals). For SNC: t = 5.042, p = 0.0040, n = 6 animals, t-test. For SNL: t = 9.36, p = 0.00023, n = 6 animals, t-test. (e) Distribution of GFP-labelled axons in the forebrain and distribution of TH-labelled axons (mean ± s.e.m. across n = 8 animals). For TS: t = 7.46, p = 0.00029, n = 8 animals, t-test with Tukey correction for multiple comparisons. ** P < 0.005, two-sided t-test with Tukey correction for multiple comparisons.

Finally, we recorded specifically from retrogradely labeled TS-projecting dopamine neurons using self-inactivating rabies virus (SiR)^17,18 (Figure 1d, also see Methods). Consistent with the activity we observed from dopamine axons in TS, we found that activity in TS-projecting dopamine neurons in lateral SN covaried with tone intensity (Figure 1d) but not with water reward value (Figure 1d). These neurons displayed increased responses to novel external stimuli as well, consistent with the idea that the signal is modulated by both external stimulus intensity and novelty (Supplemental Figure 6). These results indicate that the unique activity observed in dopamine axons in TS was not due to local modulation at the axon level in TS, but rather reflects the activity at the cell bodies of this unique population in lateral SN.

Stimulation of dopamine axons in TS causes avoidance and lesion of TS-projecting dopamine neurons reduces avoidance in a choice task

It is widely accepted that the activation of dopamine neurons is positively reinforcing. In other words, dopamine release increases the frequency of actions or decisions that elicit dopamine release^1,8. Our finding that the activity of TS-projecting dopamine neurons differs fundamentally from the activity of VS-projecting dopamine neurons suggests that TS-projecting dopamine neurons could have a different function. Therefore, we tested whether direct stimulation of dopamine axons in TS is reinforcing in a choice task (Figure 3a). First, mice were trained to enter a central port to initiate a trial and to then choose one of two side ports associated with different outcomes (Figure 3b). In this task, mice preferred the port associated with a large amount of water over a smaller amount of water, and avoided the port associated with air puff or bitter taste (quinine) (Figure 3c). In short, mice learned to develop choice biases according to the outcomes within a session.

Figure 3. Choice bias against stimulation of dopamine axons in TS.

(a) Example stimulation site (white) of ChR2-expressing dopamine axons (green) in TS. (b) Experimental design: animals enter a central port (grey) to initiate trials and then choose between water and water + × by entering one of two side ports. (c) Choice bias (mean ± s.e.m. across n = 9 animals) for extra water (t = 5.89, p = 0.00036, n = 9, t-test, Bonferroni correction), air puff (t = −4.29, p = 0.0027, n = 9, t-test, Bonferroni correction), or quinine (t = −5.96, p = 0.00038, n = 9, t-test, Bonferroni correction). Solid dots indicate mean and transparent dots indicate each animal. (d) Left: choice bias (mean ± s.e.m. across n = 9 animals) for optogenetic stimulation of dopamine axons in VS (cyan; t = 3.077, p = 0.015, n = 9, t-test, Bonferroni correction) or TS (magenta; t = −3.219, p = 0.0123, n = 9, t-test, Bonferroni correction). Solid dots indicate mean and transparent dots indicate each animal. Right: choice bias (mean ± s.e.m. across animals) for stimulation of dopamine axons in TS in GFP-expressing (t = 0.388, p = 0.711, n = 7, t-test, Bonferroni correction) or ChR2-expressing (t = −3.219, p = 0.0123, n = 9 animals, t-test, Bonferroni correction) mice. * P < 0.02, two-sided t-test. ** P < 0.005, two-sided t-test.

Next, we examined the effect of optogenetic stimulation of dopamine axons in TS in a choice task. After mice learned the choice task, the axons of VS-projecting or TS-projecting dopamine neurons were optogenetically activated using a light-gated ion channel, channelrhodopsin-2 (ChR2) in one of the two choice ports while mice received reward (Figure 3d, Supplemental Figure 7). The stimulation port was pseudo-randomly assigned in each session. Stimulation in VS biased the animals’ choices toward the port associated with stimulation (Figure 3d), consistent with the idea that dopamine acts as positive reinforcement. On the other hand, stimulation in TS caused a bias against the port associated with stimulation (Figure 3d). These results demonstrate that the optogenetic activation of dopamine axons in VS and TS have opposite effects on animals’ behavioral choices. Optogenetic activation of dopamine axons in TS caused avoidance of a behavioral choice associated with activation, instead of acting as a positive reinforcer of the choice.

Next, we tested whether ablation of TS-projecting dopamine neurons using a selective neurotoxin, 6-hydroxydopamine (6-OHDA) had an effect on avoidance behavior (Figure 4). The 6-OHDA injections in TS specifically reduced dopamine innervation of TS and dopamine cell bodies in SNL without affecting neighboring dopamine axons in the striatum or amygdala, or noradrenaline neurons in the locus coeruleus (Figure 4a, Supplemental Figure 8), consistent with our rabies-based axon-labelling data (Figure 2). The lesion mice did not exhibit differences in general locomotor activity (Supplemental Figure 9).

Figure 4. Ablation of TS-projecting dopamine neurons eliminates choice bias against a threatening stimulus.

(a) Schematic of ablation of TS-projecting dopamine neurons and examples of the effect of saline (top) or 6-OHDA (bottom) on dopamine axons in TS (“Tail”), but not more anterior striatum (“Anterior”). Axons labeled with anti-TH antibody (magenta). (b) Experimental design. (c) Normalized choice bias (mean ± s.e.m. across n = 10 animals per group) for air puff, quinine, and the reduction of water in saline (blue; air puff: t = −4.13, p = 0.0026, n = 10 animals per group; quinine: t = −10.34, p = 0.0000027, n = 10 animals per group; water reduction: t = −3.045, p = 0.014, n = 10 animals per group) and 6-OHDA (red; air puff: t = 0.31, p = 0.77, n = 10 animals per group; quinine: t = −7.78, p = 0.0000028, n = 10 animals per group; water reduction: t = −2.48, p = 0.035, n = 10 animals per group) animals. Solid dots indicate mean and transparent dots indicate each animal. For Saline × 6-OHDA: air puff: t = 2.59, p = 0.0034, n = 10 animals per group; quinine: t = 1.40, p = 0.18, n = 10 animals per group; water reduction: t = 0.059, p = 0.95, n = 10 animals per group. (d) Total number of “incorrect” choices (mean ± s.e.m. across n = 10 animals per group) of water + air puff in saline (blue) and 6-OHDA (red) animals in a session (Saline × 6-OHDA: t = 2.29, p = 0.0341, n = 10 animals per group). Solid dots indicate mean and transparent dots indicate each animal. (e) Schematic of D1 antagonist application. (f) Choice bias (mean ± s.e.m. across 6 animals) for air puff following acute application of saline (blue) or D1 antagonist (green). Saline × D1 antagonist: t = 4.44, p = 0.0068, n = 6 animals, paired t-test. Solid dots indicate mean and transparent dots indicate average across sessions for each animal. * P < 0.05 two-sided t-test, ** P < 0.005, two-sided t-test.

We examined choice bias using different outcomes at two water ports (Figure 4, Supplemental Figure 10). When choosing between different sizes of water, both control and lesion mice showed a clear preference for the larger size of water (Figure 4c). However, when given a choice between water and water plus air puff, lesion mice did not show systematic bias away from the side associated with air puff as a population (Figure 4c, Supplemental Figure 10a, b) and received more air puffs total in a session (Figure 4d). Of note, we show here results of the first session when animals experienced air puff (see Methods), and during this single session individual lesion mice displayed a variety of port biases, consistent with the idea that random choice (i.e. choosing both ports equally when both have the same value) is not necessarily a default strategy when random choice is not advantageous^19,20. Importantly, choice stickiness, or “win-stay” strategy was observed in a similar frequency in a control session (choosing between two water ports) between control and lesion mice (Supplemental Figure 10c). However, in an air puff session, whereas control mice repeated the same choice (stay) more often after the choice of a water port than after the choice of an air puff port, lesion mice showed a similar frequency of stay after the choice of either port (Supplemental Figure 10c), indicating that lesion mice did not acquire different choice preference even right after experiencing water plus air puff versus only water.

Surprisingly, the lesion animals’ immediate behavioral responses to air puff remained largely unimpaired. Lesion mice still displayed an immediate retreat with a similar retreat distance as controls in response to the first air puff (Figure 5a-c, Supplemental Figure 10d). Over the course of the session, however, the lesion mice responded much differently. While control mice continued to retreat from air puffs over many trials, lesion mice showed significantly smaller retreats after a few trials (Figure 5a-c, Supplemental Figure 11). Thus, whereas detection of air puff and initial retreat responses to air puff itself were intact, lesion mice did not maintain the retreat responses in subsequent trials. These results suggest that there is a system responsible for the initial retreat behaviors (“fixed reaction”²¹), which is independent of TS-projecting dopamine neurons, and that TS-projecting dopamine neurons are responsible for maintenance of the retreat. Consistent with a role in maintenance or reinforcement of avoidance, dopamine axons in TS (in intact mice) responded strongly to air puffs and the signals remained high during this task (Figure 5d-f, Supplemental Figure 10e), much like the signals we observed in head-fixed mice (Supplemental Figure 3).

Figure 5. Ablation of TS-projecting dopamine neurons reduces avoidance without affecting initial responses to high intensity external stimuli.

(a) Example behavioral traces from control and air puff sessions in saline (blue) and 6-OHDA (red) animals. (b) Retreat distance (mean ± s.e.m. across n = 10 animals per group) across trials. (c) Average retreat distance (mean ± s.e.m. across n = 10 animals per group) on trial 1 and for all trials in saline (blue; trial 1: t = 4.21, p = 0.0030, n = 10 animals, paired t-test; all trials: t = 3.70, p = 0.0060, n = 10 animals, paired t-test) and 6-OHDA (red; trial 1: t = 2.83, p = 0.022, n = 10 animals, paired t-test, all trials: t = −0.86, p = 0.415, n = 10 animals, paired t-test). Solid dots indicate mean and transparent dots indicate each animal. For trial 1: Saline × 6-OHDA, t = 0.25, p = 0.80, n = 10 animals per group, unpaired t-test. For all trials: Saline × 6-OHDA, t = −0.86, p = 0.42, n = 10 animals per group, unpaired t-test. (d) Average GCaMP responses in dopamine axons in TS upon water choice (cyan) or air puff choice (magenta). Solid lines indicate mean across animals and transparent lines indicate mean signal from each animal. (e) Time course of responses to water or air puff (mean ± s.e.m. across n = 11 animals). (f) Average peak GCaMP responses (mean ± s.e.m. across n = 11 animals) following water or puff on trial 1 (left; water: t = 0.64, p = 0.53, n = 13 animals, paired t-test; air puff: t = 3.03, p = 0.0058, n = 13 animals, paired t-test) as well as for all trials (right; water: t = −0.011, p = 0.99, n = 13 animals; air puff: t = 3.82, p = 0.00084, n = 13 animals). * P < 0.05, two-sided t-test, ** P < 0.005, two-sided t-test. Solid dots indicate mean and transparent dots indicate mean signal from each animal.

Notably, lesion mice were able to learn from other negative events such as bitter taste and water reduction at similar levels as control mice (Figure 4c). This suggests that TS-projecting dopamine neurons are not responsible for learning from all types of negative events. This is consistent with our observation that the activity of dopamine axons in TS increased in response to external stimuli such as tone and air puff, but not in response to the bitter taste or omission of water (Figure 1e). To determine whether the effects of lesion on choice preference are dependent on dopamine, we pharmacologically inhibited D1 dopamine receptors in TS, and observed similar effects as in 6-OHDA lesion mice (Figure 4e, f, Supplemental Figure 12). Thus, dopamine in TS is critical for avoiding air puff, and this is mediated at least partially by D1 receptor signaling.

Midbrain dopamine neurons projecting to TS reinforce avoidance of novel objects

Our results demonstrate that dopamine neurons in TS are important for learning to avoid air puff punishment. However, TS-projecting dopamine neurons are excited not only by air puff, but also by seemingly neutral novel stimuli of multiple modalities¹⁰ (Supplemental Figure 3). It is not immediately clear what air puff and novelty could have in common. To understand the function of novelty signals in TS-projecting dopamine neurons, we first examined animals’ behavioral responses to a novel object (Figure 6). When animals encounter novelty, they typically display elevated exploration, orientation, or approach to the novel stimulus compared to a familiar one^22–24. Consistent with these observations, our mice approached novel objects more frequently than familiar objects (Figure 6a). However, novel objects caused a more intricate behavior than simple approach. Mice frequently performed “bouts” of investigation in which they approached the novel object and then quickly retreated (Figure 6a, b, Figure 7a-b Saline, Supplemental Figure 13, Supplemental Video 1). The mice repeated these short bouts multiple times. The bouts became gradually longer, and the mice spent more time at the vicinity of the novel object over days. These approach-avoidance conflicts have been observed across various animal species including human beings, and have been interpreted as an unstable equilibrium of curiosity and timidity (“weal or woe” by William James)²⁵, or a sense of potential fear.

Figure 6. Dopamine axons in TS are active as animals retreat from novel objects but not as they approach novel objects.

(a) Mice exhibit bouts of approach and retreat when investigating novel objects (left). Frequency of approach to objects (mean ± s.e.m. across n = 10 animals) (right; novel × familiar: t = 3.67, p = 0.001, n = 10 animals, paired t-test; novel × no object: t = 4.41, p = 0.0017, n = 10 animals, paired t-test). Solid dots indicate mean and transparent dots indicate each animal. (b) Example GCaMP trace from dopamine axons in TS (green) aligned to animal’s distance from the novel object (black), with dotted lines indicating nearest approach per bout. (c) Left: Example GCaMP signals aligned to onset of retreat. Bouts were sorted based on time of approach, shown as a grey line. Right: Average GCaMP responses across n = 6 animals to novel object (cyan), familiar object (magenta), and no object control (black). For novel × familiar: t = 3.04, p = 0.021, n = 6 animals, paired t-test. For novel × no object: t = 3.65, p = 0.015, n = 6 animals, paired t-test. Solid dots indicate mean and transparent dots indicate each animal. * P < 0.05 two-sided t-test, ** P < 0.01 two-sided t-test.

Figure 7. Ablation of TS-projecting dopamine neurons reduces retreat from novel objects and stimulation partially reinstates avoidance of novel objects.

(a) Example paths from saline (blue) and 6-OHDA (red) mice during the first 5 minutes of a 10-minute novel object interaction. No object control paths also shown for comparison (black). (b) Left: time-course of bout duration (mean ± s.e.m. across n = 10 animals per group) in saline (blue) or 6-OHDA (red) mice during two 10-minute sessions denoted by dotted lines. For first trial, Saline × 6-OHDA: t = −1.67, p = 0.11, n = 10 animals per group, unpaired t-test. For all trials, Saline × 6-OHDA: t = 4.16, p = 0.0016, n = 10 animals per group, unpaired t-test. Right: number of approaches to (Saline × 6-OHDA: t = 0.665, p = 0.514, n = 10 animals per group, unpaired t-test) and fraction of time spent near (Saline × 6-OHDA: t = 5.77, p = 0.000018, n = 10 animals per group, unpaired t-test) a novel object (mean ± s.e.m. across n = 10 animals per group). Solid dots indicate mean and transparent dots indicate each animal. (c) Stimulation of TS dopamine axons during interaction with a familiar object. For control: first 15 minutes × stimulation: t = 0.40, p = 0.69, n = 16 sessions from 8 animals, paired t-test; first 15 minutes × last 15 minutes: t = 0.031, p = 0.96, n = 16 sessions from 8 animals, paired t-test. For ChR2: first 15 minutes × stimulation: t = 3.40, p = 0.0039, n = 16 sessions from 8 animals, paired t-test; first 15 minutes × last 15 minutes: t = 2.25, p = 0.040, n = 16 sessions from 8 animals, paired t-test. Solid dots indicate mean and transparent dots indicate each session. * P < 0.05, two-sided t-test, ** P < 0.01, two-sided t-test.

To examine the role of TS-projecting dopamine neurons in this approach-retreat behavior, we recorded signals from dopamine axons in TS as mice interacted with a novel object in a familiar environment. Signals in TS increased as mice reached the closest point of approach to the novel object in the bout (as they began to retreat), but not as they approached it (Figure 6b-c). The responses in TS were significantly larger when mice retreated from a novel object than from a familiar object or from the same location in an open field (Figure 6c). The signals in TS were not correlated with the animals’ velocity or with the initiation of movement, indicating that the signal may not be directly related to motor activity (Supplemental Figure 14). To control for motion artifacts, we recorded from control mice expressing GFP instead of GCaMP and observed no obvious artifacts that could explain the GCaMP signals (Supplemental Figure 14).

Next, we tested whether and how TS-projecting dopamine neurons regulate animals’ responses to novelty by lesioning TS-projecting dopamine neurons using 6-OHDA. When the lesion mice first encountered a novel object, they exhibited a similar behavior as control mice: an approach followed by a quick retreat (Figure 7a-b, Supplemental Figure 13, Supplemental Video 2). Strikingly, after a few bouts, the lesion animals began to spend a much longer time near the novel object per bout (Figure 7a-b, Supplemental Video 3, Supplemental Video 4). In total, lesion mice spent a significantly longer time near the object (Figure 7b), but they did not approach the object more frequently than control mice (Figure 7b). In short, mice with an ablation of TS-projecting dopamine neurons showed relatively normal novelty responses (approach-retreat) at first, but the avoidance component of this response to the novel object quickly diminished.

Importantly, like retreat from air puff, initial retreat from a novel object was independent of TS-projecting dopamine neurons. This suggests that dopamine in TS has a role in maintaining or reinforcing avoidance of objects based on their novelty. The retreats eventually diminished over days, consistent with recording data showing that responses in dopamine axons in TS decayed slowly during the first session but were much smaller in later sessions (Figure 6c, Supplemental Figure 15).

Conversely, closed-loop optogenetic stimulation of dopamine axons in TS while animals approached a familiar object reduced the duration of mice’s interaction with the object, which could be interpreted as a reinstatement of avoidance. Bout sizes became shorter following the activation of dopamine axons in TS (Figure 7c), and remained shorter after the stimulation had ended (Figure 7c). Because some dopamine neurons express a synaptic vesicular transporter for glutamate, Vglut2²⁶, which allows co-release of glutamate²⁷, we examined the function of glutamate co-release from dopamine neurons during novelty responses. We genetically knocked out Vglut2 in dopamine neurons (see Methods) and found that mice with no Vglut2 in dopamine neurons showed similar approach-retreat behaviors as control littermates (Supplemental Figure 15), suggesting that glutamate release from dopamine neurons is not essential for novel object avoidance behavior.

Discussion

When comparing inputs to dopamine neurons with different projection targets, we previously identified TS-projecting dopamine neurons based on their unique set of inputs – receiving relatively little input from the ventral striatum and more input from dorsolateral-shifted regions such as the subthalamus and globus pallidus²⁸. By recording activity from dopamine axons in VS, dorso-medial striatum (DMS), dorso-lateral striatum (DLS), and TS, we found that the activity of dopamine axons in TS is also unique¹⁰. We showed that dopamine axons in TS are excited by both aversive and neutral stimuli and that they have some characteristics of prediction error: they signal the prediction of a stimulus and their responses to the stimulus itself are suppressed by prediction. In the present study, we more systematically and parametrically examined dopamine responses in TS (focusing on unconditioned stimuli) and examined the function of this signal as a reinforcer. We found that dopamine axons in TS are monotonically modulated by the intensity and novelty of external stimuli, reminiscent of a previous proposal that the early phase of dopamine signals encodes stimulus intensity, but not aversiveness itself²⁹. In a sense, TS-projecting dopamine neuron activity could be interpreted as the extreme case of having only this phase without encoding value, which is the main phase in canonical dopamine neurons²⁹. In addition, our results indicate that dopamine axons in TS are not excited by all stimuli. Instead, we found that they are specifically excited by external stimuli, but not by ingestion-related stimuli (e.g. bitter taste). Further, we found that TS-projecting dopamine neurons encode both the novelty and intensity of stimuli of multiple modalities. Consistent with their activity, we found that TS-projecting dopamine neurons are important for both the avoidance of air puff punishment and avoidance of a novel object, but not for the avoidance of all types of negative outcomes. And finally, we found that stimulation of TS-projecting dopamine neurons elicits avoidance and that ablation of TS-projecting dopamine neurons reduced avoidance.

Combining these insights, we propose that dopamine axons in TS encode the physical salience (such as intensity and novelty) of external stimuli to signal potential external threats in the environment. Of note, threat and aversiveness have been distinguished in many studies, and for good reason. For example, a previous study found a behavioral difference between responses to external threats and aversive taste, showing that visual stimuli associated with external threats, but not aversive taste, draw attention in monkeys³⁰. These differences are potentially connected to the difference between fear and disgust caused by these stimuli^31,32. Consistent with these differences, our data showed that different brain systems underlie learning from different types of punishments, and that dopamine in TS is specifically involved in avoidance of external threats.

Dopamine in TS reinforces threat avoidance by signaling external threat

Striatal dopamine has previously been proposed to be involved in avoidance³³ within the framework of the canonical view of dopamine function. A series of studies showed that the dopamine neurons are excited by the prediction of successful avoidance of aversive events (namely, “safety”) and that this safety signal functions to promote successful avoidance³³. This theory beautifully incorporates the function of dopamine in active avoidance into the framework of value-coding: dopamine neurons signal both reward and safety as positive value. However, this rule does not apply to dopamine in TS. Contrary to this idea, dopamine axons in TS do not encode outcome values, and this system operates differently from the canonical dopamine system.

Although they encode different information, dopamine in TS may function, like canonical dopamine, as a reinforcement signal: primarily affecting future behaviors. Alternatively, dopamine in TS may regulate ongoing behaviors on a moment-by-moment basis. Our results suggest a role in reinforcement. First, in our choice tasks, the timing of optogenetic activation (when mice enter a reward port) and subsequent choice on the next trial are well separated: avoidance behavior manifests in subsequent trials without concurrent optogenetic activation of TS-projecting dopamine neurons. Similarly, in novel object exploration, the effect of optogenetic activation lingered even after the activation ended. Additionally, the initial retreats from air puff and novel objects were intact in lesion animals, and only subsequent responses were affected by lesions. These results suggest that, in our tasks, dopamine in TS primarily functions as a reinforcement signal, although we cannot formally exclude the possibility that it also affects ongoing behavior.

Optogenetic activation of dopamine axons in TS caused choice bias against the port associated with stimulation. In theory, there are at least two ways to explain this effect. One possibility is that a dopamine increase in TS may suppress preceding behaviors (i.e. ‘inhibitory’³⁴, or ‘weakening’), in a manner opposite to canonical dopamine. Alternatively, a dopamine increase may reinforce an avoidance behavior or a threat prediction by Pavlovian association³⁵. Our observations favor the latter hypothesis. After the ablation of TS-projecting dopamine neurons, mice displayed a similar number of approaches to novel objects, while retreat from novel objects differed compared to control mice. This suggests that dopamine plays a role in maintaining a high level of novelty avoidance or threat prediction, rather than discouraging approach. We therefore propose that dopamine in TS reinforces a specific type of behavior (the avoidance of threatening stimuli) by updating or maintaining a high level of threat prediction based on evidence acquired about threating stimuli.

The amygdala has long been studied in relation to fear and fear-based learning. Although they are neighboring regions, TS and amygdala are often contrasted in functional experiments^36,37. For example, even the ablation of all medium spiny neurons in the striatum hardly affected fear conditioning by electric shock³⁸, which is the most often used paradigm for amygdala studies. One possibility is that TS is more specialized for behaviors based on “threat prediction” without experiencing actual pain (see below), or that TS is especially important when animals consider both reward and threat. Accumulating studies show that amygdala is important for learning from reward as well as from threat³⁹. Consistent with this broad function, amygdala projects to both VS and TS⁴⁰ and receives dopaminergic projections from a wide range of nuclei in supramammillary areas, VTA, SNC, and SNL^28,41. By contrast, TS receives dopamine innervation primarily from the lateral part of SN. How these structures collaborate during avoidance³⁹ is an important question to address in the future.

Similarities between trial-and-error learning and novel object exploration

Although a choice task with a rigid trial-based structure and self-directed investigation of novel objects are seemingly very different behaviors, we found a potentially unifying explanation for dopamine function across them. Animals typically show “unstable equilibrium”²⁵ of approach-retreat to novelty. Multiple studies have found that dopamine in other brain areas such as dorso-medial and dorso-lateral striatum is important for exploration triggered by novelty^42,43. Our findings together with these studies suggest that dopamine plays a role on the both sides of this equilibrium. Approach-retreat behaviors in foraging have been interpreted as “risk assessment”⁴⁴. In this sense, the animals’ novelty exploration can be viewed as a sequential process of learning and decision based on value and threat (similar to sequential choice behavior). The distortion of equilibrium in such a situation potentially causes maladaptive novelty-seeking or excess fear of the “strange” in development⁴⁵, in psychiatric conditions such as addiction⁴⁶ and autism⁴⁷, and under stress⁴⁸. Our data suggests that dopamine may function in event-by-event behavioral updating not only in value-based decision-making, but also in a broader range of behaviors than was previously thought.

Conclusion

While our findings indicate that dopamine in TS and canonical dopamine systems serve different functions, we propose that these dopamine systems may function through a similar mechanism using Pavlovian prediction³⁵. Thus, a functional difference between these systems may come from the information that these dopamine signals convey and behavioral outcomes owing to this difference of information, rather than in their algorithms. Whereas canonical dopamine in VS provides bidirectional signals along the axis of value, dopamine in TS provides information along a different dimension – which includes the intensity and novelty of external stimuli (Figure 8). Interestingly, because TS is a sensory region of the striatum, receiving inputs mainly from sensory cortices and thalamus such as visual and auditory cortex, threat information carried by dopamine potentially directly impacts stimulus representation in TS.

Figure 8. Separate axes for dopamine-based reinforcement learning.

Top: a model of dopamine-based reinforcement learning using value prediction error signals conveyed to the ventral striatum (VS) primarily by VTA dopamine neurons. Bottom: a model of dopamine-based reinforcement learning using threat signals conveyed to the posterior tail of the striatum (TS) primarily by SNL dopamine neurons.

Notably, the dynamics of dopamine responses in these systems are quite different. A previous study found that dopamine responses in VS and TS are initialized in an opposite manner: responses to a novel stimulus in dopamine axons in VS are initialized at zero, whereas responses in dopamine axons in TS are initialized at a high level¹⁰, although it was not clear why their dynamics are opposite. Combined with the present finding that TS-projecting dopamine neurons reinforce threat avoidance, the functional significance of these dynamics begins to make sense. Dopamine-striatum systems seem to have cautious initialization: high for threat and low for value. A threat prediction system with high initialization may be critical to learn to avoid unknown threats, which are potentially harmful, without experiencing actual outcomes such as pain or death. Further, dopamine axons in TS do not show inhibition (“dip”) with omission of the outcome¹⁰, possibly indicating that the threat prediction system does not use active weakening but rather gradually erases the prediction.

The existence of separate systems for different types of reinforcement signals resembles the anatomically segregated reward and punishment dopamine systems observed in fruit flies⁴⁹, although the nature of the reinforcement signals are not identical. In nonhuman primates, a recent study reported that dopamine neurons in caudal-lateral SNC that project to caudate tail also did not respond to water reward but instead responded to salient visual stimuli⁵⁰, suggesting a degree of similarity of dopamine in mouse TS and primate caudate tail, although more comparative studies are needed in the future to confirm these similarities. These multiple reinforcement systems may function in parallel or cooperate, but they may also compete when animals decide between options that contain both potential value and potential threat, such as during interactions with novel stimuli. If this is the case, then understanding the balance or equilibrium between these systems will be crucial for understanding the rich behavioral dynamics that animals display in natural environments.

Online Methods

Animals

90 mice between 3 and 6 months old were used for this study. Dopamine transporter (DAT)-cre (B6.SJL-Slc6a3^{tm1.1(cre)Bkmn}/J, Jackson Laboratory; RRID:IMSR JAX:006660)⁵¹ heterozygous mice were used for recording signals from dopamine axons expressing GCaMP, for stimulation of dopamine axons expressing ChR2 and for anatomical examination of TS-projecting dopamine neurons. All mice were backcrossed with C57BL/6J (Jackson Laboratory). C57BL/6J mice were used for ablation of TS-projecting dopamine neurons using 6-OHDA. Vglut2^flox (Slc17a6^tm1Lowl/J, Jackson Laboratory 012898)²⁶ homozygous/DAT-cre heterozygous mice and their littermates (Vglut2^flox homozygous mice) were used for novelty exploration behavioral tests. Animals were housed on a 12 hour dark/12 hour light cycle (dark from 07:00 to 19:00) and performed a task at the same time each day. All procedures were performed in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and approved by the Harvard Animal Care and Use Committee.

Rabies virus production and AAV constructs

SiR-FLPo(envA)¹⁷ was amplified as previously reported⁵². Briefly, BHK/ToGG was infected by SiR-FLPo. 2 days later, the supernatant was collected to infect 5 times larger scale of BHK/ToGG. 2 days later, SiR-FLPo was harvested by filtering the supernatant with Steriflip (Millipore #SCGP00525), and was pseudotyped with envA by infecting the same large scale of BHK/EnvA-TEVp. Resulting SiR-FLPo(envA) was filtered and concentrated by ultracentrifugation at 70,000g at 4°C. Small aliquots are stored at −80°C. pAAV-EF1a-FLEX(frt)-GCaMP6f was made by inserting GCaMP6f from pGP-CMV-GCaMP6f⁵³ (Addgene #40755), cleaved with BglII and NotI and blunted, into pAAV-EF1a-fDIO-EYFP (Addgene #55641), cleaved with AscI and NheI and blunted to remove EYFP.

Surgical procedures

All surgeries were performed under aseptic conditions with animals anesthetized with isoflurane (1–2% at 0.5–1.0 l/min). Analgesia (ketoprofen, 5 mg/kg, I.P.; buprenorphine, 0.1 mg/kg, I.P.) was administered postoperatively. We used the following coordinates to target our injections and implants.

• –(VTA) Bregma: −3.0mm, Lateral: 0.6mm, Depth: between 4.5mm and 4.0mm
• –(SNC) Bregma: −3.3mm, Lateral: 1.6mm, Depth: between 4.0mm and − 3.5mm
• –(SNL) Bregma: −3.4mm, Lateral: 2.0mm, Depth: 3.5mm
• –(VS) Bregma: 1.0mm, Lateral: 1.25mm, Depth: 3.85mm
• –(TS) Bregma: −1.0mm, Lateral: 3.25mm, Depth: 2.5mm

GCaMP surgical procedure

To prepare animals for GCaMP recording in the striatum, we performed a single surgery with three key components: (1) AAV-FLEX-GCaMP6m virus infection into the midbrain, (2) head-plate installation, and (3) one or more optic fiber implants into the striatum. To express GCaMP6m⁵³ specifically in dopamine neurons, we unilaterally injected 250 nl of AAV5-CAG-FLEX-GCaMP6m (1 × 10¹² particles/ml, Penn Vector Core) into both the VTA and SNC (500 nl total). AAV1-CAG-FLEX-GFP (4.5 × 10¹² particles/ml, UNC Vector Core) was injected as a control. Virus injection lasted several minutes, and then the injection pipette was slowly removed over the course of several minutes to prevent damage to the tissue.

So that mice could be head-fixed during recording², we installed a head-plate onto each mouse with C and B Metabond adhesive cement (Parkell. Edgewood, NY). We used circular head-plates¹⁰ to ensure that the skull above the striatum would be accessible for fiber implants. Finally, during the same surgery, we also implanted optic fibers (200 μm diameter, Doric Lenses, Canada) into the VS and/or TS (1 or 2 fibers per mouse). To do this, we first slowly lowered optical fibers into the striatum. Once fibers were lowered, we first attached them to the skull with UV-curing epoxy (NOA81, Thorlabs, NJ), and then a layer of black dental adhesive (Ortho-Jet, Lang Dental, IL). After waiting fifteen minutes for this to dry, we applied a very small amount of rapid-curing epoxy (A00254, Devcon) to attach the fiber cannulas even more firmly to the underlying glue and head-plate. We used magnetic fiber cannulas (Doric, MFC_200/245) and the corresponding patch cords to allow for recordings in freely moving animals. After waiting fifteen minutes for the epoxy to dry, the surgery was complete.

To record from dopamine neurons in lateral SN regardless of projection site, we injected GCaMP6m into the midbrain (as above), but also implanted the optic fiber into the midbrain. To record from dopamine neurons in lateral SN that project to TS, we performed two surgeries. In the first surgery, we injected 250 nl of AAV5-FLEX-TVA (1 × 10¹² particles/ml, UNC vector core) and 250 nl of AAV8-FLEX(frt)-GCaMP6f (1.0×10¹³ particles/ml, UNC Vector Core) into the midbrain. After waiting 2 weeks to allow for AAV expression, we then performed a second surgery and injected 500 nl of SiR-FLPo(envA) (1.1×10⁹ plaque-forming units [pfu]/ml) into TS and implanted an optic fiber into lateral SN.

ChR2 surgical procedure

To prepare animals for unilateral ChR2 stimulation, we performed a single surgery with two key components: (1) AAV-DIO-ChR2-YFP virus infection into the midbrain, and (2) one or two optic fiber implants into the striatum. To express ChR2 specifically in dopamine neurons, we unilaterally injected 250 nl of AAV5-EF1α-DIO-ChR2-YFP⁵ (1 × 10¹² particles/ml, UNC Vector Core) into both the VTA and SNC (500 nl total). AAV1-CAG-FLEX-GFP (4.5 × 10¹² particles/ml, UNC Vector Core) was injected as a control. Virus injection lasted several minutes, and then the injection pipette was then slowly removed over the course of several minutes to prevent damage to the tissue.

We then performed optic fiber implants into VS and TS. To do this, we first slowly lowered optical fibers (200 μm diameter, Doric Lenses) into the striatum. Once fibers were lowered, we first attached them to the skull with UV-curing epoxy (Thorlabs, NOA81), and then a thick layer of black Ortho-Jet dental adhesive (Lang Dental). After waiting fifteen minutes for this to dry, we applied a very small amount of rapid-curing epoxy (Devcon, A00254) to attach the fiber cannulas even more firmly to the underlying adhesive. After waiting fifteen minutes for the epoxy to dry, the surgery was complete.

Rabies surgical procedure for anatomical examination

To label TS-projecting dopamine neurons and their axons, we first injected 250 nl of AAV5-FLEX-TVA²⁸ (1 × 10¹² particles/ml, UNC vector core) into the VTA and SNC of animals expressing Cre in dopamine neurons (DAT-cre animals). This would allow EnvA-pseudotyped rabies virus to infect these cells. After waiting 3 weeks for AAV viral expression, we injected 500 nl of SADdG-GFP(envA)⁵² (5 × 10⁷ pfu/ml, Salk Institute vector core) into TS. This virus retrogradely infected dopamine neurons projecting to TS, brightly labelling their cell bodies and axons throughout the brain due to high viral copy number. After waiting 1 week for rabies viral expression, animals were euthanized and brains were sectioned for analysis. Notably, this method is extremely sensitive compared to AAV-based approaches due to the high copy number of rabies virus.

6-OHDA surgical procedure

To ablate dopamine neurons projecting to TS bilaterally, we largely followed an existing protocol⁵⁴. We first prepared a solution of:

• –28.5 mg desipramine (Sigma-Aldrich, D3900–1G)
• –6.2 mg pargyline Pargyline (Sigma-Aldrich, P8013–500MG)
• –10 mL water
• –NaOH to pH 7.4

This solution was injected (I.P.) to animals at 10 mg/kg. Typical animals of ~25g received ~250 uL of this cocktail. This cocktail prevents dopamine uptake in noradrenaline neurons (although noradrenalin projections are not detectable in TS⁵⁵), and increases the selectivity of uptake by dopamine neurons^54,56,57. After this injection, mice were anesthetized using isofluorane as described above for injections into the brain. We prepared a solution of:

• –10 mg/mL 6-hydroxydopamine (6-OHDA) (Sigma-Aldrich, H116–5MG)
• –1mL 0.2% ascorbic acid in saline (0.9% NaCl) (Sigma-Aldrich, PHR1008–2G)

Importantly, the saline solution included a small amount of ascorbic acid to prevent 6-OHDA from breaking down. To further protect 6-OHDA from breaking down, this solution was kept on ice, wrapped in aluminum foil, and used within 3 hours of mixing. If the solution turned brown, it was discarded, as this indicates that the 6-OHDA had broken down. 6-OHDA or saline injections were performed bilaterally into the posterior tail of the striatum (TS) on each side of the brain. We injected 250 nl of 6-OHDA or saline into each TS. These injections lasted several minutes, and then the injection pipette was slowly removed over the course of several minutes to prevent damage to the tissue.

Cannula implant surgical procedure

To inhibit D1 dopamine receptors in TS, we largely followed an existing protocol⁵⁸. Importantly, we made cannulas (8 mm in length) using 23 gauge wire for the cannula and 27 ½ gauge needles to plug the cannulas between experiments. The top of the needles was plugged using epoxy to prevent contamination. We slowly lowered the cannulas into the striatum, one side at a time. Once cannulas were lowered, we first attached them to the skull with UV-curing epoxy (Thorlabs, NOA81), and then a thick layer of black Ortho-Jet dental adhesive (Lang Dental). After waiting fifteen minutes for this to dry, we applied a very small amount of rapid-curing epoxy (Devcon, A00254) to attach the cannulas even more firmly to the underlying adhesive. After waiting fifteen minutes for the epoxy to dry, the surgery was complete.

Behavioral paradigms

Head-fixed classical conditioning behavioral paradigm²

After surgery, mice were given three weeks to recover and become habituated to the installed head-plate and implanted optic fibers. Additionally, this allowed time for viral expression. After this recovery period, mice were handled for 2–3 days and water deprived. Then, mice were habituated to being head-fixed for three days. During these days, mice were head-fixed for 5–10 minutes and given unexpected water at random intervals (drawn, between 1 and 20 s, with a mean of 10 s and a normal distribution). This allowed mice to become habituated to being head-fixed. After this, mice were trained to perform a task with four trial types: 1) Odor A → reward (90%), 2) Odor A → reward omission (10%), 3) Odor B → no outcome, and 4) Free water (Supplemental Figure 4). Mice were trained in this task for 10 days before recording neuronal responses to unexpected stimuli of different modalities and intensities. To observe GCaMP signals elicited by unexpected stimuli of different modalities and intensities, we interspersed these outcomes within the context of classical conditioning.

On each day, either unexpected water (of varying sizes), unexpected tone (of varying sizes), unexpected air puff (of varying sizes), or unexpected quinine (0.001 M) were interspersed with the other trial types. To ensure that the order of stimulus presentation did not impact the data, different mice received the stimuli in a different order. To minimize the animals’ ability to use auditory or visual information, such as the click of the valve that releases water or air puff (~45 dB measured from position of the animal, with valve outside of the behavioral setup and muffled), the training was done in darkness with a constant background noise of ~50 dB.

We used a range of 5 kHz tones including 50dB, 65dB, 75dB, 90dB, and 100dB. For water size manipulation, we used 1uL, 3uL, 5uL, 10uL, and 20uL. For air puff size manipulation, we used 0.2 atm, 0.4 atm, 0.6 atm, 0.8 atm, and 1.0 atm. For quinine, we used a concentration of 0.001 M. On trials with expected water, the water deliver size was 10 μL.

Each session consisted of approximately 250 trials and took approximately 45 minutes. During these sessions, GCaMP signal was continuously sampled and the excitation laser was constantly on at approximately 0.05 mW measured at the tip of the patch cord, outside of the animal. Because unexpected outcomes came at a low frequency, only ~10% of all trials included the stimuli of interest. Each animal received each set of stimuli twice, and their responses from both days were averaged to determine that animal’s response to each stimulus.

Freely moving choice paradigm with stimulation

The behavioral apparatus (approximately 350 mm by 225 mm, a similar size as the animals’ home cages) is modified from those used in our previous experiments^59,60. Mice were first trained to drink a water reward at either side port. The next day, mice were trained to enter a central port to release water at both side ports and then trained to enter a central port and choose a side port to release water at that port. The third day, odors were introduced to instruct the mouse to choose either the left or right port (forced choice). After 5–10 days of training, mice performed this task with >90% accuracy.

After this training period, another odor was introduced to instruct mice to choose freely between the two reward ports (free choice). Each test session consists of 30 trials of forced choice followed by 100 free choice trials. We manipulated the value of one port per session (left or right, randomly) by adding a stimulus (X) to the typical water reward. These stimuli were physical (i.e. extra 5uL water, ~3 atm air puff, 0.001 M quinine) or optogenetic (i.e. VS stimulation or TS stimulation). Outcomes are applied immediately after mouse’s entry to water ports. Because mice tended to have a bias toward left or right turns, we randomly assigned each stimulus to each direction so that mice would encounter each stimulus as a “left choice” or a “right choice” at least twice over the course of the experiment. Minimum inter trial interval (ITI) was set to 5 seconds after delivery of outcome so that mice could not start the next trial within this period. We defined choice (%) as the percentage of the number of times the mouse chose the port in the total number of choice. Therefore, if the mouse initiated a trial by entering the middle port, but made no selection, such a trial would not impact their choice bias. We averaged each animal’s choice across sessions to find the “average choice” caused by the stimulus, regardless of direction.

For optogenetic stimulation, we delivered 10 pulses of 10 ms in duration, with 40 ms between pulses (i.e. 20 Hz stimulation for 500 ms). Stimulation was performed with a 473 nm laser at 20 mW (measured at the laser) and was controlled by a physical shutter rather than power modulation to increase precision. The optic fibers used were 200 μm thick (Doric) and transmitted light with >75% efficiency when tested prior to implant. Based on the width, numerical aperture of the fiber, and laser power, we estimate that, the light power is 80 mW/mm² at the tip of fiber. At lower depths, we estimate that 16 mW/mm² reaches a brain area 0.2 mm below the tip, that 4 mW/mm² of light reaches a brain area 0.5 mm below the tip, and that 0.75 mW/mm² reach a brain area 1 mm below the tip of fiber⁶¹.

Based on a more conservative estimate of light travel through biological tissue, a few percent of the light could reach 1.0 mm below the site of stimulation⁶². Because our fiber implants ranged between ~1.0 mm above the dorsal tip of the amygdala and ~2.5 mm above the dorsal tip of the amygdala (Supplemental Figure 7), and our ChR2 infection did not strongly label dopamine axons in the amygdala (for example: see Figure 3), we believe that the effects of stimulation we observed were not due to stimulation of dopamine axons in the amygdala. Furthermore, there was a negative correlation between effect size and fiber depth (Supplemental Figure 7), suggesting that the larger effects were observed in cases where the optic fiber was positioned directly above TS rather than at lower depths nearer the amygdala.

Freely moving choice paradigm with drug administration

To infuse D1 dopamine receptor antagonist (R(+)-SCH-23390 hydrochloride, Sigma, D054, 5 mg/mL)⁶³ or saline, we first head-fixed mice and then used a syringe pump (World Precision Instruments, Syringe Pump SP200I) to inject 200 nL into each side of the brain at 0.35 μL/minute. Following each injection, needles were left in the brain for 2 minutes to allow for diffusion away from the tip. After both injections, mice were released from head-fix and placed in the box to perform the choice task. Each day, each mouse was given either saline or drug at random, and then drug days were compared to saline days using a paired t-test to compare each animals’ choice behavior and retreat responses between saline and drug conditions. We analyzed the first 60 trials performed by the mice in each session to prevent off-target effects of the drug.

Freely moving choice paradigm with 6-OHDA lesion or GCaMP recording

For recording experiments and 6-OHDA lesion experiments, the same initial training method was used as in the stimulation experiments. After initial training, they were trained by only one odor signaling trial initiation. Surgery was performed after the training. The experimenter was blind to the identities (control or lesion) of mice. For the tests, all mice received sessions including air puff first, to reduce history-dependent differences in behavior. Then, they were given control, water omission, control, quinine, and control sessions in that order. This experimental design optimized our ability to measure air puff choice biases and retreat responses at the cost of potentially influencing later sessions based on history.

In this task, water reward was delivered 500 ms after the mouse’s entry to a water port (so with a ‘water delay’ of 0.5 s). Air puff was applied immediately after port entry during the water delay so that air puff does not directly prevent or delay the mouse from drinking water. In the case of water reduction, we changed the size of water reward at one port from 10 uL to 5 uL. To observe time-course of the behaviors within a session, forced choice trials were eliminated in this task. Instead, negative outcomes were applied to animal’s preferred port to ensure that they experienced them early in the session. We defined normalized choice (%) as the percentage of the number of times the mouse chose the port in the total number of choice minus general choice bias (the percentage of the number of times the mouse chose the same port on the control day without outcome manipulation minus 50). We measured “retreat distances” using the total distance traveled (based on the center of mass of the mouse, extracted from video) in the 1 second period following puff.

Novel object interaction

Mice were first habituated to a box of roughly the same dimensions as their home cage with white floor, white walls, and no objects. This habituation was done for 30 minutes a day for 5–10 days. Mice with fiber implants were connected to the recording cable during these sessions so that they would become habituated to freely moving while recordings took place. The recording cable was hung from the ceiling such that it would be able to reach all corners of the cage with minimum weight on the animal. General locomotor activity was measured after these habituation sessions.

After these habituation sessions, a novel object (a Lego) was placed in the center of the box and we measured the animals’ interactions with the object in one 10-minute session every day for one week. We compared GCaMP responses on the first day to the last day to compare responses during interactions with a “novel” object or a “familiar” object. We compared these signals to signals observed when the mouse was freely moving in the same box with no object in the center, and happened to move from the center to the periphery.

Familiar object interaction with stimulation

After habituation to a novel object (30 minutes a day for 5–10 days), we termed it a “familiar” object. We placed mice into a behavioral arena with a familiar object and recorded behavior for 15 minutes as a baseline. Then, during the next 15 minute block, we performed closed-loop stimulation of dopamine axons in TS anytime mice were within a 50 mm radius of the object. As in the choice task, light was delivered in 10 pulses of 10 ms in duration, with 40 ms between pulses (i.e. 20 Hz stimulation for 500 ms). Stimulation was performed with a 473 nm laser at 20 mW. This stimulation was repeated until mice left the imaginary radius around the object. After this 15 minute block, we stopped stimulating when mice interacted with the object, in order to observe whether any changes in behavior persisted.

GCaMP detection and analysis^{13,53,64–67}

Fiber photometry allows for recording of the activity of genetically defined neural populations in mice by expressing a genetically encoded calcium indicator and chronically implanting an optic fiber. The optic fiber (200 μm diameter, Doric Lenses) allows chronic, stable, minimally disruptive access to deep brain regions and interfaces with a flexible patch cord (Doric Lenses) on the skull surface to simultaneously deliver excitation light (473 nm, Laserglow Technologies) and collect GCaMP^53,67 or GFP fluorescence emission. Activity-dependent fluorescence emitted by cells in the vicinity of the implanted fiber’s tip was spectrally separated from the excitation light using a dichroic, passed through a single band filter, and focused onto a photodetector connected to a current preamplifier (SR570, Stanford Research Systems).

During recording, optic fibers were connected to patch cables which delivered excitation light (473 nm) and collected all emitted light. The emitted light was subsequently filtered using a 556nm beam-splitter followed by a 500 ± 20nm bandpass filter and collected by a photodetector (FDS10×10 silicone photodiode, Thorlabs) connected to a current preamplifier (SR570, Stanford Research Systems). This preamplifier output a voltage signal which was collected by a NIDAQ board (National Instruments). The NIDAQ board was connected to the same computer that was used to control odor, water, tone, and air puff delivery with Labview (National Instruments), so GCaMP signals could be readily aligned to task events such as reward delivery or tone delivery.

GCaMP signals or GFP control signals were collected as voltage measurements from current preamplifiers. The ‘dF/F’ measurement was calculated using a running median of 100 seconds, such that the voltage measurement at any point in time (‘Fa’) was subtracted from the running median voltage (‘Fb’) and then divided by the running median voltage to find dF/F = (Fa – Fb) / (Fb). To quantify responses, we looked for “peak responses” (negative or positive) by finding the point with the maximum absolute value during the one second window following the stimulus onset in each trial. To compare the response and baseline, baseline peak was obtained during 3 – 4 second before the stimulus onset.

We used this measurement because it normalized signals (i.e. in the case of low signal to noise ratio, the denominator would be larger), allowing for comparison across animals with different baseline fluorescence. Additionally, this normalization corrected for the small amount of bleaching we observed over the course of the trial in the raw fluorescence (Supplemental Figure 2). The average responses to a stimulus type were averaged within each animal, and these averages were used as the data in each figure. GCaMP signals or GFP control signals were collected through Labview during the training for offline analysis.

Video detection and analysis

We used infra-red (IR) light to illuminate the arena, and recorded video using a camera (BFLY-U3–03S2M, Point Grey Research Backfly) at 60 frames per second (fps) with H.264 video compression. The video was captured using the FlyCap2 software which accompanies the camera, and videos were processed in MATLAB (Mathworks, Natick, MA, USA). Briefly, the videos were thresholded such that the outline of the mouse could be identified, and then the center of mass, nose position, and minimum distance from the novel object were extracted for further analysis. To quantify movement velocity, we took the derivative of the position of the center of mass. Therefore, this measurement was sensitive only to large movements such as running and not sensitive to small movements such as grooming.

In the choice task, we extracted retreats from the reward port by finding the distance traveled (using the center of mass) in the 1 second period following air puff. During novel object interaction, we defined “bouts” of investigation by measuring when the mouse entered a 50 mm radius of the object. The beginning of the “approach” was defined by when the animal began moving toward the object (first increase of velocity relative to the object directly preceding crossing into 50 mm radius), regardless of the animal’s distance from the object at that time. The beginning of “retreat” was defined by when the animal began moving away from the object (first point of negative velocity relative to object after crossing into the 50 mm radius), also regardless of the animal’s distance from the object at that time. The “bout size” (duration of the bout) was determined by how long the animal remained within 50 mm of the object.

We compared the number of such bouts that animals made within the first 10-minute session of novel object investigation (Figure 6). To determine the fraction of time spent near the object, we simply measured how much of the 10-minute session each animal spent within 50 mm of the object. For this and other measurements, we required only that the mouse’s nose cross within 50 mm of the object, and not the center of mass. Because mice investigate by extending their nose toward a stimulus, their center of mass was often much further from the object than their nose.

Histology and fluorescence analysis

Animals were perfused using 4% paraformaldehyde and then brains were sliced into 100 μm thick coronal sections using a vibratome and stored in PBS. These slices were then stained with rabbit anti-tyrosine hydroxylase (TH) antibody (AB152, EMD Millipore) at 4 ºC for 2 days to reveal dopamine axons in the striatum, dopamine cell bodies in the midbrain, and other neurons expressing TH throughout the brain (such as noradrenaline neurons in the LC). Slices were then stained with fluorescent secondary antibodies and fluorescent Nissl at 4 ºC for 1 day. Slices were then mounted in anti-fade solution (VECTASHIELD anti-fade mounting medium, Vector Laboratories, H-1000) and imaged using a Zeiss LSM 700 inverted confocal microscope. The same laser settings and gain were used for all samples so that fluorescent signals could be compared across animals. Brightness was adjusted in ImageJ for figures.

To quantitatively compare across brains, we used two methods. 1) For cell body comparison, we manually counted the number of labelled cell bodies in the VTA, SNC, or SNL in each optical section. The boundaries were drawn manually for each section. 2) For axon density comparison, we first defined areas for analysis using anatomical landmarks visible by autofluorescence or fluorescent Nissl. Then, we measured the average fluorescence intensity of rabies-GFP or anti-TH antibody in each region. We divided this by the average fluorescence intensity from an unlabeled brain (in the case of rabies-GFP) or from an unstained slice of the same brain (in the case of anti-TH antibody staining).

Randomization, blinding, and data exclusion

Randomization:

For the GCaMP recording experiments, GFP control animals and GCaMP experimental animals were selected at random by the experimenter. Trial structure was pseudo-random (trials were randomly shuffled in blocks of 200 to ensure that all trial types would be presented in each session). For the 6OHDA lesion experiments, another lab member randomly selected mice to be in either Saline or 6OHDA groups. The experimenter did not know the identity of each animal until analysis was complete. For the D1 antagonist experiments, saline and 6OHDA sessions were done in a random order (randomized separately for each mouse).

Blinding:

For the 6OHDA lesion studies, the experimenter was blind to the animals’ identities (control or lesion) during data collection and analysis. The identities of the animals were revealed to the experimenter only after analysis was complete. For the D1 antagonist experiments, the experimenter was not blinded, and knew which solution was being infused (i.e. saline or D1 antagonist) at the time of the experiment.

Data exclusion:

No animals were excluded from the study: all analysis includes data from all animals. However, after applying D1 receptor antagonist, we limited our analysis to the first 60 trials performed by the mice in each session to prevent off-target effects of the drug, due to potential spread from injection site.

Statistical analyses

Data analysis was performed using custom software written in MATLAB (Mathworks, Natick, MA, USA). Any code used for analysis is available on request. All statistical tests were two-sided. For statistical comparisons of the mean, we used 1-way analysis of variance (ANOVA) and the two-sample student’s t-test, unless otherwise noted. Paired t-tests were conducted only when the same mouse’s performance was being compared across multiple drug conditions (Figure 4e-f) or different sessions (Figure 6, novel object versus familiar object). The significance level was corrected for multiple comparisons using a Holm-Sidak method unless otherwise indicated. All error bars in the figures are the standard error of the mean (s.e.m.).

No statistical methods were used to pre-determine sample sizes, but our sample sizes are similar to those reported in previous publications. To be conservative, in most cases, we compared across animals rather than across trials or across sessions. Data distribution was assumed to be normal, but this was not formally tested.

Data availability and code availability

The data that support the findings of this study are available from the corresponding author upon reasonable request. Similarly, any code used for analysis (Matlab scripts) are also available from the corresponding author upon reasonable request. More information regarding the statistical tests and methods is also available online in the Life Sciences Reporting Summary.