Dopamine dependency for acquisition and performance of Pavlovian conditioned response

Significance

Dopaminergic signaling is critical for many reward-related processes, including learning associations between stimuli and rewards (“Pavlovian learning”). Behavioral responses to reward-associated stimuli have been described as being initially directed either at the stimulus itself (“sign tracking”) or at the location of reward delivery (“goal tracking”). Although other studies have demonstrated that dopaminergic signaling is crucial for sign tracking, its role for goal-tracking behavior is less clear. Our approach uses genetic manipulations that allow determining whether dopamine is required for learning goal tracking and also which dopaminergic projections are most relevant for this behavior. Moreover, we report that instrumental learning can facilitate Pavlovian learning in mice that are impaired in Pavlovian learning but not in instrumental learning.

Abstract

During Pavlovian conditioning, pairing of a neutral conditioned stimulus (CS) with a reward leads to conditioned reward-approach responses (CRs) that are elicited by presentation of the CS. CR behaviors can be sign tracking, in which animals engage the CS, or goal tracking, in which animals go to the reward location. We investigated CR behaviors in mice with only ∼5% of normal dopamine in the striatum using a Pavlovian conditioning paradigm. These mice had severely impaired acquisition of the CR, which was ameliorated by pharmacological restoration of dopamine synthesis with l-dopa. Surprisingly, after they had learned the CR, its expression decayed only gradually in following sessions that were conducted without l-dopa treatment. To assess specific contributions of dopamine signaling in the dorsal or ventral striatum, we performed virus-mediated restoration of dopamine synthesis in completely dopamine-deficient (DD) mice. Mice with dopamine signaling only in the dorsal striatum did not acquire a CR, whereas mice with dopamine signaling only in in the ventral striatum acquired a CR. The CR in mice with dopamine signaling only in the dorsal striatum was restored by subjecting the mice to instrumental training in which they had to interact with the CS to obtain rewards. We conclude that dopamine is essential for learning and performance of CR behavior that is predominantly goal tracking. Furthermore, although dopamine signaling in the ventral striatum is sufficient to support a CR, dopamine signaling only in the dorsal striatum can also support a CR under certain circumstances.

It is widely accepted that dopamine signaling contributes strongly to reward-related processes (1, 2). Electrophysiological studies have established that midbrain dopamine neurons transiently increase their firing rate following unconditioned primary rewarding events, and electrochemical studies have confirmed that this increase in activity translates to dopamine release in projection fields of dopamine neurons in the ventral striatum (3, 4). Furthermore, when an unconditioned stimulus (US) like a food reward is paired with a conditioned stimulus (CS) during Pavlovian conditioning, the CS becomes predictive of the rewarding event and elicits increased phasic activity of dopamine neurons and dopamine release (5). Support for the hypothesis that dopamine release is critical for Pavlovian conditioning comes from studies using either local antagonism of dopamine receptors in the ventral striatum or ablation of dopamine projections to this area (6, 7). As an animal acquires the association between the CS and US, it begins to manifest a conditioned reward-approach response (CR) to the CS. Responses to the CS fall into two categories: sign tracking and goal tracking (8, 9). For sign-tracking CRs, the CS itself becomes attractive and elicits an approach toward it, whereas for goal-tracking CRs, the CS solely evokes an approach to the location of expected reward delivery (10). Systemic antagonism of dopamine receptors has been reported to impair learning of a sign-tracking CR but not of a goal-tracking CR (11). However, the same study also reported that overall motor activity and performance of sign tracking and goal tracking was reduced by dopamine receptor blockade (11). It has also been reported that local blockade of dopamine receptors in the ventral striatum severely impaired performance of sign-tracking CRs, although leaving goal-tracking CRs intact (2). Local blockade of dopamine receptors in the ventral striatum leaves open the possibility that dopamine signaling in other brain regions mediates goal-tracking CRs. These areas include the prefrontal cortex, amygdala, hippocampus, and medial part of the dorsal striatum, all of which receive dopamine innervation originating in the ventral tegmental area (VTA) of the midbrain, and the dorsal striatum, which receives most of its dopamine innervation from the substantia nigra (SN) (12, 13).

We have developed two different strains of mice that allow different approaches to assess the contributions of dopamine to behavior. The first strain lacks tyrosine hydroxylase (TH) selectively in dopamine transporter-expressing neurons, which project mainly to the striatum and to a lesser extent to other brain areas (14). These mice have severe but incomplete reduction of striatal dopamine; they have largely intact locomotion and are sufficiently motivated to eat and drink on their own (15). The second strain lacks TH expression in all dopamine neurons and will not perform tasks that require intentional movement; hence, these mice will starve without daily l-dopa injections (16, 17). These dopamine-deficient (DD) mice have conditionally inactivated Th alleles that can be reactivated by the action of Cre recombinase. Injection of a retrograde canine adenovirus-expressing Cre recombinase (CAV2-Cre) can restore dopamine signaling to the brain region that was injected with virus and enables these virally rescued DD (vrDD) mice to perform a variety of behaviors (15, 17–21). This technique allows us to explore where dopamine signaling is sufficient to support behavior that the DD mice would not normally perform.

We subjected both WT mice and mice with genetic manipulations of dopamine signaling to a Pavlovian conditioning procedure and measured their CR behaviors. We address the following questions: (i) Are mice sign trackers or goal trackers, and is dopamine required for either behavior; (ii) where in the brain is dopamine sufficient for learning a CR; (iii) can instrumental conditioning, in which a mouse must interact with the CS, facilitate Pavlovian conditioning; and (iv) is dopamine required for both learning and performance of a CR?

Results

Animal Models.

We tested two different mouse models, one in which the Th gene was inactivated in all neurons expressing the dopamine transporter (Slc6a3^Cre/+:Th KO mice; Fig. 1A), and another with inactivation of the Th gene in all dopamine neurons (Th^fs/fs:Dbh^Th/+ DD mice) followed by viral-mediated restoration of dopamine synthesis in specific brain regions (vrDD mice; Fig. 1B). Restoration of dopamine synthesis in vrDD mice was achieved by injection of CAV2-Cre virus into either the dorsal striatum (vrDD-DS mice) or ventral striatum (vrDD-VS mice). Both animal models survive without further interventions and are capable of engaging in various behavior tests (15, 19–21).

Fig. 1.

Models for dopamine deficiency and dopamine signaling restricted to striatal subregions. (A) Schematic representation of Slc6a3^Cre/+:Th KO mice, in which Cre recombinase results in functional disruption of the Th allele in neurons with expression of the Slc6a3 gene, which encodes the dopamine transporter. (B) Strategy to restore dopamine synthesis in DD mice. TH expression is restored to either the dorsal or ventral striatum by injection of CAV2-Cre into either region, which removes to stop cassette from the Th allele. (C) Dopamine levels in the dorsal and ventral striatum of Slc6a3^Cre/+:Th KO (n = 6) and WT mice (n = 6). (D) Dopamine levels in the dorsal and ventral striatum of vrDD-DS (n = 7) mice, vrDD-VS (n = 6) mice, and sham control mice (n = 6). Data are expressed as mean ± SEM. **P < 0.01.

Striatal dopamine levels in Slc6a3^Cre/+:Th KO mice were dramatically different from those in control mice (ANOVA, genotype: P < 0.01; Fig. 1C); in the dorsal striatum, the dopamine content was 4% of WT levels, whereas it was 6% of WT levels in the ventral striatum. Immunohistochemistry revealed a few TH-positive neurons in both the SN and VTA of these mice, indicating either that the dopamine transporter is not expressed in those neurons or that Cre recombinase was insufficiently active in those dopamine neurons to inactivate the conditional Th allele.

Whereas DD mice have almost no striatal dopamine (22), dopamine was restored in the striatum of vrDD-DS mice and vrDD-VS mice to 30–40% of WT levels in the CAV2-Cre–transduced brain region and to 5–10% of WT levels in the nontransduced areas (ANOVA, group: P < 0.01; Fig. 1D). Immunohistochemistry revealed that TH was predominantly restored in the dorsal striatum of vrDD-DS mice and in the ventral striatum (including the nucleus accumbens core and shell and the olfactory tubercle) of vrDD-VS mice (Fig. S1 A–C). Due to the extensive arborization of dopamine neurons (23), individually transduced neurons can restore dopamine signaling to a broad striatal area, resulting in some overlap between dorsal and ventral regions. These results indicate that we achieved partial restoration of TH expression, and hence dopamine signaling, in vrDD mice that was greater in the dorsal striatum of vrDD-DS mice and greater in the ventral striatum of vrDD-VS mice.

Predominance of Goal-Tracking Behavior in Our Pavlovian Conditioning Paradigm.

To determine whether the mice we tested are sign trackers or goal trackers, we trained a group of C57BL/6 mice for 5 d and videotaped their approach behaviors during the CS presentations. Goal-tracking CRs were much more prevalent than sign-tracking CRs during the 5-d training period (ANOVA, group: P < 0.01; Fig. S2). At all stages during training the mice made more movements towards the goal than towards the lever cue and by the fifth day virtually all of their movements were towards the goal (∼70%) compared to the lever (<5%). The resolution of our analysis did not allow us to determine whether they looked in the direction of the lever when it was extended. We conclude that the C57BL/6 mice we tested are primarily goal trackers, and we therefore focused our main analysis on this specific CR.

Dopamine Is Required to Learn a Pavlovian CR.

In the basic Pavlovian conditioning paradigm that we have used in several studies (24–26), mice receive 25 trials a day for 5 d in which the CS is a lever that is extended (visual and motor sound cues) for 10 s in an operant box with delivery of a US food reward at the end of the 10-s cue; there is a variable 60-s intertrial interval (ITI). The head-entry (HE) rate into the food hopper during the cue presentation minus the HE rate during the ITI is calculated as the CR. The male C57BL/6 mice that we use display fewer than five HEs per minute into the food hopper during an average 10-s portion of the ITI and increase their HE rate to ∼20 per minute during the 10-s cue presentation (26) within 4–5 d of training, resulting in a CR score of ∼15 (i.e., after training, they are approximately fourfold more likely to nose-poke for food when the cue is presented).

To begin to assess how much dopamine is required to learn a Pavlovian CR, we compared the performance of the severely DD mice (Slc6a3^Cre/+:Th KO), which have about 5% of normal dopamine in the striatum, with that of WT mice for 5 training days; this training is equivalent to the basic conditioning paradigm and is represented by days 1–5 in the schematic (Fig. 2A). Whereas the WT mice increased the number of conditioned HEs during the 5 d of Pavlovian conditioning, the Slc6a3^Cre/+:Th KO mice had no improvement at all (ANOVA, genotype: P < 0.01; time: P < 0.01; interaction: P < 0.01; Fig. 2B). Slc6a3^Cre/+:Th KO mice had significantly fewer conditioned HEs than WT mice on training days 3–5 (P < 0.01 each). We conclude either that 5% of normal striatal dopamine in the Slc6a3^Cre/+:Th KO mice is not able to support learning or that 5% of normal striatal dopamine is not able to support performance of this Pavlovian task, even though it is enough dopamine for normal feeding and daytime locomotion. Importantly, food rewards were consumed by Slc6a3^Cre/+:Th KO mice to the same extent as control mice. It is also worth noting that Slc6a3^Cre/+:Th KO mice have essentially normal levels of dopamine in other brain regions, including the prefrontal cortex (15). Raw data of HEs during training days 1–5 are shown in SI Materials and Methods (Fig. S3 A and B).

Fig. 2.

Pavlovian CR learning in Slc6a3^Cre/+:Th KO, vrDD-DS, and vrDD-VS mice. (A) We used two experimental procedures: one in which mice received one 5-d block of Pavlovian conditioning followed by 5 d of instrumental training, after which they received a second 5-d block of Pavlovian conditioning (Upper, Inst. Training group), and one in which mice received 15 d of uninterrupted Pavlovian conditioning (Lower, Ext. Training group). (B) Conditioned HE rates during days 1–5 of Pavlovian training by Slc6a3^Cre/+:Th KO mice (n = 21) and WT mice (n = 15). Mice from the Ext. Training and Inst. Training groups are shown together. (C) Conditioned HE rates during days 1–5 of Pavlovian training by vrDD-DS mice (n = 13), vrDD-VS mice (n = 7), and sham control mice (n = 14). Mice from the Ext. Training and Inst. Training groups are shown together. (D) Conditioned HE rates by the Inst. Training groups of Slc6a3^Cre/+:Th KO (n = 10) and WT (n = 6) mice and by the Ext. Training groups of Slc6a3^Cre/+:Th KO mice (n = 11) and WT mice at days 11–15. (E) Conditioned HE rates by the Inst. Training groups of vrDD-DS mice (n = 7) and sham control mice (n = 8) and by the Ext. Training groups of vrDD-DS mice (n = 6) and sham control mice (n = 6) at days 11–15. Data are expressed as mean ± SEM. *P < 0.05; **P < 0.01.

Dopamine in the Ventral Striatum Is Sufficient to Learn a Pavlovian CR.

We used viral rescue in the completely DD mice to ask where dopamine signaling in the striatum is sufficient to learn a Pavlovian CR. These mice have no dopamine signaling anywhere in the brain except in the brain region transduced by the CAV2-Cre virus. We tested two vrDD groups (vrDD-DS and vrDD-VS mice) and their sham controls in the basic conditioning paradigm to identify brain regions where dopamine signaling is sufficient to learn a Pavlovian association (Fig. 2C). Whereas sham control and vrDD-VS mice improved their conditioned HEs during the 5 d of Pavlovian conditioning, vrDD-DS mice did not increase their HE rate (ANOVA, group: P < 0.01; time P < 0.01; interaction: P < 0.01). HEs by vrDD-DS mice were significantly reduced on training days 2–5 (P < 0.05 each) compared with sham control mice and significantly reduced on training days 3–5 (P < 0.05) compared with vrDD-VS mice. Only on days 4–5 did vrDD-VS mice have fewer conditioned HEs than sham control mice (P < 0.05). The vrDD-VS mice have ∼30% of normal dopamine in the ventral striatum and ∼5% of normal dopamine in the dorsal striatum (Fig. 1) but essentially no dopamine in any other brain region (16, 27). Thus, at least 30% of normal dopamine in the ventral striatum is sufficient for mice to learn the association of the lever cue with delivery of a food reward, but it is not enough to restore learning fully to sham control levels. In addition, 40% of normal dopamine in the dorsal striatum is not sufficient for learning this association. Raw data of HEs during training days 1–5 are shown in SI Materials and Methods (Fig. S3 C and D).

Instrumental Training Does Not Improve the Pavlovian Performance of Slc6a3^Cre/+:Th KO Mice.

After establishing the necessity and sufficiency of dopamine signaling in the ventral striatum for learning a Pavlovian CR, we investigated if dopamine-related deficits in Pavlovian learning could be improved by instrumental training. Our idea was to give the cue more salience by requiring that the mice press the lever to deliver the food reward. One set of Slc6a3^Cre/+:Th KO and WT mice [“Instrumental (Inst.) Training” group] were first subjected to standard Pavlovian conditioning sessions for 5 d (block 1) as before; they then underwent 5 d of instrumental training; and, finally, they received 5 more days (block 2) of Pavlovian conditioning after that. Another set of mice [“Extensive (Ext.) Training” group] received 15 d of uninterrupted Pavlovian training (block 1, extra block, and block 2). This experimental design is illustrated in the schematic in Fig. 2A. Thus, all mice received a total of 15 training sessions, with one group having instrumental training in the middle session, whereas the control group had continued Pavlovian training. We reasoned that if the Slc6a3^Cre/+:Th KO mice learned that pressing the lever delivered food rewards during days 6–10, the lever might then attain salience and enhance their performance during block 2 of Pavlovian training. Because we hypothesized that the mice would learn to press the lever during the instrumental session, it was necessary to prevent them from pressing the lever during block 2 Pavlovian sessions; this was accomplished by covering the lever so that the mice had to rely on the sound of the lever extension motor as their cue.

Performance of the Pavlovian task by these mice during block 1 training is shown in Fig. 2B. During days 6–10 of instrumental training, Slc6a3^Cre/+:Th KO and WT mice earned the same number of rewards. However, although both groups increased their lever-press rate during instrumental training, Slc6a3^Cre/+:Th KO mice did not perform as well as WT mice (ANOVA, genotype: P < 0.01; time: P < 0.01; Fig. S4A). Lever-press rates by KO mice were significantly reduced on days 7–10 compared with WT mice (P < 0.05). Throughout the extended Pavlovian conditioning sessions on days 6–10 (Ext. Training group), the Slc6a3^Cre/+:Th KO mice continued to have blunted conditioned HEs and the WT mice maintained their HEs at a level similar to the last training day of the first Pavlovian conditioning block (ANOVA, genotype: P < 0.01; Fig. S4B). The number of HEs by the KO mice was fewer than that by the WT mice throughout days 6–10 (P < 0.01).

During block 2 of Pavlovian conditioning, conditioned HEs by Slc6a3^Cre/+: Th KO mice and WT mice of the Inst. Training and Ext. Training groups differed significantly and showed no change over time (ANOVA, group: P < 0.01; Fig. 2D). Throughout days 11–15 of block 2, conditioned HEs by all groups of Slc6a3^Cre/+:Th KO mice were significantly fewer in number than those of their respective WT comparison groups (P < 0.05). Thus, instrumental training did not help Slc6a3^Cre/+:Th KO mice learn the Pavlovian paradigm.

Mice with Dopamine Signaling Restricted to Dorsal Striatum Can Learn a Pavlovian CR After Instrumental Training.

The previous experiment revealed that instrumental training did not help mice with low dopamine signaling in both the dorsal and ventral striatum (the Slc6a3^Cre/+:Th KO mice) learn a Pavlovian CR. We then asked whether the same instrumental intervention would allow vrDD-DS mice to learn the Pavlovian association using the same training program (Fig. 2A).

During block 1 of Pavlovian training, all vrDD-DS mice performed poorly, as shown before in Fig. 2C. During the instrumental training sessions (days 6–10), both vrDD-DS and sham control mice from the Inst. Training group earned the same number of rewards. Although both groups increased their lever-press rate during instrumental training, vrDD-DS mice did not perform as well as sham control mice (ANOVA, group: P < 0.01; time: P < 0.01; interaction: P < 0.01; Fig. S5A). Lever-press rates of vrDD-DS mice were significantly reduced on days 7–10 (P < 0.05). Throughout the extended Pavlovian conditioning sessions on days 6–10 (Ext. Training group), vrDD-DS mice still had blunted conditioned HEs and sham control mice maintained their HEs at a level similar to the last training day of the first Pavlovian conditioning block (ANOVA, group: P < 0.01; Fig. S5B). HEs by vrDD-DS mice were fewer in number than HEs by sham control mice throughout days 6–10 (P < 0.01).

During the second Pavlovian conditioning session (block 2), conditioned HEs by the vrDD-DS mice in the Inst. Training group increased significantly compared with the Ext. Training group (ANOVA, group: P < 0.01) and approached the HE rate of the sham controls (Fig. 2E). Thus, the instrumental training allowed the vrDD-DS group of mice to perform at almost WT rates during the second Pavlovian session, whereas the group of vrDD-DS mice that received a total of 15 Pavlovian sessions still performed poorly, with a maximum CR of only five compared with >15 for vrDD-DS mice that received instrumental training or the sham controls (Fig. 2E). We conclude that the instrumental intervention restored the ability of the vrDD-DS mice to learn the Pavlovian association compared with mice that only had extensive Pavlovian conditioning. All animals, including those that could not learn the Pavlovian task, consumed all reward pellets that were delivered during conditioning sessions.

Further Analysis of Sign- and Goal-Tracking Behaviors.

To determine whether the mice that we tested in the paradigm above were sign trackers or goal trackers, we measured the number of actual lever presses (another indication of sign tracking) by all mice during block 1 of Pavlovian conditioning and by mice in the Ext. Training groups during block 2. The mice used in our study rarely produced lever presses (<0.05/min) that the apparatus could record (Fig. S6 A and B). Importantly, hiding the lever from mice of the Inst. Training groups during block 2 of the Pavlovian training did not have an impact on their CR behavior. We conclude that the C57BL/6 mice we tested are primarily goal trackers.

Dopamine Is Required for Both Learning and Maintenance of a Pavlovian CR.

To assess whether dopamine is required to learn or perform a Pavlovian CR, we took advantage of the ability to restore dopamine signaling by injection of l-dopa, which is taken up by dopamine neurons and converted to dopamine by l-amino acid decarboxylase, resulting in a bout of hyperactivity that lasts for several hours (16, 17). Thus, we trained Slc6a3^Cre/+:Th KO mice with l-dopa injections before each session and then tested them without l-dopa injections to determine whether they could maintain their CR performance without continued dopamine signaling (Fig. 3A). The Slc6a3^Cre/+:Th KO and WT mice that received l-dopa during the first 7 d of Pavlovian conditioning improved their conditioned HEs equally well over time (ANOVA, time: P < 0.01; Fig. 3B). When the same mice were then trained for an additional 3 d with saline instead of l-dopa, conditioned HEs diverged between the genotypes (ANOVA, genotype: P < 0.05; time: P < 0.05; Fig. 3C), resulting in significantly reduced HEs by saline-treated Slc6a3^Cre/+:Th KO mice on day 10 compared with WT mice (P < 0.05). This result indicates that dopamine is required to maintain a Pavlovian CR after it has been learned. Interestingly, the withdrawal of l-dopa treatment from the Slc6a3^Cre/+:Th KO mice resulted in a gradual decline of conditioned HEs over several days. The slow decline in performance is not due to residual dopamine, because the dopamine synthesized from an l-dopa injection is all metabolized within 9 h (27).

Fig. 3.

Pavlovian CR learning in l-dopa– and saline-treated Slc6a3^Cre/+:Th KO mice. (A) We used two experimental procedures. Animals received either a 7-d block of Pavlovian conditioning with l-dopa treatment followed by 3 d of Pavlovian conditioning with saline treatment (Upper) or a 10-d block of Pavlovian conditioning with saline treatment followed by 2 d of Pavlovian conditioning with l-dopa treatment (Lower). (B) Effects of l-dopa on conditioned HE rates during 7 d of Pavlovian training by Slc6a3^Cre/+:Th KO mice (n = 8) and WT mice (n = 8). (C) Conditioned HE rates during 3 d of Pavlovian training with saline treatment by the same Slc6a3^Cre/+:Th KO mice (n = 8) and WT mice (n = 8) that were previously trained with l-dopa. (D) Effects of saline on conditioned HE rates during 10 d of Pavlovian training by Slc6a3^Cre/+:Th KO mice (n = 5) and WT mice (n = 5). (E) Conditioned HE rates during 2 d of Pavlovian training with l-dopa treatment by the same Slc6a3^Cre/+:Th KO (n = 5) and WT mice (n = 5) that were previously trained with saline. Data are expressed as mean ± SEM. *P < 0.05; **P < 0.01.

As a control for the experiment described above, we trained a new cohort of WT and Slc6a3^Cre/+:Th KO mice for 10 d with saline injections. As expected, saline-treated WT mice performed better than Slc6a3^Cre/+:Th KO mice, which did not develop a CR. The control mice increased their number of conditioned HEs during 10 d of Pavlovian conditioning, but the saline-treated Slc6a3^Cre/+:Th KO mice had no improvement at all (ANOVA, genotype: P < 0.01; time: P < 0.01; interaction: P < 0.01; Fig. 3D). Saline-treated KO mice had significantly fewer conditioned HEs than saline-treated WT mice on training days 5–10 (P < 0.01 each). When these mice were then tested for 2 d with l-dopa, their performance did not change (ANOVA, genotype: P < 0.01; Fig. 3E); WT mice continued to perform well, whereas Slc6a3^Cre/+:Th KO mice showed no improvement on days 11 and 12 and were significantly different from the WT controls (P < 0.01). If the Slc6a3^Cre/+:Th KO mice had learned during the 10 d of training but were unable to perform, they should then have performed well when dopamine signaling was restored. From these two experiments, we conclude that dopamine levels greater than the 5% level observed in the Slc6a3^Cre/+:Th KO mice are required for both learning and maintenance of CR behavior.

Discussion

We evaluated the contributions of dopamine to learning an association between a cue and a food reward (CS-US association) using a paradigm that resembles Pavlovian conditioning. In classical Pavlovian conditioning, the CS was a tone and the US was salivation, an autonomic response, whereas in our paradigm, the US requires a motivated approach to the food receptacle. DD mice (without viral rescue) do not perform in our paradigm because they are not motivated to eat and will starve without daily l-dopa injections (27). The Slc6a3^Cre/+:Th KO mice with ∼5% of normal dopamine in the striatum, which is sufficient for adequate eating and locomotion, were unable to learn our Pavlovian paradigm; moreover, they could not maintain CR behavior that they had learned by training them with l-dopa injections before each session. Mice with 30% of normal dopamine in the ventral striatum were able to learn a CR, but that level of dopamine in the dorsal striatum is insufficient to support learning a CR. We conclude that 30% of normal dopamine in the ventral striatum is sufficient to learn and maintain a CR in our Pavlovian paradigm.

The CR that we observed using our Pavlovian conditioning paradigm was primarily goal-tracking CR; an approach toward lever or actually pressing the lever was rare. In fact, covering the lever so that the only cue was the sound of its motor did not impair learning. Our findings differ from studies using rats, which report that under similar experimental conditions, 30–40% of the animals develop a sign-tracking CR (2, 10). Our results are consistent with previous studies indicating that goal tracking is much more common than sign tracking in mice and that mice only develop sign tracking after more than 15 sessions of Pavlovian training (28). Our data support that finding and provide further evidence for significant differences in conditioned approach behaviors between rats and mice.

Using mice with severe depletion of striatal dopamine (only ∼5% of WT levels but sufficient dopamine for normal feeding behavior without intervention), we showed that dopamine signaling is necessary for the development of a goal-tracking CR. This observation may seem inconsistent with recent reports that systemic blockade of dopamine receptors or local blockade in the ventral striatum with the nonspecific antagonist flupenthixol did not affect the goal-tracking CR in rats (2, 11). However, systemic blockade of dopamine receptors with flupenthixol has also been reported to decrease overall activity levels and both goal- and sign-tracking performance, which complicates the interpretation of these findings (11). Therefore, our genetic models of dopamine deficiency provide an important complementary approach.

Further support for the hypothesis that dopamine signaling in the ventral striatum is sufficient for a goal-tracking CR accrues from our findings with vrDD mice. Here, we start with a mouse that has a complete lack of dopamine synthesis and then reactivate dopamine synthesis specifically in dopamine neurons that project to either the dorsal striatum (vrDD-DS mice) or ventral striatum (vrDD-VS mice); this approach allows us to infer in which brain region dopamine signaling is sufficient to enable normal behavior. It is fundamentally different from most approaches that disrupt dopamine signaling to ascertain whether dopamine is necessary for normal behavior (29–32). Although vrDD-DS mice had more dopamine in the ventral striatum than the severely DD mice (10% vs. 5% of WT levels), they did not display goal-tracking behavior. In contrast, vrDD-VS mice (∼30% of WT levels of dopamine) had significantly improved goal-tracking CR behavior, suggesting that dopamine content of 30% of WT levels in the ventral striatum is sufficient for development of goal-tracking CR behavior. However, the CR by vrDD-VS mice was not as strong as the CR made by sham control mice, suggesting that >30% of WT levels of dopamine in the ventral striatum are required for a full restoration of the CR.

We considered the possibility that dopamine signaling is not important for learning a goal-tracking CR but, rather, for its performance (11). We therefore trained the Slc6a3^Cre/+:Th KO mice in their dopamine-depleted state and then assessed their performance during sessions in which we pharmacologically restored their systemic ability for dopamine synthesis by administering l-dopa. A goal-tracking CR was not displayed during training without L-dopa, and, more importantly, there was no accelerated manifestation of goal tracking when dopamine signaling was restored. Conversely, when these mice were trained with dopamine synthesis restored by l-dopa injections, they developed a normal goal-tracking CR that degraded when the animals were retested in their DD state. We conclude that dopamine signaling is critical for both learning and maintenance aspects of a goal-tracking CR. Interestingly, the CR by the l-dopa–trained mice did not vanish immediately when animals were retested in their dopamine-deficient state but, rather, faded over several days, similar to extinction-like decline. A similar observation was made in a different model of dopamine deficiency in the context of motor learning by Beeler et al. (33), who showed that cessation of l-dopa treatment in Pitx3^−/− mice resulted in a gradual performance decline. They suggested that this poor performance might actually have a learning component and that perhaps poor motor performance in Parkinson disease is learned (33).

In Pavlovian control of behavior, predictions of future outcomes lead to an automatic-like CR that is similar to an already existing non-CR. Instrumental control of behavior is different in that a learned response is selected based on its past or current contingency with a reward. Instrumental control of behavior has been described to fall into two classes, one that is goal-directed and based on explicit expectations regarding the relationship between responses and outcomes and another in which it is more habitual (34–37). One interesting hypothesis posits that Pavlovian predictions of outcomes (rewards) underpin some types of instrumental-based control of behavior (1). One example is Pavlovian-to-instrumental transfer, in which Pavlovian cues actually modify instrumental responses (38, 39). We have previously shown that vrDD-DS mice are similar to vrDD-VS mice in their ability to learn an instrumental lever-press response (19, 20). This led us to imagine that during the establishment of instrumental control of behavior, some implicit Pavlovian-like associations might be involved. We therefore took advantage of our Pavlovian conditioning procedure in which extension of a lever serves as a CS and subjected the same mice that were incapable of developing a CR to this CS to an instrumental intervention procedure during which they had to press the lever to obtain rewards. We found that after instrumental training, the vrDD-DS mice displayed a vigorous CR to the sound of the lever CS. Furthermore, this “instrumental-to-Pavlovian” transfer was dopamine-dependent, because the Slc6a3^Cre/+:Th KO mice did not benefit from the intervention. The incentive salience hypothesis posits that dopamine is important for “incentive salience attributions to the neural representations of reward-related stimuli” (40–42). We speculate that during the development of an instrumental lever-pressing response, the cues that are associated with the act of lever pressing, like the sound of the lever extending and retracting, attain incentive salience that might then become transferred to the Pavlovian conditioning sessions in which the lever was hidden from view and touch but still retained the same sound cue associated with its presentation. Regardless of the underlying mechanism, the instrumental-to-Pavlovian transfer that we observe is akin to the axiom that some tasks are learned faster by doing than by watching. In addition, because the dorsal striatum and ventral striatum receive some overlapping innervation from the midbrain, cortex, and amygdala (43), there might be some degree of functional overlap between these striatal regions such that with training, dopamine signaling in the dorsal striatum can allow a mouse to perform tasks that would normally be performed by dopamine signaling in the ventral striatum.

We have answered the four questions posed in the Introduction by demonstrating that mice are goal trackers and require dopamine to develop a goal-tracking CR. Dopamine signaling in the ventral striatum (30% of that in WT mice) is sufficient for development of a CR, and dopamine signaling outside of the ventral striatum is unnecessary. In addition, we present an intriguing observation demonstrating that associations made during instrumental learning by mice with dopamine signaling restricted to the dorsal striatum can facilitate Pavlovian conditioning.

Materials and Methods

All drugs, animals, and surgical procedures are described in detail in SI Materials and Methods. Behavioral procedures, immunohistochemistry, and dopamine measurements. Statistical analyses are also described in SI Materials and Methods.