Abstract
Studies investigating the neural mechanisms by which associations between cues and predicted outcomes control behavior often use associative-learning frameworks to understand the neural control of behavior. These frameworks do not always account for the full range of effects that novelty can have on behavior and future associative learning. Here, in mice, we show that dopamine in the nucleus accumbens (NAc) core is evoked by novel, neutral stimuli, and the trajectory of this response over time tracked habituation to these stimuli. Habituation to novel cues prior to associative learning reduced future associative learning, a phenomenon known as latent inhibition. Crucially, trial-by-trial dopamine response patterns tracked this phenomenon. Optogenetic manipulation of dopamine responses to the cue during the habituation period bidirectionally influenced future associative learning. Thus, dopamine signaling in the NAc core has a causal role in novelty-based learning in a way that cannot be predicted based on purely associative factors.
Keywords: novelty, associative learning, prediction error, learning, reward, fear conditioning
Systems neuroscience studies have focused on the neural mechanisms of associative learning with a goal of defining how circuits in the brain encode associations between cues and predicted outcomes to control behavior. However, other experience-dependent factors, such as novelty, play an important and causal role in determining the trajectory of associative learning1,2. For example, both valenced and neutral stimuli exert the highest influence on behavior when they are novel2,3. Habituation, in which stimulus responses are reduced over repeated presentations, is a critical form of novelty-based learning that guides organisms to ignore irrelevant stimuli in their environment4. Further, novelty also potently influences associative learning and conditioned behavioral responses2,5. Novel stimuli can alter conditioned responses to previously learned cues (i.e. external inhibition) – even when no errors in prediction are present6,7. Additionally, unconditioned stimuli form stronger associations with neutral cues when the cues are novel while familiar cues impede this process – a psychological phenomenon termed latent inhibition8. While parameters such as salience and novelty are accounted for in various respects in virtually all influential associative learning models, these frameworks still do not always account for the full range of effects of novelty on behavior9–12. Thus, defining the neural underpinnings of interactions between stimulus novelty/habituation and future learning – and whether this is best explained by associative or non-associative factors – is critical to understanding fundamental neurobehavioral processes.
While dopamine is often studied in associative learning contexts13,14, work has shown that dopamine in striatal regions, such as the nucleus accumbens (NAc) core, is modulated by novelty. Extracellular dopamine levels are influenced by novelty and habituation, and basal dopamine levels correlate with attention7,15,16. While previous studies have focused on calcium imaging in ventral tegmental area (VTA) cell bodies and have suggested that dopamine neurons are critically involved in novelty detection17, the majority of work on how novelty alters the dopamine signal at its projection targets has been relegated to slow sampling techniques that only allow for the assessment of dopamine levels over long periods of time. Importantly, habituation occurs rapidly and understanding its neural correlates requires the ability to assess dopamine responses on a trial-by-trial basis. In sum, while prior work suggests that dopamine is influenced by novelty, technical limitations have prevented our ability to systematically define 1) if and how this occurs in a temporally specific fashion and the behavioral factors that influence these signals over experience in the NAc core and 2) whether the dopamine signal in the NAc core is causal to novelty effects on learning via non-associative factors and whether this affects future associative learning. The development of genetically encoded fluorescent dopamine sensors allows for direct, optical assessment of dopamine transients in vivo with a high signal-to-noise ratio. We can thus assess dopamine responses across single trials, across sessions, and across behavioral tasks within the same animals. To this end, we observed and manipulated dopamine responses during repeated presentations of neutral stimuli to understand how dopamine responses track novelty and habituation to influence future associative learning.
We show that neutral auditory and visual stimuli evoke a positive dopamine response in the NAc core in the absence of any valence-based predictions. Further, the magnitude of the dopamine response tracks the novelty of neutral stimuli whereby a response is reliably evoked during initial exposure and dissipates as a function of habituation to the stimulus. Moreover, with repeated presentations, dopamine responses to neutral cues decreased to baseline as animals habituated to the stimulus. We subsequently employed a latent inhibition paradigm to define whether these signals were causal to future associative learning. Using optogenetics to increase or decrease dopamine responses during habituation, we showed that dopamine responses during the habituation period are causal to future learning and cannot be explained solely by associative or prediction-based accounts of dopamine coding in learning and memory18–20. Our results show that dopamine in the NAc core is causal to latent inhibition. Further, we demonstrate a causal link between dopamine and novelty that influences current and future behavior that is best explained via non-associative mechanisms.
Results
Neutral cues evoke dopamine that decreases with habituation
Our first goal was to determine if novel and neutral stimuli could evoke a dopamine response and how this changed with experience (i.e., following repeated exposure). To this end, we utilized optical methods for directly recording dopamine in awake and behaving animals. A majority of work overlaying dopaminergic activity with behavioral control has utilized electrophysiology21 or calcium imaging22 to record action potentials at the soma of midbrain dopamine neurons23, or observed axonal calcium fluctuations as a proxy of dopamine release events24. Both approaches assume that dopamine itself follows the same pattern as these proxy measures. However, extracellular dopamine levels in the NAc results from both dopamine neuron firing patterns and rapid modulation of dopamine terminals by both homosynaptic mechanisms and heterosynaptic signaling via accumbal microcircuits25. These local modulatory mechanisms sculpt the timing and magnitude of dopamine transmission independent of dopamine cell body activity in the midbrain26. Indeed, recent work has shown that task-related VTA dopamine neuron spiking and dopamine release are dissociable in vivo27 highlighting the need for direct assessment of dopamine response patterns in the NAc. To this end, we used the genetically encoded dopamine sensor, dLight1.128, to record in vivo dopamine dynamics at the level of its projection targets in the NAc core (Fig. 1a; see Extended Data Fig. 1 for specific dopamine analyses conducted and representative dopamine traces). Using this approach we recorded dopamine responses during the presentation of a neutral stimulus (white noise) presented at 85 dB for 6–7 presentations on a random-time schedule for two sessions on consecutive days (Fig. 1a,b).
First, we found that novel, neutral stimuli reliably evoked dopamine transients upon first exposure (Fig. 1c,d). Next, we found that dopamine responses to the same stimulus were progressively reduced over repeated exposure (Fig. 1d; see Extended Data Fig. 2 for the second day of the exposure session). Specifically, we found that the peak of the dopamine response decreased both within (Fig. 1d) and across sessions (Fig. 1e,f). While the peak dopamine response tracked habituation to the neutral cue, there were no changes in dopamine clearance (Fig. 1g,h). These results show that in the absence of an outcome, NAc core dopamine responses track the novelty/familiarity of stimuli. That is, as the stimulus becomes more familiar during repeated exposure, the dopamine signal that the stimulus evokes diminishes.
Habituation to a neutral cue decreases future learning
Although we showed above that NAc core dopamine tracks the familiarity of a neutral stimulus, it is not known if this effect is consequential for the formation of future associations. To test this, we employed a latent inhibition paradigm8. Latent inhibition is a novelty-based learning phenomenon whereby pre-exposure to a neutral stimulus before conditioning results in a reduced learning rate for that same stimulus in the future8. This occurs because the novelty of the stimulus is reduced and thus attention to that stimulus is consequently reduced when an associative contingency is later imposed. Importantly, this is one of the main challenges to prediction-based learning models, which have been used to explain the role of dopamine in learning and memory. Prediction-based models cannot account for the change in the conditioned response based on prior exposure to the stimulus (which influences novelty and attention29).
Mice were pre-exposed to a stimulus (either tone or light, counterbalanced) for 4 consecutive sessions, a total of 33 presentations for each session, to induce habituation (Fig. 2a). Following the pre-exposure period, animals underwent fear conditioning. During these sessions, animals were presented with a pre-exposed conditioned stimulus (pre-exposed CS+) or a non-pre-exposed conditioned stimulus (CS+) – both of which were immediately followed by a footshock. Further, there was a non-pre-exposed CS− (CS−) that signaled that no shock would occur (Fig. 2b). At the end of conditioning session 1, the CS+ yielded a stronger freezing response as compared to both the CS− and the pre-exposed CS+ (Fig. 2c). Importantly, there was no difference between the freezing to the CS+ and pre-exposed CS+ on the first trial before the first shock was presented (Fig. 2d); however, following conditioning, the CS+ yielded a stronger freezing response than the pre-exposed CS+. At the end of the second session, the difference between the CS+ and pre-exposed CS+ disappeared, indicating that the observed differences were in the rate at which learning occurred (Extended Data Fig. 4). These results demonstrate a strong latent inhibition effect.
Dopamine responses track latent inhibition
As shown in the initial repeated exposure experiment, NAc core dopamine responses decreased as novelty was reduced (i.e., with increasing familiarity of the stimulus). We hypothesized that if this effect had any impact on learning rate, the dopamine response to the CS+ and the pre-exposed CS+ would also differ. Supporting our hypothesis, we found that dopamine responses to the pre-exposed CS+ were weaker as compared to the CS+ (Fig. 2e–j; see Extended Data Fig. 3–4 for additional analyses). The CS+ elicited a dopamine response which was larger than the baseline dopamine levels [during the inter-trial interval (ITI) when no stimuli were presented]; however, the dopamine response to the pre-exposed CS+ was not different from this baseline (Fig. 2h). In addition to changes in the dopamine response, there were also changes in some kinetic parameters. The time for the dopamine signal to return to baseline was increased following the presentation of the CS+ as compared to the pre-exposed CS+ (Fig. 2i,j). Supporting these results, we also found that in the mice that did not show latent inhibition, the dopamine response did not differ between the pre-exposed CS+ and CS+ (Extended Data Fig. 4a–d). Thus, even though both cues were paired identically with an aversive stimulus, there were significant differences in dopamine responses, which tracked the novelty of the cue before it had acquired value.
Consistent with the behavioral data, on the second training day these behavioral and dopamine response differences disappeared (Extended Data Fig. 4e–j). At this time, there were no longer significant differences between the CS+ and pre-exposed CS+ for peak height (Extended Data Fig. 4h), or any of the kinetic parameters measured (Extended Data Fig. 4i,j). The work presented within the current manuscript is consistent with many previous studies showing that aversive stimuli increase dopamine levels in the NAc30,31. However, work has also suggested that dopamine encodes bi-directional valence where the dopamine response to the cues predicting aversive outcomes (e.g., fear cues) is negative14. These studies present fear conditioning responses as the average of many trials, rather than a trial-by-trial analysis during the early trials, as we present here. Replicating these results, we also showed that the dopamine response to the fear conditioning cues dipped below baseline with extensive training (Extended Data Fig. 5). However, these results cannot be explained by the cue coding for negative valence as animals have learned the aversive association – and thus freeze to the cue – before this signal becomes negative. Also, the positive dopamine response to the cue was correlated with learning rate for that cue during early training in the opposite direction of what would be predicted if this were a purely associative and valence-based signal, with larger dopamine responses predicting faster learning of the aversive association.
Overall, our data supports that, like the behavioral responses, the dopamine response to a CS+ decreases following pre-exposure and this effect disappears with additional training, highlighting that pre-exposure retards the learning rate for the neutral stimulus in the future.
Latent disinhibition eliminates dopamine differences
It is well known that switching to a novel context following stimulus pre-exposure abolishes latent inhibition, an effect known as “latent disinhibition”32. We leveraged this behavioral manipulation as an additional mechanism to probe endogenous dopamine dynamics. We reasoned that if dopamine is encoding aspects of latent inhibition, eliminating the latent inhibition effect (via introducing a novel context) would abolish the differential dopamine response to pre-exposed versus novel cues. To this end, we conducted an experiment where the pre-exposure was done in a separate context than the subsequent fear conditioning session and conducted dopamine recordings within these animals at the time of fear conditioning following pre-exposure (Fig. 3a–c). When the fear conditioning occurred in a different context than the cue pre-exposure, we found that the context switch indeed abolished the behavioral latent inhibition effect as expected (Fig. 3). The reduced dopamine response to the pre-exposed cue observed in previous experiments also disappeared when the pre-exposed cue was now presented in a novel context (Fig. 3c). Thus, behaviorally manipulating latent inhibition was able to alter the dopamine response in a predictable fashion, suggesting that dopamine may be causal to its expression.
Dopamine during pre-exposure is causal to latent inhibition
Next, we wanted to confirm if the reduced dopamine signal to the pre-exposed cue is indeed causal to latent inhibition. That is, the reduced dopamine response to the cue retards the learning rate for that cue when it is paired with another stimulus subsequently. The largest dopamine response was detected during the first trial of the fear conditioning session (Fig. 4a), suggesting the latent inhibition effect is determined during the initial trials of each session. Indeed, shorter cue exposures during pre-exposure were also able to induce latent inhibition (Extended Data Fig. 6). Next, we tested if stimulating or inhibiting dopamine terminals in the NAc core via optogenetics, only at the time of the cue during the first cue-shock pairing was able to alter the behavioral effect of the pre-exposure period (Fig. 4b–e). Concurrent with the pre-exposed stimulus presentation during the first fear conditioning trial, dopamine terminals that project from the VTA and synapse in the NAc core were either stimulated (channelrhodopsin-2, ChR2 group; 20 Hz 470nm photostimulation, 1s duration) or inhibited (halorhodopsin [eNpHR3.0], NpHR group; continuous 590nm photoinhibition, 11s duration; Fig. 4b–d; Extended Data Fig. 7). We found that stimulating dopamine terminals [via ChR2 expressed selectively in dopamine terminals in the NAc] during the cue was sufficient to block the latent inhibition effect and restore a normal learning trajectory for this cue (Fig. 4e). Inhibition of dopamine terminals [via NpHR 3.0 expressed selectively in dopamine terminals in the NAc] did not result in a larger latent inhibition effect (Fig. 4e, also see Extended Data Fig. 8), likely because of a floor effect. Overall, these results show that the diminishing dopamine response to the pre-exposed cue on the first day of conditioning (after the pre-exposure period) is causal to the behavioral effects observed.
Together, these data show that dopamine responses in the NAc core are 1) positively correlated with the novelty of a stimulus regardless of valence, conditioned value, or predictions, 2) increased to aversive stimuli (footshocks), elicited by appetitive33, novel, and neutral stimuli, 3) decreased with experience, 4) can be altered by altering latent inhibition, and 5) causal to its expression.
Dopamine effects cannot be explained by associative factors
Most of the theoretical models and hypotheses for latent inhibition offer explanations that are hybrids of attentional-associative accounts12. However, some purely associative accounts have also been proposed34. Thus, there are two overarching hypotheses of how latent inhibition occurs on the behavioral level. The first is an attentional account. In this account, as novelty is reduced (via repeated exposure), the attention paid to these stimuli is also reduced as an animal is habituated to them. The core tenant of this hypothesis is that behavior in response to these cues is modulated by novelty in a non-associative fashion. The alternative hypothesis is the associative account. In this account, latent inhibition is the result of associative learning during the pre-exposure period18–20. In this framework, associations are formed between the neutral cue and the context during pre-exposure, and these associations compete with future cue-outcome associations to slow down the learning of new associations35.
All of the previous studies presented could be explained by either an associative or attentional account of latent inhibition. Even the latent disinhibition experiment could be explained through either the introduction of general novelty – which alters attention to all stimuli in an environment and thus eliminates the effect of habituation12,36 – or through cue-context associations that were made during the pre-exposure period. We specifically designed the following series of studies to test these competing hypotheses and use these experiments to better define how dopamine’s role in learning and memory is causally related to behavioral control.
Pre-exposed cues do not become conditioned inhibitors.
An associative account of latent inhibition suggests that associations formed either between cues and the absence of outcomes (i.e., cue-no outcome associations) or cues and contexts are responsible for the impaired learning during subsequent conditioning training. If cue-no outcome associations are responsible, then the pre-exposed cue might operate as a conditioned inhibitor during future learning, as it predicts that no outcome will occur. The pre-exposed cue (which would function as a conditioned inhibitor) would thus decrease the conditioned response to an excitor (a cue paired with an outcome) when presented together with that excitor in a summation test (Fig. 4f).
Our results support the attentional account since they show that the pre-exposed cue does not become an inhibitor as it does not reduce the freezing response to the excitor (the cue that was paired with a shock) (Fig. 4g). Additionally, the pre-exposed cue does not reduce the freezing response while a novel (distracting) stimulus alongside the excitor resulted in a marked reduction in the freezing response to the excitor, a purely attentional effect known as “external inhibition”6 (Fig. 4g). This suggests that the pre-exposed stimulus exerts no associative or attentional control over the conditioned response after an animal has been habituated to it.
Stimulation during pre-exposure controls subsequent learning.
Data presented thus far can be explained by novelty acting to alter attention to stimuli in a context. However, many of these findings can also be explained in part through associative learning about the relationship between the neutral cue and the context. Thus, we designed an optogenetic experiment to specifically parse these ideas from one another. In the accounts solely based on associative mechanisms, inhibiting dopamine responses during the pre-exposure period should prevent latent inhibition – as it would prevent the novel cue + context associations from forming −, conversely, increasing dopamine optogenetically should facilitate latent inhibition – as this would be necessary to facilitate the cue + context association that slows future learning. Alternatively, if dopamine responses reflect non-associative learning, as we predict, inhibiting dopamine responses during the cue pre-exposure period will facilitate latent inhibition and slow future learning - as this would signal that habituation occurred more rapidly −, whereas stimulating dopamine during pre-exposure will decrease the effect - by preventing habituation - and increase future learning.
We used optogenetics to increase or decrease the dopamine response to the cue during the pre-exposure period to determine how this influenced subsequent associative learning. Concurrent with stimulus presentation during the pre-exposure sessions, dopamine terminals that project from the VTA and synapse in the NAc core were either photo-stimulated or photo-inhibited (Fig. 5a–e; Extended Data Fig. 8). Following the pre-exposure sessions, mice underwent two fear conditioning sessions in which the pre-exposed cue (pre-exposed CS+) and a novel cue (CS+) were both paired with a shock over 6 trials. While the eYFP control group showed a significant reduction in freezing to the pre-exposed CS+ as compared to the CS+, in the ChR2 group freezing did not differ between the CS+ and pre-exposed CS+ during fear conditioning (Fig. 5c–e). Therefore, stimulating dopamine to the pre-exposed stimulus disrupted the formation of a latent inhibition effect and enhanced subsequent associative learning. Conversely, inhibiting dopamine to the pre-exposed cue further impaired future associative learning, indicating a bidirectional effect of the dopamine signal on latent inhibition (Fig. 5c–e). The latent inhibition effect in the eYFP group disappeared with additional training during the second conditioning session, whereas in the NpHR group, the freezing response to the CS+ was still stronger compared to the freezing response to the pre-exposed CS+ (Extended Data Fig. 9). Therefore, the inhibition of NAc core dopamine during pre-exposure leads to a more persistent latent inhibition effect, where pre-exposed cues show a reduced ability to acquire novel associations in the future.
To verify that the optogenetic photostimulation of NAc core dopamine terminals at the time of the pre-exposure of stimuli did indeed augment subsequent dopamine response to the pre-exposed cue during learning sessions, we combined optogenetics (with a red-shifted opsin) with optical dopamine recordings within the same animal (Fig. 6a–g). Replicating our original optogenetics results, we showed that stimulating dopamine terminals in the NAc core at the time of the pre-exposure of cues [via expressing a red-shifted excitatory opsin in TH+ VTA cell bodies (Chrimson+rTH.PI.Cre virus)] resulted in disrupted latent inhibition compared to controls (rTH.PI.Cre virus only) (Fig. 6c). We also showed that this manipulation reversed the diminished dopamine response to the pre-exposed cue during fear conditioning while this effect was intact in the control animals (Fig. 6d–g), demonstrating a causal relationship between the dopamine response to cues during pre-exposure and future dopamine signatures that occur during subsequent associative learning.
Finally, we showed that dopamine manipulations have comparable effects on future associative learning and latent inhibition, regardless of whether the subsequent associative learning is driven by aversive or appetitive stimuli (Extended Data Fig. 10). Overall, these results causally show that the dopamine signal in the NAc core is heavily influenced by novelty/familiarity and determines the associability of stimuli in future associative learning contexts. Together, these data show the parameters under which dopamine controls novelty-driven learning. These data also rule out a purely associative explanation of dopamine in latent inhibition, while demonstrating that dopamine is a critical mediator of novelty-based effects on behavior in a fashion that can be explained, at least in part, via non-associative processes.
Discussion
Here we show that dopamine in the nucleus accumbens (NAc) core is evoked by novel, neutral stimuli in isolation, and that these responses causally influence future learning for valenced stimuli. Critically, trial-by-trial dopamine response patterns tracked both the habituation to novel neutral stimuli, as well as the dopamine response patterns that were observed to habituated and novel stimuli during future associative learning. Additionally, we demonstrated that these signals were causal to this process. Optogenetically evoking or inhibiting the dopamine response to neutral cues during a habituation period bidirectionally influenced the ability of these cues to form future associations with appetitive or aversive stimuli. Together, our results demonstrate a causal temporally specific link between the dopamine signal and the novelty of a stimulus (in a valence free fashion) that influences current and future behavior. Critically, our findings challenge theories of dopamine as a purely valence-based prediction signal and highlight the causal role of dopamine in the NAc core in novelty effects on current and future behavior.
While novelty effects on behavior, such as habituation, are important for animals to learn to ignore irrelevant stimuli in their environment, they can also influence associative forms of learning. As we show here, unconditioned stimuli form stronger associations with neutral cues when the cues are novel than when an animal has been habituated to them – a psychological phenomenon termed latent inhibition35. We show here that dopamine patterns not only correlate with latent inhibition but are also causal to its development and expression. Our results demonstrate that NAc core dopamine responses evoked by a stimulus decreases as the stimulus becomes more familiar, or less novel, through repeated exposure. Further, this occurs rapidly, within the first few trials of exposure. During subsequent associative learning for a cue-footshock pairing, the dopamine signal was weaker to the pre-exposed cue compared to a novel cue and resulted in a slower rate of learning of the association between the pre-exposed cue and footshock. Latent inhibition and conditioned attention theory were first proposed by Lubow and his colleagues, who described the pre-exposure effect as being an attentional deficit caused by repeated presentations8,12. Extant work has suggested that dopamine is influenced by these processes16,31; however, it was not clear if these responses were causal to its expression or the temporal dynamics by which these responses occurred. Here we temporally linked dopamine responses evoked by pre-exposed and non-pre-exposed stimuli to future associative learning and conditioned behavior.
Previous work has implicated certain populations of VTA dopamine neurons in the expression of latent inhibition17,37. For example, Morrens et al. (2020)17 {Citation}showed that dopamine cell bodies in the VTA respond to novel but not familiar odors and that activating these neurons or the dopamine terminals in the prefrontal cortex at the time of a familiar cue accelerated associative learning. However, these studies recorded VTA cell-bodies – not dopamine responses downstream – and the authors concluded that cortical, rather than striatal, projections were mediating these effects17. Here we show that dopamine is evoked by neutral auditory and visual stimuli and response patterns track habituation on a trial-by-trial basis. Moreover, our results concluded that preventing the habituation pattern influences not only the associative processes at that moment but also future learning as well. Further, the data contained within this manuscript contrast with conclusions drawn from earlier studies15,16 which failed to show the involvement of VTA-striatum projections in detecting novel odors in the environment24. However, these previous studies examined NAc within a limited range of parameters and assessed the whole ventral striatum, rather than defined subregions which have been previously shown to have different response patterns during behavioral tasks38. Here we assessed dopamine responses to neutral stimuli across sensory modalities as well as the influence of habituation on both appetitive and aversive learning in the NAc core. Further, using optogenetics we linked the dopamine response to novelty-driven effects on learning, ultimately supporting earlier studies showing dopamine levels are modulated by novelty and attention.
One particularly striking aspect of these findings is that habituation to neutral cues at the behavioral and neural level can occur very rapidly. We show that the latent inhibition effect can be manipulated at various points in the task with optogenetic dopamine manipulations on few – even just two – trials. Thus, this work highlights the transient nature of neural signals that track novelty. However, the speed at which this occurs is likely a result of the stimulus properties of these pre-exposed stimuli. Previous studies have demonstrated that stimulus duration39, total pre-exposure time40, and stimulus intensity41 employed during pre-exposure determine the size of the latent inhibition effect. Similarly, these stimulus properties also determine the habituation rate of a stimulus42. Therefore, depending on the stimulus characteristics, habituation may be achieved faster or slower suggesting a dynamic range in the habituation of behavioral and neural responses, such as the dopamine responses measured in this study. This also explains how only a single pre-exposure to a neutral stimulus is enough for latent inhibition in paradigms where the stimuli employed are less discrete (e.g., odor or flavor), such as conditioned taste approach/aversion in mammalian43 and non-mammalian species37. Overall, our data underscores the importance of stimulus properties determining not only the size of behavioral and neural responses but also the rate and shape of the progression of those signals. Further, and maybe more importantly, these data underscore the critical importance of trial-by-trial analysis when drawing conclusions about dopamine’s involvement in novelty and novelty-related phenomenon.
While associative learning theories offer a powerful account of how animals learn relationships between stimuli, and include factors such as salience and novelty, they often fall short in being able to fully capture the effects these factors on behavior. For example, while they can predict basic associative learning, they cannot model phenomena like external inhibition or latent inhibition that are primarily driven by the effects of novelty on behavior10–12. Supporting this hypothesis, we recently showed that when a novel, unpredicted stimulus was presented during external inhibition, the dopamine response to the cue increased and optogenetically stimulating dopamine during the presentation of a CS+ resulted in an external inhibition-like decrease in the freezing response7. Thus, these results diverge from the canonical “dopamine as a prediction error” theory in several critical ways. First, prediction error for the footshock outcome is equal between the pre-exposed and non-pre-exposed cues throughout the training session as both stimuli were paired with the footshocks the same number of times. Second, we detected dopamine responses to the novel neutral cues before they acquired any predictive value. Therefore, it is not possible to attribute our results to differences in prediction error during these trials. These data show that dopamine is influenced by environmental factors that influence behavior and cannot be explained by traditional prediction-based models44,45.
The data we present here shows that dopamine’s involvement in latent inhibition may also be influenced by associative factors. Our results showed that inhibiting dopamine responses to a novel neutral cue during pre-exposure enhanced latent inhibition, rather than preventing it as one would expect if the dopamine response during that period was critical for the encoding of the cue-context association that slows future learning through competition. Similarly, stimulating the dopamine response to the cue during pre-exposure did not enhance the latent inhibition effect, rather, it eliminated the effect. Although these results clearly show that associative mechanisms are in play in latent inhibition, they are also in line with a novelty-based account of dopaminergic control of latent inhibition. For example, novelty-based accounts of dopamine predict no effect of inhibition of dopamine during pre-exposure inter-trial intervals when novelty is already minimal in the absence of any stimuli, and the disrupted latent inhibition is a result of the increased novelty in the environment due to the novel context or artificial photostimulation of dopamine. Nevertheless, novelty and associative terms such as associative strength, prediction, and prediction error are intrinsically linked to one another making it difficult to propose purely novelty-based or associative models of latent inhibition. For example, theoretical accounts like the one we proposed recently7, assumes that stronger cue-context associations reduce novelty as the cue becomes more predicted in the environment, eventually resulting in decreased attention to the pre-exposed CS. Therefore, we acknowledge that our results do not completely rule out the involvement of any associative account in the pre-exposure process.
Furthermore, replicating earlier studies of latent inhibition46, we demonstrated that the pre-exposed cue does not become an inhibitor of conditioned behavioral responses, thereby ruling out the possibility that cue-no outcome associations are formed during pre-exposure and responsible for the impaired learning in the subsequent conditioning training via the suppression of a conditioned response. This potential explanation is also not supported by theoretical models that are solely based on associative terms, such as the Rescorla-Wagner model, as these models also assume that for cues to become inhibitory and predict the absence of an outcome, the presence of an outcome should be predicted by other cues or contexts that are present9. These results are in line with non-associative and novelty-based accounts of latent inhibition. Indeed, non-associative and non-reward prediction error-based accounts similar to our own framework here were previously put forward for dopaminergic encoding47. Specifically, studies suggested that dopamine may be involved in novelty encoding in the form of novelty-induced exploration48 and saliency detection47,49. Here, in line with these accounts, we demonstrate that dopamine is involved in the non-associative processes by signaling novelty, while associative factors may be in play during the pre-exposure process.
Importantly, these data also explain a large body of human literature that has shown that dopamine deficits that characterize neurodegenerative diseases are concomitant with deficits in non-associative learning. For example, habituation is a behavioral marker for many psychiatric diseases including Parkinsonism, which is marked by dopaminergic deficits50. Specifically, Parkinson’s patients show decreased habituation to auditory stimuli51 and these deficits are alleviated by drugs that enhance dopaminergic signaling (e.g., levodopa;52). Thus, understanding the neural mechanisms that underlie non-associative learning mechanisms, such as habituation, is important in understanding psychiatric disease symptomatology50 and is critical to our understanding of how dopamine deficits influence behavior in patient populations.
Overall, the results of the present study show that dopamine tracks the novelty of a given event regardless of the origin of novelty in the environment. Dopamine has been most widely studied under behavioral conditions of associative learning where a Pavlovian or discriminative cue acquires associative strength by predicting a significant outcome, such as a reward. Here, we critically show that dopamine is involved in non-associative types of learning, such as habituation of neutral stimuli. The novelty concept, as proposed by earlier theorists (e.g.,8,12,53), is closely connected to prediction error as the source of novelty is ultimately the mismatch between prediction and actual occurrence of events. This is consistent with the literature suggesting that dopamine neurons in the VTA may compute sensory prediction errors17,54 and even support formation of stimulus-stimulus associations55. However, it is also possible that dopamine response patterns align more closely with non-associative terms, which are sensory adaptations rather than an associative process4, which the data within this manuscript support. Regardless, novelty and attention should be considered as principal components of the involvement of dopamine in associative learning and included in potential interpretation of data. Importantly, our results also suggest that more nuanced and updated understanding of predictions should be utilized when considering the role of mesolimbic dopamine in learning and memory.
Materials and Methods
Animals.
Male (N=35) and female (N=48) 6- to 8-week-old C57BL/6J mice obtained from Jackson Laboratories (Bar Harbor, ME; SN: 000664) were kept 5 per cage and maintained on a 12-hour reverse light/dark cycle, with all behavioral testing took place during the light cycle. Animals were given ad libitum access to food and water (Temperature 68–76 °F; Humidity 30%–70%). No statistical methods were used to pre-determine sample sizes but our sample sizes are similar to those reported in previous publications7. All experiments were conducted in accordance with the guidelines of the Institutional Animal Care and Use Committee (IACUC) at Vanderbilt University School of Medicine. The order of testing was counterbalanced, and experimenters were blind to experimental groups throughout behavioral experiments. We randomly assigned subjects into experimental groups when between subject design was employed. Male (N=10) and female (N=7) TH-Cre rats at around 12–16 weeks old at the beginning of the experiments were used to test the effect of photostimulation of dopamine neurons in the VTA on latent inhibition in an appetitive procedure using food rewards (Extended Data Fig. 10). These rats were bred at NIDA-IRP after the founders were obtained from the rat resource and research center (RRRC, University of Missouri). Rats were singly housed and maintained on a 12-hour light/dark cycle, with all behavioral testing took place during the light cycle. Rats were on ad libitum access to food and water unless undergoing the behavioral experiment, during which they received either 8g or 12g of chow- for females and males, respectively, daily in their home cage following training sessions. Rats were monitored to ensure they did not drop below 85% of their initial body weight across the course of the experiment. All experimental procedures were conducted in accordance with the NIDA-IRP Institutional Animal Care and Use Committee of the US National Institutes of Health (NIH) guidelines.
Apparatus.
Mice were trained and tested daily in individual operant conditioning chambers (Med Associates Inc., St. Albans, Vermont) fitted with visual and auditory stimuli including a standard house light, a white noise generator, and a 16-tone generator capable of outputting frequencies between 1 and 20 KHz (85 dB). For the optogenetic photostimulation experiments in rats, training was conducted in eight standard behavioral chambers (Coulbourn Instruments; Allentown, PA), which were individually housed in light and sound-attenuating boxes (Jim Garmon, JHU Psychology Machine Shop). Each chamber was equipped with a pellet dispenser that delivered 45-mg pellets into a recessed food port when activated. Access to the food port was detected by means of infrared detectors mounted across the opening of the recess. The chambers contained a speaker connected to an Arduino that was capable of generating many auditory sounds. A computer equipped with GS3 software (Coulbourn Instruments, Allentown, PA) controlled the equipment and recorded the responses.
Surgical Procedure.
1 hour prior to surgery, mice were administered Ketoprofen (5 mg/kg) via subcutaneous injection. Animals were anesthetized using isoflurane (5% for induction and 2% for maintenance) and placed on a stereotaxic frame (David Kopf Instruments). Ophthalmic ointment was continuously applied to the eyes throughout surgical procedures. A midline incision was then made down the scalp and a craniotomy was performed with a dental drill using aseptic technique. Using a .10-mL NanoFil syringe (WPI) with a 34-gauge needle, AAV5.CAG.dLight1.1 (UC Irvine;28) was unilaterally infused into the NAc (bregma coordinates: anterior/posterior, + 1.4 mm; medial/lateral, + 1.5 mm; dorsal/ventral, −4.3 mm; 10° angle) at a rate of 50 nL/min for a total volume of 500 nL. Following infusion, the needle was kept at the injection site for 7 minutes before being slowly withdrawn. Fiber-optic cannulas (400 μm core diameter; .48 NA; Doric) were then implanted in the NAc and positioned immediately dorsal to the viral injection site (bregma coordinates: anterior/posterior, + 1.4 mm; medial/lateral, + 1.5 mm; dorsal/ventral, −4.2 mm; 10° angle) before being permanently fixed to the skull using adhesive cement (C&B Metabond; Parkell). Follow-up care was performed according to IACUC/OAWA and DAC standard protocol. Animals were allowed a minimum of 6 weeks to recover in order to ensure efficient viral expression before commencing experiments.
The surgical procedures for the optogenetic photostimulation of the dopamine neurons in the VTA have been described previously55. Briefly, rats received bilateral infusions of 1.2μl if AAV5-EF1α-DIO-ChR2-eYFP (n=8) or AAV5-EF1α-DIO-eYFP (n=9) into the VTA at the following coordinates relative to bregma: AP −5.3mm; ML ±0.7mm; DV −6.5mm and −7.7mm (females) or −7.0mm and −8.2mm (males). Virus was obtained from the University of North Carolina at Chapel Hill (UNC Vector Core). During this surgery, optic fibers were implanted bilaterally (200μl diameter, Thorlabs) at the following coordinates relative to bregma: AP −5.3mm; ML ± 2.61; DV −7.05mm (females) and −7.55mm (males) at an angle of 15° pointed towards the midline. All procedures were conducted in accordance with the Institutional Animal Care Use Committee of the US Institutes of Health (approved protocol: 18-CNRB-108).
For the mouse optogenetic experiments, we used a viral approach to target dopaminergic cells in the VTA in combination with a terminal specific stimulation strategy, which ensures added specificity on top of the viral approach, to achieve dopamine release manipulations. AAV5.Ef1a.DIO.hchR2.eYFP (ChR2; UNC vector core), Chrimson.FLEX: AAV5-Syn-FLEX-rc[ChrimsonR-tdTomato] (Chrimson; Addgene) or AAV5-Ef1a-DIO.eNpHR.3.0-eYFP (NpHR; Addgene) and AAV9.rTH.PI.Cre.SV40 (Addgene;56) were injected into the VTA (unilaterally for ChR2 and Chrimson and bilaterally for NpHR; bregma coordinates: anterior/posterior, −3.16 mm; medial/lateral, + 0.5 mm; dorsal/ventral, −4.8 mm) of C57BL/6J mice. Unilateral (for ChR2 and Chrimson) or bilateral (for NpHR) 200um fiber optic implants were placed into the NAc core (bregma coordinates: anterior/posterior, +/− 1.4 mm; medial/lateral, + 1.5 mm; dorsal/ventral, −4.3 mm; 10° angle; at a rate of 50 nL/min for a total volume of 500 nL). This allowed for the photostimulation or photoinhibition of dopamine response only in dopamine terminals that project from the VTA and synapse in the NAc core. Control animals received AAV5.Ef1a.DIO.eYFP injections into the VTA instead of ChR2 or NpHR. Controls for the Chrimson group only received the Cre-dependent Chrimson but not the cre-inducing virus.
Histology:
Mice were deeply anaesthetized with an intraperitoneal injection of a ketamine/xylazine mixture (100 mg/kg;10 mg/kg) before being transcardially perfused with 10 mL of 1x PBS solution followed by 10 mL of cold 4% PFA in 1x PBS. Animals were subsequently decapitated, and the brain was extracted and postfixed in the 4% PFA solution stored at 4 °C for at least 48 hours before being dehydrated in a 30% sucrose in 1x PBS solution stored at 4 °C. After sinking, tissue was sectioned (35 μm slices) on a freezing sliding microtome (Leica SM2010R) and then placed in a cryoprotectant solution (7.5% sucrose + 15% ethylene glycol in 0.1 M PB) stored at −20 °C until immunohistochemical processing. For optogenetic experiments using AAV9.rTH.PI.Cre.SV40 and AAV5-Ef1a-DIO.eNpHR.3.0-eYFP/AAV5-EF1α-DIO-ChR2-eYFP/AAV5-EF1α-DIO-eYFP, we also validated the targeting of TH+ cells in the VTA via an anti-tyrosine hydroxylase (TH) antibody (mouse anti-TH; Millipore #MAB318, 1:100) and an anti-GFP antibody (chicken anti-GFP (Abcam #AB13970, 1:1000). Sections were then incubated with secondary antibodies [gfp: goat anti-chicken AlexaFluor 488 (Life Technologies #A-11039), 1:1000 and TH: donkey anti-mouse AlexaFluor 594 (Life Technologies # A-21203), 1:1000] for 2 h at room temperature. After washing, sections were incubated for 5 min with DAPI (NucBlue, Invitrogen) to achieve counterstaining of nuclei before mounting in Prolong Gold (Invitrogen). Sections were mounted on glass microscope slides with ProLong Gold antifade reagent. Fluorescent imaging was conducted using a BZ-X700 inverted fluorescence microscope (Keyence) under a dry 20x objective (Nikon). Injection site locations and optical fiber placements were determined with serial images in all experimental animals. For the TH-Cre rat optogenetic photostimulation experiments, all rats were euthanized with an overdose of carbon dioxide and perfused with phosphate buffered saline (PBS) followed by 4% Paraformaldehyde (Sigma-Aldrich Inc, NJ). Fixed brains were cut in 40μm sections to examine fiber tip position under a fluorescence microscope (Olympus Microscopy, Japan). Images of these brain slices were acquired by a fluorescence Virtual Slide microscope (Olympus America, NY) and later analyzed in Adobe Photoshop. Subjects were eliminated if viral expression was detected outside of the defined borders of the NAc core and VTA and/or the tip of the implants were identified outside of the NAc core borders based on the mouse brain atlas (Franklin & Paxinos, 2007). We excluded a total of 1 animal due to inaccurate placement of optic fiber in the NpHR group.
Fiber Photometry.
The fiber photometry system used two light-emitting diodes (490nm and 405nm; Thorlabs) controlled by an LED driver (Thorlabs). The 490nm light source was filtered with a 470nm (the excitation peak of dLight1.1) bandpass filter and the 405nm light source was used as an isosbestic control 28. Light was passed through an optical fiber (400 μm, .48 NA; Doric) that was coupled to a chronically implanted fiber optic cannula in each mouse. LEDs were controlled via a real-time signal processor (RZ5P; Tucker-Davis Technologies) and emission signals from each LED were determined by multiplexing. Synapse software (Tucker-Davis Technologies) was used to control the timing and intensity of the LEDs and to record the emitted fluorescent signals upon detection by a photoreceiver (Newport Visible Femtowatt Photoreceiver Module; Doric). LED power (125 μW) was measured daily and maintained across trials and experiments. For each event of interest (e.g., cue presentation, footshock), transistor-transistor logic (TTL) signals were used to timestamp onset times from Med-PC V software (Med Associates Inc.) and were detected via the RZ5P in the Synapse software (see below). A built-in low-pass filter on the Synapse software was set to 10 Hz to eliminate noise in the fiber photometry raw data.
Behavioral Experiments:
Latent Inhibition.
Mice received 4 consecutive pre-exposure sessions (days 1 – 4) wherein animals were presented with either a tone (85 dB, 2.5 khz frequency) or light stimulus (total of 33 stimulus presentations per session). The pre-exposure stimuli were presented for a 10 second duration with a variable inter-trial interval (35 – 55 seconds). No footshocks were paired with these stimulus presentations. During 2 consecutive sessions (sessions 5 – 6), mice were then given pseudo-random presentations of house light and tone with a 10 second stimulus duration (6 trials of each) and a variable inter-stimulus interval (60 – 100 seconds). Animals received a footshock (1 mA, .5 second duration) immediately following both house light and tone. Pre-exposure to either house light or tone across all 4 sessions was counterbalanced between animals. Therefore, for half of the animals the tone was the pre-exposed CS+ and light was the non-pre-exposed CS+ and for the other half of the animals, these stimulus roles were reversed. In a subset of animals, we employed a modified pre-exposure paradigm where they received 2 cue presentations instead of 33 and remained in the context for the remainder of the session.
Repeated stimulus exposure.
Intermixed with the CS+ presentations during the last 2 days of the latent inhibition experiment, mice were also presented with an auditory stimulus (white noise, 85 dB) a total of 12 times (6 presentations per session), for 10 seconds with a variable inter-trial interval (35 – 55 seconds) in the absence of the footshock. This was to test the dopamine response patterns during stimulus pre-exposure.
Test of conditioned inhibition following pre-exposure.
In order to test the potential inhibitory properties of cues that are pre-exposed, we ran a “summation test”. Mice received pre-exposure to a light or a tone cue (pre-exposed cue; counterbalanced) as explained above. Then received a session where they received 10 trials of tone- or light-shock pairings (excitor). Finally, for the summation test, mice received 3 trials of each test presentations: excitor alone, excitor and pre-exposed cue together, and excitor and a novel cue (white noise; external inhibition) together and freezing response to each test stimuli was scored.
Optogenetic photostimulation during cue pre-exposure.
TH-Cre rats had previously undergone appetitive training with procedures described elsewhere, utilizing four auditory and two visual cues, which were generated by Coulbourn equipment. For latent inhibition, we used two auditory stimuli generated by an Arduino to produce two very distinct sounds that would be distinguishable from cues used previously (chime and warp). All trials consisted of 10-s presentations of the chime or warp. Training began with 2 days of pre-exposure to the pre-exposed cue (warp or chime, counterbalanced). On each day, rats received 12 presentations of the pre-exposed cue. During pre-exposure, we delivered light into the brain (470nm, 1s, 20Hz) at the onset of the cue. We have previously used a greater number of pre-exposure trials to generate successful latent inhibition57. We used less pre-exposure here as we wanted to give an opportunity to see either an enhancement or reduction in latent inhibition in our experimental group. Following 2 days of pre-exposure, rats received a single critical conditioning session in which the pre-exposed cue and another novel stimulus (chime or warp, counterbalanced) were presented 6 times each followed immediately by delivery of two 45-mg sucrose pellets (5TUT; Test Diet, MO).
In a group of C57BL/6J mice, AAV5.Ef1a.DIO.hChR2.eYFP (ChR2; UNC vector core) and AAV9.rTH.PI.Cre.SV40 (Addgene) were injected into the VTA and a 200um fiber optic implant was placed into the NAc core. This allowed for photostimulation of dopamine response only in dopamine terminals that project from the VTA and synapse in the NAc core. Control animals received AAV5.Ef1a.DIO.eYFP injections into the VTA instead of ChR2. For these experiments, mice were trained utilizing auditory and visual cues generated by MedPC equipment (MED Associates, Inc). For latent inhibition experiments, all trials consisted of 10-s presentations of the tone (5 kHz at 85 dB) or house light. Training began with 4 days of pre-exposure to the pre-exposed cue (tone or house light, counterbalanced). On each day, mice received 30 presentations of the pre-exposed cue. During pre-exposure, we delivered blue laser stimulation (470nm, 1s, 20Hz, 8mW) into the NAc core at the onset of the cue for 1s. Following 4 days of pre-exposure, mice underwent two fear conditioning sessions where the pre-exposed and novel stimuli (tone or house light, counterbalanced) were paired 6 times each with a shock (1mA, 0.5 sec).
In a separate group of C57BL/6J mice, AAV5-Ef1a-DIO.eNpHR.3.0-eYFP (NpHR; Addgene) and AAV9.rTH.PI.Cre.SV40 (Addgene;56) were injected into the VTA and a 200um fiber optic implant was placed into the NAc core. Control animals received AAV5.Ef1a.DIO.eYFP injections into the VTA instead of NpHR. For latent inhibition, all trials consisted of 10-s presentations of the tone (5 kHz at 85 dB) or house light as above. Training began with 4 days of pre-exposure to the pre-exposed cue (tone or house light, counterbalanced). On each day, mice received 30 presentations of the pre-exposed cue. During pre-exposure, we delivered yellow laser stimulation (590nm, 11s, constant, 8mW) into the NAc core at the onset of the cue for 11s. Following 4 days of pre-exposure, mice underwent two fear conditioning sessions where the pre-exposed and novel stimuli (tone or house light, counterbalanced) were paired 6 times each with a shock (1mA, 0.5 sec). Another group of mice received the photostimulation and inhibition of the NAc core dopamine terminals during each inter-trial interval during the pre-exposure when no other stimuli are presented. The photostimulation parameters were identical to those used for cue stimulation/inhibition experiment described above. Finally, we also ran another experiment in a different group of mice, where the animals received photostimulation or inhibition of NAc core terminals as described above but only for the first trial of the fear conditioning session on day 5.
Optogenetic photostimulation during pre-exposure combined with dopamine recording.
In order to manipulate and record dopamine during pre-exposure and fear conditioning, respectively, in a separate group of C57BL/6J mice, Chrimson.FLEX: AAV5-Syn-FLEX-rc[ChrimsonR-tdTomato] (Chrimson; Addgene) and AAV9.rTH.PI.Cre.SV40 (Addgene) were injected into the VTA and AAV5.CAG.dLight1.1 (UC Irvine) was injected into the NAc core as described above. A 200um fiber optic implant was placed into the NAc core. Control animals received only Chrimson.FLEX: AAV5-Syn-FLEX-rc[ChrimsonR-tdTomato] injections into the VTA and AAV5.CAG.dLight1.1 into the NA core. This way, we were able to deliver a yellow laser stimulation (590nm, 11s continuous, 8mW) into the NAc core at the onset of the cue during pre-exposure and record dopamine at the same site using fiber photometry. This method has been validated previously7.
Latent disinhibition via context switch.
To test the effects of cue-context associations on latent inhibition, we ran a “latent disinhibition” experiment. For this experiment, mice were injected with AAV5.CAG.dLight1.1 (UC Irvine) into the NAc core. Following 4 days of pre-exposure to a cue in their regular context, mice received fear conditioning as described above in a novel context. For the novel context, we used MedPC boxes designed for rats with larger dimensions (11.625” L × 9.78” W × 7.35” H). Walls were added within these operant boxes made of cardboard boxes that contained spatial cues such as vertical stripes in addition to an olfactory cue (vanilla extract). All stimuli and experiment parameters were kept constant between the pre-exposure and fear conditioning contexts. NAc core dopamine responses to the cues were recorded in the new context.
Data Analysis and Statistics:
Behavioral Data Analysis.
Statistical analyses were performed using GraphPad Prism (version 8; GraphPad Software, Inc, La Jolla, CA) and Matlab (Mathworks, Natick, MA). Freezing behavior, identified as the time of immobility except respiration during the stimulus duration, was calculated and converted into percent freezing ((freezing time * 100)/stimulus duration)). Two blind reviewers scored all freezing behavior. For the statistical analyses of the freezing behavior during Session1 and Session2 of the latent inhibition experiments, we employed repeated measures ANOVA. For all other freezing data, we used a one-way ANOVA followed by Tukey’s post-hoc analysis. We identified the mice that failed to show latent inhibition based on the second trial freezing response to the pre-exposed versus non-pre-exposed CS+. The mice included in the “No Latent Inhibition” group showed higher freezing response to the pre-exposed CS+ compared to the non-pre-exposed CS+. The data from these mice is included in Extended Data Fig. 4. Alpha was 0.05 for all statistical analysis. All data were depicted as group mean ± standard error of the mean (S.E.M.).
Fiber Photometry Analysis.
The analysis of the fiber photometry data was conducted using a custom Matlab pipeline. Raw 470nm and isosbestic 405nm traces were used to compute ΔF/F values via polynomial curve fitting. For analysis, data was cropped around behavioral events using TTL pulses and for each experiment 2s of pre-TTL up to 20 seconds of post-TTL ΔF/F values were analyzed. In order to remove any movement and photobleaching artifacts, first, we used the isosbestic channel signal (405nm28) to calculate our ΔF/F (ΔF/F =F470-F405)/F405; see Extended Data Fig. 1). In addition, all fiber photometry data were converted to and reported as z-scores. We z-scored dopamine signals around the event of interests such as the CS+ using their own local baseline (2 seconds prior to the cue onset). Z-scores were calculated by taking the pre-TTL ΔF/F values as baseline (z-score = (TTLsignal − b_mean)/b_stdev, where TTL signal is the ΔF/F value for each post-TTL time point, b_mean is the baseline mean, and b_stdev is the baseline standard deviation). This allowed for the determination of dopamine events that occurred at the precise moment of each significant behavioral event. For statistical analysis, we calculated the area under the curve (AUC), peak height, time to baseline, tau, and R2 values for each individual dopamine event (58; see Extended Data Fig. 1 for the visual description of these values). The AUCs were calculated via trapezoidal numerical integration on each of the z-scores across a fixed timescale. The peak height values were the maximum values after the TTL onset. The time to baseline was computed as the seconds to going back to the 0 z-score baseline and tau was the duration to the 2/3 of the peak height. For both of the measures where individual curves did not reach the baseline or tau the minimum value was taken into the statistical analysis. Finally, for slope analysis, we computed the R2 values for the fitted curves (Linear polynomial curve) for a 15 sec duration. The duration of the data collection for the AUC, peak height, time to baseline, and tau values was determined by limiting the analysis to the z-scores between 0 time point (TTL signal onset) and the time where the dopamine peak of interest returns to baseline. Baseline dopamine responses were calculated as the z-scored dopamine values during the inter-trial interval 20 seconds prior to the CS+ presentations. Unpaired t-tests were employed to test the group differences for all fiber photometry-based dependent variables. We also calculated maximum z-scores for event fiber photometry traces and analyzed to see if these were significantly different from the critical z-score at p=0.05 level (1.645) using independent-t-tests. Data distribution was assumed to be normal but this was not formally tested.
Extended Data
Acknowledgements
This work was supported by NIH grants KL2TR002245 to M.G.K, DA055380 and DA048931 to E.S.C., GM07628 to J.E.Z. and DA045103 to C.A.S. as well as by funds from the VUMC Faculty Research Scholar Award to M.G.K., the Pfeil Foundation to M.G.K., Brain and Behavior Research Foundation to M.G.K, E.S.C and C.A.S, the Whitehall Foundation to E.S.C., and the Edward Mallinckrodt, Jr. Foundation to E.S.C. Authors would like to thank Jeff Dunning for his technical support. The opinions expressed in this article are the authors’ own and do not reflect the view of the NIH/DHHS. The authors have no conflicts of interest to report.
Footnotes
Competing Interests Statement
The authors report no conflicts of interest.
Code availability statement
Codes used for the analysis of the fiber photometry data is available at github.com/kutlugunes.
Data availability statement
All data in the manuscript or the supplementary material are available from the corresponding author upon request. Correspondence and requests for materials should be addressed to Erin S. Calipari.
References
- 1.Harris JD Studies on nonassociative factors inherent in conditioning. (Williams & Wilkins, 1943).
- 2.Lubow RE & Moore AU Latent inhibition: the effect of nonreinforced pre-exposure to the conditional stimulus. Journal of comparative and physiological psychology 52, 415 (1959). [DOI] [PubMed] [Google Scholar]
- 3.Kamprath K & Wotjak CT Nonassociative learning processes determine expression and extinction of conditioned fear in mice. Learning & memory 11, 770–786 (2004). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Harris JD Habituatory response decrement in the intact organism. Psychological bulletin. 40, 385–422 (1943). [Google Scholar]
- 5.Rescorla RA Effects of US habituation following conditioning. Journal of comparative and physiological psychology 82, 137 (1973). [DOI] [PubMed] [Google Scholar]
- 6.Pavlov IP Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex. (Oxford Univ. Press, 1927). [Google Scholar]
- 7.Kutlu MG et al. Dopamine release in the nucleus accumbens core signals perceived saliency. Current Biology 31, 4748–4761.e8 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lubow RE Latent inhibition. Psychological bulletin 79, 398 (1973). [DOI] [PubMed] [Google Scholar]
- 9.Rescorla R & Wagner A A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning II: Current Research and Theory vol. Vol. 2 (1972). [Google Scholar]
- 10.Mackintosh NJ A theory of attention: variations in the associability of stimuli with reinforcement. Psychological review 82, 276 (1975). [Google Scholar]
- 11.Pearce JM & Hall G A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological review 87, 532 (1980). [PubMed] [Google Scholar]
- 12.Schmajuk NA, Lam Y-W & Gray JA Latent inhibition: A neural network approach. Journal of Experimental Psychology: Animal Behavior Processes 22, 321 (1996). [DOI] [PubMed] [Google Scholar]
- 13.Saunders BT, Richard JM, Margolis EB & Janak PH Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nature neuroscience 21, 1072–1083 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Oleson EB, Gentry RN, Chioma VC & Cheer JF Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance. Journal of Neuroscience (2012) doi: 10.1523/JNEUROSCI.3087-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Young AMJ, Joseph MH & Gray JA Latent inhibition of conditioned dopamine release in rat nucleus accumbens. Neuroscience 54, 5–9 (1993). [DOI] [PubMed] [Google Scholar]
- 16.Joseph MH, Peters SL & Gray JA Nicotine blocks latent inhibition in rats: evidence for a critical role of increased functional activity of dopamine in the mesolimbic system at conditioning rather than pre-exposure. Psychopharmacology 110, 187–192 (1993). [DOI] [PubMed] [Google Scholar]
- 17.Morrens J, Aydin Ç, Janse van Rensburg A, Esquivelzeta Rabell J & Haesler S Cue-Evoked Dopamine Promotes Conditioned Responding during Learning. Neuron. 106, 142–153.e7 (2020). [DOI] [PubMed] [Google Scholar]
- 18.Hall G & Channell S Context specificity of latent inhibition in taste aversion learning. The Quarterly Journal of Experimental Psychology 38, 121–139 (1986). [PubMed] [Google Scholar]
- 19.Killcross S & Balleine B Role of primary motivation in stimulus preexposure effects. Journal of Experimental Psychology: Animal Behavior Processes 22, 32 (1996). [DOI] [PubMed] [Google Scholar]
- 20.Westbrook RF, Jones ML, Bailey GK & Harris JA Contextual control over conditioned responding in a latent inhibition paradigm. Journal of Experimental Psychology: Animal Behavior Processes 26, 157 (2000). [DOI] [PubMed] [Google Scholar]
- 21.Hart AS, Rutledge RB, Glimcher PW & Phillips PEM Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. Journal of Neuroscience (2014) doi: 10.1523/JNEUROSCI.2489-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.de Jong JW et al. A Neural Circuit Mechanism for Encoding Aversive Stimuli in the Mesolimbic Dopamine System. Neuron (2019) doi: 10.1016/j.neuron.2018.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Schultz W, Dayan P & Montague PR A neural substrate of prediction and reward. Science (1997) doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
- 24.Menegas W, Babayan BM, Uchida N & Watabe-Uchida M Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. eLife (2017) doi: 10.7554/eLife.21886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Nolan SO et al. Direct dopamine terminal regulation by local striatal microcircuitry. Journal of Neurochemistry (2020) doi: 10.1111/jnc.15034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Cragg SJ, Rice ME & Greenfield SA Heterogeneity of electrically evoked dopamine release and reuptake in substantia nigra, ventral tegmental area, and striatum. Journal of Neurophysiology (1997) doi: 10.1152/jn.1997.77.2.863. [DOI] [PubMed] [Google Scholar]
- 27.Mohebi A et al. Dissociable dopamine dynamics for learning and motivation. Nature (2019) doi: 10.1038/s41586-019-1235-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Patriarchi T et al. Ultrafast neuronal imaging of dopamine dynamics with designed genetically encoded sensors. Science (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Pearce JM A model for stimulus generalization in Pavlovian conditioning. The psychological review. 94, 61–73 (1987). [PubMed] [Google Scholar]
- 30.Budygin EA et al. Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience (2012) doi: 10.1016/j.neuroscience.2011.10.056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Young AMJ Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: Studies using 1 min microdialysis in rats. Journal of Neuroscience Methods (2004) doi: 10.1016/j.jneumeth.2004.03.003. [DOI] [PubMed] [Google Scholar]
- 32.Lubow RE, Rifkin B & Alek M The context effect: The relationship between stimulus preexposure and environmental preexposure determines subsequent learning. Journal of Experimental Psychology: Animal Behavior Processes 2, 38–47 (1976). [Google Scholar]
- 33.Day JJ, Roitman MF, Wightman RM & Carelli RM Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 10, 1020–1028 (2007). [DOI] [PubMed] [Google Scholar]
- 34.Bouton ME Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin 114, 80–99 (1993). [DOI] [PubMed] [Google Scholar]
- 35.Hall G & Honey RC Contextual effects in conditioning, latent inhibition, and habituation: Associative and retrieval functions of contextual cues. Journal of Experimental Psychology: Animal Behavior Processes 15, 232 (1989). [Google Scholar]
- 36.Sokolov EN Neuronal models and the orienting reflex. The central nervous system and behavior, 3rd Conference 187–276 (1960). [Google Scholar]
- 37.Jacob PF et al. Prior experience conditionally inhibits the expression of new learning in Drosophila. Current Biology 31, 3490–3503.e3 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Corbit LH, Muir JL & Balleine BW The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. Journal of neuroscience 21, 3251–3260 (2001). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Westbrook RF, Bond NW & Feyer A-M Short- and long-term decrements in toxicosis-induced odor-aversion learning: The role of duration of exposure to an odor. Journal of Experimental Psychology: Animal Behavior Processes 7, 362–381 (1981). [PubMed] [Google Scholar]
- 40.Ayres JJB, Philbin D, Cassidy S, Bellino L & Redlinger E Some parameters of latent inhibition. Learning and Motivation 23, 269–287 (1992). [Google Scholar]
- 41.Schnur P & Lubow RE Latent inhibition: The effects of ITI and CS intensity during preexposure. Learning and Motivation 7, 540–550 (1976). [Google Scholar]
- 42.Rankin CH et al. Habituation revisited: An updated and revised description of the behavioral characteristics of habituation. Neurobiology of Learning and Memory 92, 135–138 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Delacasa G & Lubow RE Latent Inhibition in Conditioned Taste Aversion: The Roles of Stimulus Frequency and Duration and the Amount of Fluid Ingested During Preexposure. Neurobiology of Learning and Memory 64, 125–132 (1995). [DOI] [PubMed] [Google Scholar]
- 44.Eshel N, Tian J, Bukwich M & Uchida N Dopamine neurons share common response function for reward prediction error. Nat. Neurosci. 19, 479–486 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Schultz W, Dayan P & Montague PR A neural substrate of prediction and reward. Science 275, 1593–1599 (1997). [DOI] [PubMed] [Google Scholar]
- 46.Killcross AS, Dickinson A & Robbins TW The on-baseline latent inhibition effect is not counterconditioning. Psychopharmacology (Berl) 118, 104–106 (1995). [DOI] [PubMed] [Google Scholar]
- 47.Redgrave P, Prescott TJ & Gurney K Is the short-latency dopamine response too short to signal reward error? Trends in neurosciences 22, 146–151 (1999). [DOI] [PubMed] [Google Scholar]
- 48.Kakade S & Dayan P Dopamine: generalization and bonuses. Neural networks. 15, 549–559 (2002). [DOI] [PubMed] [Google Scholar]
- 49.Horvitz JC Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651–656 (2000). [DOI] [PubMed] [Google Scholar]
- 50.McDiarmid TA, Bernardos AC & Rankin CH Habituation is altered in neuropsychiatric disorders—A comprehensive review with recommendations for experimental design and analysis. Neuroscience & Biobehavioral Reviews 80, 286–305 (2017). [DOI] [PubMed] [Google Scholar]
- 51.Teo C et al. Decreased habituation of midlatency auditory evoked responses in Parkinson’s disease. Movement disorders: official journal of the Movement Disorder Society 12, 655–664 (1997). [DOI] [PubMed] [Google Scholar]
- 52.Rey RD et al. The effect of levodopa on the habituation of the acoustic-palpebral reflex in Parkinson’s disease. Electromyography and clinical neurophysiology 36, 357–360 (1996). [PubMed] [Google Scholar]
- 53.Kutlu MG & Schmajuk NA Solving Pavlov’s puzzle: Attentional, associative, and flexible configural mechanisms in classical conditioning. Learning and Behavior (2012) doi: 10.3758/s13420-012-0083-5. [DOI] [PubMed] [Google Scholar]
- 54.Stalnaker TA et al. Dopamine neuron ensembles signal the content of sensory prediction errors. ELife. 8, e49315 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Sharpe MJ et al. Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nature Neuroscience (2017) doi: 10.1038/nn.4538. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Parker KE et al. A Paranigral VTA Nociceptin Circuit that Constrains Motivation for Reward. Cell. 178, 653–671.e19 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Sharpe MJ, Batchelor HM, Mueller LE, Gardner MPH & Schoenbaum G Past experience shapes the neural circuits recruited for future learning. Nat Neurosci 24, 391–400 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Yorgason JT, España RA & Jones SR Demon Voltammetry and Analysis software: Analysis of cocaine-induced alterations in dopamine signaling using multiple kinetic measures. Journal of Neuroscience Methods 202, 158–164 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All data in the manuscript or the supplementary material are available from the corresponding author upon request. Correspondence and requests for materials should be addressed to Erin S. Calipari.