A Role for Phasic Dopamine Release within the Nucleus Accumbens in Encoding Aversion: A Review of the Neurochemical Literature

Abstract

Survival is dictated by an organism’s fitness in approaching positive stimuli and avoiding harm. While a rich literature outlines a role for mesolimbic dopamine in reward and appetitive behaviors, dopamine’s involvement in aversion and avoidance behaviors remains controversial. Debate surrounding dopamine’s function in the processing of negative stimuli likely stems from conflicting results reported by single-unit electrophysiological studies. Indeed, a number of studies suggest that midbrain dopaminergic cells are inhibited by the presentation of negative or fearful stimuli, while others report no change, or even an increase, in their activity. These disparate results may be due to population heterogeneity. Recent evidence demonstrates that midbrain dopamine neurons are heterogeneous in their projection targets, responses to environmental stimuli, pharmacology, and influences on motivated behavior. Thus, in order to assemble an accurate account of dopamine function during aversive stimulus experience and related behavior, it is necessary to examine the functional output of dopamine neural activity at mesolimbic terminal regions. This Review presents a growing body of evidence that dopamine release in the nucleus accumbens encodes not only reward, but also aversion. For example, our laboratory recently utilized fast-scan cyclic voltammetry to show that real-time changes in accumbal dopamine release are detected when animals are presented with predictors of aversion and its avoidance. These data, along with other reports, support a considerably more nuanced view of dopamine neuron function, wherein accumbal dopamine release is differentially modulated by positive and negative affective stimuli to promote adaptive behaviors.

Graphical Abstract

The mesolimbic dopamine system is a subcortical pathway highly conserved across vertebrates,¹ commonly thought to underlie the generation of reward-seeking actions. Currently accepted theories suggest that transient mesolimbic dopamine release is involved in assessing the value of reward-predictive stimuli^2–4 and in generating motivated actions directed toward obtaining reward.^2,5–7 Survival, however, requires an organism to not only seek and approach positive/rewarding stimuli, but, also, to avoid negative/aversive stimuli. At question is whether and in what capacity does the mesolimbic system contribute to the avoidance of negative stimuli. Recent findings suggest that our reward-centric conceptualization of dopamine transmission is far from complete. Rather, we propose that the mesolimbic system uses transient dopamine release events to guide all advantageous behavio, regardless of whether the outcome involves the receipt of reward or the avoidance of harm. Here we will review the electrophysiological and electrochemical literature in an attempt to reconcile discrepant findings on the role of the mesolimbic system in processing aversion and its successful avoidance.

INTRODUCTION TO THE MESOLIMBIC DOPAMINE SYSTEM

The mesolimbic dopamine system is theorized to promote motivated actions by generating a teaching signal that draws animals toward favorable stimuli and, possibly, away from harmful stimuli.^8,9 The mesolimbic pathway originates from A10 dopamine neurons in the ventral tegmental area (VTA) of the midbrain and projects to motivational circuitry, most prominently the nucleus accumbens (NAcc).^10,11 Midbrain dopamine neurons work to coordinate behavior by firing in two distinct patterns. At “rest”, these cells are tonically active, firing at low frequencies (1–5 Hz); this baseline firing rate produces a dopaminergic tone on high-affinity dopamine D2 receptors in the NAcc.^12–14 In contrast, when animals are presented with motivationally relevant stimuli, for example, reward-predictive cues, VTA dopamine neurons fire in high frequency bursts (≥20 Hz),¹⁴ resulting in enhanced extracellular dopamine at terminal regions sufficient to occupy low-affinity accumbal dopamine D1 receptors.¹³ It should be noted, however, that burst activity of dopamine cells and transient dopamine release events also occur spontaneously or following the presentation of presumably nonaffective salient stimuli (e.g., visual or auditory stimuli).^15–18 Experimentally, one can detect tonic dopamine levels using techniques such as microdialysis, which allows for neurochemical detection on a time scale of minutes. However, transient fluctuations in extracellular dopamine release, like those associated with the presentation of reward-predictive cues, are best measured by tools with greater temporal precision such as fast-scan cyclic voltammetry (FSCV), an electrochemical technique that allows for the detection of dopamine on a time scale of milliseconds.

Mesolimbic dopamine signals arriving in the NAcc modulate the activity of projection neurons to control behavior. The NAcc itself is fittingly described as a limbic-motor¹⁹ and a Pavlovian-instrumental²⁰ interface, conveying the important theoretical construct that this region is critically involved in transforming information from affective environmental cues into actions devoted to obtaining favorable outcomes (i.e., approach or avoidance). The widely accepted view of NAcc circuitry suggests that neural signals pertaining to motivation-ally relevant stimuli arise from frontal and/or limbic regions and synapse on accumbal γ-aminobutyric acid (GABA)-ergic medium spiny neurons (MSNs). The established view of the basal ganglia suggests that these NAcc MSNs are largely segregated into two distinct populations, expressing either D1 or D2 receptors and projecting to either the direct- or indirect-pathway of the basal ganglia. Although it should be noted that 25% of NAcc MSNs express both D1 and D2 receptors.²¹ In general, direct pathway MSNs express dopamine D1 G_s-coupled receptors, project to the midbrain, and promote goal-directed action. Indirect pathway MSNs express dopamine D2 G_i/o-coupled receptors, project to the pallidum, and promote avoidance behavior^22–26 However, a recent report by Cui and colleagues²⁷ calls this canonical view of striatal motivational circuitry into question by demonstrating that activation of both pathways occurs during the initiation of goal-directed actions. Thus, while the current theory provides a convenient framework in which to examine meso-striatal signaling, which will be implemented in this Review, this theory is not complete and more research is necessary to fully understand the relationship between extracellular NAcc dopamine, activation of direct and indirect pathways, and stimulus-driven behavior.

DOPAMINE AS A TEACHING SIGNAL

The characteristic tonic activity of midbrain dopaminergic cells and their tendency to spontaneously burst fire, allow for dopamine cells to be bidirectionally modulated.²⁸ Indeed, electrophysiological recordings from dopamine neurons suggest that the presentation of unexpected positive/rewarding stimuli enhance the firing rate of VTA dopamine cells, while, conversely, midbrain dopamine cells are silenced by unpredicted negative/aversive stimuli.^29–31 Interestingly, after several presentations, when a stimulus becomes “expected”, modulation of dopamine cell activity at stimulus receipt now occurs at the earliest predictor of stimulus delivery, that is, stimulus predictive cues.^32,33 These observations, presented within the context of the Rescorla–Wagner model of associative learning,³⁴ led to the development of the prominent reward prediction error (RPE) theory of dopamine neural activity.^8,29

RPE states that alterations in dopamine release represent the discrepancy between the stimulus received and the expected value of that stimulus predicted by a preceding environmental cue.^8,29,35,36 This is to say that unexpected rewarding or aversive stimuli result in net positive and negative prediction errors, respectively, resulting in subsequent increases or decreases in dopaminergic cell activity. Following several presentations, stimuli become expected, and RPE theory holds that if the stimulus delivered matches an organism’s expectation, no further modulation of dopamine activity will follow. Instead, dopamine transitions to serve as a prediction signal, responding to cues that precede positive stimuli with an increase in cell firing rate, and cues that predict negative events with a decrease. Still, if there is a discrepancy between what is predicted and what actually occurs, such as omission of an expected reward, this results in alterations in dopaminergic activity commensurate with the direction and magnitude of the prediction error. RPE signals are thus theorized to update an animal’s memory of stimulus-outcome associations, thereby improving behavioral fitness. Interestingly, FSCV studies suggest that this dopaminergic signal “shift” from representing unconditioned to conditioned stimuli is much more nuanced than is typically discussed. Indeed, several reports show dopamine release at both the presentation of rewarding stimuli as well as their predictive cues within the same trial (e.g., refs 37 and 38). This is in contrast to the classic RPE experiments performed by Schultz and colleagues^29,39 wherein dopaminergic cell activation is reported distinctly at primary reward delivery early in training or reward-predictive cues late in training. These disparate observations may be due to overtraining, regional differences (i.e., electrophysiological recordings in the substantia nigra versus electrochemical recording of dopamine release in the accumbens), species differences, or a combination of these factors and deserve greater consideration within the current literature.

Altogether, the RPE hypothesis in conjunction with currently accepted theories of basal ganglia function suggests that rewards, or reward-predictive cues, enhance burst firing of midbrain dopamine neurons, increase accumbal extracellular dopamine concentrations sufficiently to occupy low-affinity D1 receptors, and activate direct pathway MSNs to initiate reward-directed appetitive behaviors. Conversely, exposure to aversive stimuli or predictive cues is hypothesized to attenuate midbrain dopamine neuron activity, reduce extracellular NAcc dopamine, and alleviate D2-mediated inhibitory tone on indirect pathway MSNs to initiate passive avoidance behavior (e.g., freezing). However, several electrophysiological reports present contradictory results.

ELECTROPHYSIOLOGICAL DATA PRESENTS A COMPLICATED PICTURE OF DOPAMINERGIC RESPONSE TO AVERSIVE STIMULI

A number of electrophysiological studies suggest that subpopulations of dopamine neurons are excited, rather than inhibited, by aversive stimuli.^{17,30,40–44} These divergent findings may suggest heterogeneity of function among dopamine neurons or variability in investigative methods. For example, several studies examining dopamine neuronal responses to negative stimuli (e.g., toe pinch) apply these stimuli while animals are under anesthesia. This design presupposes that painful stimuli are indeed experienced as aversive while under anesthesia, an assumption that cannot be verified. Further, anesthesia itself may affect dopaminergic function.⁴⁵ Additionally, it is difficult to discern if dopamine cells are responding to the application of aversive stimuli or to their removal (e.g., the termination of toe pinch), which is likely rewarding. Alternatively, dopaminergic activity in response to negative stimuli may result from rebound excitation of VTA dopaminergic cells witnessed following aversion-induced dopaminergic cell suppression.^46,47 Finally, these investigations utilize various types of negative stimuli, such as tail pinch, foot shock, a puff of air delivered to the eye, and aversive tastants (e.g., quinine). Clearly these aversive stimuli vary in transduction pathway and likely differ in sensory intensity, making it difficult to compare neurobiological findings across reports.

In a current review, Ungless and Grace⁴⁸ suggest that heterogeneity in dopamine neuronal response to aversive stimuli may be due to a misidentification of VTA dopamine neurons. Most studies select putative dopamine neurons based solely on electrophysiological properties, a practice which may result in the inclusion of nondopaminergic cells that possess similar electrophysiological profiles. Indeed, as described by Ungless and Grace, individual differences in electrodes or their distance from neuronal targets can affect features of extracellular waveforms. Further, filtering of electrophysiological data, such as high-pass filtering commonly utilized in in vivo electrophysiology to minimize cardiac signals, result in artifacts which themselves may lead to misidentification of dopaminergic cells. Considering these caveats, a population of nondopaminergic cells within the VTA has been identified with similar electrophysiological characteristics to dopamine cells, except narrower action potentials.⁴⁹ Indeed, presumed dopamine neurons excited by aversive stimuli often exhibit action potentials slightly narrower than those of VTA dopamine cells inhibited by negative stimuli.⁵⁰ To investigate the possible misidentification of nondopaminergic cells, Ungless et al.⁵¹ selected VTA dopaminergic neurons based on their electrophysiological profiles and recorded extracellular unit activity from single VTA neurons in anesthetized rats during toe pinch. In congruence with previous reports, the authors observed both excitatory and inhibitory responses to this negative stimulus. Neurons were then labeled and neurochemically characterized with immunofluorescence for tyrosine hydroxylase (TH), an enzyme involved in dopamine synthesis. As hypothesized, neurons identified as immunopositive for TH were inhibited by toe pinch, while TH-negative (TH–) neurons were excited by this aversive stimulus. These TH– cells were located close to TH+ cells and exhibited similar burst firing and spike characteristics to dopaminergic cells. However, TH- neurons had narrower action potentials, lending support to the theory that previous reports misidentified dopaminergic cells. These data highlight the need to implement more rigorous cellular identification protocols, such as complementing electrophysiological assessment with immunoflourescent verification, or excluding neurons that exhibit this narrow action potential profile.

The implementation of strict dopaminergic cell identification practices, however, reveals that there are indeed subpopulations of dopamine cells activated by aversive stimuli. Brischoux and colleagues⁴¹ found that while a majority of TH+ VTA dopamine neurons are inhibited or show no response to electrical shock to the hind paw, some dopaminergic neurons in the ventral VTA are excited by this negative stimulus. The animals utilized here, as well as in a number of other studies examining dopaminergic neuron response to negative stimuli,^42,43,51,52 were unfortunately tested under anesthesia, thus calling into question how anesthetics may alter electro-physiological responses to applied stimuli. However, these data present evidence that anatomically segregated dopaminergic cell populations may demonstrate different responses to affective stimuli. The authors take these results to suggest two functional dopamine systems within the VTA, a subsystem in the dorsal VTA that produces RPE signals, and another in the ventral VTA theorized to be activated by all salient stimuli, regardless of valence. Indeed, a similar dorsal to ventral gradient of dopamine cell function is also reported by Matsumoto and Hikosaka.³⁰ Given that various subgroups of midbrain dopamine cells receive inputs from different regions and project to diverse anatomical targets,^53,54 it is possible that dorsal VTA neurons coding for reward and aversion project preferentially to the NAcc, while ventral VTA neurons may represent a saliency signal that projects to alternate mesolimbic regions involved in attention and motor processes.⁵⁵ Although these studies present valuable information regarding heterogeneity of midbrain dopamine neurons, electrophysiological assessments cannot determine the functional result of dopamine release at terminal regions. Thus, investigating the global result of phasic dopamine neural activation (by measuring changes in accumbal dopamine concentration) during presentation of negative stimuli is a timely and relevant question that will address a growing controversy in the dopamine/reward subfield.

NAcc DOPAMINE RELEASE IN RESPONSE TO AFFECTIVE STIMULI

The majority of research investigating regional dopamine release during affective stimuli presentation and motivated behavior has focused on the NAcc. The NAcc itself is a heterogeneous structure that can be divided into distinct core and shell subregions with different afferent and efferent connections–the NAcc core receives projections from prelimbic cortex and projects to basal ganglia circuitry, while the NAcc shell receives projections from infralimbic cortex and projects to subcortical limbic structures.⁵⁶ Further, there are afferents from the accumbens core to the shell, but few fibers projecting from the shell to the core, suggesting the dopaminergic signals converging on the accumbens are capable of modulating diverse inputs to each subregion which is then integrated through core–shell projections.^57–59 In consideration of this, the current Review will discuss dopamine release data in the context of NAcc core and shell subregions in an attempt to differentiate dopaminergic function within each area.

MICRODIALYSIS STUDIES

Microdialysis, while providing a sensitive measure of extracellular dopamine, is not an ideal technique to assess for neurochemical responses to transient environmental stimuli due to its broad temporal window. However, a large body of evidence suggests that the presentation of known primary rewards such as food or drugs of abuse enhance extracellular accumbal dopamine concentrations.^60–66 Several studies also show that aversive or noxious events result in the rise of either dopamine or dopamine metabolite in the NAcc.^67–73 For example, Abercrombie and colleagues⁷⁴ found that intermittent tail shock enhances NAcc dopamine release in rats. Although they do not specify whether dopamine release was measured in the NAcc core or shell, their stereotaxic coordinates suggest that microdialysis probes were in the NAcc shell. The experimental protocol utilized in this study, however, draws into question the suitability of microdialysis for measuring discrete stimulus presentations. In this report, 10 1 mA tail shocks were delivered over 1 min every 5 min for a total test session lasting 30 min. During this time, two microdialysis samples (each 15 min in length) were collected and analyzed for dopamine. Thus, each neurochemical sample contained dialysate from 30 shock presentations and terminations, and a total of 12 min of session time consisting of no shock. This wide time frame makes it difficult to reconcile if either the presentation of tail shock or its removal is responsible for the observed increase in extracellular dopamine. A later report by Kalivas and Duffy^69b examined 20 min dialysis samples before, during, and after a more uniform foot shock exposure protocol (0.35 mA shock lasting 200 ms of every second for 20 min). This protocol revealed no change in NAcc shell dopamine levels during the 20 min foot shock presentation, but a significant increase in shell dopamine in the 20 min collection period following foot shock termination. These data suggest that relief from aversive stimuli results in NAcc shell dopamine release; however, no reduction in NAcc dopamine was observed during shock application, as would be predicted from electrophysiological data. Altogether, microdialysis data provide valuable information about prolonged changes in extracellular dopamine over longer periods of time; however, it is difficult to discern if dopamine fluctuations are due to aversive stimulus application or relief. Further, in order to examine phasic dopamine release related to discrete cues, a technique with much faster temporal resolution is required, such as FSCV.

TRANSIENT ACCUMBAL DOPAMINE RELEASE ACCOMPANIES REWARD RECEIPT, PRESENTATION OF REWARD-RELATED CUES, AND OPERANT RESPONDING FOR POSITIVE REINFORCERS

FSCV allows for the analysis of NAcc dopamine release with a temporal resolution of milliseconds, making it an ideal tool with which to examine phasic dopamine release in the accumbens. FSCV also allows for a high degree of spatial precision with carbon fiber microelectrodes measuring just 10 μm in diameter, several orders of magnitude smaller than microdialysis probes, allowing for definitive measurement of phasic dopamine release within NAcc subregions. Research employing FSCV demonstrates that the presentation of a number of rewarding stimuli produce transient increases in extracellular DA concentration (termed “transients”) within the NAcc core and/or shell.^38,75–79 These data align well with electrophysiological studies showing phasic burst firing of midbrain dopamine neurons following reward receipt.^29,80,81 Also, in support of the RPE hypothesis, phasic accumbal dopamine release is reliably observed following unexpected reward delivery or, following conditioning, in response to cues that predict reward.^{75,76,78,82–84} Indeed, our group recently used FSCV to demonstrate that phasic dopamine release is evoked in the NAcc core by the presentation of conditioned predictors of brain stimulation reward or food^85,86 (Figure 1). These dopamine signals were sensitive to changes in reward value and facilitated cue-motivated reward seeking.

Figure 1.

Accumbal dopamine release in the NAcc core evoked by a food predictive cue during a food-seeking task. Top shows voltammetric current in false color plotted against applied voltage (ordinate) and acquisition time (abscissa). Only a single cue-evoked dopamine response (green hot spots in top color plot; concentration trace over time on bottom) is observed per trial after multiple training sessions. (Figure reproduced with permission from ref 76.)

However, not only the pursuit of reward but also the avoidance of harm requires activation of motivational systems, which energize action sequences that ultimately promote survival. Yet, behaviorally relevant transient dopamine release events are almost exclusively studied during the pursuit of reward. Our group recently published data that appear to be at odds with the reward-centric nature of the RPE theory. In this report, we observed transient dopamine release events during the presentation of an environmental cue that guides motivated actions devoted to avoiding an aversive event.³⁷ Thus, it is possible that the RPE theory of dopamine neural activity is myopic in scope, and the mesolimbic dopamine pathway may use environmental associations to guide all motivated actions as animals seek valuable outcomes to maximize their behavioral fitness.

MESOLIMBIC DOPAMINE ENCODES AVERSION AND ITS CONDITIONED PREDICTORS

An elegant study by Roitman et al.^{77 fi}rst demonstrated that, in freely moving animals, phasic accumbal dopamine release oppositely encodes rewarding and aversive stimuli. Specifically, they demonstrated that passive administration of an aversive tastant (quinine) suppresses, whereas administration of an appetitive sucrose solution enhances, the frequency of NAcc shell dopamine transients.⁷⁷ Unlike previous investigations, Roitman and colleagues utilized gustatory stimuli transduced along the same sensory pathway, yet opposite in hedonic valence, allowing for a more direct comparison of dopamine release following aversive versus rewarding stimulation. Similarly, in a recent study, Willuhn and colleagues⁷⁹ analyzed NAcc dopamine transient activity within the core and shell during the presentation of either positive 50 kHz or aversive 22 kHz ultrasonic vocalizations (USVs) in rats, auditory stimuli sharing sensory transduction pathways, but opposite in affective valence. USVs are believed to serve as affective communication signals between rodents.^87,88 Indeed, previous studies show that 50 kHz USVs induce approach behavior^89,90 and activate the reward-related brain regions,⁹¹ while 22 kHz USVs lead to behavioral inhibition^89,92 and activate stress-related brain regions.⁹¹ In their investigation, Willuhn et al. found that playback of positive 50 kHz calls elicited approach behavior toward the speaker and induced phasic dopamine release in both NAcc subregions, while negative 22 kHz calls resulted in behavioral inhibition but no change in accumbal dopamine release. These data support a role for NAcc dopamine specifically in the processing of positive stimuli. However, in this particular report, the authors do not show spontaneous accumbal dopamine activity prior to 22 kHz USV presentation, making it difficult to discern if dopamine is indeed unresponsive to these negative vocalizations, or if perhaps recordings were conducted in an area of the NAcc that is dopamine-poor. Future work should aim to clarify this point. Regardless, these studies highlight a role for accumbal dopamine in the processing of primary aversive stimuli.

The experience of quinine versus that of sucrose, or 50 kHz versus 22 kHz USVs, while sharing sensory transduction mechanisms, likely do not share the same sensory intensity given that these are inherently different stimuli. Thus, in order to compare positive and negative RPEs more directly, McCutcheon and colleagues⁹³ utilized a conditioned taste aversion (CTA) paradigm to compare dopaminergic response to a sucrose solution in two groups of rats: one group which had previously received sucrose paired with an aversive stimulus (an i.p. injection of lithium chloride) and one group that had received sucrose paired with an injection of saline. In a CTA procedure, neutral or rewarding tastants are paired with stimuli that induce intestinal malaise (i.e., lithium chloride), and thus, through Pavlovian conditioning, they themselves become aversive and promote avoidance behavior. Previous reports suggest that dopaminergic transmission in the NAcc is required for lithium-induced CTAs.⁹⁴ In this study, rats that previously received sucrose in conjunction with saline (i.e., did not develop a CTA) exhibited enhanced NAcc shell dopamine release following sucrose administration. Conversely, in those animals that previously received sucrose in conjunction with lithium chloride injection, sucrose administration resulted in an inhibition of NAcc shell dopamine transients. Thus, the same stimulus is able to elicit opposite dopaminergic responses based upon prior conditioning. This observation led to speculation that a decrease in accumbal dopamine release events might also encode other conditioned aversive stimuli such as fear memories.

In a traditional Pavlovian fear-conditioning task, an initially neutral stimulus (e.g., a tone or light) is paired with an aversive unconditioned stimulus (UCS) (e.g., foot shock). During this initial fear-conditioning phase, the animal begins to freeze when the tone in presented. This freezing response is thought to be a behavioral manifestation of fear, and is commonly observed in prey animals, such as rodents, during exposure to threatening stimuli.⁹⁵ On the next day, postconditioning, the now conditioned stimulus (CS) is presented to the animal and conditioned fear (the conditioned response) is measured as the amount or magnitude of freezing behavior exhibited. The percentage of time spent freezing, however, dissipates over repeated tone presentations as the fearful memory extinguishes over repeated presentations. A wealth of evidence suggests that dopamine is involved in the acquisition and expression of conditioned fear.⁹⁶ Indeed, systemic injection of the indirect dopamine agonist amphetamine enhances conditioned fear responses and attenuates extinction of conditioned fear.^97,98 Conversely, the dopamine receptor antagonist haloperidol blocks the acquisition of conditioned fear.⁹⁹ In line with these data, studies on conditioned punishment show that dopamine agonists enhance, while dopamine antagonists diminish, the punishing efficacy of an aversive CS.¹⁰⁰ Altogether, these data suggest a physiological role for endogenous dopamine in the acquisition and expression of conditioned fear. Fear conditioning, however, requires the function of several mesolimbic terminal regions, including the prefrontal cortex, amygdala, and NAcc.¹⁰¹ Therefore, in order to determine the precise function of NAcc dopamine, local infusion of dopaminergic drugs and neurochemical monitoring is required.

Microdialysis studies demonstrate that exposure to sensory stimuli previously paired with foot shock delivery elicits NAcc dopamine release.^102–104 For example, a report by Young and colleagues¹⁰² shows that while NAcc dopamine is augmented by foot shock delivery, the presentation of a conditioned stimulus (i.e., a light cue) in conjunction with foot shock results in even greater NAcc dopamine release. In this report, the authors did not distinguish between core and shell subregions; however, their stereotaxic coordinates suggest microdialysis probe placement in the core. Further, Wilkinson et al.¹⁰⁵ found that, throughout a conditioning session, each subsequent tone-shock pairing resulted in a gradual increase in NAcc shell dopamine, suggesting that dopamine in the NAcc may be important for the development of CS–US associations. In support of this theory, NAcc dopamine decreases as the tone-shock pairing is extinguished.¹⁰⁵ Further, local NAcc core infusion of the nonselective dopamine antagonist haloperidol impairs both acquisition and extinction of conditioned fear,¹⁰⁶ suggesting that accumbal dopaminergic tone facilitates learning and maintenance of aversive CS-US pairings. Interestingly, intra-NAcc shell delivery of the D1 antagonist SCH23390 increases the expression of conditioned fear,¹⁰⁷ suggesting that phasic accumbal dopamine signaling also plays a role in fear conditioning, possibly through regulation of RPE signals.

In order to examine the role of phasic dopamine and RPE in conditioned fear, our group recently employed FSCV to examine accumbal dopamine release during conditioned fear.³⁷ Rats were conditioned to associate a 20 s tone with the delivery of foot shock. Twenty-four hours after conditioning, animals underwent a single test session during which they were presented with the CS (tone) without shock 18 times (Figure 2A). During this test session we continually assessed freezing behavior and utilized FSCV to record phasic dopamine release within the NAcc core. In agreement with previous reports, on day 2, rats exhibited the maximal amount of freezing to the first few tone presentations, however, as the session continued, the freezing response extinguished (Figure 2B). Importantly, tone presentations that resulted in freezing also suppressed dopamine release events detected in the NAcc (Figure 2C,D). These data align with previous reports from the Roitman group showing that the aversive stimuli (i.e., quinine) or conditioned aversive stimuli (i.e., sucrose that has been associated with lithium chloride) result in decreased dopamine release in the NAcc.^77,93 However, it should be noted that while studies examining aversive or conditioned aversive tastants show a decrease in dopamine in the NAcc shell, this conditioned fear investigation reports a decrease in dopamine release within the NAcc core (dopamine release in the NAcc shell was not examined).

Figure 2.

Dopamine release in the NAcc core is time-locked to the presentation of conditioned stimuli predicting aversive stimuli. (A) This fear conditioning procedure consisted of three tone-shock parings on conditioning day 1. Twenty-four hours later (on day 2), retrieval of a conditioned fear memory (measured by freezing behavior) was assessed as rats received 18 presentation of the conditioned stimulus (CS) only (a 20 s auditory tone, denoted in this figure by the trumpet symbol). (B) The freezing response elicited by fear conditioning is extinguished over the 18 successive CS presentations on day 2. (C) CS-induced decrease in NAcc core dopamine release during trial one represented by color plot (left) and associated dopamine concentration trace (right). (D) Mean ± SEM dopamine concentration trace during presentations of CS resulting in a conditioned freezing response, with CS duration represented in gray (figure reproduced with permission from ref 77).

The current literature suggests that phasic dopamine release within NAcc subregions is differentially involved in reaction to primary rewards (core and shell)^108,109 versus reward-predictive stimuli (core),^64,38,110 suggesting that dopamine release within core and shell may also differentially encode aversive stimuli and their conditioned predictors. Indeed, a concurrent report from Aragona’s group utilized a similar fear conditioning model to demonstrate that while presentation of the CS alone (a tone previously associated with foot shock) resulted in decreased transient dopamine release in the NAcc core (in agreement with work from our laboratory), CS presentation increased dopamine release in the NAcc shell.¹¹¹ Thus, fear-evoking stimuli are capable of differentially altering phasic dopamine transmission across NAcc subregions. The authors propose that the observed enhancement in NAcc shell dopamine likely reflects general motivational salience, perhaps due to relief from a CS-induced fear state when the US (foot shock) is not delivered. This reasoning is supported by a report from Budygin and colleagues¹¹² showing that, in anesthetized rats, the termination of tail pinch results in augmented dopamine release in the shell. Together with studies on rewarding and aversive tastants, these data support a role for NAcc shell phasic dopamine release in the encoding of affective valence. Enhanced NAcc shell dopamine may also serve as a positive RPE when CS presentation does not occur with the US. Indeed, increased NAcc shell dopamine release was observed mostly after the first few CS presentation, a timeline that would be congruent with both the experience of “relief” and RPE. Phasic signaling within the NAcc core, however, is proposed to represent incentive motivation, with the directionality (i.e., increases or decreases) of NAcc core dopamine shifts encoding the organization of behavioral strategies into active (e.g., reward seeking, active avoidance) or passive (e.g., freezing) forms, respectively. Interestingly, Budygin and colleagues observed an increase in NAcc core dopamine release time locked to tail pinch onset. However, given that animals were anesthetized, the behavioral impact of this core dopamine augmentation could not be verified. Thus, phasic decreases in NAcc core dopamine release may promote freezing behavior via the indirect pathway to support passive avoidance.²³ Indeed, lesions of the NAcc core decrease unconditioned and conditioned freezing.¹¹³ Still, passive avoidance behaviors, such as freezing, are not adaptive in all situations leading to the question: Does phasic NAcc core dopamine release also promote active avoidance?

NAcc CORE DOPAMINE RELEASE FACILITATES AVOIDANCE BEHAVIOR

Investigations into the neuromechanisms underlying active avoidance have traditionally utilized a conditioned avoidance behavioral procedure. The goal of this procedure is to condition an animal to the point of avoiding aversive stimuli (e.g., foot shock) by completing a task (e.g., pressing a lever) after an environmental warning cue is presented. Failure of the animal to carry out the task in a set amount of time results in repeated exposure to the aversive stimulus until an escape response is elicited. Thus, active avoidance paradigms engage negative reinforcement learning mechanisms as animals attempt to either terminate an aversive stimulus (i.e., escape) or avoid the onset of the negative stimulus altogether. Much like investigations on the neuromechanisms underlying positive reinforcement, evidence suggests that accumbens dopamine release is required for active avoidance behavior.¹¹⁴ For example, dopaminergic lesions attenuate active avoidance in a shuttling task,¹¹⁵ and dopamine depletion within the NAcc blocks all lever pressing to either escape or avoid foot shock.¹¹⁶ Further, intra-NAcc SCH23390 infusion disrupts the development of avoidance behavior,¹¹⁷ suggesting a role for phasic accumbal dopamine release.

Our group recently used FSCV to measure transient NAcc core dopamine release during ongoing avoidance behavior (Figure 3).⁷⁷ In this investigation, animals learned a signaled operant shock avoidance task wherein illumination of cue light served as a warning signal, which was presented 2 s before the onset of foot shock. Execution of a single lever press during this initial 2 s interval resulted in the avoidance of foot shock, whereas a lever press made after the initiation of shock delivery resulted in escape from foot shock (Figure 3A). All avoidance or escape responses were followed by a 20 s safety period, signaled by a unique auditory tone and illumination of the house light, during which animals received no foot shock. Animals were trained on this task until they reached a stable level of avoidance behavior with successful avoidance on >50% of trials, after which they underwent a single test session during which FSCV enabled the monitoring of NAcc core dopamine release. Under these conditions, presentation of the warning signal evoked accumbal dopamine release exclusively prior to successful avoidance responses, while a decrease in accumbal dopamine was observed prior to escape responses (Figure 3B,C). These data suggest that, prior to escape responses, the warning signal likely conveys impending foot shock and dopamine release in the NAcc core is reduced; however, prior to avoidance responses, the warning signal likely conveys the opportunity to avoid foot shock resulting in dopamine release in the NAcc core.¹¹⁸ These data suggest that transient NAcc core dopamine release events encode cues predicting negative reinforcement and may motivate actions devoted to the avoidance of the stimuli they predict through activation of the basal ganglia direct pathway. It remains unclear, however, what aspects of active avoidance behavior dopamine is encoding at the warning signal. It could be that following training the warning signal comes to represent the safety period, a positive stimulus, and in accordance with the RPE hypothesis phasic dopamine release occurs at the warning signal as the earliest predictor of safety.^37,118,119 Alternatively, accumbal dopamine release may reflect the saliency of the warning signal.⁵⁵ Further research is required to examine these possibilities.

Figure 3.

Dopamine release in the NAcc core is time-locked to the presentation of conditioned stimuli during negative reinforcement. Representative color plots (left) and dopamine concentration traces (right) show dopamine release during avoidance (top), and escape (middle) responses. Arrows indicate lever responses, lightning bolts indicate foot shocks, trumpets indicate safety periods which were accompanied by a unique auditory cue and illumination of the house light, and levers+lights indicate warning signals. (A) Warning signal presentation increases dopamine release prior to successful avoidance of foot shock. Mean ± SEM traces depict the time course of changes in subsecond dopamine release. Dashed lines represent warning stimulus onset, around which mean data are grouped. Color representations: light gray, maximum warning stimulus duration; dark gray, safety period. (B) Warning signal presentation inhibits dopamine release prior to escape responses. (C) Maximal dopamine concentration evoked by warning signal presentation with enhanced dopamine release at the warning signal positively predicting avoidance and reductions in NAcc core dopamine release (as compared to baseline) predicting escape behavior. (Figure reproduced with permission from ref 77.)

CONCLUSION

Altogether, electrophysiological data present an incomplete picture of midbrain dopamine function. Given the heterogeneity of midbrain connections and dopamine cell sub-populations within the VTA, examination of dopaminergic function also requires an analysis of regional dopamine release. Taking this into account, current microdialysis and FSCV literature supports a role for accumbal dopamine in the encoding of both positive and negative stimuli and the regulation of associated adaptive behaviors. Microdialysis studies suggest a role for NAcc dopamine in both the representation of negative stimuli as well as Pavlovian conditioning of negative stimuli. Indeed, presentation of aversive stimuli results in prolonged elevations in NAcc dopamine and dopamine release is increased further following presentation of aversion-associated cues. In congruence with microdialysis data, FSCV confirms a role for accumbal dopamine in aversive conditioning. Specifically, FSCV studies show that presentation of negative stimuli inhibits phasic dopamine release in the NAcc shell, while relief from negative stimuli enhances NAcc shell dopamine release. Interestingly, dopamine release is augmented in the NAcc core following aversion-predictive cues, implying a differential role for dopamine in core and shell subregions. Decreases in core dopamine activity likely function to disinhibit the basal ganglia indirect pathway and result in passive avoidance behavior. Conversely, discrete cues predicting negative stimuli enhance NAcc core dopamine release, theoretically activating the basal ganglia direct pathway and promoting active avoidance behavior. Therefore, the literature suggests a complex role for dopamine release in the representation and reaction to aversive stimuli. Still it is unequivocal that accumbal dopamine works to encode these stimuli and stimulate adaptive behaviors. Future research is required, however, to determine what aspects of aversive stimuli are encoded by phasic dopamine transmission.

ABBREVIATIONS

VTA

ventral tegmental area

NAcc

nucleus accumbens

Hz

hertz

FSCV

fast-scan cyclic voltammetry

MSN

medium spiny neuron

RPE

reward prediction error

TH

tyrosine hydroxylase

USV

ultrasonic vocalization

CTA

conditioned taste aversion

IP

intraperitoneal

CS

conditioned stimulus

UCS

unconditioned stimulus