Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2012 Oct 17;32(42):14804–14808. doi: 10.1523/JNEUROSCI.3087-12.2012

Subsecond Dopamine Release in the Nucleus Accumbens Predicts Conditioned Punishment and Its Successful Avoidance

Erik B Oleson 1, Ronny N Gentry 1, Vivian C Chioma 1, Joseph F Cheer 1,✉
PMCID: PMC3498047  NIHMSID: NIHMS416102  PMID: 23077064

Abstract

The mesolimbic dopamine system is believed to be a pathway that processes rewarding information. While previous studies have also implicated a general role for dopamine in punishment and its avoidance, the precise nature of subsecond dopamine release during these phenomena remains unknown. Here, we used fast-scan cyclic voltammetry to investigate whether subsecond dopamine release events in the nucleus accumbens encode cues predicting the avoidance of punishment during behavior maintained in a signaled footshock avoidance procedure. In this task, rats could initiate an avoidance response by pressing a lever within a warning period, preventing footshock. Alternatively, once footshocks commenced, animals could initiate an escape response by pressing the lever, terminating footshock. This design allowed us to assess subsecond dopamine release events during the presentation of a warning signal, safety periods, and two distinct behavioral responses. We found that release consistently increased upon presentation of the warning signal in a manner that reliably predicted successful punishment avoidance. We also observed subsecond dopamine release during the safety period, as occurs following the receipt of reward. Conversely, we observed a decrease in release at the warning signal during escape responses. Because of this finding, we next assessed dopamine release in a conditioned fear model. As seen during escape responses, we observed a time-locked decrease in dopamine release upon presentation of a cue conditioned to inescapable footshock. Together, these data show that subsecond fluctuations in mesolimbic dopamine release predict when rats will successfully avoid punishment and differentially encode cues related to aversive outcomes.

Introduction

Animals gain an adaptive advantage when their behaviors maximize reward while minimizing punishment. The mesolimbic dopamine system is theorized to promote behavioral selection by generating a teaching signal that draws animals toward favorable stimuli and, possibly, away from potentially harmful stimuli (Schultz et al., 1997). This signal originates from rapid bursts of dopamine neurons within the ventral midbrain (Schultz et al., 1997) and is conveyed as subsecond dopamine release events in terminal fields of the mesolimbic dopamine system, including the nucleus accumbens (NAc) (Wightman, 2006). While it is well accepted that rewards and their conditioned stimuli elicit such patterns of dopamine release during reward seeking (Schultz et al., 1997; Wightman, 2006), it is unknown whether dopamine release plays similar roles in the avoidance of punishment as well as in aversive outcomes (Ikemoto and Panksepp, 1999; Maia and Frank, 2011; Dayan, 2012).

Using shock avoidance procedures to study conditioned behavioral responses has a long history in the behavioral literature (Watson, 1916, 1924). While discrete conditioned cues may not be necessary for avoidance behavior (Sidman, 1953; Herrnstein, 1969), they are sufficient to facilitate performance (Rescorla and Lolordo, 1965). Precisely how conditioned stimuli facilitate avoidance remains a long-standing debate (Mowrer, 1951; Herrnstein, 1969) that recently began to encompass reinforcement-learning models featuring dopamine release as a key teaching signal (Dayan, 2012).

To investigate how subsecond dopamine release represents conditioned stimuli during avoidance behavior, we measured release in the NAc in real time using fast-scan cyclic voltammetry while well trained rats engaged in a signaled footshock avoidance procedure. In this task, a stimulus light was presented as a warning signal for 2 s before the delivery of recurring footshocks. During this warning period, a response lever, which, if pressed, produced a 20 s safety period signaled by a tone, extended into the testing chamber. Animals could initiate an avoidance response by pressing the lever within the 2 s warning period, entirely preventing footshock onset; alternatively, animals could initiate an escape response by pressing the lever after footshocks commenced, relieving ongoing punishment. This design allowed us to assess dopamine signaling during warning signal presentation, safety periods, and two distinct behavioral responses.

We found that in well trained rats (>50% avoidance), the warning signal produced surges in dopamine release in a manner that reliably predicted avoidance. By contrast, during the same sessions, we observed a decrease in dopamine release at the warning signal when animals escaped ongoing footshock. This finding led us to assess the role of subsecond dopamine release in a well accepted animal model of conditioned fear (Monfils et al., 2009). Here, we found that presentation of the fear-associated cue decreased dopamine release in a manner resembling escape behavior. These findings suggest that the warning signal initially represents footshock itself (Mowrer, 1951; Maia, 2010), which dopamine neurons compute as a decrease. As avoidance responses become more prominent however, the warning signal begins to represent successful avoidance, which is computed as an increase. Together, these data suggest that subsecond mesolimbic dopamine release differentially encodes conditioned stimuli predicting punishment and its avoidance.

Materials and Methods

Animals.

Male Sprague Dawley rats, ∼90–120 d old (300–350 g) were used as subjects. Animals were housed individually in a temperature-controlled environment on a 12 h light/dark schedule (700–1900 h) with access to food and water ad libitum. All experiments were conducted during the light period. All procedures were performed in concordance with the University of Maryland, Baltimore, Institutional Animal Care and Use Committee protocols.

Electrode fabrication.

Electrodes used for chronic intracranial implantation were fabricated according to the methods of Clark et al. (2010). A single carbon fiber (Goodfellow Corporation) was inserted into a 10 mm cut segment of fused silica (Polymicro Technologies) while submerged in isopropyl alcohol. One end of the silica tubing was sealed with a two-part epoxy (T-QS12 Epoxy, Super Glue) leaving untouched carbon fiber extending past the seal. The epoxy seal was allowed to dry, and the carbon fiber was cut to a length of 150 μm. A silver connector (Newark) was attached to the opposing side, in contact with the carbon fiber and silica tubing using silver epoxy, and was allowed to dry overnight. A final coat of two-part epoxy was then applied, to provide insulation and structural support for the electrode, and given an additional 12 h to dry.

Surgical preparation using chronic electrodes.

Animals used for footshock avoidance were anesthetized using isoflurane in O2 (4% induction, 1.5% maintenance) and implanted with a chronic voltammetry electrode aimed at the NAc core (+1.3 AP, +1.4 ML, −6.9 DV), an ipsilateral bipolar stimulating electrode (Plastics One) in the medial forebrain bundle (−2.8 AP, +1.7 ML, −8.8 DV), and a contralateral Ag/AgCl reference electrode (Sigma-Aldrich). The reference was fixed using a thin layer of dental cement, leaving the holes for the stimulating and working electrodes unobstructed. The working and stimulating electrodes were attached to the voltammetric amplifier and lowered to the edge of the target region (−6.5 DV for the working and −8.4 DV for the stimulating). At this depth, a triangular voltammetric input waveform (initial ramp, −0.4–1.3 V, 400 V/s) (Heien et al., 2003) was applied to the working electrode at 60 Hz for 30 min and then reduced to 10 Hz for the remainder of the surgery, while subsecond dopamine release was monitored. Electrical stimulation (60 biphasic pulses, 60 Hz, 300 μA, 2 ms/phase) was applied to the stimulating electrode (Plastics One) via a constant-current isolator (A-M Systems). Once electrically evoked dopamine release was detected in the NAc core, a layer of dental cement was applied to secure the electrodes in place and allowed to dry. A single Ginder implant (Ginder Scientific) was connected to the stimulating, reference, and working electrodes, and was fully insulated with cement, leaving only the screw cap of the connector exposed, to reduce noise and prevent loss of connectivity during avoidance training. Rats were injected intraperitoneally with 6 ml of saline containing 0.04 ml Carprofen, swabbed with lidocaine cream, and placed on a heating pad until consciousness was regained. All subjects were allowed a month for recovery and electrode stabilization before experimentation.

Animals used for fear conditioning followed the same surgical procedure, but instead a guide cannula (BASi) was inserted above the NAc core to allow a glass voltammetry electrode to be lowered to the desired DV coordinate on testing day (Oleson et al., 2012). These rats were allowed a minimum of 3 d for recovery before experimentation.

Footshock escape avoidance.

Subjects used in this task (n = 9) were trained daily on a 30 min footshock (0.56 mA, 0.5 ms every 2 s) escape procedure to establish the response–shock termination contingency. During each session, subjects were presented with a lever paired with a cue light; a response on the lever at any point during the session resulted in the retraction of the lever, dimming of the cue light, and termination of footshock [0.5 ms, 0.56 mA, 2 s intertrial interval (ITI)] a 20 s period signaled by a tone. Subjects were gradually shaped toward the lever (safe side, quadrant with lever, orientation toward the lever, rearing, pressing) by the experimenter as needed until acquisition. Once subjects acquired consistent escape behavior, an avoidance contingency was introduced.

Subjects performed avoidance training daily for 30 min sessions until acquisition of avoidance behavior was achieved (>50% avoidance responses for 3 consecutive sessions). At trial onset, the response lever was extended and a warning signal (cue light) was presented for 2 s. A response on the lever during the 2 s warning signal, before the initiation of footshocks, was considered an avoidance response and resulted in the retraction of the lever, dimming of the cue light, and a 20 s safety period during which a tone persisted and no footshocks were delivered. If animals failed to press within the 2 s warning signal, recurring footshocks were applied (0.5 ms, 0.56 mA, 2 s ITI) until the animal responded, which resulted in lever retraction accompanied by dimming of the cue light and onset of the 20 s safety period; this was considered an escape response.

Once subjects performed above 50% avoidance for at least 3 consecutive days (∼15 sessions), they were tested under the same conditions for a 1 h session, while dopamine release was measured using fast-scan cyclic voltammetry at chronically implanted carbon fiber microsensors in the NAc core. Voltammetric timestamps were synchronized with behavioral timestamps via transistor–transistor logic (TTL) from Med Associates output hardware.

Fear conditioning.

On conditioning day, animals (n = 4) were placed into a sound-attenuated testing chamber (Med Associates), which had been fitted with a footshock grid. Subjects were given 30 min for acclimation. Upon program initiation, subjects were presented with three consecutive tones (20 s, 3 min ITI), each culminating in a 2 s scrambled electric footshock of 0.7 mA. After three footshock presentations, the program terminated and subjects were returned to their home cage. Each session was video recorded, and freezing behavior was manually assessed every 5 s by a blind experimenter.

Twenty-four hours after conditioning, animals were again placed in a custom-built sound-attenuated testing chamber with acrylic sloped-side flooring and hooked up to a head-mounted voltammetric amplifier. A glass electrode was carefully lowered to the edge of the NAc core and allowed to cycle at 60 Hz for 45 min and reduced to 10 Hz until electrode stabilization was achieved (∼10 min). Next, an electrical stimulation (24 biphasic pulses, 60 Hz, 0.125 mA, 2 ms/phase) was applied to the stimulating electrode (Plastics One) via a constant-current isolator (A-M Systems) until electrically evoked dopamine release was detected in the target area. Once electrically evoked and spontaneous DA transients were detected, the electrode was locked in place and the animal was transferred to a new context (cylindrical in shape and made of plastic, striped radially to provide altered spatial, visual, tactile, and environmental cues). Upon program initiation, 18 presentations of the tone occurred (3 min ITI). As above all behavioral and voltammetric timestamps were synchronized via TTL logic. Each session was video recorded, and freezing behavior was manually assessed every 5 s by a blind experimenter.

Histology.

At the end of experiments, rats were terminally anesthetized with an overdose of urethane (3 g/kg), and long high-amplitude, constant-current pulses (0.6 mA for 10 s) were applied to each carbon fiber microelectrode to mark recording sites (see Fig. 3). Next, rats were transcardially perfused with 4% paraformaldehyde and their brains prepared for histological reconstruction as previously described (Oleson et al., 2012).

Figure 3.

Figure 3.

Open in a new tab

Placement of chronic and acute recording electrodes across experimental conditions.

Data analysis/statistics.

Dopamine concentrations were compared using one-way ANOVA (avoidance) or repeated-measures ANOVA (fear conditioning) following a Shapiro–Wilk test to confirm normality. Mean dopamine concentration data were compared as a function of response type using best-fit nonlinear regression. For fear-conditioning dopamine analyses, data were grouped into 5 s bins. Principal component regression was used as previously described to extract the dopamine component from the raw voltammetric data.

Results

We first optimized the design of the conditioned footshock avoidance procedure by testing the effects of footshock intensity and safety period duration on avoidance behavior. Successful avoidance increased as a function of footshock intensity (0.31–1 mA) while decreasing as a function of safety period (20–60 s). Following these preliminary studies, we opted to use 0.56 mA footshock intensity due to nonspecific effects of the large current (electrical interference with recording produced by strong behavioral responses) at the high end of the intensity range and weak avoidance behavior at the low end of the intensity range. We selected the 20 s safety period as it produced robust avoidance.

During both avoidance and escape responses, we observed an increase in dopamine release during periods of safety (Fig. 1A–C), as occurs following the receipt of reward (Schultz et al., 1997). The dopamine response to warning signal presentation, however, differed between avoidance and escape responses. When animals successfully avoided footshock, we observed a significant increase in dopamine release during warning signal presentation that reliably predicted avoidance (Fig. 1B). Accordingly, dopamine concentrations at the cue sharply decayed (r2 = 0.29; p < 0.01) when animals failed to avoid and were significantly lower during escape responses regardless of the number of footshocks received (F(2,26) = 4.988; p < 0.01 for both one-shock and two-shock escape vs avoidance; Fig. 1C). Analysis of mean dopamine concentration data from escape responses revealed warning signal presentation not only failed to increase dopamine release, but actually decreased dopamine release below baseline values (t(16) = −2.867; p = 0.011). In support of the two-process theory of avoidance, the warning signal likely elicits a fear response early in training, when escape behavior is prominent, due to its association with the aversive unconditioned stimulus that it precedes (Mowrer, 1951; Solomon and Corbit, 1974; Maia, 2010). However, our data suggest that the conditioned fear response originally elicited by the warning signal dissipates after extensive training, as the prediction of a positive outcome (avoidance) supersedes it.

Figure 1.

Figure 1.

Open in a new tab

A, Dopamine encodes conditioned stimuli during negative reinforcement. Representative color plots (left) and dopamine concentration traces (right) show avoidance (top), one-footshock escape (middle), and two-footshock escape (bottom) responses. Arrows indicate lever responses, lightning bolts indicate footshocks, trumpets indicate safety periods, levers+lights indicate warning signals. Left, Voltammetric current (z-axis) plotted against applied scan potential (Eapp; y-axis) and time (x-axis). Right, Dopamine concentration traces plotted as a function of time. Inset shows cyclic voltammogram for dopamine. B, Warning signal presentation increases dopamine release when rats successfully avoid footshock. Mean (n = 9) ± SEM traces depict the time course of changes in subsecond dopamine release as animals minimize punishment by avoiding footshock. Dashed lines represent warning stimulus onset, around which mean data are grouped. Color representations: light gray, maximum warning stimulus duration; dark gray, safety period. C, Warning signal presentation inhibits dopamine release when rats fail to avoid or escape footshock. Only one-footshock escape trials are included in the mean (n = 9). D, Maximal dopamine concentration evoked by warning signal presentation predicts successful punishment avoidance.

The finding that dopamine release was inhibited at the warning signal during escape responses prompted us to test whether release is also suppressed during classical fear associations. To assess this, we conditioned rats with a cue predicting inescapable footshock 24 h before measuring dopamine release during repeated presentations of the cue alone (Monfils et al., 2009). We observed a cue-evoked decrease in dopamine release (Roitman et al., 2008) (Fig. 2; F(11,199) = 1.681; p < 0.05 vs baseline) similar to the above-mentioned escape responses in trials where freezing was >50% during tone presentation.

Figure 2.

Figure 2.

Open in a new tab

A, B, Fear conditioning produces freezing behavior that extinguishes across repeated trials of conditioned stimulus (CS) presentation on fear-memory retrieval day. C, Representative color plot (left) and corresponding dopamine concentration trace (right) show a CS-induced decrease in dopamine release. D, Mean (n = 4) ± SEM dopamine concentration trace during presentations of a CS that produces a conditioned freezing response. Gray represents CS duration.

Discussion

Our data demonstrate that subsecond dopamine release functions to strengthen advantageous behavioral outcomes, regardless of whether they involve the procurement of reward (positive reinforcement) (Schultz et al., 1997) or the reduction of harm (negative reinforcement). However, when the subject is unable to reduce harm, instead resorting to species-specific defense responses (Bolles, 1970), this form of neural signaling is suppressed by the prediction of impending punishment. Together, these findings suggest that a value-coding population of dopamine neurons compute conditioned stimuli predicting the avoidance of punishment similarly to rewarding events, and these events are encoded oppositely from punishment itself (Matsumoto and Hikosaka, 2009; Cohen et al., 2012). These distinct patterns of accumbal dopamine release (Fig. 3) further suggest that postsynaptic targets expressing dopamine D1 (low affinity) or D2 (high affinity) receptors (Dreyer et al., 2010; Kravitz et al., 2012) may be involved in the expression of behaviors described herein. In accordance with recently published data (Kravitz et al., 2012), it is likely that high-concentration surges of dopamine release evoked by warning signal presentation facilitate successful avoidance by activating dopamine D1 receptors in target regions of the basal ganglia responsible for the generation of motor action sequences. By contrast, decreases in the frequency of dopamine release events resulting from conditioned predictors of punishment might contribute to freezing behavior via changes in the activation of dopamine D2 receptors (Kravitz et al., 2012).

By assessing dopamine release in the signaled footshock avoidance task at a time in which the avoidance response became prominent, we were able to observe how the warning signal is represented by dopamine release during the transition to successful avoidance. Our data suggest that when presented with the option to remove an aversive stimulus, animals first resort to a species-specific defense response, initially associated with a decrease in release. In accordance with the two-process theory of avoidance (Mowrer, 1951; Maia, 2010; Dayan, 2012), it is likely that at this point the warning signal comes to elicit a fear response due to its association with the aversive unconditioned stimulus that it precedes. Our footshock escape data suggest that dopamine neurons compute this conditioned fear response as a decrease in the frequency of dopamine release events. The decrease in dopamine release evoked by the conditioned fear response likely contributes to the lengthy training requirements necessary to achieve primarily avoidance behavior (∼15 sessions in the present study). Indeed, we speculate that this sharp decrease in dopamine release contributes to fear-induced species-specific defense responses, such as freezing, that might disrupt a motor pattern that is conducive to avoidance behavior (Bolles, 1970). The observations obtained during fear conditioning further support this theory, as we found that conditioned stimuli uniformly decrease behavioral output and dopamine release.

Our data further suggest that the conditioned fear response originally elicited by the warning signal dissipates after extensive training, as the prediction of a positive outcome—successful avoidance—supersedes it. During trials in which avoidance responses occur regularly, operant behavior is directed and not confounded by species-specific defense responses (freezing, attacking) that are prevalent early in avoidance learning (Bolles, 1970). As consistent avoidance behavior develops, the warning signal begins to evoke dopamine release. We speculate that, with repeated training, the interpretation of the warning signal shifts from representing fear to successful avoidance, and dopamine neurons compute this transition by increasing dopaminergic output in the form of large positive prediction error signals (Schultz et al., 1997). This interpretation is consistent with a recent report demonstrating that the representation of a conditioned cue can be transformed between appetitive and aversive stimuli over repeated pairings (Nasser and McNally, 2012).

During both escape and avoidance responses, we observed an increase in dopamine release corresponding to the safety period. This is consistent with the supposition that, even early in training, the safety period functions similarly to an appetitive stimulus (Dinsmoor, 1954, 2001). Thus, in a way, the elimination of punishment is processed similarly to the receipt of reward regardless of the representation of the preceding conditioned stimuli or whether or not punishment actually occurred.

Subsecond changes in dopamine release may contribute to the extinction of freezing behavior in animal models of conditioned fear. Consistent with this theory, recent evidence demonstrates that stimulating dopamine terminal fields in the ventral striatum promotes the extinction of fear memories (Rodriguez-Romaguera et al., 2012). We speculate that subsecond dopamine release functions as a teaching signal to alert the animal to behaviorally relevant events in the environment. In the case of conditioned fear, a decrease in dopamine release may relay information regarding unavoidable negative events to the fear network. As repeated presentations of the cue fail to coincide with an aversive event (as seen in extinction), the conditioned dopamine response may weaken, thereby updating the fear network about associations pertaining to the safety of the surrounding environment. The finding that subsecond dopamine release is decreased by conditioned predictors of punishment may appear at odds with existing microdialysis literature, which suggests that aversive stimuli increase tonic dopamine concentrations in the NAc (but see Mark et al., 1991; Saulskaya and Marsden, 1995; Wilkinson et al., 1998; Young et al., 1998; Pezze et al., 2001). As previously discussed however (McGinty et al., 2011), subsecond dopamine release is distinct from tonic dopamine transmission, which is assessed with microdialysis. Thus, it is possible that during conditioned fear, subsecond dopamine release is suppressed, whereas tonic dopamine signaling is concurrently enhanced through alternative mechanisms (McGinty et al., 2011).

These data provide several implications for the neuroscientific study of psychiatric disease. For example, opponent-process theory (Solomon and Corbit, 1974) suggests, in the context of drug addiction, investigating subsecond dopamine release during punishment avoidance may be more relevant to the neurobiological underpinnings of the disease than studying reward seeking. Furthermore, novel insights into the neural mechanisms underlying the pharmacological utility of antipsychotic drugs, many of which act on the dopamine system, may be elucidated using conditioned avoidance (Wadenberg and Hicks, 1999). Similar insights into the therapeutic interventions for post-traumatic stress disorder might also be obtained by studying fluctuations in subsecond dopamine release in animal models of conditioned fear (Pezze and Feldon, 2004).

Footnotes

This research was supported by NIH Grants DA022340 and DA025890 to J.F.C., and DA032266 to E.B.O. We thank Mr. David Bernstein for expert technical assistance, Dr. Keri Chiodo for helpful comments in the preparation of the manuscript, and Dr. Paul Phillips for assistance with all aspects of longitudinal voltammetric sensors.

The authors declare no financial conflicts of interest.

References

  1. Bolles RC. Species-specific defense reactions and avoidance learning. Psychol Rev. 1970;71:32–48. [Google Scholar]
  2. Clark JJ, Sandberg SG, Wanat MJ, Gan JO, Horne EA, Hart AS, Akers CA, Parker JG, Willuhn I, Martinez V, Evans SB, Stella N, Phillips PE. Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals. Nat Methods. 2010;7:126–129. doi: 10.1038/nmeth.1412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012;482:85–88. doi: 10.1038/nature10754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Dayan P. Instrumental vigour in punishment and reward. Eur J Neurosci. 2012;35:1152–1168. doi: 10.1111/j.1460-9568.2012.08026.x. [DOI] [PubMed] [Google Scholar]
  5. Dinsmoor JA. Punishment: I. The avoidance hypothesis. Psychol Rev. 1954;61:34–46. doi: 10.1037/h0062725. [DOI] [PubMed] [Google Scholar]
  6. Dinsmoor JA. Stimuli inevitably generated by behavior that avoids electric shock are inherently reinforcing. J Exp Anal Behav. 2001;75:311–333. doi: 10.1901/jeab.2001.75-311. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dreyer JK, Herrik KF, Berg RW, Hounsgaard JD. Influence of phasic and tonic dopamine release on receptor activation. J Neurosci. 2010;30:14273–14283. doi: 10.1523/JNEUROSCI.1894-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Heien ML, Phillips PE, Stuber GD, Seipel AT, Wightman RM. Overoxidation of carbon-fiber microelectrodes enhances dopamine adsorption and increases sensitivity. Analyst. 2003;128:1413–1419. doi: 10.1039/b307024g. [DOI] [PubMed] [Google Scholar]
  9. Herrnstein RJ. Method and theory in the study of avoidance. Psychol Rev. 1969;76:49–69. doi: 10.1037/h0026786. [DOI] [PubMed] [Google Scholar]
  10. Ikemoto S, Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Brain Res Rev. 1999;31:6–41. doi: 10.1016/s0165-0173(99)00023-5. [DOI] [PubMed] [Google Scholar]
  11. Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci. 2012;15:816–818. doi: 10.1038/nn.3100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Maia TV. Two-factor theory, the actor-critic model, and conditioned avoidance. Learn Behav. 2010;38:50–67. doi: 10.3758/LB.38.1.50. [DOI] [PubMed] [Google Scholar]
  13. Maia TV, Frank MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci. 2011;14:154–162. doi: 10.1038/nn.2723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Mark GP, Blander DS, Hoebel BG. A conditioned stimulus decreases extracellular dopamine in the nucleus accumbens after the development of a learned taste aversion. Brain Res. 1991;551:308–310. doi: 10.1016/0006-8993(91)90946-s. [DOI] [PubMed] [Google Scholar]
  15. Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 2009;459:837–841. doi: 10.1038/nature08028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. McGinty VB, Hayden BY, Heilbronner SR, Dumont EC, Graves SM, Mirrione MM, du Hoffmann J, Sartor GC, España RA, Millan EZ, Difeliceantonio AG, Marchant NJ, Napier TC, Root DH, Borgland SL, Treadway MT, Floresco SB, McGinty JF, Haber S. Emerging, reemerging, and forgotten brain areas of the reward circuit: notes from the 2010 Motivational Neural Networks conference. Behav Brain Res. 2011;225:348–357. doi: 10.1016/j.bbr.2011.07.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Monfils MH, Cowansage KK, Klann E, LeDoux JE. Extinction-reconsolidation boundaries: key to persistent attenuation of fear memories. Science. 2009;324:951–955. doi: 10.1126/science.1167975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Mowrer OH. Two-factor learning theory: summary and comment. Psychol Rev. 1951;58:350–354. doi: 10.1037/h0058956. [DOI] [PubMed] [Google Scholar]
  19. Nasser HM, McNally GP. Appetitive-aversive interactions in Pavlovian fear conditioning. Behav Neurosci. 2012;126:404–422. doi: 10.1037/a0028341. [DOI] [PubMed] [Google Scholar]
  20. Oleson EB, Beckert MV, Morra JT, Lansink CS, Cachope R, Abdullah RA, Loriaux AL, Schetters D, Pattij T, Roitman MF, Lichtman AH, Cheer JF. Endocannabinoids shape accumbal encoding of cue-motivated behavior via CB1 receptor activation in the ventral tegmentum. Neuron. 2012;73:360–373. doi: 10.1016/j.neuron.2011.11.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Pezze MA, Feldon J. Mesolimbic dopaminergic pathways in fear conditioning. Prog Neurobiol. 2004;74:301–320. doi: 10.1016/j.pneurobio.2004.09.004. [DOI] [PubMed] [Google Scholar]
  22. Pezze MA, Heidbreder CA, Feldon J, Murphy CA. Selective responding of nucleus accumbens core and shell dopamine to aversively conditioned contextual and discrete stimuli. Neuroscience. 2001;108:91–102. doi: 10.1016/s0306-4522(01)00403-1. [DOI] [PubMed] [Google Scholar]
  23. Rescorla RA, Lolordo VM. Inhibition of avoidance behavior. J Comp Physiol Psychol. 1965;59:406. doi: 10.1037/h0022060. [DOI] [PubMed] [Google Scholar]
  24. Rodriguez-Romaguera J, Do Monte FH, Quirk GJ. Deep brain stimulation of the ventral striatum enhances extinction of conditioned fear. Proc Natl Acad Sci U S A. 2012;109:8764–8769. doi: 10.1073/pnas.1200782109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Roitman MF, Wheeler RA, Wightman RM, Carelli RM. Real-time chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli. Nat Neurosci. 2008;11:1376–1377. doi: 10.1038/nn.2219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Saulskaya N, Marsden CA. Conditioned dopamine release: dependence upon N-methyl-D-aspartate receptors. Neuroscience. 1995;67:57–63. doi: 10.1016/0306-4522(95)00007-6. [DOI] [PubMed] [Google Scholar]
  27. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  28. Sidman M. Avoidance conditioning with brief shock and no exteroceptive warning signal. Science. 1953;118:157–158. doi: 10.1126/science.118.3058.157. [DOI] [PubMed] [Google Scholar]
  29. Solomon RL, Corbit JD. An opponent-process theory of motivation: I. Temporal dynamics of affect. Psychol Rev. 1974;81:119–145. doi: 10.1037/h0036128. [DOI] [PubMed] [Google Scholar]
  30. Wadenberg ML, Hicks PB. The conditioned avoidance response test re-evaluated: is it a sensitive test for the detection of potentially atypical antipsychotics? Neurosci Biobehav Rev. 1999;23:851–862. doi: 10.1016/s0149-7634(99)00037-8. [DOI] [PubMed] [Google Scholar]
  31. Watson JB. The place of the conditioned-reflex in psychology. Psychol Rev. 1916;23:89. [Google Scholar]
  32. Watson JB. Behaviorism. New Brunswick, NJ: Transaction Publications; 1924. [Google Scholar]
  33. Wightman RM. Detection technologies. Probing cellular chemistry in biological systems with microelectrodes. Science. 2006;311:1570–1574. doi: 10.1126/science.1120027. [DOI] [PubMed] [Google Scholar]
  34. Wilkinson LS, Humby T, Killcross AS, Torres EM, Everitt BJ, Robbins TW. Dissociations in dopamine release in medial prefrontal cortex and ventral striatum during the acquisition and extinction of classical aversive conditioning in the rat. Eur J Neurosci. 1998;10:1019–1026. doi: 10.1046/j.1460-9568.1998.00119.x. [DOI] [PubMed] [Google Scholar]
  35. Young AM, Ahier RG, Upton RL, Joseph MH, Gray JA. Increased extracellular dopamine in the nucleus accumbens of the rat during associative learning of neutral stimuli. Neuroscience. 1998;83:1175–1183. doi: 10.1016/s0306-4522(97)00483-1. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES