Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Apr 17.
Published in final edited form as: J Neurosci. 2012 Oct 17;32(42):14804–14808. doi: 10.1523/JNEUROSCI.3087-12.2012

Subsecond dopamine release in the nucleus accumbens predicts conditioned punishment and its successful avoidance

Erik B Oleson 1, Ronny N Gentry 1, Vivian C Chioma 1, Joseph F Cheer 1
PMCID: PMC3498047  NIHMSID: NIHMS416102  PMID: 23077064

Abstract

The mesolimbic dopamine system is believed to be a pathway that processes rewarding information. While previous studies have also implicated a general role for dopamine in punishment and its avoidance, the precise nature of subsecond dopamine release during these phenomena remains unknown. Here, we used fast-scan cyclic voltammetry to investigate whether subsecond dopamine release events in the nucleus accumbens encode cues predicting the avoidance of punishment during behavior maintained in a signaled footshock avoidance procedure. In this task, rats could initiate an avoidance response by pressing a lever within a warning period, preventing footshock. Alternatively, once footshocks commenced, animals could initiate an escape response by pressing the lever, terminating footshock. This design allowed us to assess subsecond dopamine release events during the presentation of a warning signal, safety periods and two distinct behavioral responses. We found that release consistently increased upon presentation of the warning signal in a manner that reliably predicted successful punishment avoidance. We also observed subsecond dopamine release during the safety period, as occurs following the receipt of reward. Conversely, we observed a decrease in release at the warning signal during escape responses. Because of this finding, we next assessed dopamine release in a conditioned fear model. As seen during escape responses, we observed a time-locked decrease in dopamine release upon presentation of a cue conditioned to inescapable footshock. Together, these data show that subsecond fluctuations in mesolimbic dopamine release predict when rats will successfully avoid punishment and differentially encode cues related to aversive outcomes.

Introduction

Animals gain an adaptive advantage when their behaviors maximize reward while minimizing punishment. The mesolimbic dopamine system is theorized to promote behavioral selection by generating a teaching signal which draws animals toward favorable stimuli and, possibly, away from potentially harmful stimuli (Schultz et al., 1997). This signal originates from rapid bursts of dopamine neurons within the ventral midbrain (Schultz et al., 1997) and is conveyed as subsecond dopamine release events in terminal fields of the mesolimbic dopamine system, including the nucleus accumbens (NAc) (Wightman, 2006). While it is well accepted that rewards and their conditioned stimuli elicit such patterns of dopamine release during reward seeking (Schultz et al., 1997; Wightman, 2006), it is unknown whether dopamine release plays similar roles in the avoidance of punishment as well as in aversive outcomes (Ikemoto and Panksepp, 1999; Maia and Frank, 2011; Dayan, 2012).

Using shock avoidance procedures to study conditioned behavioral responses has a long history in the behavioral literature (Watson, 1916, 1924). While discrete conditioned cues may not be necessary for avoidance behavior (Sidman, 1953; Herrnstein, 1969), they are sufficient to facilitate performance (Rescorla and Lolordo, 1965). Precisely how conditioned stimuli facilitate avoidance remains a long-standing debate (Mowrer, 1951; Herrnstein, 1969) that recently began to encompass reinforcement-learning models featuring dopamine release as a key teaching signal (Dayan, 2012).

To investigate how subsecond dopamine release represents conditioned stimuli during avoidance behavior, we measured release in the NAc in real-time using fast-scan cyclic voltammetry while well-trained rats engaged in a signaled footshock avoidance procedure. In this task, a stimulus light was presented as a warning signal for 2s prior to the delivery of recurring footshocks. During this warning period, a response lever extended into the testing chamber which, if pressed, produced a 20s safety period signaled by a tone. Animals could initiate an avoidance response by pressing the lever within the 2s warning period, entirely preventing footshock onset; alternatively, animals could initiate an escape response by pressing the lever after footshocks commenced, relieving ongoing punishment. This design allowed us to assess dopamine signaling during warning signal presentation, safety periods and two distinct behavioral responses.

We found that in well-trained rats (>50% avoidance), the warning signal produced surges in dopamine release in a manner that reliably predicted avoidance. By contrast, during the same sessions, we observed a decrease in dopamine release at the warning signal when animals escaped ongoing footshock. This finding led us to assess the role of subsecond dopamine release in a well-accepted animal model of conditioned fear (Monfils et al., 2009). Here, we found that presentation of the fear-associated cue decreased dopamine release in a manner resembling escape behavior. These findings suggest that the warning signal initially represents footshock itself (Mowrer, 1951; Maia, 2010), which dopamine neurons compute as a decrease. As avoidance responses become more prominent however, the warning signal begins to represent successful avoidance, which is computed as an increase. Together, these data suggest that subsecond mesolimbic dopamine release differentially encodes conditioned stimuli predicting punishment and its avoidance.

Materials and methods

Animals

Male Sprague-Dawley rats, ~90-120 days old (300-350g) were used as subjects. Animals were housed individually in a temperature-controlled environment on a 12-h light/dark schedule (700-1900 hr) with access to food and water ad libitum. All experiments were conducted during the light period. All procedures were performed in concordance with the University of Maryland, Baltimore's Institutional Animal Care and Use Committee (IACUC) protocols.

Electrode fabrication

Electrodes used for chronic intracranial implantation were fabricated according to the methods of Clark and colleagues (Clark et al., 2010). A single carbon fiber (Goodfellow Corporation; Oakdale, PA) was inserted into a 10mm cut segment of fused silica (Polymicro Technologies; Phoenix, AZ) while submerged in isopropyl alcohol. One end of the silica tubing was sealed with two-part epoxy (Super Glue Corp, T-QS12 Epoxy; Rancho Cucamonga, CA) leaving untouched carbon fiber extending past the seal. The epoxy seal was allowed to dry, and the carbon fiber was cut to a length of 150 μm. A silver connector (Newark; Chicago, IL) was attached to the opposing side, in contact with the carbon fiber and silica tubing using silver epoxy and allowed to dry overnight. A final coat of two-part epoxy was then applied, in order to provide insulation and structural support for the electrode, and given an additional 12 hours to dry.

Surgical preparation using chronic electrodes

Animals used for footshock avoidance were anesthetized using isoflurane in O2 (4% induction, 1.5% maintenance) and implanted with a chronic voltammetry electrode aimed at the NAc core (+1.3AP, +1.4ML, -6.9DV), an ipsilateral bipolar stimulating electrode (Plastics One; Roanoke, VA) in the medial forebrain bundle (–2.8AP, +1.7ML, –8.8DV), and a contralateral Ag/AgCl reference electrode (Sigma-Aldrich; St. Louis, MO). The reference was fixed using a thin layer of dental cement, leaving the holes for the stimulating and working electrodes unobstructed. The working and stimulating electrodes were attached to the voltammetric amplifier and lowered to the edge of the target region (-6.5DV for the working and -8.4DV for the stimulating.) At this depth, a triangular voltammetric input waveform (initial ramp: –0.4V to 1.3V, 400V/s) (Heien et al., 2003) was applied to the working electrode at 60 Hz for 30 minutes and then reduced to 10 Hz for the remainder of the surgery, while subsecond dopamine release was monitored. Electrical stimulation (60 biphasic pulses, 60Hz, 300μA, 2ms/phase) was applied to the stimulating electrode (Plastics One) via a constant-current isolator (A-M Systems; Sequim, WA). Once electrically evoked dopamine release was detected in the NAc core, a layer of dental cement was applied to secure the electrodes in place and allowed to dry. A single Ginder implant (Ginder Scientific; Nepean, ON) was connected to the stimulating, reference, and working electrodes and fully insulated with cement, leaving only the screw cap of the connector exposed, in order to reduce noise and prevent loss of connectivity during avoidance training. Rats were injected intraperitoneally with 6mL saline containing 0.04mL Carprofen®, swabbed with lidocaine cream, and placed on a heating pad until consciousness was regained. All subjects were allowed a month for recovery and electrode stabilization before experimentation.

Animals used for fear conditioning followed the same surgical procedure, but instead a guide cannula (BASi; Mount Vernon, IN) was inserted above the NAc core to allow a glass voltammetry electrode to be lowered to the desired DV coordinate on testing day (Oleson et al., 2012). These rats were allowed a minimum of 3 days for recovery before experimentation.

Footshock Escape-Avoidance

Subjects used in this task (n=9) were trained daily on a 30-minute footshock (0.56mA, 0.5ms every 2s) escape procedure in order to establish the response-shock termination contingency. During each session, subjects were presented with a lever paired with a cue light; a response on the lever at any point during the session resulted in the retraction of the lever, dimming of the cue light, and termination of footshock (0.5ms, 0.56mA, 2s ITI) for a 20s period signaled by a tone. Subjects were gradually shaped toward the lever (safe side, quadrant with lever, orientation toward the lever, rearing, pressing) by the experimenter as needed until acquisition. Once subjects acquired consistent escape behavior, an avoidance contingency was introduced.

Subjects performed avoidance training daily for 30min sessions until acquisition of avoidance behavior was achieved (>50% avoidance responses for 3 consecutive sessions). At trial onset, the response lever was extended and a warning signal (cue light) was presented for 2s. A response on the lever during the 2s warning signal, before the initiation of footshocks, was considered an avoidance response and resulted in the retraction of the lever, dimming of the cue light, and a 20s safety period during which a tone persisted and no footshocks were delivered. If animals failed to press within the 2 s warning signal, recurring footshocks were applied (0.5ms, 0.56mA, 2s ITI), until the animal responded, which resulted in lever retraction accompanied by dimming of the cue light and onset of the 20s safety period; this was considered an escape response.

Once subjects performed above 50% avoidance for at least 3 consecutive days (~15 sessions), they were tested under the same conditions for a 1-hr session, while dopamine release was measured using fast-scan cyclic voltammetry at chronically implanted carbon fiber microsensors in the NAc core. Voltammetric timestamps were synchronized with behavioral timestamps via TTL logic from Med Associates output hardware.

Fear Conditioning

On conditioning day, animals (n=4) were placed into a sound-attenuated testing chamber (Med Associates; St. Albans, VT), which had been fitted with a footshock grid. Subjects were given 30 minutes for acclimation. Upon program initiation, subjects were presented with three consecutive tones (20 s; ITI 3 min), each culminating in a 2 s scrambled electric footshock of 0.7 mA. After three footshock presentations, the program terminated, and subjects were returned to their home cage. Each session was video-recorded and freezing behavior was manually assessed every 5s by a blind experimenter.

Twenty-four hours after conditioning, animals were again placed in a custom-built sound-attenuated testing chamber with acrylic sloped-side flooring, and hooked up to a head-mounted voltammetric amplifier. A glass electrode was carefully lowered to the edge of the NAc core and allowed to cycle at 60 Hz for 45 minutes and reduced to 10Hz until electrode stabilization was achieved (~10min.) Next, an electrical stimulation (24 biphasic pulses, 60Hz, 0.125mA, 2 ms/phase) was applied to the stimulating electrode (Plastics One; Roanoke, VA) via a constant-current isolator (A-M Systems; Sequim, WA) until electrically evoked dopamine release was detected in the target area. Once electrically evoked and spontaneous DA transients were detected, the electrode was locked in place and the animal was transferred to a new context (cylindrical in shape and made of plastic, striped radially to provide altered spatial, visual, tactile environmental cues). Upon program initiation, 18 presentations of the tone occurred (ITI 3 min). As above all behavioral and voltammetric time stamps were synchronized via TTL logic. Each session was video-recorded and freezing behavior was manually assessed every 5 s by a blind experimenter.

Histology

At the end of experiments rats were terminally anesthetized with an overdose of urethane (3 g/kg), and long high amplitude constant current pulses (0.6mA for 10s) were applied to each carbon fiber microelectrode to mark recording sites (Fig. 3). Next, rats were transcardially perfused with 4% paraformaldehyde and their brains prepared for histological reconstruction as previously described (Oleson et al., 2012).

Fig. 3.

Fig. 3

Open in a new tab

Placement of chronic and acute recording electrodes across experimental conditions.

Data Analysis/Statistics

Dopamine concentrations were compared using one-way ANOVA (avoidance) or repeated-measures ANOVA (fear conditioning) following a Shapiro-Wilk test to confirm normality. Mean dopamine concentration data were compared as a function of response type using best-fit nonlinear regression. For fear conditioning dopamine analyses, data were grouped into 5s bins. Principal component regression (PCR) was used as previously described to extract the dopamine component from the raw voltammetric data.

Results

We first optimized the design of the conditioned footshock avoidance procedure by testing the effects of footshock intensity and safety period duration on avoidance behavior. Successful avoidance increased as a function of footshock intensity (0.31-1mA) while decreasing as a function of safety period (20s-60s). Following these preliminary studies, we opted to use 0.56mA footshock intensity due to nonspecific effects of the large current (electrical interference with recording produced by strong behavioral responses) at the high end of the intensity range and weak avoidance behavior at the low end of the intensity range. We selected the 20s safety period as it produced robust avoidance.

During both avoidance and escape responses, we observed an increase in dopamine release during periods of safety (Fig. 1, A-C), as occurs following the receipt of reward (Schultz et al., 1997). The dopamine response to warning signal presentation, however, differed between avoidance and escape responses. When animals successfully avoided footshock, we observed a significant increase in dopamine release during warning signal presentation that reliably predicted avoidance (Fig. 1, B). Accordingly, dopamine concentrations at the cue sharply decayed (r2=0.29; P<0.01) when animals failed to avoid and were significantly lower during escape responses irrespective of the number of footshocks received (F(2,26)=4.988; P<0.01 for both 1-shock and 2-shock escape vs. avoidance; Fig. 1C). Analysis of mean dopamine concentration data from escape responses revealed warning signal presentation not only failed to increase dopamine release, but actually decreased dopamine release below baseline values (t(16)=-2.867; p=0.011). In support of the two-process theory of avoidance, the warning signal likely elicits a fear response early in training, when escape behavior is prominent, due to its association with the aversive unconditioned stimulus that it precedes (Mowrer, 1951; Solomon and Corbit, 1974; Maia, 2010). However, our data suggest that the conditioned fear response originally elicited by the warning signal dissipates after extensive training, as the prediction of a positive outcome (avoidance) supersedes it.

Fig. 1.

Fig. 1

Open in a new tab

(A) Dopamine encodes conditioned stimuli during negative reinforcement. Representative color plots (left) and dopamine concentration traces (right) show avoidance (top), 1-footshock escape (middle) and 2-footshock escape (bottom) responses. Arrows indicate lever responses. Left: voltammetric current (z-axis) plotted against applied scan potential (Eapp; y-axis) and time (x-axis). Right: Dopamine concentration traces plotted as a function of time. Inset shows cyclic voltammogram for dopamine. (B). Warning signal presentation increases dopamine release when rats successfully avoid footshock. Mean (n=9) ± SEM traces depict the time course of changes in subsecond dopamine release as animals minimize punishment by avoiding footshock. Dashed lines represent warning stimulus onset, around which mean data are grouped. Color representations: light gray, maximum warning stimulus duration; dark gray, safety period. (C) Warning signal presentation inhibits dopamine release when rats fail to avoid, or escape footshock. Only 1-footshock escape trials are included in mean (n=9). (D) Maximal dopamine concentration evoked by warning signal presentation predicts successful punishment avoidance.

The finding that dopamine release was inhibited at the warning signal during escape responses prompted us to test whether release is also suppressed during classical fear associations. To assess this, we conditioned rats with a cue predicting inescapable footshock 24hr before measuring dopamine release during repeated presentations of the cue alone(Monfils et al., 2009). We observed a cue-evoked decrease in dopamine release (Roitman et al., 2008) (Fig. 2; F(11,199)=1.681; P<0.05 vs. baseline) similar to the above mentioned escape responses in trials where freezing was greater than 50% during tone presentation.

Fig. 2.

Fig. 2

Open in a new tab

(A,B) Fear conditioning produces freezing behavior that extinguishes across repeated trials of CS presentation on fear-memory retrieval day. (C) Representative color plot (left) and corresponding dopamine concentration trace (right) show a CS-induced decrease in dopamine release. (D) Mean (n=4) ± SEM dopamine concentration trace during presentations of a CS that produces a conditioned freezing response. Gray represents CS duration.

Discussion

Our data demonstrate that subsecond dopamine release functions to strengthen advantageous behavioral outcomes, regardless of whether they involve the procurement of reward (positive reinforcement) (Schultz et al., 1997) or the reduction of harm (negative reinforcement). However, when the subject is unable to reduce harm, instead resorting to species-specific defense responses (Bolles, 1970), this form of neural signaling is suppressed by the prediction of impending punishment. Together, these findings suggest that a value-coding population of dopamine neurons compute conditioned stimuli predicting the avoidance of punishment similarly to rewarding events, and these events are encoded oppositely from punishment itself (Matsumoto and Hikosaka, 2009; Cohen et al., 2012). These distinct patterns of dopamine release further suggest that postsynaptic targets expressing dopamine D1 (low affinity) or D2 (high affinity) receptors (Dreyer et al., 2010; Kravitz et al., 2012) may be involved in the expression of behaviors described herein. In accordance with recently published data (Kravitz et al., 2012), it is likely that high concentration surges of dopamine release evoked by warning signal presentation facilitate successful avoidance by activating dopamine D1 receptors in target regions of the basal ganglia responsible for the generation of motor action sequences. By contrast, decreases in the frequency of dopamine release events resulting from conditioned predictors of punishment might contribute to freezing behavior via changes in the activation of dopamine D2 receptors (Kravitz et al., 2012).

By assessing dopamine release in the signaled footshock avoidance task at a time in which the avoidance response became prominent, we were able to observe how the warning signal is represented by dopamine release during the transition to successful avoidance. Our data suggest that when presented with the option to remove an aversive stimulus, animals first resort to a species-specific defense response, initially associated with a decrease in release. In accordance with the two-process theory of avoidance (Mowrer, 1951; Maia, 2010; Dayan, 2012), it is likely that at this point the warning signal comes to elicit a fear response due to its association with the aversive unconditioned stimulus that it precedes. Our footshock escape data suggest that dopamine neurons compute this conditioned fear response as a decrease in the frequency of dopamine release events. The decrease in dopamine release evoked by the conditioned fear response likely contributes to the lengthy training requirements necessary to achieve primarily avoidance behavior (~15 sessions in the present study). Indeed, we speculate that this sharp decrease in dopamine release contributes to fear-induced species specific defense responses, such as freezing, that might disrupt a motor pattern that is conducive to avoidance behavior (Bolles, 1970). The observations obtained during fear conditioning further support this theory, as we found that conditioned stimuli uniformly decrease behavioral output and dopamine release.

Our data further suggest that the conditioned fear response originally elicited by the warning signal dissipates after extensive training, as the prediction of a positive outcome—successful avoidance—supersedes it. During trials in which avoidance responses occur regularly, operant behavior is directed and not confounded by species-specific defense responses (freezing, attacking) that are prevalent early in avoidance learning (Bolles, 1970). As consistent avoidance behavior develops, the warning signal begins to evoke dopamine release. We speculate that, with repeated training, the interpretation of the warning signal shifts from representing fear to successful avoidance, and dopamine neurons compute this transition by increasing dopaminergic output in the form of large positive prediction error signals (Schultz et al., 1997). This interpretation is consistent with a recent report demonstrating that the representation of a conditioned cue can be transformed between appetitive and aversive stimuli over repeated pairings (Nasser and McNally, 2012).

During both escape and avoidance responses, we observed an increase in dopamine release corresponding to the safety period. This is consistent with the supposition that, even early in training, the safety period functions similarly to an appetitive stimulus (Dinsmoor, 1954, 2001). Thus, in a way, the elimination of punishment is processed similarly to the receipt of reward regardless of the representation of the preceding conditioned stimuli or whether or not punishment actually occurred.

Subsecond changes in dopamine release may contribute to the extinction of freezing behavior in animal models of conditioned fear. Consistent with this theory, recent evidence demonstrates that stimulating dopamine terminal fields in the ventral striatum promotes the extinction of fear memories (Rodriguez-Romaguera et al., 2012). We speculate that subsecond dopamine release functions as a teaching signal to alert the animal to behaviorally relevant events in the environment. In the case of conditioned fear, a decrease in dopamine release may relay information regarding unavoidable negative events to the fear network. As repeated presentations of the cue fail to coincide with an aversive event (as seen in extinction), the conditioned dopamine response may weaken, thereby updating the fear network about associations pertaining to the safety of the surrounding environment. The finding that subsecond dopamine release is decreased by conditioned predictors of punishment may appear at odds with existing microdialysis literature, which suggests that aversive stimuli increase tonic dopamine concentrations in the NAc (Saulskaya and Marsden, 1995; Wilkinson et al., 2000; Young et al., 1998; Pezze et al., 2002; but see Mark et al., 1991). As previously discussed however (see McGinty et al., 2011), subsecond dopamine release is distinct from tonic dopamine transmission, which is assessed with microdialysis. Thus, it is possible that during conditioned fear, subsecond dopamine release is suppressed whereas tonic dopamine signaling is concurrently enhanced through alternative mechanisms (McGinty et al., 2011).

These data provide several implications for the neuroscientifc study of psychiatric disease. For example, opponent-process theory (Solomon and Corbit, 1974) suggests, in the context of drug addiction, investigating subsecond dopamine release during punishment avoidance may be more relevant to the neurobiological underpinnings of the disease than studying reward seeking. Furthermore, novel insights into the neural mechanisms underlying the pharmacological utility of antipsychotic drugs, many of which act on the dopamine system, may be elucidated using conditioned avoidance (Wadenberg and Hicks, 1999). Similar insights into the therapeutic interventions for post-traumatic stress disorder might also be obtained by studying fluctuations in subsecond dopamine release in animal models of conditioned fear (Pezze and Feldon, 2004).

Acknowledgements

This research was supported by NIH grants DA022340 and DA025890 to JFC and DA032266 to EBO. The authors would like to thank Mr. David Bernstein for expert technical assistance, Dr. Keri Chiodo for helpful comments in the preparation of the manuscript and Dr. Paul Phillips for assistance with all aspects of longitudinal voltammetric sensors.

Footnotes

Any Conflict of Interest: none

References

  1. Bolles RC. Species-specific defense reactions and avoidance learning. Psychological review. 1970;77:32. [Google Scholar]
  2. Clark JJ, Sandberg SG, Wanat MJ, Gan JO, Horne EA, Hart AS, Akers CA, Parker JG, Willuhn I, Martinez V, Evans SB, Stella N, Phillips PE. Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals. Nat Methods. 2010;7:126–129. doi: 10.1038/nmeth.1412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012;482:85–88. doi: 10.1038/nature10754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Dayan P. Instrumental vigour in punishment and reward. Eur J Neurosci. 2012;35:1152–1168. doi: 10.1111/j.1460-9568.2012.08026.x. [DOI] [PubMed] [Google Scholar]
  5. Dinsmoor JA. Punishment: I. The avoidance hypothesis. Psychological review. 1954;61:34. doi: 10.1037/h0062725. [DOI] [PubMed] [Google Scholar]
  6. Dinsmoor JA. Stimuli inevitably generated by behavior that avoids electric shock are inherently reinforcing. Journal of the Experimental Analysis of Behavior. 2001;75:311. doi: 10.1901/jeab.2001.75-311. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dreyer JK, Herrik KF, Berg RW, Hounsgaard JD. Influence of phasic and tonic dopamine release on receptor activation. J Neurosci. 2010;30:14273–14283. doi: 10.1523/JNEUROSCI.1894-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Heien ML, Phillips PE, Stuber GD, Seipel AT, Wightman RM. Overoxidation of carbon-fiber microelectrodes enhances dopamine adsorption and increases sensitivity. Analyst. 2003;128:1413–1419. doi: 10.1039/b307024g. [DOI] [PubMed] [Google Scholar]
  9. Herrnstein RJ. Method and theory in the study of avoidance. Psychological review. 1969;76:49. doi: 10.1037/h0026786. [DOI] [PubMed] [Google Scholar]
  10. Ikemoto S, Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Brain Res Rev. 1999;31:6–41. doi: 10.1016/s0165-0173(99)00023-5. [DOI] [PubMed] [Google Scholar]
  11. Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci. 2012 doi: 10.1038/nn.3100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Maia TV. Two-factor theory, the actor-critic model, and conditioned avoidance. Learning & Behavior. 2010;38:50–67. doi: 10.3758/LB.38.1.50. [DOI] [PubMed] [Google Scholar]
  13. Maia TV, Frank MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci. 2011;14:154–162. doi: 10.1038/nn.2723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Mark GP, Blander DS, Hoebel BP. A conditioned stimulus decreases extracellular dopamine in the nucleus accumbens after the development of a learned taste aversion. Brain Res. 1991;551:308–10. doi: 10.1016/0006-8993(91)90946-s. [DOI] [PubMed] [Google Scholar]
  15. Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 2009;459:837–841. doi: 10.1038/nature08028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. McGinty VB, Hayden BY, Heilbronner SR, Dumont EC, Graves SM, Mirrione MM, du Hoffmann J, Sartor GC, España RA, Millan EZ, Difeliceantonio AG, Marchant NJ, Napier TC, Root DH, Borgland SL, Treadway MT, Floresco SB, McGinty JF, Haber S. Emerging, reemerging, and forgotten brain areas of the reward circuit: Notes from the 2010 Motivational Neural Networks conference. Behav Brain Res. 2011;225:148–57. doi: 10.1016/j.bbr.2011.07.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Monfils MH, Cowansage KK, Klann E, LeDoux JE. Extinction-reconsolidation boundaries: key to persistent attenuation of fear memories. Science. 2009;324:951–955. doi: 10.1126/science.1167975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Mowrer OH. Two-factor learning theory: summary and comment. Psychological review. 1951;58:350. doi: 10.1037/h0058956. [DOI] [PubMed] [Google Scholar]
  19. Nasser HM, McNally GP. Appetitive-aversive interactions in Pavlovian fear conditioning. Behav Neurosci. 2012;126:404–422. doi: 10.1037/a0028341. [DOI] [PubMed] [Google Scholar]
  20. Oleson EB, Beckert MV, Morra JT, Lansink CS, Cachope R, Abdullah RA, Loriaux AL, Schetters D, Pattij T, Roitman MF. Endocannabinoids Shape Accumbal Encoding of Cue-Motivated Behavior via CB1 Receptor Activation in the Ventral Tegmentum. Neuron. 2012;73:360–373. doi: 10.1016/j.neuron.2011.11.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Pezze MA, Heidbreder CA, Feldon J, Murphy CA. Selective responding of nucleus accumbens core and shell dopamine to aversively conditioned contextual and discrete stimuli. Neuroscience. 2001;108:91–102. doi: 10.1016/s0306-4522(01)00403-1. [DOI] [PubMed] [Google Scholar]
  22. Pezze MA, Feldon J. Mesolimbic dopaminergic pathways in fear conditioning. Prog Neurobiol. 2004;74:301–320. doi: 10.1016/j.pneurobio.2004.09.004. [DOI] [PubMed] [Google Scholar]
  23. Rescorla RA, Lolordo VM. Inhibition of avoidance behavior. Journal of Comparative and Physiological Psychology. 1965;59:406. doi: 10.1037/h0022060. [DOI] [PubMed] [Google Scholar]
  24. Rodriguez-Romaguera J, Do Monte FH, Quirk GJ. Deep brain stimulation of the ventral striatum enhances extinction of conditioned fear. Proc Natl Acad Sci U S A. 2012;109:8764–8769. doi: 10.1073/pnas.1200782109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Roitman MF, Wheeler RA, Wightman RM, Carelli RM. Real-time chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli. Nat Neurosci. 2008;11:1376–1377. doi: 10.1038/nn.2219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Saulskaya N, Marsden CA. Conditioned dopamine release: dependence upon N-methyl-D-aspartate receptors. Neuroscience. 1995;67:57–63. doi: 10.1016/0306-4522(95)00007-6. [DOI] [PubMed] [Google Scholar]
  27. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  28. Sidman M. Avoidance conditioning with brief shock and no exteroceptive warning signal. Science. 1953 doi: 10.1126/science.118.3058.157. [DOI] [PubMed] [Google Scholar]
  29. Solomon RL, Corbit JD. An opponent-process theory of motivation: I. Temporal dynamics of affect. Psychological review. 1974;81:119. doi: 10.1037/h0036128. [DOI] [PubMed] [Google Scholar]
  30. Wadenberg MLG, Hicks PB. The conditioned avoidance response test reevaluated: is it a sensitive test for the detection of potentially atypical antipsychotics? Neuroscience & Biobehavioral Reviews. 1999;23:851–862. doi: 10.1016/s0149-7634(99)00037-8. [DOI] [PubMed] [Google Scholar]
  31. Watson JB. The place of the conditioned-reflex in psychology. Psychological review. 1916;23:89. [Google Scholar]
  32. Watson JB. Behaviorism. Transaction Pub.; New Brunswick, NJ: 1924. [Google Scholar]
  33. Wilkinson LS, Humby T, Killcross AS, Torres EM, Everitt BJ, Robbins TW. Dissociations in dopamine release in medial prefrontal cortex and ventral striatum during the acquisition and extinction of classical aversive conditioning in the rat. Eur J Neurosci. 1998;10:1019–26. doi: 10.1046/j.1460-9568.1998.00119.x. [DOI] [PubMed] [Google Scholar]
  34. Wightman RM. Detection technologies. Probing cellular chemistry in biological systems with microelectrodes. Science. 2006;311:1570–1574. doi: 10.1126/science.1120027. [DOI] [PubMed] [Google Scholar]
  35. Young AM, Ahier RG, Upton RL, Joseph MH, Gray JA. Increased extracellular dopamine in the nucleus accumbens of the rat during associative learning of neutral stimuli. Neuroscience. 1998;83:1175–83. doi: 10.1016/s0306-4522(97)00483-1. [DOI] [PubMed] [Google Scholar]

RESOURCES