Figure 2.

Theoretical illustration of dopamine neuron activity during three different self-administration phases, which is based on phasic dopamine release that occurs in the NAc during drug self-administration, extinction, or reinstatement (Phillips et al., 2003; Stuber et al., 2005). (A) In the acquisition or maintenance phases, animals are placed in a specific context in which they learn that active lever presses or nose pokes elicit intravenous drug administration (this is sometimes paired with a cue, which becomes associated with the rewarding properties of the drug). Once acquired, as shown by steady increases in daily active lever or nose pokes (lower left graph, black line), the maintenance phase is initiated and the animals’ active lever presses or nose pokes reach a plateau. The inset in panel (A) shows the activity of a midbrain dopamine neuron during drug self-administration. The firing of the dopamine neuron is precisely timed with reward prediction errors (RPEs) such that an unexpected delivery of drug elicits phasic dopamine neuron firing. When these rewards are repeatedly preceded by a cue, phasic dopamine neuron firing shifts from occurring at the time of reward to occurring at the time of exposure to the reward-predicting cue. (B) The extinction phase refers to extinguishing the behavior learned during the acquisition/maintenance phase (lower middle graph, black line), which occurs by eliminating drug delivery after the animal completes a learned behavioral response to obtain the drug (e.g., an active lever press or nose poke). This behavior is performed in the absence of a cue in tests requiring cue-reinstatement. The inset in panel (B) shows that omitting the expected reward (negative RPE) inhibits dopamine neuron firing. However, drug-associated contexts (i.e., house light, lever or nose poke hole) are likely to induce phasic firing of dopamine neurons until these learned associations are updated during extinction training (McFarland and Ettenberg, 1997; Crombag and Shaham, 2002; Stuber et al., 2005). (C) Drug-associated cues reinstate the learned behavior of active nose pokes (lower right graph, panel C). If the reward is omitted following the cue presentation (negative RPE), the tonic firing of dopamine-neurons would be inhibited at the time the reward is expected (inset, panel C). If this reward omission is repeated, dopamine neurons no longer show phasic firing patterns during reward-predicting cue re-exposure (Schultz et al., 1997; Schultz, 2016). The precise timing of dopamine-neuron firing controls cue-reward learning making the phasic firing of dopamine neurons an important neural mechanism responsible for the adaptability of reward-seeking behaviors.