Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation

Abstract

Dopamine in the nucleus accumbens (NAc) is an important neurotransmitter for reward-seeking behaviors such as intracranial self-stimulation (ICSS), although its precise role remains unclear. Here, dynamic fluctuations in extracellular dopamine were measured during ICSS in the rat NAc shell with fast-scan cyclic voltammetry at carbon-fiber microelectrodes. Rats were trained to press a lever to deliver electrical stimulation to the substantia nigra (SNc)/ventral tegmental area (VTA) after the random onset of a cue that predicted reward availability. Latency to respond after cue onset significantly declined across trials, indicative of learning. Dopamine release was evoked by the stimulation but also developed across trials in a time-locked fashion to the cue. Once established, the cue-evoked dopamine transients continued to grow in amplitude, although they were variable from trial to trial. The emergence of cue-evoked dopamine correlated with a decline in electrically evoked dopamine release. Extinction of ICSS resulted in a significant decline in goal-directed behavior coupled to a significant decrease in cue-evoked phasic dopamine across trials. Subsequent reinstatement of ICSS was correlated with a return to preextinction transient amplitudes in response to the cue and reestablishment of ICSS behavior. The results show the dynamic nature of chemical signaling in the NAc during ICSS and provide new insight into the role of NAc dopamine in reward-related behaviors.

Intracranial self-stimulation (ICSS) was discovered in 1954 (1). In this paradigm, a rat depresses a lever to deliver an electric shock to electrodes implanted within the brain. Extensive mapping studies by Olds and Olds later showed that the neuroanatomical region supporting ICSS centered in the posterior MFB region of the lateral hypothalamus (2). This finding provoked considerable interest, because it identified a brain reward pathway that could be centrally activated without the need for sensory stimulation (3, 4). Although a role for several neurotransmitters has been implicated in ICSS, dopamine appears to play a primary role (5, 6), leading to the view that dopaminergic signaling is essential during goal-directed behaviors. Indeed, it was postulated that increased dopaminergic neurotransmission was necessary for the reinforcement of reward-related behavior (7).

More recently, electrophysiological studies in primates have provided new insight into the role of dopaminergic neurons in reward processing (8). In response to unexpected rewards, dopamine neurons exhibit phasic firing. However, when an animal learns that a cue predicts reward, the burst of neuronal firing switches to the onset of the cue (9–12). Responses to the cue increase with repeated trials, and these paired responses of midbrain dopamine neurons follow the expectations of models of associative learning in which dopamine signaling is a reward-prediction error (12, 13). Similar responses to conditioned stimuli that predict reward have also been observed for midbrain dopaminergic neurons in rats (14).

A phasic increase in dopamine neuronal firing should lead to a dopamine concentration transient in terminal areas such as the nucleus accumbens (NAc). Indeed, using fast-scan cyclic voltammetry at carbon-fiber microelectrodes, we have previously shown that cues that predict cocaine (15), liquid reward (16), and food reward (17) evoke a transient increase in NAc dopamine. Dopamine transients also occur in the NAc shell during ICSS in response to conditioned stimuli that predict reward availability and to the intracranial stimulus (ICS) (18). These responses were obtained in animals trained with a fixed time-out between trials. Here, we expand that work and examine whether this cue-evoked dopamine release correlates with behavioral indices of learning when the cues that predict the availability of ICS are presented with a variable time out between trials. Because ICSS is learned quickly in comparison with other reward-based paradigms (19), behavioral correlates of learning can be investigated in a single training session, thus enabling quantification of changes in dopamine release during acquisition of ICSS. Dopamine was monitored with a carbon-fiber microelectrode in the NAc shell while learning was evaluated as the rate of responding after onset of an audiovisual cue. Extracellular dopamine concentration transients, time-locked to cue onset predicting ICS availability, were monitored during regular ICSS (maintenance), extinction, and reinstatement. The results support the concept that rapid dopamine signaling is dynamic and reflect a learned association between cue-related events and ICS.

Results

Dopamine Release During ICSS.

Rats (n = 9) that reached criterion responding during initial training were examined during ICSS by using the VTO paradigm illustrated in Fig. 1A. In the first VTO phase (maintenance), the lever and cues were presented simultaneously for 50 trials. As seen in the color plot for a representative trial (Fig. 1B), the cyclic voltammetric data recorded after the lever press show that the stimulation evoked dopamine release. The dopamine concentration increase after the lever press was confirmed with principal component regression (Fig. 1B Upper). Additionally, in the delay after cue-onset/lever extension but before the lever-press, a small dopamine transient was observed (Fig. 1B, between 0–1 s).

Fig. 1.

Dopamine and behavioral changes during maintenance phase. (A) Temporal sequence during the first 50 ICSS trials (maintenance phase). (B) Voltammetric response from one trial. Dopamine increased immediately after t = 0 s, the time of the cue onset/lever out (red dashed line) and again after the lever press (black dashed line). (C) Maximum cue-evoked dopamine concentration increased with each cue (n = 9 animals). (D) Latency to press decreased across trials. (E) There was a linear correlation between latency and maximum cue-evoked dopamine concentration.

Although not seen on every trial, cue-evoked dopamine release was observed in all animals. The mean dopamine amplitude associated with each subsequent cue/lever extension (trial) during the maintenance phase increased in a linear fashion (r² = 0.047, P < 0.0001) (Fig. 1C). The latency to press the lever after its extension decreased significantly over trials and was fit to a parabolic curve (r² = 0.064, P < 0.05) (Fig. 1D). During the first five trials, the average latency to press for all animals was 5.3 ± 0.9 s, and this latency decreased on the last five trials to 1.4 ± 0.3 s. The decreased latency with trial number was inversely correlated with the amplitude of cue-evoked dopamine and fit to a parabolic curve with a significant linear correlation between latency to press and dopamine concentration (r² = 0.022, P < 0.05) (Fig. 1E).

Dopamine Release During the Maintenance-Delay Phase.

After the 50th trial, the maintenance-delay phase began, with lever extension shifted so that it was delayed 2 s relative to the onset of the cues (Fig. 2A). With this delay, cue-evoked dopamine became more clearly resolved from electrically evoked dopamine as shown for a single representative animal in Fig. 2B. The initial increase in dopamine began immediately after 0 s, i.e., at the onset of the compound cue, reached a maximum, and then fell before the lever extension. The lever extended 2 s after cue-onset, and electrically evoked dopamine release was observed after the lever press. In this animal, cue-evoked dopamine was not seen in every trial (Fig. 2C). However, when evaluated across the maintenance-delay phase for this animal (trials 51–200), the amplitude of cue-evoked dopamine was found to significantly increase in a linear fashion (r² = 0.42, P < 0.0001). This increase occurred even though there was no significant change in latency to lever-press across these trials (Fig. 2D). In contrast to cue-evoked dopamine, extracellular dopamine after the electrical stimulation decreased significantly over trials 51–200 (r² = 0.13, P < 0.0001) (Fig. 2E).

Fig. 2.

Dopamine and behavioral changes during maintenance-delay phase. (A) Temporal sequence used for trials 51–200 (maintenance delay phase). (B) Voltammetric data recorded during a single trial. The dopamine concentration rise begins at t = 0 s with cue onset (red dashed line) and again after the lever press (black dashed line). (C and D) Maximum cue-evoked extracellular dopamine concentration increased with trial number in this animal (C), whereas latency to press remained constant (D). (E) Electrically evoked extracellular dopamine decreased during trials 51–200.

Similar results were obtained in eight other animals. Fig. 3A shows the average of all trials in another animal and the increase in dopamine after the cue is clearly observed along with the second increase after the lever-press. The average amplitude of electrically evoked dopamine release during ICSS from all animals (n = 7 rats, two ICSS rats were excluded as stimulated dopamine release was not significantly elevated after stimulation) decreased across the maintenance and maintenance-delay phases (Fig. 3B). Although electrically evoked dopamine was initially high, there was a significant attenuation in dopamine concentration across trials (in two animals, the stimulated release actually increased in the first three trials) (Fig. 3B). Superimposed on the experimental data are a line computed with a neurochemical model that predicts dopamine release with repeated stimulation (see Fig. 3D and Discussion) (20).

Fig. 3.

Dopamine and behavioral changes in all trials. (A) Averaged voltammetric data recorded in the NAc shell of a single animal during trials 51–200. Dopamine concentration begins to rise at t = 0 s with cue onset (red dashed line) and after the lever press (black dashed line). (B) Points: average maximal electrically evoked dopamine concentration decreases over trials 1–200. During trials 4–50, stimulated dopamine release decreased significantly. During trials 151–200, stimulated dopamine did not decrease further. Solid line: simulation of maximal dopamine release to stimulus trains repeated at 17.5 s intervals, the average between ICS trials (contributions of the cue-evoked dopamine responses were not included). Although the simulation includes terms for short term facilitation and depression, they are ineffective on this time scale. The long term depression used an amplitude of 0.999 and a time constant of 12 min (20). (C) Average of the maximal cue-evoked dopamine concentration increased with trial number (n = 9 animals). (D) Average latency to press did not change during trials 51–200.

The average cue-evoked dopamine concentration increased over trials (1–200) and could be fit to a parabolic curve, leveling off at later trials (r² = 0.15, 95% confidence interval) (Fig. 3C). In the final trials (196–200), the dopamine concentration reached a plateau of 142 ± 22 nM. Latency to lever press (1–200) was fit to a parabolic curve (r² = 0.16) (Fig. 3D). Across all animals, the latency to lever-press after lever extension remained constant after the maintenance phase (trials 1–50), with a value of 0.8 ± 0.2 for trials 51–55 and 0.9 ± 0.1 s for trials 196–200, values that are not significantly different (Fig. 3D). There was no significant relationship between latency to press and cue-evoked dopamine (data not shown).

Dopamine Release During Extinction and Reinstatement of ICSS.

Some of the rats (n = 8) that had completed the maintenance-delay VTO phase were then tested in another paradigm that consisted of 30 VTO ICSS maintenance-delay trials, followed by 30 trials in which the electrical stimulation was not delivered when the lever was pressed (extinction), and finally, a reinstatement phase in which the stimulation was restored. The temporal fluctuations of dopamine concentration were extracted by principal component regression and are shown for one representative animal in Fig. 4 A, D, and G. As expected, the dopamine signals at cue-onset and during the electrical stimulation were readily apparent during maintenance (Fig. 4A). During extinction, stimulated release was eliminated, and this was accompanied by a profound decrease in the amplitude of cue-evoked dopamine (Fig. 4D). Both cue and electrically evoked dopamine were restored during reinstatement (Fig. 4G).

Fig. 4.

Dopamine and behavioral changes during extinction. (A, D, and G) Dopamine concentration during single trial in an animal during maintenance, extinction and reinstatement, t = 0 s is cue-onset. Remaining panels: pooled data from eight animals. (B) During maintenance, maximal amplitude of cue-evoked extracellular dopamine and (C) latency to press were constant. (E and F) During extinction, the maximal concentration of cue-evoked dopamine decreased significantly (E), whereas latency to press significantly increased (F). (H and I) During reinstatement, the maximal concentration of cue-evoked dopamine rapidly returned to preextinction values (H) as did the latency to press (I).

When examined on a trial by trial basis for all animals tested, the maximum cue-evoked dopamine concentration remained fairly constant during the maintenance trials; there was no significant correlation between dopamine concentration and trial number (rats were allowed 30 maintenance trials, however only presses 11–30 are shown because of variability in rest time between phases which increases the variability of both response time and magnitude of cue-evoked dopamine release) (Fig. 4B). During these trials, rats pressed with stable rates that did not change (mean latency after lever extension was 1.1 ± 0.1 s) (Fig. 4C). During extinction, cue-evoked dopamine concentrations decreased, and the dopamine concentration plotted against trial number could be fit to a linear decline (r² = 0.140, P < 0.0001) (Fig. 4E). At the same time, latency to lever press rapidly increased (r² = 0.331 P < 0.0001) (Fig. 4F) and there was a significant, inverse, linear correlation between extracellular dopamine and latency to press (r² = 0.273, P < 0.0001, data not shown). At the beginning of the reinstatement phase, rats were primed 0–3 times (data not shown) to resume lever pressing. Once ICSS behavior was reestablished, cue-evoked dopamine concentrations rapidly returned to preextinction values after trial 1 and then maintained a constant level (trial 1 differs significantly from trials 2–10 (P < 0.05, unpaired t test)). At the same time, latency to press significantly decreased (r² = 0.074, P < 0.0001) (Fig. 4I).

Discussion

A central role for dopamine in reward-based behaviors has long been recognized (19). Our high speed recordings of the chemical dynamics of dopamine during ICSS resolve this into different components. Initially dopamine transients are only seen at stimulus delivery, and these closely resemble the dopamine responses to noncontingent electrical stimulation. With repeated trials, dopamine transients develop at the cues that predict reward availability, and these grow with increasing trials while the stimulus-evoked release diminishes. The development of cue-evoked dopamine correlates with a decline in latency to press the lever with repeated trials, indicative of learning. During the extinction phase, when the electrical stimulus was withheld, the cue-associated dopamine transient amplitude decreased whereas the latency to press dramatically increased. Upon reinstatement of the electrical stimulus, the cue-associated dopamine transients rapidly reemerged, and the latency to press diminished. The appearance of a dopamine signal associated with a random cue that predicts reward is consistent with the firing patterns of dopaminergic neurons during reward based behaviors that have been shown to follow the theories of reward-prediction error (12, 14).

Whereas cue-evoked dopamine transients increased in concentration with trial number, electrically evoked dopamine release gradually decreased, again resembling dopamine neuronal responses seen in reward-prediction experiments in which a switch of dopamine signaling from the reward to the cue occurs (21). However, unlike natural rewards, the reinforcement in ICSS involves directly depolarizing neuronal networks. Although calculations and experiments indicate direct depolarization is less likely with stimuli delivered to the cell bodies (22–24), electrically evoked dopamine release could arise from transsynaptic activation of glutamatergic or cholinergic afferents in the VTA. Indeed, by using much different stimulation parameters and locations, it has been shown that ICSS can be supported by stimulations that activate descending, nondopaminergic fibers and secondarily effect dopamine neurotransmission (25–27).

The diminished amount of dopamine release evoked by the stimulation has been reported in other ICSS studies (28). Stimulation-evoked dopamine release declines because of a restricted releasable pool of dopamine (20, 29, 30). A mathematical model proposed by Montague and coworkers predicts diminished dopamine release over the long term of the 200 ICSS trials that is quite similar to our experimental results. Autoreceptor interactions can also affect release amplitude of closely spaced dopamine release events (31), and thus, the cue-evoked release could further modulate the stimulated release.

As the association between the cues that predict ICS availability and reward was established, the amplitude of the cue-evoked dopamine signal increased, and it was inversely correlated with the latency to press. However, the relationship between dopamine and our measure of learning was not linear because the amplitude of cue-associated dopamine continued to increase during the maintenance-delay phase, eventually reaching a plateau (∼150 nM). Thus, it appears that a floor effect had been achieved for the behavioral measure. During this portion of the behavioral paradigm, extracellular dopamine release after cue-onset in some cases exceeded levels of electrically evoked dopamine release. This concentration is sufficient to activate the D1 receptors (32) that have been shown to be important in ICSS (18).

Dopamine neurons are activated by reward-predicting stimuli that cause phasic firing that lasts for ≈200 ms (33). Consistent with a burst evoking release, the initiation of the dopamine rise in response to the cue is immediate as it is in response to the electrical stimulation. Prior work using amperometry, a technique with higher temporal resolution, shows that it takes ≈15 ms for dopamine to diffuse out of the synapse and reach the probe (34). However, when used with fast-scan cyclic voltammetry, the electrode has a delayed response to reach the peak (∼ 0.2 s) as evidenced by the maximal dopamine evoked by the 0.4 s electrical stimulations at the lever press that maximizes at 0.6 s (35). Taking these delays into account, the cue-evoked dopamine transients are likely the result of burst firing observed with cues that predict reward in electrophysiological studies (14).

The increase in cue-evoked dopamine amplitude with trial number can be observed even in the results from a single animal. The variability in dopamine release between consecutive responses is striking, even though the latency to press remains constant. The fluctuations in cue-evoked dopamine release were not due to a lowered electrode sensitivity as the dopamine response to cues increased across trials. Instead, the data reveal the complexity of chemical signaling during behavior. Unlike conventional chemical probes that provide an average concentration over a relatively large region, the carbon-fiber electrode reports temporal fluctuations from a microscopic local environment immediately adjacent to the electrode (36). Although the NAc shell functions as a unit that may influence behavior, the fluctuations in amplitude of dopamine release appear to indicate that the behavior is not specific to a single set of terminals. Thus, terminal release varies from trial to trial much like the firing pattern of dopaminergic neurons in response to reward predictors when examined on a trial-by-trial basis, e.g., middle panel of figure 12 in ref. 10. Cue-evoked chemical signaling mimics neuronal activity, whereby the sum of dopamine transients across trials reflects the chemical message of cue-reward (ICS) associations.

Extinction trials were done in animals that showed stable ICSS and cue-related dopamine release. During the extinction phase, cue-evoked dopamine transients in the NAc shell rapidly diminished whereas the latency to press increased. Upon reinstatement of the association between cues and electrical stimulation, ICSS resumed with a partially restored, cue-evoked dopamine transient apparent at the first press. The latency to lever-press rapidly diminished whereas the cue-evoked dopamine returned to preextinction values on subsequent trials. These results are quite similar to the restoration of cue-associated dopamine transients during reinstatement of cocaine self-administration after its extinction (37). This rapid reacquisition of performance and dopamine signaling provides strong evidence that extinction did not eliminate all original associations between the cue, the response requirement, and the reward (38). Thus, rapid dopamine signaling in the NAc follows the expectations of reward-prediction error theory in which cue-evoked dopaminergic signals in the shell reflect “errors” when the brain fails to predict the onset of predictive cues (12). Consistent with this, the concentration of dopamine released in response to the cue grows during formation of the association between cue-reward and/or cue response requirement to a limiting value (14). However, when the cue is no longer associated with the ICSS reward (extinction), the acquired dopamine signal rapidly disappears.

Although dopamine's release during the acquisition of cue-evoked ICSS is revealed by this study, further studies are needed to fully understand the complete neural circuitry underlying this behavior (19). Cue-evoked dopamine signaling may involve activation of ascending GABAergic neurons projecting from the VTA (39, 40) or activation of descending neurons. Indeed, the pedunculopontine tegmentum (PPTg), a site that is a major input to dopaminergic neurons in the VTA, show phasic activity to the onset of cues (41). During ICSS, extracellular acetylcholine levels increase in both the PPTg and the VTA (42–44). This could activate phasic firing of dopaminergic neurons leading to the dopamine transients we observe in the NAc shell. The role of cue-evoked dopamine transients may be to potentiate corticostriatal postsynaptic potentials, a function established for dopamine in rats undergoing ICSS (45). Future studies will be required to evaluate dopaminergic activity in the NAc core during similar behaviors, as discussed in prior work (18). Indeed, using a similar protocol, we previously reported stimulus evoked dopamine changes in the NAc core, but, over a limited set of trials, these were unaccompanied by cue-evoked dopamine signals (46).

Taken together, the data presented here suggest a complex role of NAc dopamine in ICSS. As reported previously, activation of dopaminergic neurons facilitates the initiation of ICSS-behavior in tasks that do not involve a discrete audiovisual cue or extended periods between trials (3, 28). Our chemical measurements reveal two aspects of dopamine signaling in the shell. First, cues that predict ICS contingent on a response evoke transient dopamine concentrations that are high enough to activate D1 receptors. This D1 activation is highly significant, because it has been linked to neural processing related to long term potentiation, a change in synaptic strength linked to learning (45). Second, like individual dopaminergic cell bodies, dopaminergic terminals at one location do not respond in the same way during all trials as the behavior is learned. This finding reveals the stochastic nature of chemical signaling in the brain.

Materials and Methods

Surgical Procedures.

Surgery for voltammetric recordings followed previously described procedures (47). Briefly, a guide cannula (Bioanalytical Systems, West Lafayette, IL) was implanted above the NAc shell (1.7 mm anterior, 0.8 lateral, coordinates relative to bregma), and a bipolar stimulating electrode (Plastics One, Roanoke, VA) was lowered to the substantia nigra/ventral tegmental area (VTA, 5.2 mm posterior, 1 mm lateral and 7.8 mm dorsoventral). The bipolar stimulating electrode tips were 1 mm apart. This tip separation allowed for centering in the VTA-region. These coordinates assure activation of the neurons projecting to the NAc shell (48). An Ag/AgCl reference electrode was placed in the contralateral hemisphere (coordinates from ref. 49). For detailed surgical procedures, see supporting information (SI) Materials and Methods.

ICSS.

Rats (n = 9) were trained to criterion on an FR-1 schedule, lever continuously presented. After this rats were trained to lever press on a variable time-out (VTO) schedule, FR-1 (Fig. 1A). The VTO-schedule comprised of a maintenance and a maintenance-delay phase. When the animal depressed the lever, a stimulus train (24 biphasic pulses, 60 Hz, 125–150 μA, 2 ms per phase) was delivered to the stimulating electrode on average 150 ms later. In the maintenance phase, the lever was presented with an audiovisual cue for 50 trials. In the maintenance-delay phase, the audiovisual cue preceded lever-out by 2 s (trials 51–200) (Fig. 2A). Each trial finished after lever depression or if the animal failed to lever press after 15 s. The intertrial interval varied between 5 and 25 seconds. See SI Materials and Methods for details.

Next, some animals (n = 8) were tested under extinction conditions. After a rest interval they were given another 30 maintenance-delay trials with the same protocol. The next 30 trials (extinction) were identical except that depression of the lever had no consequence (i.e., no electrical stimulation). Finally, the reinstatement phase followed and consisted of 0–3 operator delivered “priming” stimulations, and another 30 trials identical to those in the maintenance-delay phase.

Fast-Scan Cyclic Voltammetry.

Carbon-fiber microelectrodes were prepared with T650 fibers (6-μm diameter; Amoco Corporation) inserted into a pulled glass pipette (A-M Systems). The carbon fiber was allowed to extend 50–100 μm beyond the glass tip. The carbon-fiber electrode was held at −0.4 V versus Ag/AgCl, and every 100 ms a cyclic voltammogram was acquired. The applied potential was ramped to +1.3 V and back in a triangular fashion at 400 V/s (50). Timing, voltage application, and data collection was achieved with an interface board (National Instruments) in a Pentium IV computer running custom-designed LABVIEW (National Instruments) software. The interface board also controlled the stimulations.

The background-subtracted voltammograms were plotted with the abscissa as acquisition time of the cyclic voltammogram, the ordinate as the applied potential, and the current in false color (51). Dopamine oxidation occurs at approximately +0.6 V vs. Ag/AgCl. Carbon-fiber electrodes were postcalibrated for dopamine concentration in vitro in a flowcell system.

Principal Component Regression.

Principal component regression was used to extract the dopamine concentration from the voltammetric data (52, 53), see SI Materials and Methods.

Verification of Carbon-Fiber Microelectrode Placement.

Electrode placement was verified for each electrode; see SI Materials and Methods and Fig. S1.