Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Sep 1.
Published in final edited form as: Eur Neuropsychopharmacol. 2008 May 21;18(9):628–638. doi: 10.1016/j.euroneuro.2008.04.005

Certain or uncertain cocaine expectations influence accumbens dopamine responses to self-administered cocaine and non-rewarded operant behavior

Manoranjan S D’Souza 1, Christine L Duvauchelle 1,*
PMCID: PMC2630114  NIHMSID: NIHMS68200  PMID: 18499407

Abstract

Uncertainty and errors in predicting natural rewards influence associative learning and dopamine activity. The present study was conducted to determine the influence of cue-induced cocaine uncertainty, certainty and prediction error on nucleus accumbens dopamine (NAcc DA) in rats. For Certainty training, distinctive sensory cues were present during cocaine availability and alternate cues were paired with non-reinforced (saline) operant sessions. For Uncertainty training, all cues were equally associated with both cocaine and non-reinforcement. After training, animals self-administered cocaine or saline in the presence of conditioned cues while NAcc DA responses were assessed using in vivo microdialysis. Findings revealed cocaine-stimulated NAcc DA increased significantly less in Certainty- compared to Uncertainty-trained animals, and cocaine-paired cues in the absence of cocaine (Negative Prediction Error) resulted in a significant depression of baseline NAcc DA. These findings provide support for enhanced DA activity during cocaine uncertainty or the development of conditioned cocaine tolerance in subjects certain of a cocaine outcome.

Keywords: uncertainty, prediction error, conditioned tolerance, nucleus accumbens, dopamine, expected nonreward

1. Introduction

Drug-associative learning plays an important role in drug-use relapse and maintenance of drug addiction. Several reports suggest that stimuli previously associated with drug-rewards lead to intense craving among abstinent addicts (Ehrman et al. 1992; Childress et al. 1999; Foltin and Haney 2000; Kilts et al. 2004), and are believed to perpetuate further drug use (Gawin 1991). Similarly, experimental studies in animals have shown that learned associations between environmental cues and cocaine result in persistent cocaine seeking and enhancement of cocaine-stimulated behaviors (Duvauchelle et al. 2000; Ito et al. 2002; Ciccocioppo et al. 2004).

Over the last few years, a number of studies involving both human and animal subjects have examined the role of the midbrain dopamine neurons with respect to associative learning involving natural rewards. Primary reinforcers do not increase neuronal bursting in well-trained animals, (Mirenowicz and Schultz 1994; Roitman et al. 2004). However, using reward-associated cues, single unit studies in non-human primates show that midbrain dopaminergic neurons are activated by unexpected rewards (Ljungberg et al. 1992; Mirenowicz and Schultz 1994) and depressed by non-occurrence of expected rewards (Hollerman and Schultz 1998) and by expected non-reward (Matsumoto and Hikosaka 2007). According to contemporary learning theories, errors in predicting reward and uncertainty of reward occurrence play an important role in associative learning (Kamin 1969; Rescorla and Wagner 1972; Pearce and Hall 1980). Errors of reward prediction occur when expected rewards either fail to occur in the presence of cues consistently associated with rewards (negative prediction error) or occur in the presence of cues not associated with reward (positive prediction error). Reward uncertainty exists in the absence of an accurate predictor for reward and is thought to be crucial for the establishment of associative learning (Kamin 1969). Uncertainty is associated with increased activation in dopamine terminal regions in human fMRI studies (Berns et al. 2001; Knutson et al. 2001) and enhanced dopaminergic neuronal firing in non-human primates. Indeed, when the probability of reward is 50%, uncertainty and midbrain neuronal responses are at their greatest (Fiorillo et al. 2003; Dreher et al. 2006).

Associative learning studies utilizing natural reward (as referenced above), are thought to provide a framework for understanding drug-associative learning (Kelley and Berridge 2002; Di Chiara 2005). However, empirical work indicates that the drug experience leaves a more significant impact on neural systems than does nutrient ingestion. For example, cues associated with drug rewards are more resistant to extinction compared to natural reward-associated cues (Weiss et al. 2001; Ciccocioppo et al. 2004). Also, repetitive drug use results in sensitization (Di Chiara et al. 1999) and morphological neuronal changes, such as increased spine density (Li et al. 2003) but repeated ingestion of natural rewards does not (Robinson et al. 2001; Di Chiara 2005). In view of these differences, there is an essential need to specifically study neural mechanisms involved in drug-associative learning.

The nucleus accumbens (NAcc) is a major terminal region of midbrain dopaminergic neurons arising from the ventral tegmental area (VTA) (Fallon and Moore 1978). The NAcc has been known to play a critical role in drug-associative learning and drug reward (Weiss et al. 2000; Weiss et al. 2001; Phillips et al. 2003). Enhancement in NAcc DA has been reported following exposure to cues previously associated with cocaine (Kiyatkin and Stein 1996; Di Ciano et al. 1998; Duvauchelle et al. 2000). Similarly studies have shown that cues predictive of cocaine reward have a dopamine dependent influence on NAcc firing rates (Nicola and Deadwyler 2000; Carelli and Ijames 2001; Ghitza et al. 2003). However, the impact of cue-induced cocaine predictions on basal and cocaine-stimulated NAcc DA responses has not been thoroughly examined.

The present study utilized a cue conditioning technique to produce cue-induced cognitive states of ‘Certainty’ and ‘Uncertainty’ with regards to cocaine reinforcement. ‘Certainty’ cue training consisted of pairing certain sensory cues with self-administered cocaine and alternate cues with non-reinforcement (saline). As indicated above, since midbrain neural responses are greatest when the probability of reward is 50% (Fiorillo et al. 2003; Dreher et al. 2006), we used that percentage in our cocaine ‘Uncertainty’ training. For instance, ‘Uncertainty’ cue training consisted of equal pairings of two sets of sensory cues with both cocaine and non-reinforced operant sessions. After cue-training sessions were completed (16 total sessions; 8 cocaine and 8 non-reinforced), an in vivo microdialysis test session within the operant chamber was employed to test effects of ‘uncertainty’, ‘certainty’ and ‘prediction error’ on cocaine- and non-stimulated NAcc DA responses and locomotor activity. ‘Certainty’ effects were determined under the same cue conditions as during training, and ‘Prediction Error’ effects were determined with mismatched cue/reinforcement conditions. For instance, cocaine-associated cues were presented in conjunction with a non-reinforced lever response (negative prediction error), and a cocaine injection was self-administered in the presence of saline-associated cues (positive prediction error). ‘Uncertainty’ effects were tested in animals that had undergone ‘Uncertainty’ cue training. In that condition, either cue set was equally predictive of either cocaine- or non-reinforcement, thus the presentation of either cue was conceptualized as a signal producing maximal uncertainty.

2. Materials and methods

2.1 Animals

Male Sprague Dawley rats (Charles River, Houston) weighing approximately 300 g at the beginning of the experiments were used. The rats were housed individually after surgery in polypropylene cages and maintained on a 12 hr. reversed light/dark cycle (lights on 7:00 p.m. to 7:00 a.m.). Animals were handled daily for 2 weeks prior to the start of the experiment. Food and water was available ad libitum in the home cage except during the food-training phase.

2.2 Apparatus

Food training, self-administration sessions and in vivo microdialysis test sessions were conducted in one-lever operant chambers (28 × 22 × 21 cm) located within sound-attenuating compartments (Med Associates, St. Albans, VT). Sensory cues and lighting conditions introduced during cocaine and non-reinforced operant conditioning and test sessions (see Cue Conditioning/Self-Administration and Test Session below) were absent during food-reinforced training. Chamber set-up, construction and controls were as previously described (Ikegami et al. 2007).

2.3 Food Training and Surgery

Animals were food restricted (≈ 6 g of standard rat chow per day, adjusted as needed to maintain, not decrease body weight) and trained to lever press for food on a FR1 schedule of reinforcement. Each lever response resulted in dispensing one sucrose pellet (45 mg; P.J. Noyes, Lancaster, NH). After the lever press response for food was acquired (approx 3 days), 10-min. food-reinforced operant sessions (FR1) were conducted for the next 6 days without food restriction.

After the completion of food training sessions, animals were implanted with a chronic silastic intravenous (i.v.) jugular catheter (0.6 mm o.d.) under pentobarbital sodium (Nembutal®, 50 mg/kg, i.p.) anesthesia. Atropine sulfate (250µg/rat, s.c.) was given prophylactically to prevent respiratory tract secretions. Supplemental chloral hydrate (80 mg/kg, i.p.) was given, if necessary, to prolong anesthesia. Catheters were implanted such that the free end of the catheter with a cannula termination (Plastics One) passed subcutaneously on the side of the neck, out an incision in the animal’s head and mounted on the skull. Animals were also stereotaxically implanted with a unilateral guide cannula (21 g) aimed above the NAcc (flat skull; AP: + 1.7 mm; ML:± 1.7 mm; DV:− 2.5 mm). The catheter cannula and the guide cannula were affixed to the skull with four stainless steel screws and dental acrylic cement. Animals underwent a minimum of one-week recovery prior to the beginning of the experiments. After the surgery, animals received 0.1 ml of saline containing 67.0 mg/ml of the antibiotic, Timentin and 30 U/ml heparin through their i.v. catheters daily for the next week. Animals continued receiving the same solution daily without the Timentin component through the duration of the experiment to maintain catheter patency (Emmett-Oglesby t al. 1993).

2.4 Cue Conditioning/Self-Administration Sessions

Animals were weighed daily and cocaine concentrations were altered accordingly for cocaine sessions, so that each lever press resulted in the delivery of 0.5 mg/kg cocaine hydrochloride in a volume of 0.1 ml of isotonic saline. During saline sessions, an equal volume of saline was infused. A cue light above the lever was illuminated for the 6-sec duration of each infusion. After each infusion, there was a 20-sec “time-out” period, during which time the lever was retracted, the stimulus light turned off and no infusions could be delivered. Training consisted of 16 alternating days of cocaine and saline availability (8 sessions each for cocaine and saline) during one-hour conditioning/self-administration sessions. During the first 30 min of each session, the chamber was darkened and the lever was retracted while animals habituated to the neutral environment. After 30 min, the house light was illuminated, discriminatory stimuli in the form of sensory cues (see below) and the lever were inserted, and cocaine or saline was then available for a total of 30 min.

Sensory Cues

Visual and olfactory environmental cues were introduced into the operant chamber immediately following the 30 min darkened habituation period. Visual cues consisted of either black or white felt wall coverings attached to the sides of the clear Plexiglas operant chamber. Olfactory cues consisted of an oil-based scent (cinnamon or rose) saturated on a cotton ball located under the grid floor of the operant chamber.

Training Assignments

Animals were assigned to undergo ‘Certainty’ conditioning with specific cues sets (olfactory + visual) consistently associated with cocaine or non-reinforced (saline) sessions, or one of the two types of ‘Uncertainty’ conditioning, where each of the two distinct cue sets was equally associated with cocaine and saline sessions. For ‘Uncertainty 1’, the two sets of cues were alternated daily between reinforcement conditions. For example, the cues for the first cocaine session (Day 1) was used on the second saline session (Day 4); the cue for first saline session (Day 2) was used for second cocaine session (Day 3), and so on. The ‘Uncertainty 2’ group had one set of cues associated with the first four cocaine sessions and another set of cues associated with the first four saline sessions. For the last eight sessions, the cue/reinforcement sets were switched. It should be noted that the Certainty group was assigned the largest number of animals (n=29) due to the need for this group to be subdivided into 4 groups for the dialysis testing segment of the experiment.

2.5 In vitro recovery and microdialysis probe implantation

Microdialysis probes were constructed as previously described (Duvauchelle et al. 2000), with an active membrane length of 2.5 mm at the probe tip. Prior to probe recovery, all probes were flushed with nanopure water. Recovery values for each probe were calculated by comparing the peak heights of samples to those from a standard as previously described (Ikegami et al. 2007). The mean (± SEM) recovery of probes used in the experiment was 14.04 + 0.6%.

Within 12 hrs after the 16th self-administration session, animals were briefly anesthetized with isoflurane (Abbott Laboratories, IL) and implanted with a microdialysis probe through the previously implanted guide cannula. Each microdialysis probe was connected to a 1.0 ml gastight Hamilton 1000 series syringe mounted on a syringe pump (Razel®, Model A), and freshly prepared Ringer’s solution was pumped through the probe. Animals implanted with the probe remained in a holding chamber overnight with the syringe pump speed set at 0.261 µl/min. Bedding, food, and water were available in the holding chamber. One hour prior to the test session, the pump speed was changed to 1.63 µl/min.

2.6 Testing NAcc DA and Locomotor Activity Responses

Conditions

Animals were tested 24 hrs after the completion of training (at least 12 hrs post probe implantation) in one of six different testing conditions: Certainty/Cocaine; Certainty/Saline; Positive Prediction Error/Cocaine; Negative Prediction Error/Saline; Uncertainty/Cocaine and Uncertainty/Saline. Certainty and Prediction Error conditions had undergone Certainty cue training and the Uncertainty condition had undergone Uncertainty 1 or Uncertainty 2 cue training. For the Certainty/Cocaine and Certainty/Saline conditions, animals were tested with the same cue/reinforcement conditions as during training. For the Prediction Error conditions, animals were tested with mismatched cues and reinforcement. For example, in the Negative Prediction Error group, cocaine-associated cues accompanied a non-reinforced lever response. For the Positive Prediction Error condition, non-reward associated cues were presented prior to a self-administered cocaine injection. For the Uncertainty condition, all cue sets had been equally associated with cocaine and non-rewarded sessions.

Test Session

Animals were placed in the operant chamber with the lever retracted for the first 30 min (Baseline). After 30 min., the house light illuminated, the lever extended into the chamber and the assigned cues were introduced into the chamber. Animals were allowed to respond once on the lever and then received either a single injection of cocaine (1.5 mg/kg) or saline (approx 0.1 ml) infused over a 6-sec interval. The lever was then retracted for the remainder of the session. In vivo microdialysis samples were continuously collected at 10 min intervals across the entire test session, comprising 3 10-min Baseline and 3 10-min Test (e.g., post-cocaine or saline injection) samples. Locomotor activity units (photobeam breakages) were assessed in correspondence with dialysis sampling.

2.7 Assay of dialysate

The dialysates were analyzed for DA concentration using HPLC and electrochemical detection. The HPLC had a Shizeido capcell C-18 narrow bore column, ESA model 5200 A Coulochem II detector, a Model 5041 cell and a Model 5020 Guard cell. The mobile phase composition was as follows: sodium dihydrogen phosphate (75 mM), citric acid (4.76 mM), SDS 1 g/l, EDTA (0.5 mM), MeOH 8% and acetonitrile 11% (v/v), pH 5.6. The analytical cell potential was set at + 200 mV (oxidation), guard cell potential at 400 mV, and the pump speed at 0.2 ml/min. The detection limit of DA was 0.05 pg/ul with a signal to noise ratio of 3:1. The amount of DA within each sample was determined by comparison with standards prepared and analyzed on the day of sample analysis. Prior to correcting for probe recovery, mean (± SEM) basal NAcc DA concentrations (first baseline sample) for all animals was calculated at 0.6 ± 0.05 nM (n=54). Data were collected and analyzed using an ESA Model 500 Data station.

2.8 Histology

After the experiment, animals were euthanized by an overdose of pentobarbital sodium and brains were perfused using 0.9% saline and 10% formalin. The brains were carefully removed and stored in a 10% formaldehyde/30% sucrose solution. Probe placements within the NAcc were verified using the atlas of Paxinos and Watson (Paxinos and Watson 1998). Fig 1 depicts probe tracts from cresyl violet stained coronal sections (60 µm).

Fig. 1. Histological Diagram.

Fig. 1

Open in a new tab

Tracings represent active membrane of dialysis probe in the nucleus accumbens (NAcc) regions (Paxinos and Watson 1998). Coronal sections of probe traces in the NAcc ranged from +2.20 mm through +1.2 mm anterior to Bregma. Active membrane region of dialysis probes were located in core and shell subterritories of the NAcc (n=54).

2.9 Statistical Analyses

Behavioral and NAcc DA data were compared using two- and three-way repeated measures ANOVAs. Lever responses and locomotor activity across training days for Certainty, Uncertainty 1 and Uncertainty 2 groups were compared using three-way repeated measures ANOVAs (Training Condition [Certainty, Uncertainty 1, Uncertainty 2] × Treatment [Cocaine, Saline] × Session Days [8]). To compare the magnitude of NAcc DA responses between differently conditioned and cued groups (Certainty, Uncertainty and Prediction Error), data from animals receiving cocaine versus saline during the test session were separately analyzed using two-way ANOVAs (Cue Condition × Time). Dopamine concentration in nM was corrected according to probe recovery rates and converted to percent of baseline for data analyses (overall baseline averaged from within-subject means of three baseline measurements). Locomotor activity during the test session was also analyzed using two-way repeated measures ANOVAs (Cue Condition × Time). Post hoc analyses (Fishers LSD) were used to detect specific group, treatment or time differences (e.g., at least p <0.05) when main and/or interaction effects were indicated by overall analyses.

3. Results

3.1 Operant Sessions: Certainty and Uncertainty Cue Training

Lever Responses

Three-factor repeated measures ANOVA (Training Condition × Treatment × Session Days) showed no significant overall Training, Treatment, or Training × Treatment interaction effects [F(2,104) =0.64; F(1,104)=0.11; F(2,104)=0.88, respectively, all n.s.]. However, significant Session Days [F(7,728)=12.42], Training Condition × Session Days [F(14,728)= 3.84; both p<0.0001] and Training Condition × Treatment × Session Days [F(14,728)=2.01; p=0.015] interactions were observed. Posthoc analyses showed that for Certainty trained animals, no significant differences between response rates during matched-order cocaine and saline self-administration sessions occurred until the last session of each, when responses for cocaine were significantly higher than for saline (see Fig 2A). Animals trained under Uncertainty 1 and 2 conditions elicited comparable response rates for cocaine and saline throughout all sessions (see Figs 2B and 2C).

Fig. 2. Lever presses during cocaine- and non-reinforced operant sessions for Certainty (A), Uncertainty 1 (B), and Uncertainty 2 (C) groups.

Fig. 2

Open in a new tab

A. Animals trained under ‘Certainty’ conditions (n=29) had distinct sets of olfactory/visual cues paired consistently with cocaine- and non-reinforced operant sessions. These animals showed a gradual increase in cocaine lever responses as sessions progressed. Lever responses for cocaine on Day 15 (last cocaine session) were significantly greater (* = p<0.05) compared to non-reinforced (saline) responses on Day 16. B. In the Uncertainty 1 group (n=11), cue sets were equally paired with both cocaine- and non-reinforcement and were frequently interchanged over the course of training sessions. C. For Uncertainty 2 animals (n=15), cue pairings with either cocaine- or non-reinforced operant sessions were consistent for the first 8 trials, but were switched for the last 8 sessions.

Locomotor Activity

A three-factor ANOVA (Training Condition × Treatment × Session Days) showed significant Training Condition [F(2,104)=7.83; p=0.0007], Treatment [F(1,104)=117.01; p<0.0001], Session Days [F(7,728)=50.01; p<0.0001], and significant interaction effects [Training Condition × Session Days, F(14,728)=2.99; p=0.0002; Treatment × Session Days, F(7,728)=23.62; p<0.0001; Training Condition × Treatment × Session Days, F(14,728)=2.05, p=0.013]. Posthoc tests revealed that cocaine- and non-stimulated locomotor activity were significantly higher in the Uncertainty 1 compared to most Certainty trained operant sessions and some Uncertainty 2 sessions. Activity levels in the Uncertainty 2 groups were also higher than Certainty groups during a few cocaine and saline sessions (see Fig 3).

Fig. 3. Locomotor activity totals during cocaine and non-reinforced operant sessions.

Fig. 3

Open in a new tab

Photobeam breakages between sets of photocells within the operant chamber were assessed as locomotor activity units. On several occasions, locomotor activity during Uncertainty 1 training was significantly higher during both cocaine- and non-reinforced trials than matched sessions for the other training conditions (*, ** = significantly different than Certainty trained animals at p<0.05, 0.01; +, ++ = significantly different than animals trained under Uncertainty 2 conditions).

3.2 Test Session

A one-way ANOVA on response latency (elapsed time between cue presentation and lever press) at the start of the Test Session did not significantly vary between the differently-cued groups [F (2,51)= 0.73, n.s.; mean latency (+/− SEM) = 63.7 (+/− 9.8) sec]. In addition, NAcc DA mean baseline nM concentrations did not significantly differ between groups receiving cocaine at test [F(2,22)= 1.86, n.s.], or groups receiving saline at test [F(2,26)= 1.15, n.s.], thus conversion of data to baseline percentages did not obscure between-group comparisons. Uncertainty 1 and 2 groups showed no significant differences in NAcc DA response to cocaine [F(1,11)=0.036; n.s.] or saline [F(1,11)=2.24; n.s.] or locomotor response to cocaine [F(1,11)=0.47; n.s.] or saline [F(1,11)=0.85, n.s.] during the test session. Therefore, data were combined into a single Uncertainty group for comparisons with the Certainty and Prediction Error cue conditions in NAcc DA and locomotor activity levels. (Dialysis data from 1 animal trained under Uncertainty 2 conditions could not be used due to dialysate collection errors during the test session).

NAcc DA: Cocaine Effects

A two-factor repeated measures ANOVA (Cue Condition × Time) showed significant Cue Condition [F(2,22)=3.85; p=0.04], Time [F(5,110)=68.37; p<0.001] and Cue Condition × Time interaction effects [F(10,110)=3.99; p=0.0001]. Post hoc tests reveal that cocaine-stimulated NAcc DA responses were significantly greater in the Uncertainty group for the first two test intervals (e.g., 10 and 20 min post-injection) compared to both Certainty and Positive Prediction Error groups (see Fig 4A). Cocaine-stimulated NAcc DA responses in the Certainty and Positive Prediction Error conditions were comparable.

Fig. 4. A–B: NAcc DA before and after self-administered cocaine (A) and saline (B) in the presence of cues associated with Certainty, Uncertainty and Prediction Error.

Fig. 4

Open in a new tab

Timeline represents DA levels (% of baseline mean ± SEM) at 10 min intervals, from 30 min before to 30 min after a single operant response (e.g., cocaine 1.5 mg/kg or saline 0.1 ml). A. Following cocaine self-administration (1.5 mg/kg), all three groups showed significantly greater enhancement in NAcc DA compared to baseline following cocaine self-administration. In addition, NAcc DA was significantly greater in the presence of Uncertainty cues compared to Certainty and Positive Prediction Error cues (** p<0.01; * p<0.05). No significant differences in baseline DA nM levels were detected between Certainty (n=6), Uncertainty (n=13) and Positive Prediction Error (n=6) groups [F(2, 22)=1.86, n.s.]. B. There was a significant decrease from baseline in the first tested interval (10 min) following a non-reinforced operant response in the Negative Prediction Error compared to the Certainty cued group (** = p<0.01). No other group differences were observed during the test session. No significant differences in baseline DA nM levels were detected between Certainty (n=8), Uncertainty (n=13) and Negative Prediction Error (n=8) groups [F(2,26)=1.15, n.s.].

NAcc DA: Non-Reinforcement (Saline) Effects

A two-factor repeated measures ANOVA (Cue Condition × Time) showed no overall significant Cue Condition or Interaction effects [F(2,26)=0.55; and F(10,130)=1.65; both n.s.], but significant effects of Time [F(5,130)=5.36; p=0.0002]. Post hoc analyses revealed that after cue presentation, NAcc DA levels were significantly decreased in the Negative Prediction Error condition compared to all baseline measurements and compared to the Certainty condition. Certainty and Uncertainty conditions showed gradual, yet comparable reductions in DA levels across the testing interval (see Fig 4B).

Locomotor Activity: Cocaine Effects

A two-factor repeated measures ANOVA (Cue Condition × Time) showed no significant Cue Condition or Interaction effects [F(2,21)=0.43; F(10,105)=0.87, respectively, both n.s.], but significant effects of Time [F(5,105)=22.59; P<0.0001]. Post hoc tests revealed significantly increased locomotor activity immediately following the self-administered cocaine injection for all groups. There were no significant between-group differences in locomotor response to cocaine (see Fig 5A).

Fig. 5. A–B: Locomotor activity before and after self-administered cocaine (A) and saline (B) in the presence of cues associated with Certainty, Uncertainty and Prediction Error.

Fig. 5

Open in a new tab

Combined locomotor data were assessed concurrently with in vivo microdialysis data (e.g., same animals as depicted in Fig 4A–B). Data points represent photobeam breakage means (± SEM) A. Cocaine (1.5 mg/kg) resulted in locomotor activity levels significantly greater than baseline in the presence of Certainty, Uncertainty and Positive Prediction Error cues, but there were no significant differences between groups. (Activity data for 1 animal in Uncertainty group lost due to computer error). B. Cues did not differentially alter activity levels after a non-reinforced lever press. All groups showed significantly diminished activity levels at the last test interval compared to the first baseline measure. (Activity data for 1 animal in Certainty group and 4 animals in Uncertainty group lost due to computer error).

Locomotor Activity: Non-reinforcement (Saline) Effects

A two-factor repeated measures ANOVA (Cue Condition × Time) showed no significant Cue Condition effects [F(2,21)=0.112; n.s.], but significant Time and Interaction effects [F(5,105)=12.02; P<0.0001; F(10, 105)=2.3; p=0.018]. Post hoc tests revealed that after the non-reinforced lever response, all groups showed significantly decreased locomotor activity compared to baseline levels (see Fig 5B).

4. Discussion

For the present study, it was hypothesized that specific cue-associative training procedures would produce expectations of ‘certainty’ or ‘uncertainty’ of impending cocaine and subsequently influence behavioral and neurochemical responses to cocaine and non-reward. Indeed, different types of cue training had distinct effects on operant responding, locomotor activity and NAcc DA responses. For example, even though all experimental groups underwent alternating days of cocaine and non-reinforced operant sessions during the course of cue training, groups trained under ‘Certainty’ cue conditions (e.g., consistent cues paired with cocaine and non-rewarded operant sessions) showed gradual progression towards preferential cocaine self-administration, but ‘Uncertainty’-trained animals (e.g., all cues equally associated with both cocaine and non-reinforcement) did not. Also, locomotor activity during Uncertainty training sessions was significantly higher during both cocaine- and non-reinforced sessions during Certainty training. After completion of all cue-training sessions, Certainty animals self-administering saline showed a significant decrease in basal NAcc DA levels when cocaine-associated cues were present (Negative Prediction Error condition), but no DA changes occurred when cues signaling “no reward” were present. On the other hand, cocaine-stimulated NAcc DA was of significantly greater magnitude in Uncertainty animals, compared to Certainty groups in the presence of either paired cue.

4.1 Behavioral Effects of Cues during Associative Training

Patterns of lever responding observed during the different cue-training procedures of the present study suggest that cues paired with operant sessions can signal impending reinforcement and influence learning of stimulus/response outcomes. For example, as can be seen in Fig 2A, the Certainty groups showed a gradual increase in cocaine responses and a decrease in non-reinforced responses over the course of the training sessions, indicating the development of discriminative learning. Though the same alternating schedule of cocaine and saline self-administration sessions was used during both types of Uncertainty training (Figs 2B and 2C; Uncertainty 1 and Uncertainty 2), the number of cocaine- and non-reinforced responses remained comparable between all matched sessions, with no discernable trend towards discriminative learning by the end of the conditioning sessions. In addition, response patterns across sessions differed between the Uncertainty 1 and 2 groups, conceivably reflecting different schedules of cue pairings between the groups. As previously noted (see Training Assignments), in the Uncertainty 1 group, the cues associated with cocaine or non-rewarded sessions were switched every 2 days, while the Uncertainty 2 group had consistent cue-reinforcement pairings for the first 8 sessions, but were switched for the last 8 sessions. The resulting patterns of responding suggest that paired cues influence performance variables during discriminative operant learning.

It is reasonable to assert that discriminative learning between cocaine and non-reinforced operant sessions was particularly difficult to achieve in the current study that utilized alternating days of cocaine and saline availability. During initial non-reinforced sessions in the Certainty group, higher response rates likely reflected extinction responding due to the previous day’s cocaine availability. Notably, preferential responding for cocaine in the Certainty training group was observed only between the last matched sessions (Day 15 versus Day 16), and was not a robust finding. However, previous work using a similar cue-training paradigm with alternating cocaine/saline sessions reported that, as the number of training sessions increase (e.g., beyond the timecourse of the present study), response rates for cocaine gradually and significantly increase over saline level responding (Ikegami et al. 2007). Overall, the present data suggest that consistent cued associations can assist in the development of cocaine discriminative learning, while inconsistently paired cues can interfere with this process.

The effects of cue training procedures were also reflected in locomotor activity levels. Fig 3 illustrates that animals trained with the Uncertainty 1 procedure (which included the most frequent interchanges between cue/reinforcer pairings) showed the highest level of locomotor activity during both cocaine- and non-reinforced operant sessions. Though cue-associated activity enhancement has previously been shown in cocaine-paired environments in the presence (Duvauchelle et al. 2000) and absence of cocaine (Brown and Fibiger 1992; Fontana et al. 1993; Bell et al. 1997; Duvauchelle et al. 2000), no previous work has specifically reported effects of cocaine reward uncertainty on locomotor activation. One interpretation of these findings is that anticipatory excitation produced by uncertain signaling of cocaine reward exceeds that of cues predicting reliable cocaine delivery. Since increased locomotor activity corresponds with increased NAcc and striatal DA levels (D'Souza and Duvauchelle 2006), this line of reasoning is consistent with studies showing maximal uncertainty accompanied by the highest rate of midbrain DA neuronal firing (Fiorillo et al. 2003). Another possibility is that the Certainty animals are expressing conditioned tolerance to the hyperlocomotor effects of cocaine. For example, drug tolerance can develop to self-administered drugs more readily than to experimenter-administered drugs (Dworkin et al. 1995) and cocaine tolerance effects have been reported during self-administration procedures (Emmett-Oglesby et al. 1993; Li et al. 1994). In addition, drug-conditioned cues can evoke the expression of drug tolerance (Siegel 1977). Therefore, because Certainty-trained animals receive consistent cue/drug training, they may develop conditioned tolerance to cocaine’s stimulatory effects at a faster rate than Uncertainty animals. Therefore, differences in cocaine-induced activity levels may be the result of increased tolerance to the locomotor-activating effects of cocaine in Certainty rats rather than the enhancement of activity in the Uncertainty groups.

Certainty groups also exhibited lower activity levels than both Uncertainty groups during non-reinforced operant sessions. In this circumstance, it is possible that cue-induced “certain non-reinforcement” during these sessions served to suppress activity levels in the Certainty group. Though the effects of conditioned non-reinforcement have not been thoroughly studied, this interpretation is consistent with behavioral learning theory (Rescorla and Wagner 1972; Rescorla 2006), which would predict decremental effects of non-reinforcement on conditioned excitation. In addition, cues predicting no-reward inhibit midbrain DA neuronal activity (Matsumoto and Hikosaka 2007), and may also contribute to decreased motoric output.

4.2 Cue-Training Effects on Baseline and Cocaine-Stimulated NAcc DA Levels

After cue-training sessions were completed, animals in the Uncertainty Test condition showed the greatest increase in cocaine-stimulated NAcc DA, while NAcc DA responses in the presence of cues associated with cocaine (Certainty) and cues associated with non-reinforcement (Positive Prediction Error) were comparable (see Fig 4A). In addition, a non-reinforced lever response was followed by a significant depression in baseline NAcc DA levels in the presence of cocaine-associated cues (Negative Prediction Error) compared to the Certainty and Uncertainty groups (see Fig 4B)

These findings are the first to show the influence of uncertainty and prediction errors regarding cocaine reward on mesolimbic dopamine levels. These results are theoretically consistent with various single unit opamine neuronal firing studies. For instance, similar to our findings in the cocaine-receiving Uncertainty condition, maximal neuronal DA activation occurs in the presence of maximum uncertainty of natural reward (Fiorillo et al. 2003). Though animals presented with Uncertainty cues in the absence of cocaine would also be expected to show increased neuronal bursting, this effect would not be expected as detectable using the microdialysis procedure (Floresco et al. 2003). However, in the cocaine-reinforced group, a speculative view could assert that uncertainty-evoked neuronal bursting in addition to the presence of cocaine-enhanced DA may have increased DA levels to the point of detection through microdialysis.

Our dialysis data also corresponded to findings of DA firing depression in response to an unexpected loss of reward (e.g., a negative prediction error) (Hollerman and Schultz 1998; Tobler et al. 2003), when we observed a significant decrease in basal NAcc DA levels when cocaine cues were followed by a non-reinforced operant response. In addition, the current findings of suppressed cocaine-stimulated NAcc DA response in the presence of non-reward associated cues (e.g., the Positive Prediction Error condition) are consistent with single unit DA findings showing decreased DA neuronal firing when cues predicting no-reward are present (Matsumoto and Hikosaka 2007) Since the no-reward cue signals a distinct outcome, this scenario should not be confused with “unexpected reward”, in which DA neuronal firing increases when reinforcement occurs at a moment of no expectations of impending reward (Ljungberg et al. 1992). Thus, the present results, and those of Matsumoto and Hikosaka, are at odds with the idea that a "better than expected" outcome is always marked by increased dopamine or neural activity (Montague et al. 1996).

Our findings are also consistent in theory with fMRI clinical work using natural and monetary rewards. Human fMRI studies of uncertainty show preferential activation of the NAcc region with uncertainty regarding juice reward, when larger uncertain rewards were chosen over smaller certain rewards (Matthews et al. 2004) and when the probability of monetary reward was 50% (Dreher et al. 2006). Similarly, fMRI studies also report a depression in NAcc activity following omission of expected monetary reward (Knutson et al. 2001; Abler et al. 2005). Yet, it should be noted that these different procedures likely reflect functionally distinct aspects of the dopamine system in response to conditioned associations. Indeed, direct comparisons of the current findings with existing literature of uncertainty and prediction error are not possible due to obvious differences between cocaine reinforcement and natural reward and disparities in time resolution for in vivo microdialysis versus electrophysiological and imaging data.

In fact, consistent with the previous suggestion of tolerance to locomotor activation during conditioning trials, the findings noted above could also be attributed to differences in the development of conditioned tolerance between the differently cued groups. For instance, as demonstrated with opiate drugs (Hinson and Siegel 1983) and alcohol (Siegel 1987; Larson and Siegel 1998), cues consistently paired with drugs elicit compensatory responses that attenuate drug effects, thus contributing to tolerance. In addition, decreasing the frequency with which cues result in drug reinforcement (Siegel 1991) can interfere with the development of conditioned tolerance. Correspondingly, in the present study, distinctive cue sets were consistently paired with either cocaine or non-reinforcement (saline) during Certainty training, but during Uncertainty training, all cues were equally paired with cocaine and non-reinforcement. Therefore, it could be asserted that 1) conditioned tolerance resulted in the attenuated cocaine-induced NAcc DA response observed in Certainty trained animals, and 2) the significantly greater NAcc DA response to cocaine in the Uncertainty group could be considered a direct demonstration of hindered tolerance development. Moreover, our observations of a significant depression in basal NAcc DA levels when saline was self-administered in the presence of cocaine-paired cues (Negative Prediction Error; Fig 4B) might further support that notion. For instance, as previously discussed by Siegel (Siegel and Ramos 2002), in the absence of drugs, many withdrawal symptoms are simply conditioned compensatory responses to drug effects. In the current study, it is conceivable that all presentations of cocaine-associated cues decreased NAcc DA levels in Certainty trained animals. Hence, in the absence of cocaine, the effect was revealed as below basal DA levels in the Negative Prediction Error group, while in the presence of cocaine, the NAcc DA response to cocaine was attenuated (e.g., Certainty/Cocaine), as referred to above.

As previously stated, NAcc DA responses to cues and cocaine in the Certainty and Positive Prediction Error groups were not significantly different in the current study. Yet, in previous work with similar groups and a shorter conditioning period (Ikegami et al. 2007), significant group differences were observed. The study also revealed that the magnitude of behavioral and NAcc DA responses to cocaine and saline self-administration were experience-dependent. Therefore, diverging results between past and present work are likely to be due to differences in training duration and other procedures, such as cocaine test dosages. Nevertheless, the present findings confirm effects of cocaine and cue conditioning and reveal that, in the presence or absence of cocaine reinforcement, cue-induced certainty, uncertainty and prediction error can have a critical impact on NAcc DA.

4.3 Cue-Training Effects on Baseline and Cocaine-Stimulated Activity Levels

There was a significant increase in locomotor activity following cocaine self-administration in all the three groups (see Fig 5A), consistent with previous work showing the psychomotor activating effects of cocaine (Wise and Bozarth 1987). However, though NAcc DA plays an important role in cocaine-induced hyperlocomotor activity (Di Chiara and Imperato 1988; Pettit et al. 1990), our present findings show that increased locomotor activity does not completely correspond with NAcc DA levels. For example, in the presence of Uncertainty cues, the NAcc DA response to cocaine self-administration was significantly greater than the other cued groups, yet locomotor activity was comparable between all cued groups. Thus, our findings indicate cue-associated dissociations between NAcc DA levels and locomotor activity. Other studies have also reported asynchrony between NAcc DA and locomotor activity, especially after repeated cocaine administration (Kuczenski et al. 1991; Segal and Kuczenski 1992; Ikegami et al. 2007). It has been suggested that other regions such as dorsal striatum, VTA, ventral pallidum and olfactory tubercle also play a critical role in mediating cocaine-induced locomotor effects (Mayfield et al. 1992; Gong et al. 1996; Borgland et al. 2004; Chambers et al. 2004). Therefore, it is conceivable that pathways influencing cocaine-induced locomotor activity may differ from those supporting conditioned responses developed under the influence of cocaine.

4.4 Conclusions

Our findings show that behavioral and neurochemical responses to cocaine are not merely pharmacological responses, but are also influenced by drug-associative learning experiences. In the present study, it is likely that associative conditioning produced cued expectations, such as certainty and uncertainty of impending cocaine intake. However, whether the opposing cued expectations enhanced drug-stimulated behaviors and DA levels or produced physiological responses to decrease anticipated drug effects (e.g., drug tolerance) are yet to be determined. Another possibility is that cocaine administered under conditions of uncertainty is aversive and (or) induces a stress-mediated enhancement of NAcc DA (Imperato et al. 1991; Doherty and Gratton 1997; Weiss et al. 1997) independent of expectations of reinforcement. Future work to disentangle these issues will help to determine implications of the conditioned effects on clinical applications and how past drug experiences can influence treatment scenarios.

Acknowledgements

This project was supported by NIH grant DA14640 (C.L.D.), and a University of Texas Waggoner Center for Alcohol and Addiction Research Bruce-Jones Graduate Fellowship (M.S.D.).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Abler B, Walter H, Erk S. Neural correlates of frustration. Neuroreport. 2005;16:669–672. doi: 10.1097/00001756-200505120-00003. [DOI] [PubMed] [Google Scholar]
  2. Bell SM, Stewart RB, Thompson SC, Meisch RA. Food-deprivation increases cocaine-induced conditioned place preference and locomotor activity in rats. Psychopharmacology. 1997;131:1–8. doi: 10.1007/s002130050258. [DOI] [PubMed] [Google Scholar]
  3. Berns GS, McClure SM, Pagnoni G, Montague PR. Predictability modulates human brain response to reward. J Neurosci. 2001;21:2793–2798. doi: 10.1523/JNEUROSCI.21-08-02793.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Borgland SL, Malenka RC, Bonci A. Acute and chronic cocaine-induced potentiation of synaptic strength in the ventral tegmental area: Electrophysiological and behavioral correlates in individual rats. J Neurosci. 2004;24:7482–7490. doi: 10.1523/JNEUROSCI.1312-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Brown EE, Fibiger HC. Cocaine-induced conditioned locomotion: Absence of associated increases in dopamine release. Neuroscience. 1992;48:621–629. doi: 10.1016/0306-4522(92)90406-r. [DOI] [PubMed] [Google Scholar]
  6. Carelli RM, Ijames SG. Selective activation of accumbens neurons by cocaine-associated stimuli during a water/cocaine multiple schedule. Brain Res. 2001;907:156–161. doi: 10.1016/s0006-8993(01)02604-x. [DOI] [PubMed] [Google Scholar]
  7. Chambers RA, Sheehan T, Taylor JR. Locomotor sensitization to cocaine in rats with olfactory bulbectomy. Synapse. 2004;52:167–175. doi: 10.1002/syn.20017. [DOI] [PubMed] [Google Scholar]
  8. Childress AR, Mozley PD, McElgin W, Fitzgerald J, Reivich M, O'Brien CP. Limbic activation during cue-induced cocaine craving. Am J Psychiatry. 1999;156:11–18. doi: 10.1176/ajp.156.1.11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Ciccocioppo R, Martin-Fardon R, Weiss F. Stimuli associated with a single cocaine experience elicit long-lasting cocaine-seeking. Nat Neurosci. 2004;7:495–496. doi: 10.1038/nn1219. [DOI] [PubMed] [Google Scholar]
  10. D'Souza MS, Duvauchelle CL. Comparing nucleus accumbens and dorsal striatal dopamine responses to self-administered cocaine in naive rats. Neurosci Lett. 2006;408:146–150. doi: 10.1016/j.neulet.2006.08.076. [DOI] [PubMed] [Google Scholar]
  11. Di Chiara G. Dopamine in disturbances of food and drug motivated behavior: A case of homology? Physiol Behav. 2005;86:9–10. doi: 10.1016/j.physbeh.2005.06.020. [DOI] [PubMed] [Google Scholar]
  12. Di Chiara G, Imperato A. Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proc Natl Acad Sci U S A. 1988;85:5274–5278. doi: 10.1073/pnas.85.14.5274. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Di Chiara G, Tanda G, Bassareo V, Pontieri F, Acquas E, Fenu S, Cadoni C, Carboni E. Drug addiction as a disorder of associative learning. Role of nucleus accumbens shell/extended amygdala dopamine. Ann NY Acad Sci. 1999;877:461–485. doi: 10.1111/j.1749-6632.1999.tb09283.x. [DOI] [PubMed] [Google Scholar]
  14. Di Ciano P, Blaha CD, Phillips AG. Conditioned changes in dopamine oxidation currents in the nucleus accumbens of rats by stimuli paired with self-administration or yoked-administration of d-amphetamine. Eur J Neurosci. 1998;10:1121–1127. doi: 10.1046/j.1460-9568.1998.00125.x. [DOI] [PubMed] [Google Scholar]
  15. Doherty MD, Gratton A. Nmda receptors in nucleus accumbens modulate stress-induced dopamine release in nucleus accumbens and ventral tegmental area. Synapse. 1997;26:225–234. doi: 10.1002/(SICI)1098-2396(199707)26:3<225::AID-SYN4>3.0.CO;2-9. [DOI] [PubMed] [Google Scholar]
  16. Dreher JC, Kohn P, Berman KF. Neural coding of distinct statistical properties of reward information in humans. Cereb Cortex. 2006;16:561–573. doi: 10.1093/cercor/bhj004. [DOI] [PubMed] [Google Scholar]
  17. Duvauchelle CL, Ikegami A, Asami S, Robens J, Kressin K, Castaneda E. Effects of cocaine context on nacc dopamine and behavioral activity after repeated intravenous cocaine administration. Brain Res. 2000;862:49–58. doi: 10.1016/s0006-8993(00)02091-6. [DOI] [PubMed] [Google Scholar]
  18. Duvauchelle CL, Ikegami A, Castaneda E. Conditioned increases in behavioral activity and accumbens dopamine levels produced by intravenous cocaine. Behav Neurosci. 2000;114:1156–1166. doi: 10.1037//0735-7044.114.6.1156. [DOI] [PubMed] [Google Scholar]
  19. Dworkin SI, Mirkis S, Smith JE. Response-dependent versus response-independent presentation of cocaine: Differences in the lethal effects of the drug. Psychopharmacology (Berl) 1995;117:262–266. doi: 10.1007/BF02246100. [DOI] [PubMed] [Google Scholar]
  20. Ehrman RN, Robbins SJ, Childress AR, O'Brien CP. Conditioned responses to cocaine-related stimuli in cocaine abuse patients. Psychopharmacology (Berl) 1992;107:523–529. doi: 10.1007/BF02245266. [DOI] [PubMed] [Google Scholar]
  21. Emmett-Oglesby MW, Peltier RL, Depoortere RY, Pickering CL, Hooper ML, Gong YH, Lane JD. Tolerance to self-administration of cocaine in rats: Time course and dose-response determination using a multi-dose method. Drug Alcohol Depend. 1993;32:247–256. doi: 10.1016/0376-8716(93)90089-9. [DOI] [PubMed] [Google Scholar]
  22. Fallon JH, Moore RY. Catecholamine innervation of the basal forebrain. Iv. Topography of the dopamine projection to the basal forebrain and neostriatum. J Comp Neurol. 1978;180:545–580. doi: 10.1002/cne.901800310. [DOI] [PubMed] [Google Scholar]
  23. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  24. Floresco SB, West AR, Ash B, Moore H, Grace AA. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nature Neuroscience. 2003;6:968–973. doi: 10.1038/nn1103. see comment. [DOI] [PubMed] [Google Scholar]
  25. Foltin RW, Haney M. Conditioned effects of environmental stimuli paired with smoked cocaine in humans. Psychopharmacology (Berl) 2000;149:24–33. doi: 10.1007/s002139900340. [DOI] [PubMed] [Google Scholar]
  26. Fontana DJ, Post RM, Pert A. Conditioned increases in mesolimbic dopamine overflow by stimuli associated with cocaine. Brain Res. 1993;629:31–39. doi: 10.1016/0006-8993(93)90477-5. [DOI] [PubMed] [Google Scholar]
  27. Gawin FH. Cocaine addiction: Psychology and neurophysiology. Science. 1991;251:1580–1586. doi: 10.1126/science.2011738. [DOI] [PubMed] [Google Scholar]
  28. Ghitza UE, Fabbricatore AT, Prokopenko V, Pawlak AP, West MO. Persistent cue-evoked activity of accumbens neurons after prolonged abstinence from self-administered cocaine. J Neurosci. 2003;23:7239–7245. doi: 10.1523/JNEUROSCI.23-19-07239.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Gong W, Neill D, Justice JB., Jr Conditioned place preference and locomotor activation produced by injection of psychostimulants into ventral pallidum. Brain Res. 1996;707:64–74. doi: 10.1016/0006-8993(95)01222-2. [DOI] [PubMed] [Google Scholar]
  30. Hinson RE, Siegel S. Anticipatory hyperexcitability and tolerance to the narcotizing effect of morphine in the rat. Behav Neurosci. 1983;97:759–767. doi: 10.1037//0735-7044.97.5.759. [DOI] [PubMed] [Google Scholar]
  31. Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci. 1998;1:304–309. doi: 10.1038/1124. [DOI] [PubMed] [Google Scholar]
  32. Ikegami A, Olsen CM, D'Souza MS, Duvauchelle CL. Experience-dependent effects of cocaine self-administration/conditioning on prefrontal and accumbens dopamine responses. Behav Neurosci. 2007;121:389–400. doi: 10.1037/0735-7044.121.2.389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Imperato A, Puglisi-Allegra S, Casolini P, Angelucci L. Changes in brain dopamine and acetylcholine release during and following stress are independent of the pituitary-adrenocortical axis. Brain Res. 1991 Jan 4;538:111–117. doi: 10.1016/0006-8993(91)90384-8. [DOI] [PubMed] [Google Scholar]
  34. Ito R, Dalley JW, Robbins TW, Everitt BJ. Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J Neurosci. 2002;22:6247–6253. doi: 10.1523/JNEUROSCI.22-14-06247.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kamin LJ. Selective association and conditioning. Fundamental issues in instrumental learning. WKH N.J. Mackintosh. Halifax, Can, Dalhousie University Press; 1969. pp. 42–64. [Google Scholar]
  36. Kelley AE, Berridge KC. The neuroscience of natural rewards: Relevance to addictive drugs. J Neurosci. 2002;22:3306–3311. doi: 10.1523/JNEUROSCI.22-09-03306.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kilts CD, Gross RE, Ely TD, Drexler KP. The neural correlates of cue-induced craving in cocaine-dependent women. Am J Psychiatry. 2004;161:233–241. doi: 10.1176/appi.ajp.161.2.233. [DOI] [PubMed] [Google Scholar]
  38. Kiyatkin EA, Stein EA. Conditioned changes in nucleus accumbens dopamine signal established by intravenous cocaine in rats. Neurosci Lett. 1996;211:73–76. doi: 10.1016/0304-3940(96)12731-2. [DOI] [PubMed] [Google Scholar]
  39. Knutson B, Adams CM, Fong GW, Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci. 2001;21:RC159. doi: 10.1523/JNEUROSCI.21-16-j0002.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Knutson B, Fong GW, Adams CM, Varner JL, Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport. 2001;12:3683–3687. doi: 10.1097/00001756-200112040-00016. [DOI] [PubMed] [Google Scholar]
  41. Kuczenski R, Segal DS, Aizenstein ML. Amphetamine, cocaine, and fencamfamine: Relationship between locomotor and stereotypy response profiles and caudate and accumbens dopamine dynamics. J Neurosci. 1991;11:2703–2712. doi: 10.1523/JNEUROSCI.11-09-02703.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Larson SJ, Siegel S. Learning and tolerance to the ataxic effect of ethanol. Pharmacol Biochem Behav. 1998;61:131–142. doi: 10.1016/s0091-3057(98)00072-0. [DOI] [PubMed] [Google Scholar]
  43. Li DH, Depoortere RY, Emmett-Oglesby MW. Tolerance to the reinforcing effects of cocaine in a progressive ratio paradigm. Psychopharmacology (Berl) 1994;116:326–332. doi: 10.1007/BF02245336. [DOI] [PubMed] [Google Scholar]
  44. Li Y, Kolb B, Robinson TE. The location of persistent amphetamine-induced changes in the density of dendritic spines on medium spiny neurons in the nucleus accumbens and caudate-putamen. Neuropsychopharmacology. 2003;28:1082–1085. doi: 10.1038/sj.npp.1300115. [DOI] [PubMed] [Google Scholar]
  45. Ljungberg T, Apicella P, Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol. 1992;67:145–163. doi: 10.1152/jn.1992.67.1.145. [DOI] [PubMed] [Google Scholar]
  46. Matsumoto M, Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature. 2007;447:1111–1115. doi: 10.1038/nature05860. [DOI] [PubMed] [Google Scholar]
  47. Matthews SC, Simmons AN, Lane SD, Paulus MP. Selective activation of the nucleus accumbens during risk-taking decision making. Neuroreport. 2004;15:2123–2127. doi: 10.1097/00001756-200409150-00025. [DOI] [PubMed] [Google Scholar]
  48. Mayfield RD, Larson G, Zahniser NR. Cocaine-induced behavioral sensitization and d1 dopamine receptor function in rat nucleus accumbens and striatum. Brain Res. 1992;573:331–335. doi: 10.1016/0006-8993(92)90783-6. [DOI] [PubMed] [Google Scholar]
  49. Mirenowicz J, Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol. 1994;72:1024–1027. doi: 10.1152/jn.1994.72.2.1024. [DOI] [PubMed] [Google Scholar]
  50. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci. 1996;16:1936–1947. doi: 10.1523/JNEUROSCI.16-05-01936.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Nicola SM, Deadwyler SA. Firing rate of nucleus accumbens neurons is dopamine-dependent and reflects the timing of cocaine-seeking behavior in rats on a progressive ratio schedule of reinforcement. J Neurosci. 2000;20:5526–5537. doi: 10.1523/JNEUROSCI.20-14-05526.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Paxinos G, Watson C. The rat brain in stereotaxic coordinates. Academic Press. 1998 doi: 10.1016/0165-0270(80)90021-7. [DOI] [PubMed] [Google Scholar]
  53. Pearce JM, Hall G. A model for pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev. 1980;87:532–552. [PubMed] [Google Scholar]
  54. Pettit HO, Pan HT, Parsons LH, Justice JB., Jr Extracellular concentrations of cocaine and dopamine are enhanced during chronic cocaine administration. J Neurochem. 1990;55:798–804. doi: 10.1111/j.1471-4159.1990.tb04562.x. [DOI] [PubMed] [Google Scholar]
  55. Phillips PE, Stuber GD, Heien ML, Wightman RM, Carelli RM. Subsecond dopamine release promotes cocaine seeking. Nature. 2003;422:614–618. doi: 10.1038/nature01476. [DOI] [PubMed] [Google Scholar]
  56. Rescorla RA. Deepened extinction from compound stimulus presentation. J Exp Psychol Anim Behav Process. 2006;32:135–144. doi: 10.1037/0097-7403.32.2.135. [DOI] [PubMed] [Google Scholar]
  57. Rescorla RA, Wagner AR. Classical Conditioning ii: Current Research and Theory. New York, Appleton Century Crofts: AM Black and WF Prokasy; 1972. A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and non reinforcement; pp. 64–69. [Google Scholar]
  58. Robinson TE, Gorny G, Mitton E, Kolb B. Cocaine self-administration alters the morphology of dendrites and dendritic spines in the nucleus accumbens and neocortex. Synapse. 2001;39:257–266. doi: 10.1002/1098-2396(20010301)39:3<257::AID-SYN1007>3.0.CO;2-1. [DOI] [PubMed] [Google Scholar]
  59. Roitman MF, Stuber GD, Phillips PE, Wightman RM, Carelli RM. Dopamine operates as a subsecond modulator of food seeking. J Neurosci. 2004;24:1265–1271. doi: 10.1523/JNEUROSCI.3823-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Segal DS, Kuczenski R. Repeated cocaine administration induces behavioral sensitization and corresponding decreased extracellular dopamine responses in caudate and accumbens. Brain Res. 1992;577:351–355. doi: 10.1016/0006-8993(92)90297-m. [DOI] [PubMed] [Google Scholar]
  61. Siegel S. Morphine tolerance acquisition as an associative process. J Exp Psychol Anim Behav Process. 1977;3:1–13. doi: 10.1037//0097-7403.3.1.1. [DOI] [PubMed] [Google Scholar]
  62. Siegel S. Pavlovian conditioning and ethanol tolerance. Alcohol Alcohol Suppl 1. 1987:25–36. [PubMed] [Google Scholar]
  63. Siegel S. Tolerance: Role of conditioning processes. NIDA Res Monogr. 1991:213–229. [PubMed] [Google Scholar]
  64. Siegel S, Ramos BM. Applying laboratory research: Drug anticipation and the treatment of drug addiction. Exp Clin Psychopharmacol. 2002;10:162–183. doi: 10.1037//1064-1297.10.3.162. [DOI] [PubMed] [Google Scholar]
  65. Tobler PN, Dickinson A, Schultz W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci. 2003;23:10402–10410. doi: 10.1523/JNEUROSCI.23-32-10402.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Weiss F, Ciccocioppo R, Parsons LH, Katner S, Liu X, Zorrilla EP, Valdez GR, Ben-Shahar O, Angeletti S, Richter RR. Compulsive drug-seeking behavior and relapse. Neuroadaptation, stress, and conditioning factors. Ann N Y Acad Sci. 2001;937:1–26. doi: 10.1111/j.1749-6632.2001.tb03556.x. [DOI] [PubMed] [Google Scholar]
  67. Weiss F, Imperato A, Casu MA, Mascia MS, Gessa GL. Opposite effects of stress on dopamine release in the limbic system of drug-naive and chronically amphetamine-treated rats. 1997:219–222. doi: 10.1016/s0014-2999(97)01264-8. 1997 Oct 22. [DOI] [PubMed] [Google Scholar]
  68. Weiss F, Maldonado-Vlaar CS, Parsons LH, Kerr TM, Smith DL, Ben-Shahar O. Control of cocaine-seeking behavior by drug-associated stimuli in rats: Effects on recovery of extinguished operant-responding and extracellular dopamine levels in amygdala and nucleus accumbens. Proc Natl Acad Sci U S A. 2000;97:4321–4326. doi: 10.1073/pnas.97.8.4321. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Weiss F, Martin-Fardon R, Ciccocioppo R, Kerr TM, Smith DL, Ben-Shahar O. Enduring resistance to extinction of cocaine-seeking behavior induced by drug-related cues. Neuropsychopharmacology. 2001;25:361–372. doi: 10.1016/S0893-133X(01)00238-X. [DOI] [PubMed] [Google Scholar]
  70. Wise RA, Bozarth MA. A psychomotor stimulant theory of addiction. Psychol Rev. 1987;94:469–492. [PubMed] [Google Scholar]

RESOURCES