Abstract
Background
Addiction is characterized by an inability to stop using drugs, despite adverse consequences. One contributing factor to this compulsive drug taking could be the impact of drug use on the ability to extinguish drug seeking after changes in expected outcomes. Here we compared effects of cocaine, morphine, and heroin self-administration on two forms of extinction learning: standard extinction driven by reward omission and extinction driven by reward over-expectation.
Methods
In Experiment 1, we trained rats to self-administer cocaine, morphine, or sucrose for 3 hr/day (limited access). In Experiment 2, we trained rats to self-administer heroin or sucrose for 12 hr/day (extended access). Three weeks later, we trained the rats to associate several cues with palatable food reward, after which we assessed extinction of the learned Pavlovian response, first by pairing two cues together in the over-expectation procedure and later by omitting the food reward.
Results
Rats trained under limited access conditions to self-administer sucrose or morphine demonstrated normal extinction in response to both over-expectation and reward omission, whereas cocaine-experienced rats or rats trained to self-administer heroin under extended access conditions exhibited normal extinction in response to reward omission but failed to show extinction in response to over-expectation.
Conclusions
The specific long-lasting effects of cocaine and heroin show that drug exposure induces long-lasting deficits in the ability to extinguish reward seeking after changes in expected outcomes. These deficits were not observed in a standard extinction procedure but instead only affected extinction learning driven by a more complex phenomenon of over-expectation.
Keywords: addiction, cocaine, morphine, heroin, self-administration, extinction, rat, orbitofrontal
INTRODUCTION
Drug addiction is characterized by an inability to stop using drugs, often despite a reported lack of enjoyment of the drugs and a high probability of adverse consequences. Drug-seeking behavior is even resistant to extinction-based treatments specifically designed to extinguish the addict’s response to drug-associated cues (1, 2) that often provoke relapse during abstinence (3, 4).
According to theoretical accounts, extinction depends on new memories formed when outcomes predicted by cues and events in the environment fail to materialize (5, 6). The mismatch between expected and actual outcomes results in teaching signals - prediction errors - that are thought to drive this new learning, which then suppresses or modulates expression of the original learned behaviors (7). A critical part of this process is appropriate signaling of the expectations regarding likely outcomes. Failure to appropriately signal outcome expectancies would result in impaired extinction learning, since such a failure could weaken (or even reverse) the sign of the resultant error signal. Thus drug-induced dysfunction in the signaling of these outcome expectancies could result in impaired extinction learning in some settings. This hypothesis is supported by recent findings showing that exposure to cocaine causes long-lasting structural and functional changes in the orbitofrontal cortex (OFC), a region critical to signaling information about expected outcomes (9–15). Accordingly, exposure to psychostimulants, particularly cocaine, often impairs OFC-dependent behaviors (9, 10, 13, 16–22). The list of impaired behaviors includes Pavlovian over-expectation (23), a task in which learning depends directly upon signaling of outcome expectancies (24). We have found that these deficits are associated with disrupted signaling of outcome expectancies in OFC (23, 25).
To further address this hypothesis--the specificity of the extinction learning deficit, and whether effects on OFC function might be observed for drugs other than cocaine--we trained rats to self-administer cocaine, morphine, or heroin versus an oral sucrose solution. Three weeks after the end of this training, we tested these same rats in a Pavlovian over-expectation task (24). This task independently assesses extinction of conditioned responses driven by summation and over-expectation, an OFC-dependent learning process, versus that induced by simple reward omission, which we have found to be independent of OFC (26, 27). As previously reported (23), we found that rats trained to self-administer cocaine under limited access conditions were unable to extinguish a previously learned behavior in response to over-expectation but showed normal extinction of the same behavior in response to reward omission. Rats trained to self-administer heroin under extended access conditions showed a similar pattern of behavior. However, morphine-experienced rats with limited access showed no deficit in the ability to extinguish in response to over-expectation or omission of reward. These results have implications for understanding how drug exposure disrupts fundamental non-experiential mechanisms that normal subjects access to modulate behavior and learning.
MATERIALS AND METHODS
Subjects
Rats (Charles River Labs; Experiment 1: 43 Long-Evans; Experiment 2: 23 Sprague-Dawley) weighing 250–275 g upon arrival were housed individually and placed on a 12-hr light/dark schedule. During testing, rats were food deprived to 85% of their baseline weight. All testing followed the guidelines outlined in the Guide for the Care and Use of Laboratory Animals (8th edition; http://grants.nih.gov/grants/olaw/Guide-for-the-Care-and-Use-of-Laboratory-Animals.pdf).
Drugs
Cocaine hydrochloride, morphine sulfate and heroin (diacetylmorphine hydrochloride) (NIDA, Bethesda, MD) were dissolved in sterile 0.9% saline.
Surgery
Rats were anaesthetized with ketamine (100 mg/ kg, i.p., Sigma) and xylazine (10 mg/kg, i.p., Sigma) and catheters were implanted in the right jugular vein, passed subcutaneously to the top of the skull, and attached to a 22-gauge cannula (Plastics One) and mounted to the rat’s skull. Carprofen (0.1 mg/kg, s.c., Pfizer) was given after surgery as an analgesic (9). Catheters were flushed every day with sterile 0.9% saline + the antibiotic Gentamicin (0.08 mg/mL, BioSource International).
Self-Administration (SA)
In Experiment 1, rats self-administered cocaine (0.75 mg/kg/infusion; n= 16), morphine (1 mg/kg/infusion; n=15), or sucrose (10% w/v; n = 11), for 3 hr/day, for 14 consecutive days. In Experiment 2, rats self-administered heroin (0.1 mg/kg/infusion; n=13) or sucrose (10% w/v; n = 10), 12 hr/day, for 14 days. A fixed ratio 1 (FR1) schedule of reinforcement was used; presses on the active lever delivered a 4 (Experiment 1) or 3.5 (Experiment 2) s infusion of the drug. Each chamber was equipped with two levers, located 8–9 cm above the floor. Presses on the retractable (active) lever activated the infusion pump to deliver drug or sucrose; presses on the stationary (inactive) lever were not reinforced.
For cocaine and morphine SA in Experiment 1, sessions lasted 3 hr, with 15-min timeout periods after each hour. Each session began with the insertion of the active lever. Each infusion was accompanied by the retraction of the active lever, which was followed by a 40-sec timeout period. Pressing on the inactive lever had no programmed consequences. At the end of each session the active lever was retracted. The number of cocaine or morphine infusions was limited to 20/hr to prevent overdose. After 20 infusions, the active lever was retracted for the remainder of the hour. For heroin SA in Experiment 2, procedures were the same as those for cocaine or morphine SA, except that daily sessions lasted 12 hr, and the number of heroin infusions was limited to 100/day.
Over-expectation
Three weeks after the end of SA training, all rats underwent Pavlovian over-expectation training. Food reinforcers (45 mg sucrose pellets - plain, banana-flavored, or grape-flavored; Bio-Serv) were delivered to a food cup recessed in the center of one wall. White noise or a tone, each measuring approximately 76 dB, was delivered via a wall speaker. Also mounted on that wall were a clicker (2 Hz) and a 6-W bulb that could be illuminated to provide a light stimulus during the otherwise dark session. Procedures were identical to those described previously (26).
Training began with simple conditioning. Rats were shaped to retrieve food pellets and then they underwent 10 conditioning sessions. In each session, the rats received eight 30-sec presentations of three different auditory cues (A1, A2, and A3) and a visual cue (V1). Cues were presented in a blocked design (counterbalanced); V1 consisted of a cue light, and A1, A2 and A3 consisted of a tone, clicker, or white noise (counterbalanced). Two differently flavored sucrose pellets (banana (O1) and grape (O2), counterbalanced) were used as rewards. V1 and A1 terminated with delivery of three pellets of O1, and A2 terminated with delivery three pellets of O2. A3 was not paired with food.
After completion of conditioning, rats received four consecutive days of compound training in which A1 and V1 were presented together as a 30-s compound cue terminating with three pellets of O1, and V1, A2, and A3 continued to be presented as before. Cues were again presented in a blocked design (counterbalanced). For each cue, there were 12 trials on the first three days of compound conditioning and six trials on the last day of compound conditioning.
One day after the last compound training session, rats received an extinction probe test session consisting of eight non-reinforced presentations (extinction conditions) of the three auditory cues (A1, A2, A3), with the order mixed and counterbalanced.
Behavioral measures
The primary measure of conditioning to cues was the percentage of time that each rat spent with its head in the food cup during the 30-sec conditioned stimulus (CS) presentation, as indicated by disruption of the photocell beam. This is a standard measure of Pavlovian conditioned responding, routinely employed by us for this task (23, 26, 27, 30). Other measures of responding at the food cup, such as frequency of food cup entries, are generally very strongly correlated with this measure (31). In the current experiment, food cup entries were strongly correlated with percent time in food cup in all groups during the last conditioning day (r values = 0.31 to 0.38, p values < 0.05). We also measured the percentage of time that each rat showed rearing behavior during the 30-s CS period. This is a standard approach we have used previously to factor out and reduce variance within groups related to competing behaviors (23, 26, 27, 30); there was no significant difference in the amount of rearing behavior among all auditory cues or different groups during the last conditioning day (p's >0.05). To correct for time spent rearing, the percentage of responding during the 30-s CS was calculated as follows: % of responding = 100 * ((% of time in food cup) / (100 – (% of time of rearing)). For normalized percentage of responding during presentation of cues A1+V1 and A2 in compound training sessions, we calculated for each compound training session the corresponding normalized value using this formula: % of responding = 100 * ((% of responding to cue in compound training session) / (% of responding to corresponding auditory cue in the last day of conditioning session)), then we averaged these values.
Statistical analyses
Data were recorded by Coulbourn GS2 software and processed in Matlab. These data were analyzed by ANOVAs using STATISTICA software with Tukey HSD post-hoc testing when appropriate (p < 0.05).
RESULTS
Experiment 1
Self-administration training
Rats in all groups responded at high rates on the active lever compared to the inactive lever across sessions (Fig. 1). ANOVA’s (session × lever) showed significant lever by session interactions in cocaine and sucrose groups (cocaine: F13,195=3.5, p 0.05; sucrose: F13,130=2.2, p<0.05), and a significant main effect of session and lever in morphine group (session: morphine: F13,182=13.8, p 0.01; lever: morphine: F1,14=4.6, p 0.05). Separate ANOVA’s also showed significant effects of session on the number of reward delivery (cocaine: F13,195=56.0, p 0.01; sucrose: F13,130=1.9, p<0.05; morphine: F13,182=4.0, p<0.01).
Conditioning training in the over-expectation task
Rats in all groups progressively increased responding to A1 and A2 versus A3 across sessions (Fig. 2a). A three-factor ANOVA (group × cue × session) showed a significant interaction between cue and session (F18,1026=11.6, p<0.01); there was no significant main effect nor any interactions involving group (p's>0.2). Moreover, sucrose, cocaine and morphine rats learned to respond to A1 and A2 equally, as indicated by the lack of statistical effects of either cue or group in an analysis restricted to these two cues (p's>0.2).
Rats were also trained during the initial 10-d period to associate a visual cue (cue light, V1; data not shown) with three pellets of O1. Rats in the sucrose, cocaine and morphine groups also showed similar responding to V1 in all phases. ANOVA’s (group × session) showed a significant main effect of session during initial conditioning (F9,351=14.5, p<0.01), and there were neither significant main effects nor any interactions involving group (p's>0.1). Moreover, sucrose-, cocaine-, and morphine-trained rats learned to respond to A1, A2 and V1 in a similar way, as indicated by the lack of statistical effects of either cue or group in an analysis restricted to these three cues (p values > 0.05).
Compound training in the over-expectation task
Rats in all groups maintained elevated responding to A1+V1 and A2, compared to A3 (Fig. 2b). A three-factor ANOVA (group × cue × session) showed a significant main effect of cue (F2, 114=101.8, p<0.01). Although there were no effects involving group on the raw response rates (p's>0.1), when responding during the four compound training sessions was compared to that during the last conditioning session, there was a significant increase in responding to A1+V1 cue in the sucrose (F1, 10=9.6, p<0.05) and in the morphine groups (F1, 14=7.9, p<0.05), but not in the cocaine group (p=0.10). Notably this increase was specific to the A1+V1 compound cue; neither group showed any change in responding to A2 (p’s>0.2).
We also examined responding on the first trial of compound training, since theoretical accounts and our own previous work suggest that summation should be strongest on this trial (23, 27). Consistent with this notion, we found that both sucrose and morphine groups exhibited a spontaneous increase in responding to A1+V1 in their raw response rates on this trial (sucrose: F2,30=40.2, p<0.01 for all cues, post-hoc A1 vs A2: p=0.002; morphine: F2,42=67.2, p<0.01 for all cues, post-hoc A1 vs A2: p=0.03). This increase was not observed in the cocaine group (F2,45=25.2, p<0.01 for all cues, post-hoc A1 vs A2: p=0.69). Additionally, when response rates during compound training were normalized to the last day of training; there was a significant difference between A1+V1 and A2 in the sucrose (F1, 10=5.2, p<0.05), and morphine groups (F1, 14=4.9, p<0.05), but not in cocaine group (p=0.8) (Fig. 2b, right).
Probe test in the over-expectation task
Rats in all groups showed elevated responding to A1 and A2, compared to A3, and this responding extinguished across the session (Fig. 2c). Consistent with this impression, a 3-factor ANOVA (group × cue × trial) showed significant interaction between cue and trial (F14, 819=2.00, p<0.05). Notably, there were no significant interactions involving group and trial (p’s>0.2), indicating that extinction of responding as a result of reward omission in the probe test was unaffected by prior self-administration of either cocaine or morphine.
However, in addition to these effects, which were similar between groups, rats in the sucrose and morphine groups also showed less responding to the over-expected A1 cue than to A2 control cue in the probe test (Fig. 2c bar graphs). Thus, there was a significant interaction between group and cue (F4, 78=2.52, p<0.05). A step-down ANOVA comparing responding to A1 and A2 showed that rats in the sucrose and morphine groups showed less responding to the overexpected A1 cue than to the A2 control cue (sucrose: F1, 10=5.1, p<0.05; morphine: F1, 14=4.9, p<0.05). This extinction of responding as a result of the prior over-expectation in compound training was not evident in rats in the cocaine group, which responded at similarly high levels to these two cues (Fig. 2c bar graph; p=0.85). Importantly, this difference in responding between A1 and A2 in the sucrose and morphine groups was evident on the first trial in the probe test, reflecting a significant and spontaneous decline in responding to A1 when it was presented without the V1 in the probe test. This was evident in a comparison of responding to A1 and A2 on this first trial versus responding on the final day of conditioning (Fig. 2c, line plots). This comparison showed that responding to A1 declined significantly on the first trial of the probe test in the sucrose (F1, 10=6.0, p<0.05) and morphine groups (F1, 14=16.9, p<0.01), but not the cocaine group (p=0.55), while responding to A2 did not change significantly in any group (p's>0.2). Finally, to test if summation was related to the subsequent extinction, we compared the difference in conditioned response to A1 and A2 during the first trial of the first day of compound training to the difference in responding to A1 and A2 during the first trial of the probe test. We found an inverse correlation between these two difference scores in the sucrose and morphine groups (sucrose: r2=−0.67, p<0.05; morphine: r2=−0.53, p<0.05) but not in the cocaine group (r2=0.4, p>0.05) (Figure 3).
Experiment 2
The results obtained in Experiment 1 showed an important dissociation between the effects of a psychostimulant, cocaine, versus an opiate, morphine, in a critical cognitive function that depends on the OFC. In order to further assess the specificity of the cocaine effect, we tested another opiate, heroin, which is the most widely used opiate in drug self-administration studies (33, 34), in the same Pavlovian over-expectation task. Additionally, to provide a stronger test of specificity, we used a long access procedure, which usually leads to escalation of drug self-administration in both humans and laboratory animals (35, 36).
Self-administration training
Rats in all groups responded at high rates on the active lever compared to the inactive lever across sessions (Fig. 3). ANOVA’s (session × lever) showed a significant interaction (heroin: F13,156=2.4, p 0.05; sucrose: F13,117=5.3, p<0.05). Separate ANOVA’s also showed significant effects of session on the number of reward delivery (heroin: F13,156=8.6, p 0.01; sucrose: F13,117=16.01, p<0.01).
Conditioning training in the over-expectation task
Rats in both groups progressively increased responding to A1 and A2 versus A3 across sessions (Fig. 4a). A three-factor ANOVA (group × cue × session) showed a significant interaction between cue and session (F18, 378=13.8, p<0.01); there was no significant interactions involving group (p's>0.08). Moreover, both groups learned to respond to A1 and A2 equally, as indicated by the lack of statistical effects of cue in an analysis restricted to these two cues in each group (p's>0.1). Rats in both groups progressively increased responding to V1 as well. ANOVA’s showed a significant main effect of session during initial conditioning (F9, 189=6.3, p<0.01), but no significant interaction or main effect of group (p’s>0.4). Moreover, both groups learned to respond to A1, A2 and V1 in a similar way, as indicated by the lack of statistical effects of either cue or group in an analysis restricted to these three cues (p values>0.05).
Compound training in the over-expectation task
Rats in both groups maintained elevated responding to A1+V1 and A2, compared to A3 (Fig. 4b). A three-factor ANOVA (group × cue × session) showed a significant main effect of cue (F2, 42=112.8, p<0.01). Moreover, the spontaneous increase responding to A1+V1 was evident in the raw response rates of the sucrose group (F2,27=32.0, p<0.01 for all cues, post-hoc A1 vs A2: p=0.001) but not the heroin group (F2,36=25.6, p<0.01 for all cues, post-hoc A1 vs A2: p=0.20). When response rates during compound training were normalized to the last day of training; there was a significant difference between A1+V1 and A2 in the sucrose (F1, 9=7.9, p<0.05), but not in heroin group (p=0.8) (Fig. 4b, right).
Probe test in the over-expectation task
Rats in both groups showed elevated responding to A1 and A2, compared to A3, and this responding extinguished across the session (Fig. 4c). Accordingly, a 3-factor ANOVA (group × cue × trial) showed significant interaction between cue and trial (F14,294=7.4, p<0.01). Notably, there were no significant interactions involving group and trial (p’s>0.41), indicating that extinction of responding as a result of reward omission in the probe test was unaffected by prior self-administration of heroin.
However, in addition to these effects, which were similar between groups, rats in the sucrose group also showed less responding to the overexpected A1 cue than to the A2 control cue in the probe test (Fig. 4c bar graphs). There was a significant interaction between group and cue (F2, 42=3.4, p <0.05). A step-down ANOVA comparing responding to A1 and A2 showed that rats in the sucrose group showed less responding to the overexpected A1 cue than to the A2 control cue (sucrose: F1,9=6.9, p<0.05). This extinction of responding as a result of the prior over-expectation in compound training was not evident in rats in the heroin group, which responded at similarly high levels to these two cues (Fig.4c bar graph; p=0.7). Importantly, this difference in responding between A1 and A2 in the sucrose group was evident on the first trial in the probe test, reflecting a significant and spontaneous decline in responding to A1 when it was presented without V1 in the probe test. This was evident in a comparison of responding to A1 and A2 on this first trial versus responding on the final day of conditioning (Fig. 4c, line plots). This comparison showed that responding to A1 declined significantly on the first trial of the probe test in the sucrose group (F1,9=10.4, p<0.05), but not in the heroin group (p=0.1), while responding to A2 did not change significantly in any group (p's>0.09). Finally, we examined the relationship between the difference in the conditioned response to A1 and A2 during the first trial of the first day of compound training to the difference in responding to A1 and A2 during the first trial of the probe test. We again found an inverse correlation between these two difference scores in the sucrose group (r2=−0.73, p<0.05) but not in the heroin group (r2=0.01, p>0.05) (Figure 6).
DISCUSSION
We found that rats trained to self-administer cocaine under limited access conditions showed a deficit in an OFC-dependent learning task not observed in controls trained to self-administer a natural reinforcer, sucrose, or in rats trained to self-administer morphine in the same experimental access conditions. This impairment did generalize to a heroin, under experimental conditions that lead to escalation of drug self-administration. Notably deficits were observed several weeks after the end of drug use, thus it is possible that effects are related to neuroadaptations that occur with the cessation of drug use or abstinence.
Interestingly, the effect of cocaine or heroin was specific to behavior and learning that required the integration of prior reward expectancies. Thus cocaine- and heroin-experienced rats showed normal acquisition and maintenance of conditioned responding during training and normal extinction in the probe test. However, they failed to exhibit summation in response to the compound cue, and also failed to show extinction as a result of over-expectation at the start of the probe test. This pattern of results suggests a selective deficit in the ability to integrate prior knowledge to generate a novel prediction in response to compound cue delivery. This capacity has been closely linked to OFC function and its modulation of downstream learning mechanisms, both in this specific setting (26, 27) and more generally in other tasks (37). Thus our results confirm previous studies that implicate drug-induced OFC dysfunction in addiction (9, 13, 19, 20, 38–45).
Importantly cocaine and heroin did not cause a specific learning deficit, rather they caused a deficit or a change in how information in the environment was summed or integrated. However, our results illustrate that the loss of this function can impact learning. Specifically learning deficits can be expected when the prediction that is violated goes beyond simple reinforcement history, instead requiring inference or access to a model-based representation of the task or environment. The resultant deficits can appear as either impaired or facilitated learning. In the current study, learning that required integration of prior reinforcement histories was impaired while similar learning that did not require integration remained unaffected. However, we have recently shown using sensory pre-conditioning that an inferred value estimate can serve to block learning (46). In this setting, prior self-administration of cocaine promoted learning in the form of inappropriate unblocking (10). In the real world, it is likely that learning often involves such inferred or integrated value predictions, particularly for extinguishing a complex behavior, such as drug use, in which one can imagine the full expectation likely would hinge upon a multitude of predictive cues in any particular situation. Failure to integrate appropriately might lead to an especially weak prediction and inappropriate updating of the associative relationships – less learning when the outcome is of low value and more learning perhaps when it is of high value. This might occur in addition to other impairments in guiding behavior according to delayed or low probability adverse outcomes, credit assignment, and so forth.
Our data also address a fundamental debate in addiction field, which is whether different classes of drugs of abuse have substantial differences in neurobiological and behavioral effects (33). Opiates and psychostimulants, while both addictive, have very different pharmacological mechanisms of action (47–49). Accordingly, there are both similarities and differences in the molecular and behavioral effects of psychostimulant and opiate exposure, even within the prefrontal cortex. For example, at the structural level, their effects are somewhat similar; both psychostimulants and opiates are associated with decreases in gray matter concentration generally (43, 50, 51) and with declines in glucose metabolism after withdrawal from the drugs (52–55). However, differences have also been reported. Chronic psychostimulant exposure has been shown to decrease spine density in OFC (56), whereas chronic opiate exposure increases spine density (57). At the behavioral level, the OFC is involved in both cocaine and heroin relapse in rat models (58, 59), and both opiate and cocaine users show similar impairments on the Iowa Gambling Task (17, 60), an OFC-dependent task, but opiate users do not show impairments in another OFC-dependent task, probabilistic reversal learning, in which cocaine users show altered performance (61). Here we see a similar partial dissociation of effects.
Acknowledgments
This work was supported by funding from NIDA. This article was prepared in part while GS and YT were employed at UMB. The opinions expressed in this article are the author's own and do not reflect the view of the National Institutes of Health, the Department of Health and Human Services, or the United States government.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication., As a service to our customers we are providing this early version of the manuscript., The manuscript will undergo copyediting, typesetting, and review of the resulting proof, before it is published in its final citable form. Please note that during the production, process errors may be discovered which could affect the content, and all legal disclaimers, that apply to the journal pertain.
Financial Disclosures: The authors report no biomedical financial interests or potential conflicts of interest.
REFERENCES
- 1.Conklin CA, Tiffany ST. Applying extinction research and theory to cue-exposure addiction treatments. Addiction. 2002;97:155–167. doi: 10.1046/j.1360-0443.2002.00014.x. [DOI] [PubMed] [Google Scholar]
- 2.Franken IH, de Haan HA, van der Meer CW, Haffmans PM, Hendriks VM. Cue reactivity and effects of cue exposure in abstinent posttreatment drug users. J Subst Abuse Treat. 1999;16:81–85. doi: 10.1016/s0740-5472(98)00004-x. [DOI] [PubMed] [Google Scholar]
- 3.O'Brien CP, McLellan AT, Alterman A, Childress AR. Psychotherapy for cocaine dependence. Ciba Found Symp. 1992;166:207–216. doi: 10.1002/9780470514245.ch12. discussion 216–223. [DOI] [PubMed] [Google Scholar]
- 4.Wikler A. Dynamics of drug dependence. Implications of a conditioning theory for research and treatment. Arch Gen Psychiatry. 1973;28:611–616. doi: 10.1001/archpsyc.1973.01750350005001. [DOI] [PubMed] [Google Scholar]
- 5.Pavlov I. Conditioned Reflexes. London: Oxford University Press; 1927. [Google Scholar]
- 6.Todd TP, Vurbic D, Bouton ME. Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning. Neurobiol Learn Mem. 2014;108C:52–64. doi: 10.1016/j.nlm.2013.08.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical Conditioning II. Appleton-Century-Crofts; 1972. pp. 64–99. [Google Scholar]
- 8.Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci. 2009;10:885–892. doi: 10.1038/nrn2753. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Calu DJ, Stalnaker TA, Franz TM, Singh T, Shaham Y, Schoenbaum G. Withdrawal from cocaine self-administration produces long-lasting deficits in orbitofrontal-dependent reversal learning in rats. Learn Mem. 2007;14:325–328. doi: 10.1101/lm.534807. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wied HM, Jones JL, Cooch NK, Berg BA, Schoenbaum G. Disruption of model-based behavior and learning by cocaine self-administration in rats. Psychopharmacology (Berl) 2013;229:493–501. doi: 10.1007/s00213-013-3222-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lucantonio F, Stalnaker TA, Shaham Y, Niv Y, Schoenbaum G. The impact of orbitofrontal dysfunction on cocaine addiction. Nat Neurosci. 2012;15:358–366. doi: 10.1038/nn.3014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Everitt BJ, Hutcheson DM, Ersche KD, Pelloux Y, Dalley JW, Robbins TW. The orbital prefrontal cortex and drug addiction in laboratory animals and humans. Ann N Y Acad Sci. 2007;1121:576–597. doi: 10.1196/annals.1401.022. [DOI] [PubMed] [Google Scholar]
- 13.Jentsch JD, Olausson P, De La Garza R, 2nd, Taylor JR. Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology. 2002;26:183–190. doi: 10.1016/S0893-133X(01)00355-4. [DOI] [PubMed] [Google Scholar]
- 14.Schoenbaum G, Saddoris MP, Ramus SJ, Shaham Y, Setlow B. Cocaine-experienced rats exhibit learning deficits in a task sensitive to orbitofrontal cortex lesions. Eur J Neurosci. 2004;19:1997–2002. doi: 10.1111/j.1460-9568.2004.03274.x. [DOI] [PubMed] [Google Scholar]
- 15.Winstanley CA. The orbitofrontal cortex, impulsivity, and addiction: probing orbitofrontal dysfunction at the neural, neurochemical, and molecular level. Ann N Y Acad Sci. 2007;1121:639–655. doi: 10.1196/annals.1401.024. [DOI] [PubMed] [Google Scholar]
- 16.Nelson A, Killcross S. Amphetamine exposure enhances habit formation. Journal of Neuroscience. 2006;26:3805–3812. doi: 10.1523/JNEUROSCI.4305-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bechara A, Dolan S, Denburg N, Hindes A, Anderson SW, Nathan PE. Decision-making deficits, linked to a dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia. 2001;39:376–389. doi: 10.1016/s0028-3932(00)00136-6. [DOI] [PubMed] [Google Scholar]
- 18.Rogers RD, Everitt BJ, Baldacchino A, Blackshaw AJ, Swainson R, Wynne K, et al. Dissociable deficits in the decision-making cognition of chronic amphetamine abusers, opiate abusers, patients with focal damage to prefrontal cortex, and tryptophan-depleted normal volunteers: evidence for monoaminergic mechanisms. Neuropsychopharmacology. 1999;20:322–339. doi: 10.1016/S0893-133X(98)00091-8. [DOI] [PubMed] [Google Scholar]
- 19.Schoenbaum G, Setlow B. Cocaine makes actions insensitive to outcomes but not extinction: implications for altered orbitofrontal-amygdalar function. Cerebral Cortex. 2005;15:1162–1169. doi: 10.1093/cercor/bhh216. [DOI] [PubMed] [Google Scholar]
- 20.Simon NW, Mendez IA, Setlow B. Cocaine exposure causes long-term increases in impulsive choice. Behavioral Neuroscience. 2007;121:543–549. doi: 10.1037/0735-7044.121.3.543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Mendez IA, Simon NW, Hart N, Mitchell MR, Nation JR, Wellman PJ, et al. Self-administered cocaine causes long-lasting increases in impulsive choice in a delay discounting task. Behavioral Neuroscience. 2010;124:470–477. doi: 10.1037/a0020458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Roesch MR, Takahashi Y, Gugsa N, Bissonette GB, Schoenbaum G. Previous cocaine exposure makes rats hypersensitive to both delay and reward magnitude. Journal of Neuroscience. 2007;27:245–250. doi: 10.1523/JNEUROSCI.4080-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lucantonio F, Takahashi YK, Hoffman AF, Chang CY, Bali-Chaudhary S, Shaham Y, et al. Orbitofrontal activation restores insight lost after cocaine use. Nature Neuroscience. 2014;17:1092–1099. doi: 10.1038/nn.3763. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lattal KM, Nakajima S. Overexpectation in appetitive Pavlovian and instrumental conditioning. Animal Learning and Behavior. 1998;26:351–360. [Google Scholar]
- 25.Stalnaker TA, Roesch MR, Franz TM, Burke KA, Schoenbaum G. Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making. Eur J Neurosci. 2006;24:2643–2653. doi: 10.1111/j.1460-9568.2006.05128.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Takahashi Y, Roesch MR, Stalnaker TA, Haney RZ, Calu DJ, Taylor AR, et al. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron. 2009;62:269–280. doi: 10.1016/j.neuron.2009.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Takahashi YK, Chang CY, Lucantonio F, Haney RZ, Berg BA, Yau H-J, et al. Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning. Neuron. 2013;80:507–518. doi: 10.1016/j.neuron.2013.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Caprioli D, Celentano M, Dubla A, Lucantonio F, Nencini P, Badiani A. Ambience and drug choice: cocaine- and heroin-taking as a function of environmental context in humans and rats. Biol Psychiatry. 2009;65:893–899. doi: 10.1016/j.biopsych.2008.12.009. [DOI] [PubMed] [Google Scholar]
- 29.Weeks JR, Collins RJ. XXII. Patterns of intravenous self-injection by morphine-addicted rats. Res Publ Assoc Res Nerv Ment Dis. 1968;46:288–298. [PubMed] [Google Scholar]
- 30.Haney RZ, Calu DJ, Takahashi YK, Hughes BW, Schoenbaum G. Inactivation of the central but not the basolateral nucleus of the amygdala disrupts learning in response to over-expectation of reward. Journal of Neuroscience. 2010;30:2911–2917. doi: 10.1523/JNEUROSCI.0054-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.McDannald MA, Lucantonio F, Burke KA, Niv Y, Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. Journal of Neuroscience. 2011;31:2700–2705. doi: 10.1523/JNEUROSCI.5499-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Burke KA, Franz TM, Miller DN, Schoenbaum G. The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature. 2008;454:340–344. doi: 10.1038/nature06993. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Badiani A, Belin D, Epstein D, Calu D, Shaham Y. Opiate versus psychostimulant addiction: the differences do matter. Nat Rev Neurosci. 2011;12:685–700. doi: 10.1038/nrn3104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.De Vries TJ, Shippenberg TS. Neural systems underlying opiate addiction. J Neurosci. 2002;22:3321–3325. doi: 10.1523/JNEUROSCI.22-09-03321.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Ahmed SH, Walker JR, Koob GF. Persistent increase in the motivation to take heroin in rats with a history of drug escalation. Neuropsychopharmacology. 2000;22:413–421. doi: 10.1016/S0893-133X(99)00133-5. [DOI] [PubMed] [Google Scholar]
- 36.Ahmed SH, Koob GF. Transition from moderate to excessive drug intake: change in hedonic set point. Science. 1998;282:298–300. doi: 10.1126/science.282.5387.298. [DOI] [PubMed] [Google Scholar]
- 37.Schoenbaum G, Esber GR. How do you (estimate you will) like them apples? Integration as a defining trait of orbitofrontal function. Curr Opin Neurobiol. 2010;20:205–211. doi: 10.1016/j.conb.2010.01.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Winstanley CA, Green TA, Theobald DE, Renthal W, LaPlant Q, DiLeone RJ, et al. DeltaFosB induction in orbitofrontal cortex potentiates locomotor sensitization despite attenuating the cognitive dysfunction caused by cocaine. Pharmacol Biochem Behav. 2009;93:278–284. doi: 10.1016/j.pbb.2008.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Winstanley CA, Bachtell RK, Theobald DE, Laali S, Green TA, Kumar A, et al. Increased impulsivity during withdrawal from cocaine self-administration: role for DeltaFosB in the orbitofrontal cortex. Cereb Cortex. 2009;19:435–444. doi: 10.1093/cercor/bhn094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Harvey RC, Dembro KA, Rajagopalan K, Mutebi MM, Kantak KM. Effects of self-administered cocaine in adolescent and adult male rats on orbitofrontal cortex-related neurocognitive functioning. Psychopharmacology (Berl) 2009;206:61–71. doi: 10.1007/s00213-009-1579-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Goldstein RZ, Volkow ND, Wang GJ, Fowler JS, Rajaram S. Addiction changes orbitofrontal gyrus function: involvement in response inhibition. Neuroreport. 2001;12:2595–2599. doi: 10.1097/00001756-200108080-00060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Childress AR, Mozley PD, McElgin W, Fitzgerald J, Reivich M, O'Brien CP. Limbic activation during cue-induced cocaine craving. Am J Psychiatry. 1999;156:11–18. doi: 10.1176/ajp.156.1.11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Franklin TR, Acton PD, Maldjian JA, Gray JD, Croft JR, Dackis CA, et al. Decreased gray matter concentration in the insular, orbitofrontal, cingulate, and temporal cortices of cocaine patients. Biol Psychiatry. 2002;51:134–142. doi: 10.1016/s0006-3223(01)01269-0. [DOI] [PubMed] [Google Scholar]
- 44.Maas LC, Lukas SE, Kaufman MJ, Weiss RD, Daniels SL, Rogers VW, et al. Functional magnetic resonance imaging of human brain activation during cue-induced cocaine craving. Am J Psychiatry. 1998;155:124–126. doi: 10.1176/ajp.155.1.124. [DOI] [PubMed] [Google Scholar]
- 45.Renshaw PF, Daniels S, Lundahl LH, Rogers V, Lukas SE. Short-term treatment with citicoline (CDP-choline) attenuates some measures of craving in cocaine-dependent subjects: a preliminary report. Psychopharmacology (Berl) 1999;142:132–138. doi: 10.1007/s002130050871. [DOI] [PubMed] [Google Scholar]
- 46.Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, et al. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science. 2012;338:953–956. doi: 10.1126/science.1227489. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Gysling K, Wang RY. Morphine-induced activation of A10 dopamine neurons in the rat. Brain Res. 1983;277:119–127. doi: 10.1016/0006-8993(83)90913-7. [DOI] [PubMed] [Google Scholar]
- 48.Johnson SW, North RA. Opioids excite dopamine neurons by hyperpolarization of local interneurons. J Neurosci. 1992;12:483–488. doi: 10.1523/JNEUROSCI.12-02-00483.1992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Harris JE, Baldessarini RJ. Uptake of (3H)-catecholamines by homogenates of rat corpus striatum and cerebral cortex: effects of amphetamine analogues. Neuropharmacology. 1973;12:669–679. doi: 10.1016/0028-3908(73)90120-2. [DOI] [PubMed] [Google Scholar]
- 50.Fein G, Di Sclafani V, Meyerhoff DJ. Prefrontal cortical volume reduction associated with frontal cortex function deficit in 6-week abstinent crack-cocaine dependent men. Drug Alcohol Depend. 2002;68:87–93. doi: 10.1016/s0376-8716(02)00110-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Lyoo IK, Pollack MH, Silveri MM, Ahn KH, Diaz CI, Hwang J, et al. Prefrontal and temporal gray matter density decreases in opiate dependence. Psychopharmacology (Berl) 2006;184:139–144. doi: 10.1007/s00213-005-0198-x. [DOI] [PubMed] [Google Scholar]
- 52.Volkow ND, Fowler JS, Wolf AP, Hitzemann R, Dewey S, Bendriem B, et al. Changes in brain glucose metabolism in cocaine dependence and withdrawal. Am J Psychiatry. 1991;148:621–626. doi: 10.1176/ajp.148.5.621. [DOI] [PubMed] [Google Scholar]
- 53.London ED, Cascella NG, Wong DF, Phillips RL, Dannals RF, Links JM, et al. Cocaine-induced reduction of glucose utilization in human brain. A study using positron emission tomography and [fluorine 18]-fluorodeoxyglucose. Arch Gen Psychiatry. 1990;47:567–574. doi: 10.1001/archpsyc.1990.01810180067010. [DOI] [PubMed] [Google Scholar]
- 54.Volkow ND, Fowler JS, Wolf AP, Gillespi H. Metabolic studies of drugs of abuse. NIDA Res Monogr. 1990;105:47–53. [PubMed] [Google Scholar]
- 55.Volkow ND, Hitzemann R, Wang GJ, Fowler JS, Wolf AP, Dewey SL, et al. Long-term frontal brain metabolic changes in cocaine abusers. Synapse. 1992;11:184–190. doi: 10.1002/syn.890110303. [DOI] [PubMed] [Google Scholar]
- 56.Crombag HS, Gorny G, Li Y, Kolb B, Robinson TE. Opposite effects of amphetamine self-administration experience on dendritic spines in the medial and orbital prefrontal cortex. Cereb Cortex. 2005;15:341–348. doi: 10.1093/cercor/bhh136. [DOI] [PubMed] [Google Scholar]
- 57.Robinson TE, Gorny G, Savage VR, Kolb B. Widespread but regionally specific effects of experimenter- versus self-administered morphine on dendritic spines in the nucleus accumbens, hippocampus, and neocortex of adult rats. Synapse. 2002;46:271–279. doi: 10.1002/syn.10146. [DOI] [PubMed] [Google Scholar]
- 58.Fanous S, Goldart EM, Theberge FRM, Bossert JM, Shaham Y, Hope BT. Role of orbitofrontal cortex neuronal ensembles in the expression of incubation of heroin craving. Journal of Neuroscience. 2012;32:11600–11609. doi: 10.1523/JNEUROSCI.1914-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Fuchs RA, Evans KA, Parker MP, See RE. Differential involvement of orbitofrontal cortex subregions in conditioned cue-induced and cocaine-primed reinstatement of cocaine seeking in rats. Journal of Neuroscience. 2004;24:6600–6610. doi: 10.1523/JNEUROSCI.1924-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Grant S, Contoreggi C, London ED. Drug abusers show impaired performance in a laboratory test of decision making. Neuropsychologia. 2000;38:1180–1187. doi: 10.1016/s0028-3932(99)00158-x. [DOI] [PubMed] [Google Scholar]
- 61.Ersche KD, Roiser JP, Robbins TW, Sahakian BJ. Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology (Berl) 2008;197:421–431. doi: 10.1007/s00213-007-1051-1. [DOI] [PMC free article] [PubMed] [Google Scholar]