Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction

Abstract

We hypothesize that drug addiction can be viewed as the endpoint of a series of transitions from initial voluntary drug use through the loss of control over this behaviour, such that it becomes habitual and ultimately compulsive. We describe evidence that the switch from controlled to compulsive drug seeking represents a transition at the neural level from prefrontal cortical to striatal control over drug-seeking and drug-taking behaviours as well as a progression from ventral to more dorsal domains of the striatum, mediated by its serially interconnecting dopaminergic circuitry. These neural transitions depend upon the neuroplasticity induced by chronic self-administration of drugs in both cortical and striatal structures, including long-lasting changes that are the consequence of toxic drug effects. We further summarize evidence showing that impulsivity, a spontaneously occurring behavioural tendency in outbred rats that is associated with low dopamine D_2/3 receptors in the nucleus accumbens, predicts both the propensity to escalate cocaine intake and the switch to compulsive drug seeking and addiction.

1. Introduction

The central hypothesis guiding our research during the past decade or more has been that drug addiction can be understood in terms of the operation of the brain's learning and memory systems (Robbins & Everitt 1999; Everitt et al. 2001; Everitt & Robbins 2005). In particular, that chronically self-administered drugs may in some way pathologically subvert these memory systems and so lead to the establishment of compulsive drug-seeking habits (Everitt & Robbins 2005). Initially, our approach to the understanding of drug addiction built upon the clinical insight embodied within the DSM-IV (1994) criteria for ‘substance abuse’ and ‘substance dependence’. Therefore, we began experimentally to decompose and augment the DSM-IV diagnostic framework in terms of specified learning and cognitive processes deriving from animal learning theory and that increasingly have been attributed to the operation of specific neural, especially limbic cortical–striatal, systems (Everitt et al. 2001).

The early focus of much experimental drug addiction research was to understand the reinforcing, or ‘rewarding’, effects of abused drugs; this has led to great advances in defining the primary molecular targets of addictive drugs as well as, more recently, the adaptations in these targets that develop with chronic drug self-administration (Nestler 2004; Koob & Le Moal 2005). However, it has been appreciated for some time that the molecular and neurochemical correlates of acute and chronic drug administration must be interpreted in behavioural and cognitive terms if the psychological processes and neurobiological mechanisms determining human drug addiction are to be specified. Therefore, we and others have increasingly come to view drug addiction as the endpoint of a series of transitions from initial drug use—when a drug is voluntarily taken because it has reinforcing, often hedonic, effects—through the loss of control over this behaviour, such that it becomes habitual and ultimately compulsive. We have recently reviewed the evidence that these transitions depend upon interactions between Pavlovian and instrumental learning processes (Everitt & Robbins 2005). Furthermore, we have hypothesized that the ‘switch’ from voluntary drug use to habitual and progressively compulsive drug use represents a transition at the neural level from prefrontal cortical to striatal control over drug-seeking and drug-taking behaviours, as well as a progression from ventral to more dorsal domains of the striatum, mediated at least in part by its stratified dopaminergic innervation (Everitt & Robbins 2005).

We summarize here recent evidence in support of the hypothesis that drug-seeking habits are associated with a shift from ventral to dorsal striatal control over behaviour. We also address the major issue of vulnerability to drug addiction—that some individuals are more likely than others to take drugs, lose control over and escalate their drug intake, and ultimately to seek drugs compulsively. Experimental models of addiction, rather than drug self-administration, must reflect such individual differences and also incorporate extended periods of drug taking if the underlying neural mechanisms of addiction are to be identified (Koob & Le Moal 2005).

2. From voluntary to habitual drug seeking: the shift from ventral to dorsal striatum

With drugs such as cocaine, there is wide, though not universal, agreement that the dopaminergic innervation of the nucleus accumbens shell (AcbS), and even more ventral regions of the striatum, such as the olfactory tubercle, underlies its primary reinforcing effects (Di Chiara et al. 2004; Wise 2004; Ikemoto et al. 2005) as measured in drug self-administration procedures. We term this ‘drug taking’ to distinguish it from ‘drug-seeking’ behaviour, which must often be maintained over long periods of time and is profoundly influenced by environmental stimuli associated with self-administered drugs through Pavlovian conditioning (Everitt et al. 2001). In humans, these conditioned stimuli (CSs) can induce subjective states such as craving, as well as drug seeking and relapse after abstinence (Grant et al. 1996; Childress et al. 1999; Garavan et al. 2000). We have therefore developed a general model of drug seeking—a second-order schedule of cocaine reinforcement—in which this behaviour is sensitive not only to the contingency between instrumental responses and drug administration, but also to the presence of drug-associated CSs that have a powerful effect on performance by acting as conditioned reinforcers (Everitt & Robbins 2000). We have established that the acquisition of this cocaine-seeking behaviour depends upon the integrity of the nucleus accumbens core (AcbC) and its afferents from the basolateral amygdala (BLA). Thus, selective lesions of the BLA prevented the acquisition of cocaine seeking under a second-order schedule (Whitelaw et al. 1996), as expected given its fundamental role in conditioned reinforcement (Cador et al. 1989; Cardinal et al. 2002). Similarly, selective lesions of the AcbC also greatly impaired the acquisition of cocaine seeking (Ito et al. 2004). Simple drug taking, on the other hand, was unimpaired by BLA or AcbC lesions (Whitelaw et al. 1996; Ito et al. 2004). As we have reviewed elsewhere, the AcbC region of the ventral striatum is also a key locus not only for conditioned reinforcement but also for other Pavlovian influences on appetitive behaviour, including approach, as measured in autoshaping procedures, and Pavlovian–instrumental transfer, the process by which Pavlovian CSs energize ongoing instrumental behaviour, an example of conditioned motivation, each of which also depends upon processing in sub-regions of the amygdala (Cardinal et al. 2002; Cardinal & Everitt 2004).

These observations indicate that the BLA and AcbC may function together as nodes within a limbic cortical–ventral striatopallidal system that underlies the acquisition of drug seeking. This notion is further supported by the observation that disconnecting these structures by unilateral pharmacological blockade of dopamine and AMPA receptors in the BLA and AcbC, respectively, on opposite sides of the brain also greatly diminished cocaine seeking (Di Ciano & Everitt 2004). Thus, the acquisition and early performance of cocaine seeking that is maintained over protracted periods of time by the contingent presentation of drug-associated conditioned reinforcers depends upon the integrity of the AcbC and its afferent input from the BLA. It can be assumed that, at this stage, drug seeking is under the control of instrumental response–outcome contingencies in which animals respond in a voluntary and goal-directed way for intravenous cocaine infusions. Indeed, we have clear evidence that this is so from studies using a ‘seeking–taking’ chained schedule in which animals perform a drug-seeking response in the initial link of the chain, which then gives access to a drug-taking response in the second link, performance of which delivers cocaine (Olmstead et al. 2001). After a limited experience with the drug, cocaine seeking was shown to be a goal-directed action in that its performance was sensitive to devaluation of the drug-taking link (Olmstead et al. 2001). This devaluation effect demonstrates that cocaine seeking at early stages of acquisition and performance is mediated by the knowledge of the contingency between the seeking response and its outcome.

This response–outcome process can be contrasted with a second, stimulus–response (S–R) instrumental process in which seeking behaviour is a response habit triggered and maintained by drug-associated stimuli (Everitt et al. 2001). Tiffany (1990), O'Brien & McClellan (1996) and ourselves (Robbins & Everitt 1999; Everitt et al. 2001; Everitt & Robbins 2005) have all advanced the hypothesis that drug addiction encompasses changes in the associative structure underlying drug seeking such that it becomes ‘automatic’ or habitual. We have additionally hypothesized that the progressive engagement of dorsal striatal mechanisms underlies this transition (Everitt & Robbins 2005), based upon evidence that the dorsolateral (DL) striatum mediates S–R habit learning—evidence that has been considerably strengthened through the recent studies of instrumental responding for food and its resistance to reinforcer devaluation, a canonical test of the development of S–R habits (Yin et al. 2004, 2006). What then is the evidence that the dorsal striatum mediates the performance of well-established drug-seeking habits that depend initially upon ventral striatal mechanisms during acquisition? Correlative evidence came from in vivo microdialysis measurement of extracellular dopamine in rats that had attained over a two-month period stable responding under a second-order schedule of cocaine seeking. While self-administered cocaine increased dopamine release in the AcbS, AcbC and caudate–putamen, extracellular dopamine was increased selectively in the AcbC in response to unexpected (i.e. non-response contingent) presentations of a cocaine-associated stimulus. However, during a prolonged period of cocaine seeking maintained by contingent presentations of the same cocaine-associated CS, dopamine release was increased only in the dorsal striatum, not in the AcbC or AcbS (Ito et al. 2000, 2002). Furthermore, dopamine release in the dorsal striatum was subsequently shown to be causally important for the maintenance of drug seeking, since it was greatly reduced by dopamine receptor blockade at this site, whereas this treatment was without effect in the AcbC (Vanderschuren et al. 2005). This apparent shift from ventral to dorsal striatal control over drug seeking provides support for the hypothesis that it is under S–R, or ‘habit’, control owing to accumulated evidence implicating the dorsal striatum in habit learning (Yin et al. 2004, 2006). Moreover, the fact that the second-order schedule we have principally used is of a fixed-interval type encourages this view, since interval schedules are known to result in the more rapid development of S–R habits through the weaker relationship between response and outcome that obtains, when compared with ratio schedules (Dickinson 1985). In addition, studies with orally self-administered cocaine and alcohol have shown the more rapid development of habitual drug seeking compared with the seeking of a natural sweet reward (Dickinson et al. 2002, Miles et al. 2003).

These observations supporting the increasing importance of the dorsal striatum in well-established, or habitual, drug seeking raise the issue of how, in neural terms, such a shift in the locus of control from ventral to dorsal striatum might occur. The observations of Haber et al. (2000) in primates and, more recently, Ikemoto (2007) in rats provide a possible neuroanatomical basis. They showed that ventral tiers of the striatal complex regulate the dopaminergic innervation of more dorsal tiers through so-called ‘spiralling’ connections with the midbrain (figure 1a). Thus, the AcbS projects to dopamine neurons in the ventral tegmental area that innervate not only the shell but also the more dorsally situated AcbC. Neurons in the AcbC innervate dopamine neurons projecting to both the AcbC and the immediately dorsal regions of the dorsomedial caudate–putamen and so on, in a serially cascading pattern ultimately to encompass more lateral parts of the dorsal striatum—the site at which dopamine release is increased during habitual drug seeking and where dopamine receptor antagonist infusions impair this behaviour.

Figure 1.

Within striatum serial processing underlies the establishment of cocaine-seeking habits. (a) Schematic of the intrastriatal dopamine-dependent spiralling circuitry functionally connecting the ventral with the dorsal striatum in the rat (modified from Belin & Everitt 2008). The spiralling loop organization is depicted as the alternation of pink and black arrows from the ventral to the more dorsal parts of the circuit, i.e. from the AcbS (yellow) to the AcbC (light blue) via the ventral tegmental area (pink) and from the AcbC, via the substantia nigra to the dorsal striatum (dark blue). (b) Cocaine seeking is dose dependently impaired by bilateral infusions of the DA receptor antagonist α-flupenthixol (depicted as green dots) into the DL striatum. α-Flupenthixol infusions into the DL striatum dose dependently decreased responding on the active lever under a second-order schedule of cocaine reinforcement, but had no effect on responding on the inactive lever (Belin & Everitt 2008). (c) Disconnecting the AcbC from the dopaminergic innervation of the dorsal striatum impairs habitual cocaine seeking. In unilateral AcbC-lesioned rats, the AcbC relay of the loop is lost on the one side of the brain. However, on the non-lesioned side, the spiralling circuitry is intact and functional. When α-flupenthixol (green dots) is infused in the DL striatum contralateral to the lesion it blocks the DAergic innervation from the midbrain, impairing the output structure of the spiralling circuitry on the non-lesioned side of the brain. Therefore, this asymmetric manipulation disconnects the core of the nucleus accumbens from the DL striatum bilaterally and greatly diminishes cocaine seeking (figure adapted with permission from Belin & Everitt (2008)).

We tested the hypothesis that the serial cascade of striato-nigro-striatal connectivity underlies this progressively greater control over well-established cocaine-seeking behaviour by the DL striatum using a novel ventral–dorsal striatal ‘disconnection’ (figure 1b). Thus, the AcbC was selectively lesioned on one side of the brain and combined with dopamine receptor blockade in the contralateral DL striatum, thereby functionally disconnecting serial interactions between these ventral and dorsal striatal domains bilaterally (Belin & Everitt 2008). This disconnection greatly and selectively decreased cocaine seeking in rats tested some weeks after stable responding had been attained under a second-order schedule of reinforcement (figure 1c). Two important additional observations underlined the specificity of the ventral–dorsal striatal disconnection effect. (i) The same animals were trained to perform a novel chain-pulling response for sucrose under a fixed ratio 1 schedule of reinforcement and again underwent the disconnection manipulation, or bilateral dorsal striatal dopamine receptor antagonist infusions (in non-lesioned rats), immediately after acquisition when the behaviour was under response–outcome control; neither manipulation had any effect (Belin & Everitt 2008). (ii) In separate groups of animals, either bilateral dorsal striatal dopamine receptor blockade or AcbC-dorsal striatal disconnection was performed at a much earlier stage of acquisition of the cocaine seeking, second-order schedule when responding was under ratio, rather than interval, control and when response–outcome mechanisms dominate performance. Again, neither manipulation had any effect on cocaine seeking (D. Belin & B. J. Everitt 2008, unpublished observations).

Taken together, the above results clearly indicate the devolution of control over drug seeking from ventral to dorsal striatum and also strongly suggest that this shift is progressive. Other data also support the notion of this shift. For example, using autoradiographic methods, Porrino and colleagues showed the development of neuroadaptations in D_2/3 dopamine receptors and other neurochemical, or metabolic, markers in the dorsal striatum following chronic, but not acute, cocaine self-administration by monkeys (Letchworth et al. 2001; Porrino et al. 2004). At earlier stages of training, these adaptations were largely restricted to the more ventral, nucleus accumbens region. The DL striatum has also been shown to be involved in ‘relapse’ to a cocaine-seeking habit, since neural inhibition induced by γ-aminobutyric acid receptor agonist infusion into this area, but not into the AcbS or AcbC, prevented the reinstatement of cocaine seeking after protracted withdrawal (Fuchs et al. 2006; See et al. 2007). Moreover, the presentation of drug cues to human cocaine addicts both induced drug craving (that has been shown to be correlated with the activation of the amygdala and limbic prefrontal cortical areas; Grant et al. 1996; Childress et al. 1999; Garavan et al. 2000) and also marked the activation of the dorsal striatum (Garavan et al. 2000; Volkow et al. 2006). These observations therefore strongly indicate a link between limbic cortical mechanisms and the engagement of the dorsal striatum in long-term drug abusers exposed to drug cues, whereas the results of our ventral/dorsal striatal disconnection experiments reveal that this recruitment is mediated by antecedent limbic cortex-dependent activity in the AcbC and its regulation of dorsal striatal dopaminergic projections.

It seems likely that the ventral to dorsal striatum shift is not specific to drug seeking, but would apply equally to the control over instrumental responding for natural reinforcers under appropriate conditions. Indeed, lesion or inactivation of the AcbC, dorsomedial or DL striatum in rats responding for ingestive reinforcers does not globally impair instrumental behaviour, but instead has major effects that depend upon the response–outcome or S–R associative structure underlying the behaviour. Lesions or N-methyl-d-aspartate receptor blockade of the AcbC (Kelley et al. 1997; Corbit et al. 2001) or dorsomedial striatum (Yin et al. 2004, 2005), impair instrumental behaviour under response–outcome control, but actually enhance the development of S–R habits in which responding persists after reinforcer devaluation (Yin et al. 2004). By contrast, DL striatal lesions, inactivation or dopamine denervation return previously habitual responding to response–outcome control, reinstating sensitivity to reinforcer devaluation (Yin et al. 2004; Faure et al. 2005) or action–outcome contingency degradation (Yin et al. 2006). These observations emphasize that response–outcome and S–R learning mechanisms are probably engaged not serially, but in parallel, with DL striatum-dependent S–R mechanisms eventually dominating the control over behaviour.

However, it is possible that the shift from ventral to dorsal striatal control occurs more rapidly in animals seeking drugs owing to the effects of these agents themselves on the plasticity mechanisms involved, particularly in the case of psychomotor stimulants which so powerfully increase dopamine transmission. Thus, an amphetamine sensitization treatment regimen leads to the more rapid instantiation of habit learning in animals responding for food (Nelson & Killcross 2006). Moreover, animals that had escalated their cocaine intake showed, when subsequently challenged with cocaine, a marked enhancement of stereotyped behavioural responses that have long been known to depend upon the dorsal, rather than the ventral, striatal dopaminergic innervation (Ferrario et al. 2005). Finally, Schoenbaum and colleagues have demonstrated a shift in the balance of associative encoding from ventral to dorsal striatum correlated with concomitant enhancement of cue-evoked neuronal firing in the dorsal striatum (Takahashi et al. 2007). This finding resonates with the observation of drug-associated CS-induced activation of the dorsal striatum in human cocaine abusers (Garavan et al. 2000; Volkow et al. 2006). Therefore, the unique properties of drugs as reinforcers, especially stimulant drugs, might accelerate, or more effectively consolidate, the development of drug seeking as a S–R habit (Everitt et al. 2001).

But it should also be appreciated that while all animals responding for drugs or natural reinforcers will engage the dorsal striatal habit mechanism under appropriate reinforcement contingencies, not all individuals that take addictive drugs will become addicted. That is, not all individuals self-administering drugs will escalate their intake and go on to develop compulsive drug seeking that persists in the face of negative or aversive outcomes, the key characteristics of addiction, or substance dependence, in DSM-IV. Thus, some individuals are vulnerable in terms of these characteristics, a proportion that is often estimated to be approximately 20% or less of those initially exposed to addictive drugs (Anthony et al. 1994). We have shown recently that impulsivity is a behavioural characteristic that both predicts the escalation of cocaine intake and the progression to compulsive drug seeking, as well as an increased propensity to relapse after abstinence.

3. Impulsivity, ventral striatal dopamine receptors and vulnerability to addiction

Studies of human addicts have implicated individual differences in impulsivity, or other traits, such as ‘sensation seeking’, in the vulnerability to drug use and abuse (Verdejo-Garcia & Perez-Garcias 2007); although it has never been clear whether the impulsivity observed in drug addicts predates the onset of addictive behaviour or is a consequence of protracted exposure to drugs (Dom et al. 2006; Zilberman et al. 2007). We have investigated this issue experimentally by defining in rats an operational measure of the human trait of impulsivity as premature responses in a five-choice serial reaction-time task (5-CSRTT; Dalley et al. 2007). A proportion (less than 10%) of the outbred Lister-hooded strain of rats in our study were impulsive on this task in that they showed high levels of anticipatory responses made before the presentation of a food-predictive, brief light stimulus—especially under conditions when the stimulus presentation was delayed after trial onset (Dalley et al. 2007). This impulsivity we term ‘waiting impulsivity’, as it is related to impulsivity measured as an inability to tolerate delays of reinforcement (Robinson et al. 2008), but appears different from the response impulsivity measured by the stop-signal reaction time in STOP-signal reaction-time tasks (Eagle et al. 2008; E. S. J. Robinson, D. M. Eagle, D. Economidou, D. E. H. Theobald, A. C. Mar, E. R. Murphy, T. W. Robbins & J. W. Dalley 2008, unpublished observations).

When allowed to self-administer cocaine intravenously, the impulsive rats showed a marked escalation of their cocaine intake compared with non-impulsive controls. They did not acquire cocaine self-administration more rapidly, but responded at a much higher rate for their cocaine infusions than did non-impulsive subjects (Dalley et al. 2007). This is in marked contrast to rats showing high locomotor responses to novelty (a ‘sensation-seeking’ phenotype), which self-administer cocaine at low doses that do not sustain self-administration in those rats that show low locomotor responses to novelty (Piazza et al. 1989). Thus, high impulsivity predicts the tendency to escalate cocaine intake, which is reminiscent of one of the diagnostic characteristics of substance dependence in DSM-IV which describes the taking of drugs in larger quantities than intended as a core symptom.

Highly impulsive animals were investigated neurobiologically using positron emission tomography to measure binding in the striatum of the selective, high affinity D_2/3 dopamine receptor antagonist [¹⁸F]fallypride (Mukherjee et al. 1999). Impulsive animals showed markedly reduced fallypride binding within the ventral, but not the dorsal striatum. Moreover, the reduced D_2/3 dopamine receptor availability in the ventral striatum was correlated with impulsivity on the 5-CSRTT (Dalley et al. 2007). Thus, low D_2/3 dopamine receptors in the ventral striatum, encompassing the nucleus AcbC and shell regions, are correlated both with impulsivity and the marked propensity to escalate cocaine intake. This observation, taken together with the role of the AcbS in the reinforcing and stimulant effects of cocaine, as well as involvement of the AcbC in the acquisition of drug-seeking behaviour (Ito et al. 2004) and the ability to tolerate delays to reinforcement (Cardinal et al. 2001), indicates the importance of the ventral striatum in the neural mechanisms underlying the propensity to seek and work for cocaine over extended periods of time. Nader and colleagues, in studies of socially housed monkeys, have also demonstrated the significant relationship between D₂ dopamine receptors in the striatum and cocaine self-administration (Czoty et al. 2004; Nader et al. 2006); these studies are reviewed comprehensively by Nader et al. (2008).

Impulsive rats having self-administered cocaine subsequently showed reduced levels of impulsivity (Dalley et al. 2007)—perhaps related to the reduction in impulsive behaviour following treatment with the stimulant methylphenidate seen in humans with attention-deficit hyperactivity disorder. Moreover, during a subsequent period in which animals had no access to cocaine self-administration (‘enforced abstinence’), impulsivity returned to near pre-cocaine self-administration levels (figure 2a). Since impulsivity predicts the escalation of drug intake, we investigated whether the return of impulsivity during abstinence might also be associated with a greater propensity to relapse, as suggested by de Wit & Richards (2004). In these experiments (Economidou et al. 2007; D. Economidou, J. W. Dalley & B. J. Everitt 2007, unpublished observations), rats were trained in a cocaine seeking–taking task. Once responding had stabilized, a punishment contingency was introduced whereby on a random basis 50% of the seeking responses were followed by the presentation of the cocaine taking link in the chain, but 50% were followed by mild footshock (Pelloux et al. 2007). This schedule of unpredictable cocaine taking and aversive footshock outcomes results in the suppression of drug-seeking responses, especially after a limited, or non-escalated, history of cocaine self-administration; it can therefore be viewed as the development of ‘abstinence’, in that animals voluntarily withhold their drug-seeking responses when the punishment contingency is present. After having attained abstinence in this way and some two weeks after their last cocaine infusion, groups of impulsive and non-impulsive rats were reintroduced to the test boxes and again allowed to respond on the seeking lever, which always resulted in access to the taking lever, responses upon which resulted in the presentation of the cocaine-associated CS, but no drug. While both groups of animals reinstated their drug-seeking responses, i.e. ‘relapsed’, responding by impulsive rats was markedly and significantly greater than that of non-impulsive subjects (figure 2b). Thus, impulsivity not only confers an increased predisposition to escalate cocaine self-administration but also an increased propensity to relapse to a drug-seeking habit after abstinence. The neurobiological mechanisms underlying this persisting vulnerability to seek and take drugs after abstinence are an important area for future research.

Figure 2.

(a) The return of impulsive behaviour in highly impulsive rats following withdrawal and abstinence from cocaine self-administration. The data shown are premature responses in a 5-CSRTT. (i) Highly impulsive rats respond prematurely before any cocaine experience and (ii) their impulsivity is reduced following sessions of cocaine self-administration; (iii) but premature responding returns to pre-cocaine levels following an extended period of withdrawal (J. W. Dalley 2007, unpublished observations). Filled circles, high; open circles, low. (b) Impulsive rats show an increased propensity to relapse after abstinence. (i) Impulsive and non-impulsive rats were trained to seek and take cocaine under a chained schedule. (ii) Subsequently, 50% of the seeking responses were followed unpredictably by punishment and 50% by access to the taking lever; this results in the suppression of drug seeking. (iii) Following a further preiod of abstinence, when no cocaine was available rats were returned to the self-admininistration setting in which seeking responses resulted in the presentation of a cocaine-associated conditioned reinforcer, but no drug. (iv) Impulsive rats showed much higher numbers of seeking responses than low impulsive rats and hence showed a greater propensity to ‘relapse’. Filled circles, high; open circles, low.

4. From impulsivity to compulsive drug seeking in addiction

Demonstrating that impulsivity is a factor underlying the tendency to escalate drug intake and to relapse after abstinence leaves open the important issue of whether impulsivity is also a vulnerability marker for drug addiction and the compulsive drug seeking this entails. There are relatively few accepted models of compulsive drug seeking, or indeed compulsive behaviour in general, in animals. Perseverative responding in reversal learning tasks may provide one interesting example because this form of compulsion is persistently enhanced following even relatively brief periods of cocaine treatment (Jentsch et al. 2002; Calu et al. 2007). In theoretical terms, we have suggested (Everitt & Robbins 2005) that compulsive drug seeking can be characterized as a maladaptive S–R habit in which the ultimate goal of the behaviour has been devalued, perhaps through tolerance to the rewarding effects of the drug. Instead, drug seeking is increasingly controlled by a succession of drug-associated discriminative stimuli, which also function as conditioned reinforcers when presented as a consequence of instrumental responses, as in the second-order drug-seeking schedule described above. Central to drug addiction, then, is the persisting quality of these habits, which we have suggested (Everitt & Robbins 2005) may correspond to the subjective state of ‘must do!’—the persistent reinitiation of habitual acts—not least to distinguish it from the subjective state of excessive ‘wanting’ embodied in the incentive salience sensitization view of addiction (Robinson & Berridge 1993; see Robinson & Berridge 2008).

In attempting to model drug addiction in animals, we have tried to capture the compulsive quality of drug seeking by measuring its persistence despite negative or aversive outcomes, as in the DSM-IV. In developing such behavioural procedures, we have shown that compulsive drug seeking only emerges following an extended, or chronic, history of cocaine taking (Deroche-Gamonet et al. 2004; Vanderschuren & Everitt 2004; Pelloux et al. 2007). In the study by Deroche-Gamonet et al. (2004), three addiction-like behavioural criteria were measured in rats, namely (i) increased motivation to take the drug, (ii) inability to refrain from drug seeking, and (iii) maintained drug use despite aversive consequences. After approximately 40 days of cocaine self-administration, but not at earlier times, some 17% of subjects developed these addiction-like criteria showing increased break points under a progressive ratio of cocaine reinforcement, persistent responding during signalled periods of drug unavailability and, perhaps most importantly, persistence of the instrumental nose-poke response for cocaine even when it was punished by mild footshock. In the study by Pelloux et al. (2007), rats were trained on the seeking–taking chained schedule with intermittent punishment of the seeking response (i.e. on 50% of the seeking bouts, randomly occurring) to achieve suppression of drug seeking, or abstinence, as described previously. In this study too, whereas all rats suppressed their cocaine seeking after a limited history of cocaine self-administration, after an extended history 17–20% of subjects were completely resistant to punishment, continuing to seek and take drugs despite the ongoing, daily experience of the negative outcome. The proportion of rats compulsively seeking drugs, then, was similar both to that in the Belin study and to the addiction-vulnerable subgroup of human subjects, often estimated to be less than 20% of the population that initially use drugs (Anthony et al. 1994).

However, the origins of this propensity to seek cocaine compulsively have not been established. We hypothesized that impulsivity, which we have shown to be associated with low D_2/3 dopamine receptor availability in the ventral striatum and to predict the escalation of cocaine intake (Dalley et al. 2007), might also confer a vulnerability to develop compulsive drug seeking and addiction following extended access to cocaine. Rats were screened both for impulsivity in the 5-CSSRT and also for the sensation-seeking phenotype of high locomotor responsiveness to novelty (HR rats) which has earlier been suggested to be an addiction vulnerability phenotype (Piazza et al. 1989). The resultant groups were then studied in the acquisition of cocaine self-administration, and for the emergence of the three addiction-like behavioural criteria identified by Deroche-Gamonet et al. (2004), but particularly persistent responding for the drug in the face of punishment. As expected, and as reported previously (Piazza et al. 1989), the HR rats more readily acquired cocaine self-administration and showed an upward shift in the cocaine dose–response curve as compared both with rats with low responses to novelty and also high impulsivity (Belin et al. 2008). We also confirmed our earlier finding that impulsivity is not associated with more rapid acquisition of cocaine self-administration, but instead with the escalation of cocaine intake. However, and in marked contrast, it was high impulsivity, but not high reactivity to novelty, that predicted the switch (Leshner 1997) from controlled to compulsive cocaine taking (Belin et al. 2008). Highly impulsive rats displayed higher addiction scores and much greater resistance to punishment than rats with high or low responses to novelty or low impulsivity (electronic supplementary material, figure S1). In fact, highly impulsive rats did not differ from rats showing the three addiction-like behavioural criteria in any of their addiction-like behaviours after the extended period of cocaine self-administration. Therefore, it seems, perhaps counter-intuitively, that the propensity to acquire cocaine self-administration when first encountering the drug and the vulnerability to develop compulsive cocaine intake (addiction), depend upon distinct and seemingly orthogonal behavioural characteristics—novelty/sensation-seeking versus impulsivity, respectively—each of which might have a genetic or environmental basis. The results of this study also provide experimental evidence that high levels of impulsivity can antedate the onset of compulsive drug use, thereby emphasizing the importance of pre-existing impulsivity seen in individuals addicted to drugs (Jentsch & Taylor 1999; Dom et al. 2006).

Now that we have established models of compulsive drug seeking and addiction, it will be possible to investigate not only predisposing factors, such as impulsivity, but also the underlying neurobiological mechanisms. There are several current views about the origins of compulsion within the brain, which are often thought of as being, but in reality are not, mutually exclusive. The neuroadaptations occurring during behavioural sensitization to stimulant drugs have been argued to underlie an extreme incentive motivational state of drug ‘wanting’ (Robinson & Berridge 1993). According to this view, addicts experience this state especially when exposed to drug-associated cues, which leads to over-activation of the sensitized dopaminergic innervation of the nucleus accumbens, in which plasticity-associated structural changes in dendritic spines have also been observed (Ferrario et al. 2005). This hypothesis is discussed in detail by Robinson & Berridge (2008). One interpretation of compulsive drug seeking, then, is that it is a behavioural manifestation of this potentiated motivational state, which has been demonstrated in some studies as an increased break point under progressive ratio schedules of reinforcement (for a review, see Vezina 2004). However, as noted above, a sensitization treatment regimen with amphetamine also leads to the more rapid instantiation of S–R habits (Nelson & Killcross 2006) and it is not easy at the behavioural level to differentiate between an increased tendency to repeat drug-seeking responses elicited and maintained by drug-associated stimuli—the ‘must do!’ of compulsive habits discussed above—from an increased desire for a drug, which might also seem counter-intuitive given the development of tolerance to its rewarding or reinforcing effects.

Perhaps more directly related to the notion of drug seeking as a compulsive habit, however, is the observation of reductions in dopamine D₂ receptors in the dorsal striatum in abstinent alcoholics, cocaine, heroin and methamphetamine addicts (Volkow & Wise 2005) and also following chronic, but not acute, cocaine self-administration in monkeys (Moore et al. 1998; Nader et al. 2002). The consequences of this change in striatal dopamine D₂ receptors for the plasticity underlying instrumental learning and performance is unclear, but an intriguing putative sequential mechanism within the striatum and involving its dopaminergic innervation might be considered. Thus, the early vulnerability to escalate cocaine intake seen in impulsive rats is predicted by low D_2/3 dopamine receptor levels in the ventral, but not the dorsal striatum (Dalley et al. 2007). However, this escalated intake may lead to more rapid neuroadaptations, including downregulated D₂ dopamine receptors, in the dorsal striatum and mediated in part by aberrant engagement of the spiralling striato-nigro-striatal circuitry. This would lead to more rapid consolidation of drug-seeking habits that are difficult to relinquish, despite negative outcomes and are more readily reinstated after abstinence following exposure to response-eliciting drug-associated stimuli.

An alternative account may be provided by the impact of negative reinforcement, as has been suggested to underlie obsessive–compulsive disorder, whereby drug-seeking habits are maintained by the motivation to alleviate or avoid (self-medicate) the negative emotional state and dysregulation resulting from tolerance to, and withdrawal from, drugs taken in increasing amounts over time (Koob & Le Moal 2001). These counter-adaptations are prevalent in the central and extended amygdala and their motivational impact on drug self-administration is described in detail by Koob & Le Moal (2008). These mechanisms are not of course mutually exclusive. Addiction to drugs may reflect a combination of increased incentive motivation mediated by the upregulation of ventral striatal dopamine transmission, by ‘hyper-consolidated’ habit learning mediated by upregulated dorsal striatum, dopamine-dependent mechanisms and the drive engendered by negative emotional states in extra-striatal networks.

However, we and others have also hypothesized an additional neurobiological mechanism perhaps arising in part as the direct or indirect consequence of toxic drug effects. This mechanism may be implicated in a shift in balance of behavioural control processes from the prefrontal cortex to the striatum, thereby promoting compulsive habitual behaviour. There are abundant data suggesting prefrontal cortical, especially orbitofrontal (OFC), dysfunction in addicts, which are also increasingly supported by experimental studies in animals (Schoenbaum et al. 2006; Everitt et al. 2007; Olausson et al. 2007) and humans (see Garavan et al. 2008). Thus, in cocaine and methamphetamine abusers, reduced activity of the OFC that correlates with reduced D_2/3 dopamine receptors in the striatum (Volkow et al. 2001), and reduced grey matter volume in this region (Matochik et al. 2003) have been reported. There are also growing numbers of reports of impaired behavioural and cognitive functions, including poor behavioural adjustment (Bechara 2005) and impaired probabilistic reversal learning in cocaine abusers (Ersche et al. 2008), possibly due to reduced inhibitory control. Deficits have been reported in decision-making cognition on computerized versions of a gambling task, when stimulant abusers chose the most favourable option less frequently than control subjects and chose significantly against the odds in risky conditions, suggesting difficulties in estimating outcome probabilities (Rogers et al. 1999; Ersche et al. 2005). Similar changes in behaviour are seen in individuals with OFC damage (Rogers et al. 1999) and this has encouraged the view that chronic drug taking may actually be a causal factor in inducing such prefrontal cortex-dependent deficits. But suboptimal prefrontal cortical, including OFC and anterior cingulate cortex, function (Volkow & Fowler 2000; Kaufman et al. 2003; Hester & Garavan 2004) may also represent a pre-existing vulnerability trait that results in poor decisions and/or a lack of sensitivity to the consequences of such decisions, and hence drug abuse leading to addiction.

Experimental studies primarily involving psychostimulant treatment of rats and monkeys even after brief periods of exposure have supported the view that disrupted OFC function may indeed be a consequence of toxic drug effects during an addict's history of drug abuse (Jentsch & Taylor 1999; Schoenbaum et al. 2006). Short term, usually experimenter-, and not self-, administered cocaine or amphetamine enhanced the development of impulsivity (Jentsch & Taylor 1999; Roesch et al. 2007). Reversal learning was impaired by cocaine treatment in monkeys (Jentsch et al. 2002) and rats (Schoenbaum et al. 2004). Rats having self-administered and then been withdrawn from cocaine exhibited both increased extinction responding and a marked deficit in reversal learning during withdrawal (Calu et al. 2007). Schoenbaum and colleagues have emphasized the similarity between OFC lesions and these apparently long-lasting effects of relatively short-term treatment with cocaine, and also showed that the deficit in reversal learning is reflected in a change in the properties of OFC neurons, which do not develop appropriate responses to cues predicting outcomes (Stalnaker et al. 2006).

It is remarkable that even brief periods of drug exposure, whether experimenter administered or self-administered, can result in enduring changes in behaviour indicative of OFC dysfunction. However, in the great majority of imaging and neuropsychological investigations of addicts, there has been an exceptionally long history of drug abuse and often poly-drug abuse. These drug-addicted individuals must represent, therefore, a relatively small proportion of the much larger number of individuals in a population who have abused drugs over varying periods of time, but who have not made the transition to an addicted state as characterized by compulsive drug use. Thus, it would seem unlikely that experimental groups of rats receiving the fairly modest exposure to stimulant drugs in the experiments described above would in any sense fulfil the criteria for addiction, yet the changes in behaviour indicative of OFC dysfunction are seen in the entire population of treated experimental animals. It will be important, therefore, to investigate neurobiologically those models that capture chronic drug self-administration (Dalley et al. 2005a,b) and the compulsive drug seeking that develops in vulnerable sub-populations of rats if we are to understand the mechanisms underlying the interaction between predisposing behavioural traits and chronic drug exposure in the development of drug addiction.

The majority of the theorizing and evidence summarized above comes from studies of the neural and psychological basis of the seeking and taking of stimulant drugs such as cocaine. There is clearly a major need for studies of other drugs, especially opiates and alcohol, at both the psychological and neurobiological levels before the generalizability of the mechanisms so defined becomes clear and the gaps in our understanding are filled.