Addictive behaviour in experimental animals: prospects for translation

Abstract

Since the introduction of intravenous drug self-administration methodology over 50 years ago, experimental investigation of addictive behaviour has delivered an enormous body of data on the neural, psychological and molecular mechanisms of drug reward and reinforcement and the neuroadaptations to chronic use. Whether or not these behavioural and molecular studies are viewed as modelling the underpinnings of addiction in humans, the discussion presented here highlights two areas—the impact of drug-associated conditioned stimuli—or drug cues—on drug seeking and relapse, and compulsive cocaine seeking. The degree to which these findings translate to the clinical state of addiction is considered in terms of the underlying neural circuitry and also the ways in which this understanding has helped develop new treatments for addiction. The psychological and neural mechanisms underlying drug memory reconsolidation and extinction established in animal experiments show particular promise in delivering new treatments for relapse prevention to the clinic.

This article is part of a discussion meeting issue ‘Of mice and mental health: facilitating dialogue between basic and clinical neuroscientists'.

1. Introduction

Experimental studies of addictive behaviour in animals would seem to have obvious importance in increasing our understanding of disease mechanisms and received a boost following the pivotal description of addiction as a brain disease by Leshner [1]. They further provide an opportunity to develop new medications for addiction, for which there is a major unmet need [2]. Yet, there is a contemporary mood that ‘animal models’ of brain disorders, while seemingly of great importance, have shown poor translation from animals to humans leading industry to withdraw from, especially, treatment development for psychiatric disorders [3]. In fact, the pharmaceutical industry has never had treatments for addiction high on its list of priorities for development (with one or two notable exceptions) despite the morbidity and mortality associated with the disorder and its enormous personal, family, economic and societal impact [4].

However, translational studies of addiction stand on firm ground if in animals, behavioural rather than subjective measures of drug use (e.g. craving, liking) are used to enable contact—homology or analogy—to be made with clinical and human experimental studies. Animals will self-administer drugs that are addictive in humans, often showing patterns of drug taking and foraging that resemble patterns of behaviour seen in humans. More than 50 years of advances in research on drug self-administration have enabled a detailed understanding of the molecular and cellular basis of the reinforcing effects of stimulants, opioids, alcohol and other drug classes and, increasingly, circuit level explanations of drug seeking and relapse [5–8]. Yet, it has been suggested that ‘to anoint rodents engineered or trained to avidly self-administer drugs as a model of addiction risks leading translational neuroscience astray. This is because, at a minimum, such ‘models’ are too reductive, the critical brain structures too evolutionarily distant and they would fail to capture relevant human risk genotypes' [3, p.1384]. With some selective examples, it will be argued here that this is perhaps too pessimistic a view. Experimental investigation of addictive behaviour in animals has delivered a mechanistic understanding of addiction in humans—for example, why people take drugs, the nature of the adaptations they trigger in the brain [9], and more recently explaining why some individuals compulsively seek and take these drugs [10,11]—leading to advances in theory that have survived direct test in clinical populations. The vulnerability to develop the behavioural characteristics of addiction has been demonstrated in behaviourally heterogeneous rat populations [12–15] that have directly translated to addiction in humans, including sibling studies [16–19], and have begun to define endophenotypes for the disorder, at least in the case of stimulant addiction. There are pharmacological and psychological treatment leads that have been developed in animal experiments that are on the verge of translation to the clinic [5]. However, it must be acknowledged that there remains a reluctance to invest in expensive clinical trials with novel pharmacological treatments much for the reasons Hyman suggests [3]. These include the continued utilization of simplistic animal models [11] of addiction that are correctly considered unlikely to deliver effective treatments or mechanistic explanations of a disorder that affects only those users with a pre-existing vulnerability, and only after a protracted history of self-administered drug exposure. In that sense, to see the self-administration of drugs—i.e. drug taking—as a ‘model of addiction’ likely underestimates the complexity of this neuropsychiatric disorder at aetiological, behavioural and neural levels of analysis.

A selective and cursory overview of the neural correlates of addiction in humans serves to emphasize this point. While much of the experimental focus of experimental studies (and many earlier clinical studies) has been on the brain's reward system, with the mesolimbic dopamine system at its core (and that is undoubtedly important in mediating the reinforcing effects of addictive drugs), contemporary clinical imaging reveals that there are widespread anatomical and functional changes in the brains of those addicted to drugs. The seminal finding of reduced D2 dopamine receptors, initially identified in the dorsal striatum, of humans addicted to several classes of drugs, including stimulants, opiates and alcohol [20], further implicated adaptations in the dopamine system that were also shown to be highly correlated with reduced metabolic activity of the orbital prefrontal cortex (PFC) [21], thus bringing dysfunction in limbic cortical–dorsal striatal systems into view. Stimulant abusers have been reported to show grey matter loss in anterior cortical areas including the insula, ventromedial PFC, inferior frontal gyrus and pregenual anterior cingulate gyrus, as well as the anterior thalamus [18], with reports of even more widespread cortical and striatal grey matter loss in the brains of alcoholics [22–25].

Functional and PET imaging studies have further revealed changes in dorsal striatal and cortical function that are correlated with alterations in psychological processes including inhibitory control, decision making and habitual behaviour that contribute to compulsivity, as well as more familiar subjective measures, such as craving and its physiological correlates [26]. These neural correlates of behaviour and subjective states have both driven changes in and reflect the evolution of the definition of symptoms and hence the diagnosis of substance use disorders (SUD) now embodied in the Diagnostic and Statistical Manual of Mental Disorders (DSM5) [27]. Such widespread neural correlates in the brains of people addicted to drugs clearly indicate that exclusively, or even primarily, considering activity in the nucleus accumbens (NAcb) dopamine system and the associated enhanced motivation for addictive drugs as the key to understanding and treating addiction is too narrow a view.

The revised DSM5 symptom-based classification of SUD describes varying degrees of severity and no longer refers to drug dependence as in DSM-IV [28]. Although the pharmacological criteria of drug tolerance and withdrawal are still included among the 11 symptoms, they are not required for the diagnosis of severe SUD. Along with craving, the majority of symptoms reflect different aspects of compulsive behaviour and failures of control that are considered to belong to neurobehavioural continuums. The alternative dimensional approach to defining psychiatric disorders encapsulated by the NIH Research Domain Criteria (rDOC) [29] further emphasizes the requirement for objectively measured changes in neurobehavioural systems, for example those that might underlie compulsive drug use, rather than symptom clusters.

This has considerable implications for the ways in which translational studies of addictive behaviour in animals are undertaken. Our experimental approach, like that of other groups, has always sought to understand the symptoms of drug addiction and abuse in humans, as captured by the evolving DSM, but in terms of the underlying neurobehavioural and neurocognitive systems [26]. This is not a trivial undertaking because it needs to go beyond the self-administration of drugs or measuring behavioural responses to non-contingent drugs, even though these have been immensely useful in defining the neural basis of drug reinforcement and associated learning.

We have found it important to make a distinction between the taking as opposed to the seeking of drugs. Drug taking, i.e. self-administration under low response requirements, is directly controlled by the reinforcing properties of the drug, and the performance of taking responses, both in naturalistic and under experimental conditions, requires a specific set of motor skills. Drug seeking, or foraging for drugs, over sometimes long delays results in eventual access to the drug and the opportunity to make a taking response (i.e. using motor skills whether a lever press of loading a syringe or pipe) and subsequent drug self-administration [30,31]. Drug seeking is the predominant behaviour of individuals addicted to drugs because they spend large amounts of time acquiring drugs. Drug seeking is increasingly controlled by drug-associated stimuli and may even become divorced from the rewarding properties of the drug that decrease over time through tolerance (e.g. [32]).

A large volume of prior research has made clear that the psychological processes and neural mechanisms underlying seeking (appetitive) and taking (consummatory) behaviour are quite distinct, but they interact [33]. Drug seeking and taking are two independent components of complicated chains of instrumental behaviour, the performance of which requires skills and flexibility but that are determined by, and subordinate to, either of two competing psychological processes, described by contemporary animal learning theory as depending upon: (i) action–outcome (A–O) or (ii) stimulus–response (S–R) associations [34]. The former underpins goal-directed behaviour, within which a behavioural sequence is initiated under explicit goal-directed cognitive schemata using a representation of the motivational value of the outcome from the outset. The latter underpins habitual responding, within which a behavioural sequence is enacted with no representation of the motivational value of the outcome, but the performance of which relies, as for goal-directed behaviour, on skills and more flexible strategies. As discussed earlier, because drug-seeking responses are, by nature, more distal in a behavioural sequence from the drug goal, they are far more likely to be controlled by S–R mechanisms than drug-taking responses.

This too has had a marked impact on our understanding of drug-seeking behaviour and the growing evidence that there is a transition from goal-directed to habitual control over drug seeking over the course of a long drug use history [35], itself a requirement for addiction to develop: one or few instances of drug self-administration do not result in addiction, it takes time and quantity of drug exposure, as well as the associated history of Pavlovian–instrumental interactions for compulsive drug seeking to emerge. Environmental stimuli associated with addictive drugs through Pavlovian conditioning both elicit craving (in humans) [36] and profoundly influence the instrumental behaviour of drug seeking [2,37]. Thus, understanding Pavlovian conditioning mechanisms and the neural systems that underlie their impact on instrumental seeking behaviour, including mediating long delays to reinforcement, is important. Not all individuals exposed to drugs become ‘addicted’, by which is meant becoming compulsive in their pursuit and use of drugs and so individual vulnerability and its neural basis in animals is a research area of great interest that is meaningful in terms of understanding addiction vulnerability in humans [13,38]. Finally, potential treatments for addiction can emerge from understanding and reducing, for example, the impact of drug cues that powerfully elicit relapse to drug seeking and taking, either by pharmacological or by psychological means [2].

2. Drug cues, drug seeking, craving and relapse

An important mechanism by which drug conditioned stimuli (CSs) can influence instrumental drug seeking is conditioned reinforcement, through which Pavlovian CSs acquire a representation of the reinforcing properties of a drug and are able to reinforce seeking behaviour themselves when presented response-contingently. There is a clear distinction between this process and other Pavlovian influences on behaviour such as ‘sign-tracking’ (or Pavlovian approach behaviour) [39] and Pavlovian–instrumental transfer (PIT, previously termed Pavlovian motivation) [40,41]. Both the latter involve CS presentations that are not contingent upon instrumental responses and elicit either an automatic approach response (sign-tracking) or potentiate ongoing instrumental responding (PIT). Conditioned reinforcement, PIT and sign-tracking depend upon dissociable components of limbic corticostriatal circuitry; the abundant associated data have been reviewed in detail elsewhere [26,42–45].

In our own research, we have investigated the impact of conditioned reinforcers on drug seeking in second-order schedules of reinforcement for cocaine, heroin and alcohol [46–48]. Rats will work for long periods of time for an infusion of, or access to, these reinforcers at high levels of responding and these seeking responses decrease dramatically if response-contingent CS presentation is omitted (figure 1) (reviewed in [49]), thereby demonstrating the response-invigorating effects of conditioned reinforcement in the mediation of delays to drug reward. Non-contingent presentations of the same CS have much less effect and may even decrease seeking behaviour [50]. In widely used ‘extinction–reinstatement’ [51] or ‘incubation of craving’ [52] procedures, it is also the conditioned reinforcing properties of the CS that underlie ‘relapse’. Rats learn instrumentally to respond for the CS in the absence of the primary reward (self-administered drug) after either a period of instrumental (not CS) extinction (extinction–reinstatement) or a period of abstinence (incubation of craving) when the behavioural impact of the conditioned reinforcer increases with time in abstinence.

Figure 1.

The effects of drug CSs acting as conditioned reinforcers on cocaine, heroin and alcohol seeking. (a–c) Drug-seeking instrumental responses during a fixed interval of 15 min (open columns on the left of each panel) and the impact of presenting drug-associated CSs response-contingently, i.e. as conditioned reinforcers (shaded bars) over several sessions of daily testing. Introduction of the CS (in the second-order schedule of reinforcement) results in a marked increase in the vigour of seeking responses for cocaine (a), heroin (b) and alcohol (c). These effects are prior to the first drug infusion and, therefore, measure the seeking of the drug and not the effects of drug on instrumental behaviour or conditioned reinforcement. Omitting presentations of the CS results in a marked decrease in responding (open columns on the right). Data are mean + s.e.m. of responses/fixed interval 15 min in the presence (coloured bars) or absence (white bars) of drug CS presentation. (Online version in colour.)

It is perhaps worthwhile pointing out the difference between these different ways of measuring drug seeking and the impact of CSs. In second-order schedule procedures, rats from the outset must use CSs on a daily basis to mediate delays to reinforcement, as they forage for drugs. Extinction–reinstatement procedures [51,53] by contrast may look straightforward, but are psychologically more complex. There are three phases: (i) rats learn to take, but not seek drugs and each infusion is associated with a CS presentation; (ii) rats then undergo lever press (i.e. instrumental) extinction—they learn a new association of lever press–no drug and lever press–no CS, i.e. two new Pavlovian and instrumental inhibitory associations; (iii) in the key test (‘relapse’ phase) rats now learn that lever presses result only in conditioned reinforcement. So, there are three separate learning phases and in the final test phase, rats learn to respond with conditioned reinforcement and these responses will never be reinforced by the drug. The procedure undoubtedly taps into an aspect of inhibitory control (inhibiting lever presses in the absence of drug) and this is reflected in the extensive information we now have on the underlying circuitry, in which a PFC–nucleus accumbens pathway is key [53], as well as the adaptations in glutamate homeostasis seen after cocaine self-administration and withdrawal that can be remediated with N-acetylcysteine [54].

The ‘incubation of craving’, first demonstrated by Grimm et al. for cocaine [52], revealed that after extended access to (i.e. long sessions of) cocaine self-administration (drug taking) followed by increasing periods of enforced or voluntary (i.e. when an alternative reinforcer is offered as mutually exclusive choice) abstinence [55], reinstatement of the taking response in a ‘relapse’ test is greatly increased [56], i.e. responding with conditioned reinforcement has ‘incubated’ during the drug-free period. Again, in the test session, rats are learning for the first time that the taking response, now termed seeking behaviour, is reinforced only by the CS as drug is no longer delivered. Intriguingly, in fig. 1 of Grimm et al. [52], the data are described as ‘Persistence of a cocaine-seeking habit as a function of time since the last day of self-administration of cocaine’, which might be closer to what this phenomenon reflects than was perhaps intended at the time (see below).

The incubation phenomenon has been investigated neurally in great detail (e.g. [57,58]). Thus, while the impact of conditioned reinforcement on responding in this procedure was initially shown to depend upon the basolateral amygdala (BLA, as expected from our prior studies on conditioned reinforcement and cocaine-seeking acquisition under a second-order schedule), the incubation effect was shown instead to depend upon extracellular signal-related kinase (ERK) phosphorylation in the central amygdala [57]. Incubation of cue reactivity has since been shown to be associated with a number of time-dependent adaptations during the withdrawal period in a number of brain loci, including changes in excitatory transmission in NAcb medium spiny neurons associated with alterations in AMPA receptor subunit composition [58] and the unsilencing of synapses in the BLA–NAcb shell pathway [59]. In extending this approach to the incubation of methamphetamine craving, AMPA receptor changes were also observed in the NAcb core [60], but the incubation phenomenon was further associated with increased expression of several proteins, including BDNF and glutamate receptors, selectively in dorsal striatal neurons activated by drug CSs [61], bringing dorsolateral striatum (DLS) mechanisms and the S–R processes they mediate into focus. More recently, dorsomedial striatal neuronal ensembles have also been shown to play a role in the incubation of methamphetamine craving after choice-based abstinence [62]. The neural picture is thus increasingly complex and it remains to be seen whether these various demonstrations of incubation at a neural level can be brought together in a circuit-based explanation, or whether the incubation responses to different drug-associated stimuli established in different ways are underpinned by separate mechanisms and circuits.

Using second-order methodology to study drug seeking [49] has enabled us to make progress in defining the underlying psychological processes and neural circuitry in both the acquisition and long-term maintenance of cue-controlled drug seeking, as opposed to drug self-administration. It has also provided a way to explore putative pharmacological and psychological treatments that will decrease drug seeking and relapse by diminishing the impact of conditioned reinforcement. These findings have recently been reviewed extensively [2,26,31,63] and will be considered briefly here with an emphasis on their possible translational relevance.

In summary, circuitry involving the BLA and nucleus accumbens core (NAcbC) is necessary for the acquisition [64,65] and initial performance [66] of cocaine seeking. However, when the behaviour is well established over several weeks, dopamine-dependent mechanisms in the anterior dorsolateral striatum (aDLS) exert dominant control over seeking (but not taking) behaviour [67,68], consistent with the hypothesis that initially goal-directed cocaine seeking emerges as a S–R habit over time and extended training [69]. The temporal nature of this transition has been further demonstrated by timed interventions in the dorsomedial striatum and aDLS at different stages of acquisition and performance [70]. Moreover, the recruitment of the aDLS control over seeking depends upon the ventral striatum and is likely mediated by the spiralling circuitry [71] that links the nucleus accumbens with dorsal striatum dopaminergic mechanisms [72]. Indeed, in vivo voltammetry during CS-elicited cocaine seeking confirmed the dependence of aDLS dopamine release upon antecedent ventral striatal processing [73]. It should be emphasized that under these conditions, the aDLS is dominant in its control over drug seeking, as compared to the importance of ventral and dorsomedial striatal (DMS) mechanisms earlier in training and the lack of involvement of the aDLS at that time. This should not be taken to mean that the NAcb and DMS are no longer engaged, but in having recruited the aDLS, their role is subordinate to it in functional terms [73]. Extensive research on the striatal basis of goal-directed and habitual responding for food further emphasizes the parallel engagement of DMS and DLS, but relative dominance of one over the other and shifts between them when probed directly by reinforcer devaluation and inactivation of each independently [74–76].

In more recent work, we have shown that functional recruitment of dopamine-dependent aDLS control over cocaine seeking depends upon the BLA, but the maintenance of the cocaine-seeking habit depends upon the central amygdala (CeN) and its dopamine-dependent functional interaction with the aDLS [77]. However, there is no direct amygdala–aDLS connectivity and so the circuitry must involve other nodes. Using in vivo electrophysiology, we have established that the BLA influence on aDLS neuronal activity is mediated by antecedent glutamatergic mechanisms in the NAcbC and thence via a polysynaptic route involving the substantia nigra and its dopaminergic innervation of the DLS [77]. The pathways linking the CeN to the DLS have not been established directly to date, but there is a well-established projection from the CeN to the substantia nigra that has previously been shown to have a functional role in conditioned orienting [78], while CeN interacting with the aDLS has also been shown to play a key role in habitual responding for food [79].

It is not simply the idiosyncrasies of second-order schedules of reinforcement that have revealed and emphasized the progressive importance of dorsal striatal processes and habits in drug seeking. Using a seeking–taking chained schedule of cocaine reinforcement that we established to investigate the involvement of A–O versus S–R associations in instrumental cocaine seeking, Zapata et al. [80] confirmed our earlier finding that cocaine seeking is initially goal-directed using a devaluation procedure, and went on to show that after extended training cocaine seeking eventually became dependent on the aDLS, the inactivation of which restored goal-directedness (i.e. sensitivity to reinforcer devaluation). It was also shown that alcohol seeking involves a transition from goal-directedness to habitual control over time and that this involves a progression from the DMS to the DLS [81], with habitual responding depending upon DLS AMPA and dopamine D2 receptors [82]. These behavioural data indicating a transition from ventral-to-dorsal striatal engagement in well-established cocaine and alcohol seeking, especially but not only in behaviour supported by conditioned reinforcers, are paralleled by a number of neural studies showing a similar progression from ventral to dorsal striatum in neuroadaptations to long-term cocaine self-administration [83,84].

3. Ventral to dorsal striatal processing in imaging studies in humans

Do these briefly summarized data translate to imaging and other clinical studies of drug addiction? More or less contemporaneously with our initial studies on amygdala involvement in cocaine seeking, the earliest functional imaging studies of cocaine addiction revealed metabolic activation of the amygdala, orbitofrontal cortex and other limbic structures in response to cocaine CSs that elicited craving responses [85,86]. Subsequently, stimulant drug cues were shown to increase dopamine release in the ventral striatum of healthy volunteers after just three prior doses of amphetamine paired with discrete cues, but in those with cocaine use disorders, similar drug CS presentation increased dopamine release in the dorsal striatum. Craving was induced by these cues in both situations [87,88]. These data led Leyton and co-workers in an important recent study [89] to investigate whether stimulant cues induce dopamine release in the dorsal striatum only in individuals with drug use disorders (addiction), or whether this can occur in cocaine users explicitly not meeting DSM criteria of addiction. The results emphatically show that cocaine cues (personalized videos) that led to the opportunity to take cocaine in recreational cocaine users increased extracellular dopamine levels in the dorsal striatum and, therefore, prior to any diagnosable SUD [89]. From a translational perspective, this is precisely what our own data, summarized above, and other animal experimental studies, predict: in no sense are rats seeking cocaine under the control of drug CSs in a second-order schedule of reinforcement ‘addicted’, but the maintenance of this persistent seeking behaviour depends upon dorsal striatum, dopamine-dependent S–R habit mechanisms. We have further hypothesized that these habits are important building blocks of later-emerging compulsive drug seeking that is a key characteristic of addiction [26]. Cox et al. [89] similarly speculated that cue-induced dopamine release in the dorsal striatum is associated with ‘an accumulation of dorsal striatum related habits' that in their turn can be modulated by motivational processes (for review, see [26]).

Clinical imaging data have also supported our hypothesis of a shift from ventral-to-dorsal striatal processing during the establishment of addiction [31]. Thus, in former heroin addicts, functional coupling between the ventral and the dorsal striatum was revealed to be increased and associated with decreased functional coupling between the striatum and the PFC [90], suggesting diminished top–down control over striatal function. A similar shift in activation from the ventral to the dorsal striatum was demonstrated in response to alcohol cues in alcohol-dependent subjects when compared with recreational alcohol drinkers [91]. A link to the dominance of habitual behaviour in addiction was further shown in alcohol-dependent individuals who displayed an overreliance on S–R learning that was associated with increased activation of the posterior putamen, a region mediating habitual behaviour, and decreased activation of the ventromedial PFC and anterior putamen, a region involved in goal-directed learning [92]. Intriguingly, the ventral-to-dorsal striatal transition has also been demonstrated in a behavioural addiction—internet gaming disorder. Those with the disorder showed higher CS-induced activations than healthy controls in both ventral and dorsal striatum. But, activity in the left ventral striatum was, in fact, negatively correlated with CS-elicited craving, which was instead positively correlated with activations in the right dorsal striatum (putamen) and left caudate nucleus [93]. These data indicate that the intrastriatal transitions we have demonstrated in rats seeking cocaine and recently heroin (R Hodebourg, JE Murray, M Fouyssac, M Puaud, BJ Everitt, D Belin 2017, unpublished), and seen in humans addicted to drugs, may not be restricted to drug-induced plasticity in this circuitry. In human subjects engaged in learning a virtual maze task that revealed individual differences in spatial versus stimulus–response navigational strategies, response learners who had greater use of abused substances than spatial learners (double the lifetime alcohol consumption, a greater number of cigarettes smoked and a greater lifetime use of cannabis) also showed increased dorsal striatal grey matter volume and activity measured using fMRI, while spatial learners had increased hippocampal grey matter and activity [94]. Finally, cocaine-addicted individuals and also their non-cocaine abusing siblings had a significantly enlarged left putamen [18,95], suggesting that greater dorsal striatal (putamen) volume may be associated with a predisposition to acquire drug seeking and taking habits (see below). Furthermore, cocaine-addicted subjects showed reduced white matter connectivity of the right inferior frontal gyrus that correlated with impulsivity on the stop signal reaction-time task [19], a relationship also seen in non-drug-abusing siblings [96] and further suggestive of a cocaine addiction endophenotype.

4. Prospects for treatment of attenuating the motivational effects of drug cues

Whatever the mechanisms underlying the Pavlovian–instrumental interactions that contribute to the development of maladaptive habits, it has been apparent for some time that decreasing the impact of drug CSs on drug seeking in animals may have considerable utility if translated to the clinic to prevent relapse to drug use and thereby prolong abstinence. There are several possible ways of achieving this. The increased understanding of the neural and neurochemical basis of CS effects on behaviour indicates that pharmacological treatments might be used to reduce or prevent the effects of the CS on drug seeking and, in humans, decrease craving. Psychological treatments such as cue exposure therapy—essentially CS extinction through non-reinforced presentations—which have been in use for many years, can decrease subjective and physiological measures of craving in the clinic, but rapidly lose their effectiveness in the real world [97,98]. This may be partly explained by the marked context dependence of extinction learning (CS extinction in the therapeutic setting does not transfer to the drug-use setting) but may also reflect that the conditioned reinforcing effects of CSs, which are not restricted to exteroceptive cues, are quite resistant to extinction. However, CS extinction may be more effective when preceded by a brief CS exposure (memory ‘reactivation’, i.e. brief CS memory retrieval) in so-called super-extinction procedures [99,100]. Finally, as discussed extensively in this issue, memory reconsolidation-based methods established in animal experimental studies have recently emerged as a potential treatment approach to addiction [101–103] and other psychiatric disorders including phobias [104] and post-traumatic stress disorder [105].

5. Pharmacological approaches to reducing cue-elicited drug seeking and relapse

Our initial approach was to explore treatments that reduced drug seeking under a second-order schedule, because this behaviour depends, for its vigour, upon response-contingent CS presentations and provides an opportunity to study the impact of any treatment before and after the self-administration of drug [49]. Our initial breakthrough was to show that an antagonist and an inverse agonist at the D3 dopamine receptor both had the ability to markedly decrease cocaine seeking [106,107]. The antagonist was further shown to be effective in reducing conditioned responses to CSs associated with several drugs, including nicotine and heroin, in a number of procedures [108]. The D3 receptor antagonist had very limited effects on cocaine reinforcement (i.e. self-administration under continuous reinforcement) and did not impair locomotor activity, being devoid of what would be viewed as the unacceptable side-effects associated with D1 or D2 dopamine receptor antagonists. However, compounds from this class were subsequently shown to have unfavourable cardiovascular effects [109] and they have not been developed further as treatments for addiction, revealing some of the risks associated with drug development even when the preclinical lead is strong.

In demonstrating disturbances in glutamate homeostasis following cocaine and heroin self-administration [110], Kalivas and colleagues have highlighted this as a potential therapeutic target and gone on to demonstrate that the cysteine pro-drug N-acetylcysteine (NAC), a substrate for the cysteine–glutamate antiporter, prevents cued relapse in an extinction–reinstatement procedure [111]. Subsequently, we showed that it is also effective in reducing both cocaine and heroin seeking when well established, as well as restoring control after voluntary abstinence in the face of punishment in rats with a history of escalated cocaine self-administration, an effect that was associated with adaptations in a plasticity gene, zif268, in the DLS [112]. While open-label clinical trials showed early promise in cocaine addiction (see [113]), as did placebo-controlled clinical trials of cocaine and nicotine addiction [111], subsequent clinical trials have disappointingly not confirmed this early promise [114,115]. However, NAC may show more promise as a treatment adjunct to reduce craving or cue reactivity [116–118] as discussed in detail by Kalivas and Kalivas [119], perhaps emphasizing that specifying the treatment target (e.g. craving versus use) in clinical trials is especially important. It cannot be overstated that the potential of this treatment emerged from experimental investigations in rats across several behavioural procedures, suggesting that an animal experimental drug development pipeline can deliver therapeutic leads that show clinical promise.

More recently, we have shown that a highly selective µ-opioid receptor antagonist, GSK1521498, is effective in reducing cocaine, heroin and alcohol seeking as assessed in rats responding for these drugs under second-order schedules [47,48] (figure 2). The effects are only seen in the presence of response-contingent CS presentations and not when seeking responses are made in the absence of the CS, suggesting an interaction with the conditioned reinforcement process. These data are salient because they strongly implicate µ-opioid transmission in incentive motivational processes. Additional advantages for the treatment of opioid addiction is that in addition to reducing CS-induced drug seeking and relapse (as naltrexone has been shown to do in clinical trials) it should also diminish the impact of a lapse as it antagonizes the reinforcing effect of self-administered heroin (it is without effect on the reinforcing effects of cocaine) [48], although this may carry the risk of increasing drug intake and attendant mortality under treatment. The same compound, in addition to decreasing CS-controlled alcohol seeking, also reduced compulsive alcohol seeking (responding for alcohol under the threat of intermittent seeking punishment) and alcohol drinking [12] (figures 2 and figure 3). Antagonists at the µ-opioid receptor such as nalmefene are already in clinical use to decrease volumes of alcohol drunk in drinking bouts in alcohol-dependent subjects [120]. Again, then, here is a potential treatment that may diminish the propensity to relapse and also the impact of a lapse to drinking. The compound is well tolerated in humans after chronic treatment and decreased the subjective response to alcohol, but it has yet to enter into a clinical trial [121].

Figure 2.

The µ-opioid receptor antagonist, GSK1521498, decreased cocaine, heroin and alcohol seeking under second-order schedules of reinforcement and on voluntary alcohol consumption. The highly selective µ-opioid receptor antagonist, GSK1521498, was effective in reducing cocaine (in a), heroin (b) or alcohol (c(i)) seeking in rats responding for these drugs under second-order schedules. GSK1521498 also reduced alcohol intake (c(ii)) during the 20 min drinking period earned by prior alcohol-seeking responses reinforced by the alcohol-associated CS during the prior 15 min fixed interval. Data are mean + s.e.m. seeking responses/fixed interval 15 min; alcohol intake is expressed as grams per kilogram body weight. GSK1521498 was given at three different doses (0.1, 1, 3 mg kg⁻¹) and injected intraperitoneally (IP) 20 min before session. (Online version in colour.)

Figure 3.

Persistent compulsive alcohol-seeking phenotype in rats with a preference for alcohol (P rats) and its reduction by µ-opioid antagonism. (a) Rats were trained on a seeking–taking chained task to respond for alcohol, and when a stable baseline was established, seeking responses were punished probabilistically by mild electric foot-shocks of increasing intensity, from 0.25 to 0.30, 0.35, 0.40 mA, before stabilizing at 0.45 mA for six consecutive daily sessions. The arrow indicates the first session with a 0.45 mA foot-shock. Based on the persistence of alcohol seeking during the last three punishment sessions, measured as the number of completed seeking–taking cycles, a cluster analysis enabled the segregation of subgroups of rats: compulsive (C, in black), in which behaviour persisted despite unpredictable adverse outcomes (i.e. foot-shock punishment), and non-compulsive (NC, in grey), which ceased seeking under punishment and ‘abstained’. (b) Compulsive (in black, on the left) and non-compulsive (in grey, on the right) rats were tested under extinction (no reward was available) on the seeking lever only. Alcohol-seeking responses were greatly decreased, especially in compulsive rats, by systemic administration of the selective µ-opioid receptor antagonist GSK1521498. Data are mean seeking lever responses + s.e.m.); GSK1521498 was administered at the dose of 1 mg kg⁻¹, intraperitoneally 20 min before session. White bars, black and grey bordered bars represent the seeking responses in vehicle-injected, compulsive and non-compulsive rats, respectively. Black and grey bars represent the seeking responses in GSK1521498-treated compulsive and non-compulsive rats, respectively.

It should be acknowledged at the outset that most experimental demonstrations of the effects of drugs from several drug classes to reduce drug seeking and relapse involve acute treatments, whereas in the clinic such treatments will likely have to be given chronically to promote abstinence and decrease relapse. Few animal experiments have investigated the effects of chronic dosing on preventing drug seeking and relapse and this is an obvious challenge to translation, but one that has initially been met in trials with N-acetylcysteine. However, the present climate is not encouraging for the development by pharmaceutical companies of anti-relapse medications and it can only be hoped that this might change given the major unmet need.

6. Targeting drug memories in the prevention of drug seeking and relapse

The putative problems of chronic dosing and the compliance necessary to continue an abstinence-promoting treatment may, however, be avoided if the associative memories encoded by drug CSs could be erased or suppressed with a single treatment or very few treatments. This is the prospect provided by psychological therapies targeting memory reconsolidation and extinction. These topics have been reviewed extensively (see current issue, also [2,102]) and the focus here will be on the degree to which these treatments that have been developed in theory and in practice in animal experiments may successfully translate to the clinic. Memory reconsolidation is the process by which brief retrieval, or ‘reactivation’, of a memory by brief presentations of a CS (or context) that are insufficient to engage extinction—results in the memory becoming destabilized in the brain. The process by which it becomes re-stabilized to persist has been termed ‘reconsolidation’ and disrupting it leads to amnesia—i.e. the loss of behavioural response to the CS when tested subsequently [122–124]. The great majority of experimental investigations of reconsolidation have been on conditioned fear and these have yielded considerable understanding of the molecular and neurochemical mechanisms, including the fundamental requirement of new protein synthesis, the expression of a key protein (ZIF268, the protein product of the immediate-early gene zif268), the necessary activation of NMDA receptors and the ability of β-adrenoceptor antagonists to prevent reconsolidation in many instances [101,124–126].

Pavlovian fear presents a very tractable method for studying reconsolidation, as a very small number of CS–US (foot-shock) pairings is required to establish a persistent memory and the retrieval conditions to achieve destabilization are relatively straightforward—often a single CS presentation [127]. Thus, NMDA or β-adrenoceptor blockade (and other treatments) in association with memory reactivation result in amnesia and the loss of conditioned fear when the CS is again encountered. There is no amnestic effect of the same treatment given at the same time but in the absence of reactivation, hence it is a retrieval-dependent deficit. The effect seems to be persistent, leading to suggestions that the memory has been erased [124]

It is relatively unproblematic to demonstrate reconsolidation of a drug memory in a comparable Pavlovian procedure such as conditioned place preference in which there are few CS–drug pairings, followed by a simple CS–context exposure to reactivate the memory coupled with an amnestic treatment, and an equally simply preference test to measure the amnestic effect [128–130]. It is more challenging to demonstrate this phenomenon in a drug-seeking setting that involves several days (usually at least 10) of instrumental drug self-administration and as many as 300–500 discrete pairings of CS and drug US. What reactivation parameters would destabilize such a memory? The success of preventing Pavlovian drug memory reconsolidation can be measured only by the loss of effect of the CS on drug seeking, itself underpinned by an instrumental memory that might persist even when the Pavlovian memory has been diminished or erased. Nevertheless, we ( Jonathan Lee, Amy Milton and our colleagues) showed that brief memory reactivation by presenting the drug CS in association with knockdown of zif268, or NMDA receptor antagonist treatment or, in some circumstances, β-adrenoceptor blockade could prevent drug memory reconsolidation and lead to significant reductions in drug seeking in several procedures: (i) the impact of the CS acting as a conditioned reinforcer in rats responding under a second-order schedule [125]; (ii) in an acquisition of a new response procedure—the most precise demonstration of the loss of conditioned reinforcing properties of the CS following reconsolidation blockade [125,131,132]; (iii) in an abstinence–reinstatement procedure, where the effect tended to be smaller, but significant and no different from the effect of CS omission itself [133]. Reconsolidation blockade has also been shown for alcohol–CS [134] and heroin withdrawal–CS memories [135]. This is a brief summary of work from the Cambridge laboratory over the past decade or more; there are many other demonstrations in several laboratories [103,126,136], and also, of course, some failures that have been discussed informatively elsewhere [124]. Memory reconsolidation is a complex process, the precise retrieval conditions required successfully to destabilize the memory remain unclear and there is as yet no definitive biomarker for destabilization. This would appear to be a very unpromising basis for translation to clinical populations.

However, that reconsolidation-based treatments can successfully be translated to the clinic has been shown emphatically by the dramatic success in treating specific phobias, as summarized by Merel Kindt in this issue [104,137]. The outcomes of attempts to apply similar treatments to the addiction clinic have been more mixed, but with reasons now for optimism. In a very well-designed study with smokers treated with memantine, an NMDA receptor antagonist, at CS-induced memory reactivation, there was no effect on smoking levels, cue salience or reactivity to smoking-associated stimuli assessed in the post-treatment phase and even some indication of a slightly worse outcome in terms of relapse latency [138]. The authors discussed in detail the problems of knowing whether the reactivation protocol resulted in memory destabilization—pointing out that a smoker of 2 years will have undergone about 146 000 CS–nicotine pairings—and hence the difficulty in understanding whether memantine was indeed without therapeutic utility or just not administered in conjunction with a destabilized memory. However, a double-blind placebo-controlled trial of propranolol given at cocaine CS memory reactivation did provide evidence of albeit transient reductions in craving and cardiovascular reactivity on subsequent presentation of the same cues [139]. These data suggest that β-adrenoceptor blockade might be used in conjunction with drug memory reactivation, but that the treatment parameters need to be manipulated to optimize destabilization. This potential has been confirmed in a combined animal and human study of nicotine (smoking) memory reconsolidation blockade by propranolol [140]. Rats either underwent nicotine place preference conditioning or were trained to respond instrumentally for nicotine. Treatment with propranolol in association with memory reactivation induced by non-contingent injection of the US (i.e. nicotine) and not the CS, resulted in subsequent impaired conditioned place preference at test and diminished CS-reinforced and nicotine-induced reinstatement in a relapse test after abstinence. This reconsolidation-blockade effect was then demonstrated in a population of smokers who also received propranolol treatment in association with nicotine-induced (i.e. US-induced) memory reactivation; there was reduced preference for nicotine and nicotine cues and of nicotine craving induced by nicotine in the smokers [140]. These data indicate the potential of reconsolidation-based therapies in the treatment of addiction and also that memory destabilization might more effectively be achieved by US- (drug), rather than CS-based reactivations, perhaps because this results in the stronger prediction error that is required for memory destabilization to occur [124,141,142].

The reconsolidation approach involves a combined psychological and pharmacological treatment protocol but with the advantage that very few drug treatment sessions are required, thereby avoiding problems of treatment compliance and adaptations to chronic treatment. The reconsolidation phenomenon and the demonstration that fully consolidated memories can become labile under certain retrieval conditions are also beginning to have an impact on cue extinction therapies. This follows from the demonstration that brief fear memory reactivation an hour or so before extinction (repeated non-reinforced CS presentations) leads to enhanced extinction and reduced spontaneous recovery, reinstatement and renewal of the fear memory following CS, context or US exposure in both rats [99] and humans [143]. A delay of 6 h between reactivation and extinction prevents the effect, suggesting initially that destabilization of the memory by reactivation to induce reconsolidation mechanisms results in the original memory being ‘overwritten’ by the new CS–noUS extinction memory [144]. There remains considerable debate as to whether the phenomenon does indeed depend upon engaging reconsolidation mechanisms prior to extinction, or whether extinction itself is rendered more effective by the prior retrieval event [145]. This so-called super-extinction effect, though not always replicable [124], has now been successfully deployed in the treatment of addiction. Thus, rats self-administering cocaine or heroin subjected to a protocol of brief CS exposure followed by extinction repeated over several days were shown to have much lower levels of drug seeking at subsequent test [100]. This approach was then translated to a heroin-dependent inpatient population who were briefly shown heroin paraphernalia and an explicit drug use video (reactivation), followed by long exposure to the video (extinction) soon afterwards or after a delay of 6 h. In the retrieval–short delay extinction group, but not the delayed group, there was a significant reduction in craving and physiological responses to heroin cues as well as relapse measured up to 6 month post-treatment [100]—a truly remarkable demonstration of translation from animal experimental studies of addiction treatment directly to the clinic. More recently, this memory updating procedure has been compared with extinction alone in a randomized clinical trial of smokers, showing that retrieval–extinction resulted in ‘substantially attenuated craving to both familiar and novel smoking cues and reduced the number of cigarettes smoked per day by participants 1 month after treatment relative to extinction training alone’, the authors concluding that this approach indeed has the potential to enhance relapse prevention [146].

There is still much research needed to understand the underlying mechanisms of super-extinction, to define the retrieval conditions that optimize memory destabilization in reconsolidation-based treatment procedures, and also to increase the range or pharmacological treatments that can be used safely and effectively to block reconsolidation. However, the clinical rewards for persisting with this approach would appear to be great.

7. Compulsive drug seeking and its treatment: a translational challenge

A major challenge for understanding addictive behaviour and its treatment concerns the compulsive nature of drug use: can this be measured in animal experimental procedures in a way that is relevant to the human disorder, and would this enable the development of treatments that would decrease or even prevent compulsive drug seeking and taking in addicted individuals? There is considerable interest in procedures that measure compulsion in animals (primarily rats) seeking and taking drugs. Compulsive behaviour can be defined as the maladaptive persistence of responding despite adverse consequences [147] and this can be recognized in several of the criteria of SUD in DSM5. The origins of compulsivity in addiction are likely complex and have been suggested to include withdrawal via a negative reinforcement mechanism and allostasis [148] and stress [149], sensitization to the effects of addictive drugs (although this may be more important in the early stages of drug use) [150] and, as a result of imaging and psychological studies of clinical populations, the progressive loss of top–down inhibitory control over drug use as a result of dysfunction of the PFC (see above and [26]).

Compulsive drug use in animals has generally been measured according to its persistence in the face of an aversive outcome. Wolffgramm and Heyne's demonstration of persistent alcohol drinking in rats made the important observation that this occurred only after a very long period of drinking alcohol and that the chronically elevated intake was not affected by quinine adulteration at this stage, whereas intake was reduced at an earlier (non-addicted, in their terms) stage [151]. With intravenous drugs, taste adulteration is not an option to test persistent drug use and so punishment, usually mild foot-shock [14,152], and also aversive CSs [153] have been used to probe the persistence of responding despite negative outcomes. In our own work, we have again exploited the power of separating seeking and taking instrumental responses so as to avoid the interpretational complications of associating foot-shock with the self-administered drug. This might devalue the drug if delivered after a taking response and a drug infusion and the shock may also come to predict the resultant drug-induced increase in ventral striatal dopamine through counter-conditioning, thereby also decreasing its aversiveness [154].

Thus, we developed a modified seeking–taking chained schedule of cocaine reinforcement in which a seeking response is never reinforced, but instead allows a rat to gain access to a taking response that is always reinforced by drug—cocaine in our initial studies [155]. Under this schedule, seeking responses are directly related to the dose of cocaine (not inversely related, as in the case of the taking response) [155] and cocaine seeking is initially goal-directed [156], but emerges as an S–R habit under dorsal striatal control after an extended self-administration history [80]. To measure compulsive cocaine seeking, we introduced intermittent and unpredictable punishment of the seeking response, such that on some trials cocaine seeking resulted in the opportunity to make a taking response and receive i.v. cocaine, but on a random 50% of the trials the outcome of seeking responses, but never taking responses, was a single mild foot-shock and no presentation of the taking lever [14]. Under this procedure, rats must therefore run the risk of punishment in order to gain the opportunity to take cocaine; it thereby taps into some aspects of drug seeking in people who compulsively seek drugs. Key findings from these studies include: (i) all rats suppress their cocaine seeking after a brief cocaine taking history, i.e. they abstain from drug seeking and use; (ii) after a long drug history—that does not require escalation of cocaine intake—only a subset of rats (about 20%) persist in seeking cocaine despite punishment, i.e. are compulsive [14]. This individual vulnerability was also seen in a related study using an electric grid as a barrier to the taking response [152]; (iii) the development of compulsive drug seeking is related to the level of drug intake [14,157]; (iv) the ability to withhold seeking responses under punishment is increased by the availability of an alternative, concurrently available ingestive reinforcer [158]; (v) pre-existing trait impulsivity predicts CS-induced relapse after abstinence [159]. Trait impulsivity was further shown to be an important vulnerability factor in the development of compulsive cocaine self-administration in the three-criteria model of cocaine addiction, which does not use separate seeking and taking responses [13]. We have recently demonstrated compulsive alcohol seeking in rats that show a preference for alcohol when again a subgroup of compulsive individuals emerged after an extended alcohol taking and drinking history and, further, that this compulsive phenotype was stable over a 10 month period [12] (figure 3).

There is only a limited amount of data on the neural mechanisms underlying compulsive cocaine seeking in rats. Reduced forebrain serotonin (and striatal dopamine) levels were seen in compulsive versus non-compulsive rats, despite a very similar cocaine history [160]. Pharmacologically reducing central serotonin or treatment with a 5-HT2C receptor antagonist resulted in the emergence of punishment-resistance in rats after a brief cocaine history at a time when none displayed compulsivity. Moreover, a 5-HT2C receptor agonist reduced compulsive cocaine seeking in compulsive rats, as did treatment with the serotonin-selective reuptake inhibitor, citalopram [160], suggesting that SSRIs may be used clinically to reduce compulsive cocaine use. While clinical trials with SSRIs have not been generally successful in the treatment of cocaine addiction [161–163], at higher doses fluoxetine was shown to decrease the likelihood of relapse in patients who were abstinent at the start of treatment, while those with detectable blood levels of fluoxetine showed lower craving [164]. As we have discussed previously [160], higher doses of SSRIs such as those used in the treatment of obsessive-compulsive disorder, might have clinical utility in reducing compulsive drug use [165,166]. Compulsive alcohol seeking, as well as alcohol intake, was significantly reduced by the µ-opioid receptor antagonist GSK152498 (figure 3), again suggesting clinical utility.

The corticostriatal systems underlying compulsive drug seeking have been relatively little studied, but significant advances have been made. A discrete zone of the dorsal striatum is required specifically for mediating cocaine seeking under punishment in this task, but not unpunished seeking, which is subserved by an equally discrete zone in the mid-lateral anterior dorsal striatum [167]. Pre-training lesions of the anterior cingulate, prelimbic, infralimbic, orbitofrontal or anterior insular cortices were without effect on the development of compulsive cocaine seeking, while lesions of the BLA, although resulting in persistent seeking under punishment, also significantly reduced conditioned fear, which is not seen in rats that have become compulsive after a long cocaine self-administration history [168]. Together, these data suggest that any impairment in top–down inhibitory control mechanisms that might be associated with compulsivity are emergent, arising as a consequence of chronic drug exposure, rather than pre-existing [168]. Functional imaging data also suggest this to be the case [18]. This notion is further supported by the demonstration that long-term cocaine seeking in the seeking–taking with intermittent punishment task introduced by Pelloux et al. [14] is associated with decreased ex vivo intrinsic excitability of deep-layer pyramidal neurons in the prelimbic cortex and that this was most evident in the subgroup of rats that were compulsive (a proportion very similar to that seen by Pelloux et al.) [14]. In an ambitious study, it was further shown that optogenetic stimulation of this area of prelimbic cortex reduced compulsive cocaine seeking, while optogenetic inhibition of this area in non-compulsive rats resulted in increased responding under punishment [15]. These data show rather convincingly that chronic cocaine self-administration is associated with reduced prelimbic neuronal excitability and that this is causally involved in compulsive cocaine seeking.

Although clinical practice is some way from adopting optogenetic manipulation of the brain, this finding that optogenetic stimulation of a hypo-excitable prelimbic cortex reduced cocaine seeking in compulsive rats has been related to the hypofrontality seen in individuals addicted to cocaine [21]. Thus, Bonci and co-workers in Italy [169] have translated into clinical treatment an attempt to increase prefrontal cortical activity by transcranial magnetic stimulation (TMS). Cocaine-addicted patients recruited to the study were assigned as a treatment group or as controls in an open-label study. They received repetitive TMS (rTMS) of the right dorsolateral PFC and this treatment was repeated on subsequent occasions as required. Of course, rTMS of this general area of frontal cortex in no sense targeted the functional equivalent, if any, of the prelimbic cortical area targeted in the rat study, but was intended to modulate frontal circuitry in general, but the two bodies of work might ultimately be tapping into analogous functional networks. The results revealed significantly higher numbers of cocaine-free urines and lower cocaine craving in the rTMS subjects, some of whom had repeated rTMS sessions in order to maintain abstinence or reduce cocaine use. As the authors argue, the study supports the safety and potential efficacy of rTMS in treating individuals addicted to cocaine. If these preliminary data are taken in the context of a meta-analysis of a several studies involving rTMS of the dorsolateral PFC in SUD that provided clear evidence of decreased craving [170], there is a strong case for double-blind, placebo-controlled trials of the kind now underway independently at National Institute on Drug Abuse (NIDA), in Rome and Mexico City. Time will tell whether the data emerging from them will provide a definitive answer to the promising preliminary data from Terraneo et al. [169].

8. Conclusion

The data summarized here on measuring the effects of drug-associated stimuli on drug seeking and relapse, including manipulations of drug memories through reconsolidation blockade or extinction, as well as compulsive drug seeking provide some of the evidence that experimental investigation of addictive behaviour in rats (but also in mice and primates) can provide translationally relevant and important data. They give insights into the underlying neural circuitry and mechanisms of drug-seeking characterizing addictive behaviour. They have also indicated new treatment approaches that are already showing signs of promise in the clinic. This review has focused on the approaches used in our laboratory because the behavioural methodologies are somewhat distinctive, but we have also pointed to the rich source of data using approaches developed in other laboratories. We hope that the different approaches across the world of addiction research are not viewed as being in competition, but as part of a common endeavour to understand addiction as a disorder and provide hope to those who are addicted to drugs by helping to develop much needed treatments.

Data accessibility

This article has no additional data.

Competing interests

We declare we have no competing interests.

Funding

The research presented here was funded by the Medical Research Council (RG82507).