The transition to compulsion in addiction

Abstract

Compulsion is a cardinal symptom of drug addiction (severe substance use disorder). However, compulsion is observed in only a small proportion of individuals who repeatedly seek and use addictive substances. Here, we integrate accounts of the neuropharmacological mechanisms that underlie the transition to compulsion with overarching learning theories, to outline how compulsion develops in addiction. Importantly, we emphasize the conceptual distinctions between compulsive drug-seeking behaviour and compulsive drug-taking behaviour (that is, use). In the latter, an individual cannot stop using a drug despite major negative consequences, possibly reflecting an imbalance in frontostriatal circuits that encode reward and aversion. By contrast, an individual may compulsively seek drugs (that is, persist in seeking drugs despite the negative consequences of doing so) when the neural systems that underlie habitual behaviour dominate goal-directed behavioural systems, and when executive control over this maladaptive behaviour is diminished. This distinction between different aspects of addiction may help to identify its neural substrates and new treatment strategies.

In severe addiction, individuals spend much time and effort in procuring drugs for subsequent use that may not be in rich supply or immediately available, despite considerable risk of personal, physical and social harm. Such compulsive drug seeking, despite adverse or negative consequences, occurs in the absence of the drug. Drug seeking therefore cannot be simply the pharmacological effect of the taken substance, although the drug may further promote such behaviour. Drug taking, by contrast, is profoundly affected by the self-administered drug from the first ingestion. This distinction between two types of compulsive behaviour — compulsive drug taking and compulsive drug seeking — is made in this Review, as these behaviours may be governed by different psychological processes and dissociable neural circuits. The distinction also has implications for the way animal models of addiction are developed and used.

Indeed, the fifth edition of the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM-5)¹ has also subtly changed the way in which psychiatrists might consider addiction — now referred to as ‘substance use disorder’ — by abandoning some criteria that focused mostly on behaviours related to drug dependence, and placing more diagnostic emphasis on the compulsive quality of drug seeking. Thus, the notion that compulsivity is central to addiction is supported by the listing in the DSM-5 of symptoms that include an excessive time taken to search for drugs, the neglect of other goal-directed behaviours (such as employment and family activities) and even a failure to avoid physical self-harm, as well as subjective correlates of drug-seeking behaviour, such as craving. In this Review, we mostly use the term ‘addiction’ rather than the more cumbersome ‘severe substance use disorder’ as defined in the DSM-5.

Here, we describe addiction as a manifestation of contrasting behaviours of compulsive drug seeking and taking (BOXES 1,2) within a learning theory framework (BOX 3). We integrate this approach with recent findings from circuit neuroscience and discuss how distinct and dissociable neural and psychological mechanisms may underlie these distinct facets of addictive behaviour. We outline how the transition from controlled to compulsive drug seeking may involve an imbalance in the activity of frontostriatal circuits underlying goal-directed behaviour. We further discuss an optogenetic model involving direct self-stimulation of the mesolimbic dopamine (DA) system that provides evidence for a gain of function in cortical ‘top-down’ control. We contrast this model with evidence from human and animal studies of addiction that indicates a predominance of habitual over goal-directed behaviour associated with a loss of executive control by the prefrontal cortex (PFC). Finally, we suggest ways of resolving these discrepant views in the context of the distinction between drug seeking and drug taking with important theoretical and methodological implications for the field.

Box 1. Animal models of drug taking.

Self-administration

Laboratory animals will readily self-administer drugs that are addictive in humans by, for example, learning to press a lever for an intravenous infusion¹⁴⁰. This ‘controlled drug taking’ is usually performed under continuous or low-ratio schedules of reinforcement (that is, all or most responses deliver an infusion) and the rate of infusions is titrated. Drinking alcohol is a consummatory response that may be preceded by a nose-poke or lever-press taking response (see BOX 2 for more on seeking–taking chains¹⁴¹). For intravenous drugs, the lever press is effectively both the taking response and the consummatory response, as it is followed only by the drug’s effect in the brain. Drug self-administration or taking has been used to define the reinforcing or rewarding mechanisms of addictive drugs and the neuroadaptations that follow extended use, particularly in the dopamine reward circuitry^142,143.

Optogenetic dopamine neuron self-stimulation

Optogenetic dopamine neuron self-stimulation (oDASS) has been proposed to facilitate mechanistic investigations of drug reward focusing on the mesolimbic dopamine circuit. Mice will vigorously self-stimulate dopamine neurons in the ventral tegmental area (VTA) through oDASS²⁵. Such self-stimulation is occluded by injections of addictive drugs, readily supports adaptive behaviours such as cue-associated reward seeking and yields synaptic adaptations in the VTA and ventral striatum typically seen with addictive drugs. oDASS may also occlude drug self-administration, suggesting overlapping underlying neural circuits.

Loss of control over use: escalation of intake

Loss of control over drug taking can be reflected in escalation of drug intake that occurs after long-term, but not short-term, daily self-administration sessions for intravenous drugs, especially cocaine^81,144, or, in the case of alcohol, persistent drinking over several weeks¹⁴⁵. Brain adaptations to long-term drug intake may raise reward thresholds (hedonic set points)^146,147 and the efficacy of the reinforcing effects of cocaine148 (reviewed elsewhere¹⁴⁹).

However, escalation of intake neither predicts nor is necessary for compulsive drug taking (for example, as in the three-criteria model (see below)), but rather it is a consequence of developing the three behavioural criteria of addiction⁴⁷. Escalation of cocaine intake also does not predict development of compulsive cocaine seeking¹¹⁸. Intermittent access to cocaine or alcohol, rather than extended access, might favour the transition to compulsive cocaine intake¹⁵⁰ and alcohol intake^151,152, as well as compulsive alcohol seeking⁸².

Another model of loss of control over taking is the altered inter-infusion interval of cocaine seen during periods of unlimited ‘binge’ access¹⁵³. In addition, the progressive disruption of titration, as characterized by a burst-like pattern of infusions, is a predictive behavioural marker of the future transition to compulsivity in vulnerable individuals¹⁵⁴.

Addiction-like behaviour in rats

Compulsive drug taking can be defined as uncontrolled use despite negative consequences. This implies a ready drug supply and focuses on behaviour most proximal to the drug, such as drinking, smoking or self-injecting. Compulsive alcohol taking is measured in animals that persist in drinking despite quinine adulteration, emerges only after a long drinking history and is evident even after 9 months of abstinence, is specific to alcohol and is independent of physical dependence^145,151,152.

For intravenous drugs, punishment-resistant instrumental taking responses can be taken as a measure of compulsion^47,155,156. In the three-criteria model of addiction-like behaviour^47,157, rats are defined as compulsive if they meet three criteria: persistent responding for the drug despite receiving mild footshock punishment; persistent responding when the drug is signalled to be unavailable (but not under extinction conditions); and increased motivation for the drug under a progressive ratio. This model captures individual vulnerability to addiction: only ~20% of rats initially exposed to cocaine develop punishment resistance after 100 days of drug use. About 50% of animals persist in oDASS responding punished with mild footshock, demonstrating another form of compulsion⁴⁸.

Box 2. Animal models of drug-seeking behaviour.

Drug seeking in instrumental chained schedules

The most important feature of the seeking–taking chained schedule of drug reinforcement is that seeking responses on one lever (usually scheduled under random intervals) are never reinforced and instead only give access to a second, ‘taking’ lever, responses which deliver a drug intravenously. Such schedules have made possible investigation of the associative (goal-directed or habitual) structure underlying drug-seeking behaviour^80,158,159.

For alcohol, responses on a single lever can be used to measure seeking behaviour, as they are followed by a nose-poke taking response that gives access to alcohol and the consummatory response of drinking^51,70,82,99.

Cue-controlled drug seeking

Other procedures emphasize the impact of drug-associated conditioned stimuli (CSs) on drug seeking, particularly those in which CSs act as conditioned reinforcers because they are presented response contingently. In second-order schedules, seeking responses are reinforced by drug-associated CSs that bridge the delay to receiving an intravenous infusion of a drug or access to alcohol. Thus, seeking responses less tightly or less reliably deliver the drug and may thus be more susceptible to habitual control than are taking responses. Second-order schedules have been used to study cocaine, heroin and alcohol seeking^160–162, and to probe the neural circuitry underlying CS-controlled drug seeking and the transition from goal-directed behaviour to seeking habits¹⁶³.

Incubation of craving

Cue-controlled seeking after abstinence has revealed the phenomenon called the ‘incubation of craving’, whereby a drug-conditioned reinforcer (a drug-associated CS delivered response contingently) supports seeking behaviour that progressively increases the longer the period of abstinence following long-access self-administration sessions¹⁶⁴. This procedure models the considerable influence of drug cues on relapse after abstinence and, in some sense, models an aspect of the loss of control over drug seeking, and has been suggested to reflect enhanced cue-induced craving during abstinence in humans. A complex neural circuitry is involved in the incubation effect that may also be drug specific. Incubation of craving has been reviewed extensively^165,166 and is not considered in detail here.

Extinction–reinstatement

This is a procedure that models relapse. After a limited period of drug taking, the instrumental lever press and the drug-associated CS are extinguished (that is, the drug is no longer made available). The reinstatement of responding is then assessed in a relapse test, during which lever presses result in CS presentation alone (which acts as a conditioned reinforcer), or following stress or a non-contingent injection of the drug. This procedure has revealed a rich circuitry involving the nucleus accumbens and the medial prefrontal cortex, as well as molecular mechanisms underlying relapse^167–169. However, we do not cover this topic in this Review, as the procedure is not designed to examine the neural basis of drug taking or seeking, except in the complex postinstrumental-extinction state. Moreover, instrumental extinction is not generally a feature of voluntary abstinence in addiction in humans.

Compulsive drug seeking

Compulsive drug seeking has been operationalized as persistent instrumental seeking behaviour despite the risk of, or actual, punishment of the seeking, but not the taking, response^118,170. In seeking–taking schedules for cocaine or alcohol, mild footshock punishment of the seeking response occurs randomly on completion of 50% of the seeking intervals, instead of access to the taking response. Taking responses are never punished, and drug delivery is never associated with shock, thereby contrasting with models of compulsive drug taking in BOX 1. Only 20% of individuals develop compulsive drug seeking, thus capturing individual differences in vulnerability to addiction. In a new procedure based on optogenetic dopamine neuron self-stimulation in a seeking– taking chain, lever pressing during a random interval gives access to a taking lever that delivers optogenetic dopamine neuron self-stimulation, which is then punished. When seeking responses are punished, they continue in a fraction of a population of mice, indicating differential vulnerability to compulsivity⁴⁸.

Box 3. Actions and habits, learning theory and circuits.

Three major aspects of learning may contribute to drug seeking and drug taking: Pavlovian conditioning, instrumental goal-directed learning and stimulus–response (habit) learning.

In Pavlovian conditioning, previously neutral stimuli (such as drug paraphernalia or drug-associated contexts) are conditioned to the effects of a drug because of their predictive relationship. Pavlovian conditioned stimuli (CSs) can motivate seeking and taking through Pavlovian–instrumental transfer (PIT), and can reinforce and sustain seeking responses by acting as conditioned reinforcers. PIT, conditioned reinforcement and their neural bases are not considered here but are reviewed elsewhere^{91,98,171,172}. Notably, drug-associated CSs also capture attention and can elicit sign tracking^90,173.

Goal-directed (instrumental) behaviours depend on the value of the goal (so-called action–outcome associations). Drug self-administration might be preceded by ‘foraging’ for the drug (or drug seeking), especially if it is only occasionally available. Entire sequences of behaviour may be maintained by drug-predictive stimuli; such stimuli become goals or subgoals in themselves and are termed ‘conditioned reinforcers’⁴².

The great majority of neural data come from studies with food reinforcement. Plasticity in the posterior dorsomedial striatum is required for acquiring and performing goal-directed action, as is the basolateral amygdala, which updates goal-value information^81,171,174. By contrast, prelimbic projections to the posterior dorsomedial striatum are required to acquire, but not to perform, goal-directed action175. The lateral–ventral orbitofrontal cortex (OFC) controls the flexibility of instrumental behaviour when action–outcome or stimulus–outcome contingencies are devalued, or when Pavlovian stimuli guide choice^176,177, whereas in rats the medial OFC is necessary for retrieving representations of the outcome in the absence of the expected outcome to guide goal-directed action¹⁷⁸. In primates, certain OFC regions are required for revaluing rewards, whereas others re-evaluate rewards in behavioural choice and compare stimulus values in choice settings¹⁷⁹ (that is, in goal-directed behaviour that requires decision-making).

Habitual behaviour is underpinned by stimulus–response associations and often occurs after extensive training, when behaviour becomes autonomous; that is, independent of outcome value and elicited by particular stimuli or contexts¹⁸⁰. An instrumental behaviour can be diagnosed as habitual when it is performed even after goal devaluation (as with drug tolerance) or with contingency degradation. Habits place lower demand on limited cognitive resources and thus free them up for flexible adaptation to changing environmental contingencies, which requires effort and goal-directed control¹⁸¹.

Plasticity in the anterior dorsolateral striatum is required for acquiring and performing food-reinforced stimulus– response habits, as impairing its function reinstates seeking behaviour sensitive to outcome devaluation or contingency degradation^171,182. The infralimbic cortex is also involved in habit learning183; optogenetic inhibition of the infralimbic cortex prevents habit acquisition^184,185. However, no direct projections connect the infralimbic cortex to the anterior dorsolateral striatum, so how these areas interact remains to be established.

Notably, the nucleus accumbens (NAc) is not required for the acquisition or performance of instrumental behaviour. However, the core and shell of the NAc are differentially required for the impact of CSs on behaviour and for the influence of outcome value on responding¹⁷². Lesion or inactivation of the NAc does not affect instrumental learning or sensitivity to contingency degradation^{57,81,185,186}. By contrast, the NAc core and shell are dissociably required for conditioned reinforcement¹⁸⁷, PIT⁹¹ and sign tracking^172,187.

Recent findings obtained with projection-specific transgenic mouse lines suggest that neurons projecting to the NAc core mediate association, whereas shell projectors encode motivation¹⁸⁸. As a result, self-stimulation of shell projectors is fully supported only after conditioning has occurred. In the case of optogenetic dopamine neuron self-stimulation, both streams are activated together, explaining the very strong reinforcement.

Everyday behaviour probably consists of a mix of goal-directed and habitual behaviour. Goal-directed and habitual behaviour are often well coordinated, but may also compete. People who abuse stimulants may rely more on habitual control than on goal-directed control for laboratory-based tasks, suggesting an imbalance of these learning systems³⁷.

Drug taking and reinforcement

Animal models of self-administration (BOX 1) show that all addictive drugs are positively reinforcing². Acquisition of this instrumental behaviour requires DA signalling in the mesolimbic pathway, particularly by neurons of the ventral tegmental area (VTA) that project to the medial shell of the nucleus accumbens (NAc; part of the ventral striatum)^3–5 (FIG. 1). This mesolimbic pathway has also long been implicated in the mediation of both the anticipation of and the reinforcing effects of many natural reinforcers including, notably, food⁶. Whereas both food and alcohol share a common consummatory response of ingestion, there is no equivalent consummatory response for intravenous drugs other than the self-administrating lever press. The clear separation between instrumental and consummatory behaviour for food has made possible a detailed analysis of the separate circuits involved in eating and instrumental responding for food^7,8. However, the confounding of instrumental and consummatory (that is, instrumental self-administration) behaviour for intravenous drugs has made the analysis of these components both difficult and more ambiguous. It has been addressed by the introduction of so-called seeking–taking instrumental chained procedures (FIG. 2; BOX 2), which have largely illuminated the instrumental aspects of drug seeking, rather than drug taking, as is apparent in this Review.

Titrated.

Adjusted in intensity, to measure the effect of a stimulus on behaviour.

Fig. 1. Neural circuits engaged in drug seeking, drug taking and the transition to compulsion in addiction.

a| Addictive drugs of different pharmacological classes have a common initial effect of increasing levels of dopamine in the nucleus accumbens (NAc) — particularly dopamine released by neurons projecting from the ventral tegmental area (VTA). This effect is viewed as crucial for initial drug reinforcement. Drug taking depends on plasticity of projections from the medial prefrontal cortex (mPFC) and orbitofrontal cortex (OFC) to the dorsomedial striatum (DMS). b | Initially, drug seeking is goal-directed and depends on the DMS and afferents from the mPFC and OFC. The NAc is not required for instrumental drug-taking behaviour but has a major role in mediating the reinforcing effects of drug-associated conditioned stimuli (CS) on seeking responses. The NAc is functionally related to the DMS via the serially looping circuitry that involves the VTA and substantia nigra par compacta (SNc). c | When drug seeking is well established, it is under the dominant control of the dorsolateral striatum (DLS; putamen in primates), which receives its major cortical afferents from the motor cortex (MC). The DLS may be recruited through the recurrent circuitry that links the NAc with the VTA and, progressively, via the substantia nigra, with the DLS. Note that back-projecting medium spiny neurons (MSNs) express dopamine D1 receptors and preferentially synapse onto GABA interneurons in the midbrain (green). The acquisition of CS-controlled drug seeking depends on the basolateral amygdala (BLA) and its projection to the NAc (shown in part b). The maintenance of established drug seeking habits does not depend on the BLA (shown in faded purple in part c) but instead depends on the central amygdala (CeA; purple in part c). The CeA has direct projections to the SNc and can therefore influence the dopaminergic innervation of the anterior DLS (aDLS). d | Compulsive drug seeking depends on the loss of prefrontal cortical ‘top-down’ control over the striatal mechanisms underlying drug-seeking habits (denoted by shading of the DLS and grey shading of the mPFC and OFC). This model may be contrasted with the ‘gain-of-function’ model derived from optogenetic dopamine neuron self-stimulation studies (FIG. 2). Thus, two models are discussed in the main text: one that sees compulsion as an excessively goaldirected action, possibly mediated by the OFC, and a contrasting model that builds on the role of habits in compulsion and the failure to disengage the DLS (see also FIG. 4). D2R, dopamine D2 receptor.

Fig. 2. Assessing compulsive drug-taking and drug-seeking behaviours in animal models.

a| In an operant chamber with an active lever and an inactive lever, responding on the active lever results in drug infusion (drug taking), and a presented light stimulus becomes a drug conditioned stimulus through Pavlovian conditioning (left panel). Here, compulsive drug taking is defined as persistent responding when the lever press is punished at the same time as drug infusion (right panel). b | In a seeking–taking chained schedule of reinforcement, the animal learns to press a seeking lever under a random interval (for example, lasting 60 s on average, but ranging from 45 to 120 s). The lever press is never reinforced, but gives access to a second, taking lever, pressing on which results in drug infusion (usually after each press). During the test for compulsive seeking, pressing the seeking lever results in either access to the taking lever or mild footshock punishment delivered randomly on completion of half of the trials. Compulsive seeking is measured as persistent responding on the seeking lever under probabilistic punishment¹¹⁸.

In the case of drug taking, each substance has its own range of molecular targets, potentially creating a complex interoceptive experience mediated by a neural network including the insula⁹. Moreover, the increase in the level of mesolimbic DA that is elicited by most addictive drugs mediates reinforcement through distinct cellular mechanisms. For example, cocaine, a potent stimulant, blocks DA reuptake by neuronal terminals, whereas nicotine can directly depolarize DA neurons¹⁰. Alcohol has complex actions, including at GABA receptors and glutamate receptors, that cause DA transients in the NAc¹¹. Opioids, by contrast, inhibit inhibitory interneurons that express the µ-opioid receptor and thereby disinhibit the firing of the VTA DA neurons¹². There are transient increases in DA release in the NAc of mice injected with heroin (visualized using genetically encoded DA sensors)¹³ as a result of enhanced DA neuron activity¹⁴. Whether opioid drug reinforcement depends on mesolimbic DA has been questioned, however, because heroin self-administration persists after DA depletion and during blockade of DA receptors in the NAc^15–17. Furthermore, in individuals addicted to heroin who are receiving methadone substitution, intravenously administered heroin results in a marked euphoric ‘high’ but does not increase the level of striatal DA¹⁸ (reviewed in REF.¹⁹).

The initial mesolimbic DA level increase caused by a single dose of cocaine (and to varying extents by other addictive drugs³) potentiates excitatory afferents onto DA neurons in the VTA, thus leaving a trace that outlasts the drug’s presence in the brain^20,21. This trace can be mimicked by optogenetically activating VTA DA neurons²². Repeated exposure to a drug can also induce synaptic transmission and plasticity in the NAc²³. Excitatory synapses between neurons in the medial PFC (mPFC) or ventral hippocampus and DA D1 receptor (D1R)-expressing medium spiny neurons (MSNs) in the NAc are strengthened²⁴. This form of plasticity has been linked to locomotor sensitization and cue-elicited reinstatement of taking responses and is also observed after optogenetic DA neuron self-stimulation (oDASS)25 (FIG. 3; BOX 1). Afferents onto DA D2 receptor (D2R)-expressing MSNs, particularly those from the basolateral amygdala (BLA), are also potentiated after extended access to high doses of cocaine²⁶.

Escalation of drug intake.

Increase of drug intake during extended (long-access) periods of self-administration.

Fig. 3. Circuits undergoing gain of function with oDASS.

a| Mice expressing channel rhodopsin in dopamine transporter-expressing neurons learn to self-stimulate dopamine neurons in the ventral tegmental area (VTA) — known as optogenetic dopamine neuron self-stimulation (oDASS). b | Once acquired, these adaptive behaviours are observed in all animals and depend on a potentiation of medial prefrontal cortex (mPFC) and ventral hippocampus (vHipp) to nucleus accumbens (NAc) afferents, particularly onto medium spiny neurons (MSNs) expressing dopamine D1 receptors (D1Rs). Synapses between basolateral amygdala (BLA) neurons and their dopamine D2 receptor-expressing targets in the NAc may also undergo potentiation, akin to observations with extended-access cocaine self-administration. c | In the last stage of the test, every third lever press is punished, yielding two distinct groups: one that perseveres with oDASS and the other that ceases oDASS. Ex vivo quantification of the synaptic strength of the orbitofrontal cortex (OFC)-to-dorsomedial striatum (DMS) projection shows enhanced connectivity in compulsive mice. Similar neural changes were observed in a seek–take version of oDASS. SNc, substantia nigra pars compacta.

Many drugs, particularly opioids and alcohol, cause an aversive withdrawal syndrome on the abrupt termination of long-term use that the animal learns to avoid (that is, through negative reinforcement)²⁷. The circuit adaptations that mediate this negative reinforcement are triggered by stress mediators, such as corticotropin-releasing factor and noradrenaline in the extended amygdala, as well as by adaptations in the NAc itself²⁸. For example, an upregulation of the dynorphin–κ-opioid receptor system in the brains of individuals addicted to alcohol or opioids has been argued to underlie dysphoria associated with withdrawal²⁹. Notably, stress mediators eventually lead to a potentiation of excitatory projections onto D2R-expressing MSNs, such as those from the paraventricular thalamus, from the lateral or medial habenula or from the BLA^30,31.

These initial positively and negatively reinforcing adaptations are prevalent as a result of drug self-administration and withdrawal, but do not necessarily reflect compulsion. However, they may contribute to the loss of control over drug taking (BOX 1) and the transition to compulsive drug taking or compulsive drug seeking.

Compulsion is central to addiction

Learning theory and the dopamine system.

Focus on a behavioural manifestation of addiction makes possible a useful theoretical analysis in terms of learning theory, which has helped to bridge computational accounts of behaviour and their implementation by discrete neural systems in the brain (BOX 3). Much of this learning theory was originally established in studies with experimental animals, but it has translated remarkably well to behavioural and functional neuroimaging studies of addiction in humans.

This learning theory perspective posits that addiction is a consequence of alterations in the activity of limbic– corticostriatal circuits (FIG. 1) that arise in part from the excessive DA modulation described above. Studies using positron emission tomography show that, whereas recreational stimulant use (without addiction) increases the release of DA in the ventral striatum³², addiction is associated with increased DA release in response to drug-associated cues in the putamen (which together with the caudate makes up the dorsal striatum)³³. This cue-elicited DA release occurs despite the evidence that striatal dopaminergic transmission is blunted overall in people who are addicted^34,35.

Burgeoning evidence suggests that addiction to drugs is associated with a general bias to a habitual (also known as ‘model-free’) mode of behaviour, as distinct from goal-directed (or ‘model-based’) behaviour. Habitual behaviour is generally associated with activity in the dorsolateral striatum (DLS; or the putamen in primates), whereas goal-directed behaviour is associated with activity in the dorsomedial striatum (DMS; or the caudate in primates) and the ventral striatum³⁶ (FIG. 1). This is true of addiction to stimulants^37,38, nicotine³⁹ and alcohol⁴⁰. In individuals addicted to alcohol⁴⁰, brain areas implicated in goal-directed action (namely the ventromedial PFC and anterior striatum) were less active than in controls, whereas the posterior putamen, which is implicated in habit learning, was more engaged. These findings may be consistent with evidence of larger putamen volume in stimulant drug abusers and their relatives, although the functional link of such differences with habit learning has yet to be demonstrated⁴¹.

Reward thresholds.

The minimal stimulation intensities required to produce a reinforcing effect.

Natural reinforcers.

Rewards such as food and sex that motivate behaviour in animals and humans. They may be distinguished from artificial rewards such as addictive drugs that may nevertheless depend on the same neural systems in the brain.

Interoceptive.

Relating to internal, or bodily, states that are interpreted by the brain through a process called ‘interoception’. In the case of drugs, bodily changes (such as increases in heart rate caused by stimulants) are an important component of the subjective effects of drugs.

Stimulant.

A drug that increases arousal and activity. Amphetamine is a typical example of a stimulant, sometimes called a ‘psychomotor stimulant’.

Locomotor sensitization.

Enhanced motor responses to the same does of a stimulant drug that follows intermittent, repeated dosing.

Cue-elicited reinstatement.

Reinstatement of performance of previously extinguished drug-taking responses, supported by drug cues acting as conditioned reinforcers.

Extended amygdala.

Neuroanatomical term that includes the centromedial amygdala, bed nucleus of the stria terminalis and, according to some, the shell of the nucleus accumbens and a group of neurons in the basal forebrain that links these structures.

Dysphoria.

A state of unhappiness or suboptimal mood in humans.

Pavlovian–instrumental transfer.

(PIT). Transfer of learning whereby conditioned stimuli associated with a reward can increase a separately trained instrumental response for that reward (specific transfer) or for other rewards (general transfer).

Compulsive behaviour can be understood as a narrowing of the range of goals that persists in the face of adverse consequences. This apparent goal devaluation, including tolerance to the euphoric effects of the drug, further suggests that addictive behaviour itself indeed has habitual qualities. A major question arising from the learning theory perspective is therefore whether compulsive drug taking (which is synonymous with compulsive drug use) and compulsive drug seeking (or ‘foraging’, which often occurs when resources are scarce and at personal risk) are goal-directed or habitual. Moreover, habit-like behaviour can also be perseverative to the extent that it can be said to be ‘out of control’. Thus, to relate habits to compulsion, a second factor — that of a lack of top-down executive control over behaviour — probably also has to be postulated⁴².

Loss of top-down control.

Such a lack of top-down control can readily be postulated in addiction (but see the ‘gain-of-function’ scenario discussed later). Impairments in frontal functioning in addiction may result either from effects of the drugs themselves or constitutional, predisposing factors^43,44. Thus, for example, loss of grey matter in the orbitofrontal cortex (OFC) has been observed as a function of the duration of drug use in addicted individuals, whereas a loss of white matter in the right inferior frontal gyrus, associated with impairments in response inhibition, has been observed not only in stimulant-addicted individuals but also in their non-drug-taking siblings^44,45.

Recreational drug taking does not inevitably lead to fronto-executive impairments⁴⁴ nor to addiction; only a relatively small proportion of people who use stimulant drugs, for example, become addicted⁴⁶, and individual differences must therefore also be taken into account. However, for those individuals who do become addicted, there must presumably be some progressive change in the functioning of neural circuitries that mediates learning processes and their executive control.

Sign tracking.

Behaviour whereby the animal approaches a conditioned stimulus predictive of reward — as opposed to approaching a reward (or goal) directly (‘goal tracking’).

Contingency degradation.

Degradation of the predictive relationship between responses and outcomes; for example, by presentation of ‘free’ (that is, response-independent) outcomes or extinction.

Quinine.

A bitter crystalline compound present in cinchona bark that is used to adulterate an otherwise readily ingested liquid.

Compulsive drug taking

Compulsive drug taking refers to self-administration in the face of punishment or aversive consequences; for example, persistent drug taking despite mild footshock or, in the case of an orally consumed drug such as alcohol, adulteration (for example, with quinine).

In the three-criteria model of addiction to stimulants⁴⁷ (BOX 1), rats were considered to show ‘addiction-like behaviour’ when they exhibited each of the three main behavioural features of addiction: first, a reduced ability to inhibit taking responses when cocaine was signalled as available; second, a heightened motivation for cocaine as assessed under a progressive ratio schedule of reinforcement; and third, persistent cocaine-taking responses when cocaine infusions were punished by footshock (compulsion). Rats showing all three criteria were classed as showing addiction-like behaviour. Rats showing the first and second criteria were not necessarily compulsive. Hence, rats showing none, one or two of the criteria made up the majority of the population and could be described as resilient⁴⁷. Unbiased clustering algorithms for several behavioural parameters after the introduction of punishment have arrived at similar classifications in oDASS studies⁴⁸.

There are large individual differences in the three-criteria model; only about 20% of rats that have an extended history of cocaine self-administration are compulsive in this sense⁴⁷, matching the human data⁴⁶. Furthermore, in rats showing addiction-like behaviour, long-term depression in the NAc was permanently impaired, whereas in non-addicted rats that maintained controlled drug intake, long-term depression progressively reduced⁴⁹. Further studies will be required to identify the circuitry and mechanisms involved in these behavioural and plasticity changes. Notably, measuring compulsive opioid taking in the three-criteria model is not possible, because opioid drugs exert analgesic effects that confound any interpretation of perseverance in the face of punishment.

There has been surprisingly little analysis of the goal-directed or habitual nature of this drug-taking behaviour or its neural basis (FIG. 1). Nevertheless, some evidence suggests that in rats showing compulsive methamphetamine taking (as defined by the three-criteria model), there is an imbalance in the activity of different corticostriatal circuits, with more activity in OFC–DMS connections and less engagement of the mPFC– ventrolateral striatum circuitry⁵⁰. Moreover, rats that initially choose alcohol over saccharin go on to display compulsive alcohol drinking, as assessed by resistance to footshock punishment or by perseverance despite quinine adulteration of alcohol⁵¹. In the latter study, expression of the GABA transporter GAT3 was selectively decreased in the brains of rats that preferred alcohol, and GAT3 expression is also decreased in the central amygdala (CeA) of alcohol-dependent humans, suggesting that pre-existing differences in GABAergic mechanisms are involved in the development of compulsive alcohol use. By contrast, a study of alcohol drinking in mice showed that a decrease in the activity of an mPFC–dorsal periaqueductal grey pathway during initial alcohol exposure predicted greater tolerance for quinine adulteration and compulsion (although the distribution of alcohol drinking remained unimodal)⁵². Further studies may reveal how this PFC–brainstem projection interacts with the circuits described above in the emergence of compulsive alcohol taking, perhaps implicating the CeA, which projects to similar brainstem regions as does the PFC⁵³.

Progressive ratio schedule.

A behavioural procedure whereby the number of required responses increases after each reward delivery. The number of responses at which the animal ceases to respond is called the ‘break point’.

Analgesic.

Relating to the effects of a drug that relieves pain (for example, morphine).

Value updating.

The perception of a change in value of a reinforcer after an antecedent manipulation; for example, the value of food is decreased following ingestion of the food to satiety.

Incubation of cocaine craving.

The increase in instrumental responding for a drugassociated conditioned stimulus that occurs the longer the period of abstinence from drug taking.

Another tool for studying compulsive responding with primary reinforcement is that of mice trained in an oDASS protocol in which a mouse’s attempts to self-stimulate VTA neurons are punished on a certain proportion of occasions with footshocks^25,48 (FIG. 3). In a yoked punishment design in which the total number of electric shocks received by each group was matched (so that quantification of shock-related neuronal expression of the immediate early gene Fos was unbiased), the activity of OFC projections to the dorsal striatum was enhanced in compulsive mice compared with non-compulsive mice. As in the three-criteria stimulant-addiction model, there were individual differences in punishment resistance, such that a bimodal distribution emerged in which about 50% of mice persisted in responding when oDASS was associated with footshock. Thus, strong activation of mesolimbic DA neurons can overcome the suppressant effect of punishment, and this effect may be related to compulsive stimulant taking.

Indeed, the enhanced activity of OFC–dorsal striatum projections in animals that compulsively selfadministered methamphetamine⁵⁰ also emerged in mice exposed to a sensitizing regimen of non-contingent, systemically administered cocaine⁵⁴. Furthermore, optogenetic inhibition of OFC neurons that project to the dorsal striatum in compulsive rats decreased responding, mirroring the permanently potentiated OFC–dorsal striatum transmission in punishment-resistant mice. Artificial potentiation of OFC–dorsal striatum synapses (through high-frequency optogenetic stimulation) also caused compulsive responding in punishment-sensitive mice, whereas depotentiation of these synapses in compulsive mice had the converse effect⁴⁸.

Given the likely role of the OFC in goal-directed behaviour and value updating, the oDASS model might represent a form of compulsive responding that has a goal-directed basis. A recent study argues that OFC neurons specifically projecting to the dorsal striatum encode the integrated value of reward⁵⁵. Thus, in the oDASS compulsion model, a gain of function in the OFC–striatal pathway might promote overestimation of the value of the drug experience relative to punishment, biasing instrumental behaviour towards drug consumption.

Controlled and compulsive drug seeking

Controlled drug seeking.

The acquisition (that is, early stages) of cocaine seeking that explicitly relies on a drugconditioned stimulus to reinforce responding depends on circuitry that links the BLA and the NAc core56–58 (FIG. 1). Hence, selective lesion or inactivation of either the BLA or the NAc, or their disconnection, has no effect on cocaine self-administration, but impairs or abolishes the acquisition of cocaine seeking under a second-order schedule of reinforcement^56–58. This difference emphasizes the distinction between drug taking (or the reinforcement of taking) and drug seeking at a neural circuit level. It is also consistent with the established function of this circuitry to mediate conditioned reinforcement⁵⁹, which is the process that also underlies cued reinstatement of extinguished responding (a relapse procedure⁶⁰) and the incubation of cocaine craving⁶¹ (BOX 1).

Synaptic potentiation of excitatory afferents selectively onto D1R-expressing MSNs may represent a cellular mechanism of cue-associated seeking and locomotor sensitization^24,62. Daily cocaine exposure during a short interval (less than 3 hours) potentiated afferents from the mPFC and ventral hippocampus, whereas BLA afferents, especially those that projected to D2R-expressing MSNs, underwent plasticity only with extended cocaine access²⁶. These drug-adaptive responses may relate to the physiological role of these projections. Neurons in the BLA-to-NAc projection that express the marker cholecystokinin (CCK) synapse preferentially onto D2R-expressing MSNs to encode aversive stimuli⁶³. Similarly, the expression of the genes Ppp1r1b and Rspo2 marks neurons preferentially activated by reward-related or aversion-related stimuli, respectively^63,64. It may be of interest to investigate whether these projections that encode aversive stimuli are involved in negative reinforcement.

Taken together, these findings indicate a circuit and synaptic basis of drug-seeking behaviour that is dissociable from the neural mechanisms of drug reinforcement that depend on activity in the NAc and underlie drug-taking behaviour.

From controlled to compulsive drug seeking.

To understand the distinction between controlled and compulsive drug seeking, it is important to consider two theoretical points. First, there may be an imbalance between the control of goal-directed actions versus habits over drug-seeking behaviour. Second, such an imbalance might represent enhanced habit formation together with decreased executive control over the performance of drug-seeking habits, or enhanced motivated performance of, and loss of executive control over, goal-directed actions. These potential theoretical explanations of compulsion were previously more vividly conceptualized as a distinction between the ‘must do’ and ‘must have’ nature of drug seeking⁶⁵.

The neural circuitry that underlies goal-directed actions and stimulus–response habits has been identified through detailed analyses of instrumental responding for food rewards. In such analyses, devaluing the reinforcer or degrading the contingency does not prevent food-seeking behaviour in some animals, signifying a shift to habitual behaviour. This identified circuitry therefore provides the backdrop for studies of drug seeking^66–68 (FIG. 1; BOX 3).

Direct pathway.

Projection of dopamine D1 receptor-expressing medium spiny neurons in the striatum to the midbrain. The indirect pathway involves a striatal projection of dopamine D2 receptor-expressing medium spiny neurons to the pallidum.

Consistent with its goal-directed function, the posterior DMS (pDMS) in rats is required for the acquisition of cocaine seeking⁶⁹ and for alcohol seeking at a time when responding is still sensitive to reinforcer devaluation⁷⁰. Optogenetic inhibition of direct pathway MSNs (which express D1R) in the DMS decreased responding for access to alcohol⁷¹, whereas optogenetic stimulation of corticostriatal inputs in vivo at frequencies that induced long-term depression in DMS slices increased responding for alcohol — an effect that was blocked by a D2R antagonist, indicating a link between neuronal plasticity in the DMS and goal-directed drug seeking⁷¹.

Nevertheless, much converging evidence suggests a prominent role of the anterior DLS (aDLS; putamen in primates) in the transition from drug taking to compulsive drug seeking in the absence of the drug^42,65,72. This role is especially supported in studies of drug cue-related reactivity in functional MRI studies of addiction in humans. Whereas people who drink alcohol or take cannabis on a recreational basis show increased activation in the ventral striatum on seeing drug-related cues, individuals who are addicted to alcohol, cannabis or stimulants exhibit enhanced cue reactivity in the dorsal striatum^73–75. Furthermore, in primates, long-term alcohol exposure resulted in increased dendritic spine density and enhanced glutamatergic transmission in the putamen, but not in the caudate (DMS)⁷⁶. The excitability of striatal neurons was also enhanced in monkeys following heavy drinking, whereas GABAergic transmission was selectively suppressed in the putamen of these monkeys. These changes together indicate that prolonged heavy drinking results in a shift in the balance of inhibitory and excitatory transmission that promotes synaptic activation of putamen output neurons⁷⁶.

Similarly, when rats learn over the course of several weeks to seek cocaine maintained by presentations of a cocaine-related cue, aDLS circuitry has a dominant role in controlling performance. Extracellular DA levels in the aDLS, but not in the NAc, are correlated with well-established cocaine-seeking behaviour⁷⁷. Moreover, DA receptor blockade in the aDLS, but not in the NAc or pDMS, reduces cocaine seeking, but not taking, behaviour in animals that had been performing this behaviour over several weeks^69,78. These observations imply that seeking eventually becomes reliant on the habit system. In the same task, heroin-seeking behaviour, over an extended course of training, similarly becomes dependent on dopaminergic transmission in the aDLS⁷⁹.

Seeking–taking chained procedures are tasks in which animals must make one instrumental response (such as a lever press) to ‘seek’ the drug that enables them to then perform another distinct action, such as a nose poke or a different lever press to ‘take’ the drug (resulting in an intravenous drug infusion or the opportunity to drink alcohol or another addictive drug). One study trained animals to perform a seeking–taking chained procedure for alcohol and observed the same shift from pDMS to aDLS control in 4–7 weeks. This change was associated with a loss of sensitivity to reinforcer devaluation, indicating a transition from goal-directed to habit control⁷⁰. Similarly, with the same behavioural paradigm, cocaine seeking became dependent on the aDLS and insensitive to reinforcer devaluation. When the aDLS was then inactivated, taking the habit system offline, goal-directed cocaine seeking sensitive to devaluation was reinstated⁸⁰. Together, these observations imply that DMS-dependent goal-directed seeking behaviour is progressively dominated by habitual seeking behaviour that relies on DLS activity.

It is neither necessarily aberrant nor maladaptive for the instrumental seeking of drugs, or of any reward, to become habitual. In normal circumstances, habitual seeking of a food reward can return rapidly to goal-directed control in the presence of negative outcomes or negative feedback^72,81. As Gremel et al.⁶⁸ emphasize, it is not an individual who is either goal-directed or habitual; rather, it is the nature of control over an individual’s instrumental actions within a particular context or in the presence of particular conditioned stimuli. Although drug seeking may initially become habitual, it is the transition to compulsive drug seeking that establishes it as maladaptive, and this transition may entail a loss of executive control over its performance and a loss of goal-directed behaviour that also ensures its dominance. It is perhaps the difficulty in relinquishing drug-seeking habits and making possible a return to goal-directed control, as is readily the case for food-seeking habits, that further characterizes the maladaptive nature of such drug-seeking habits⁸².

Underlying the ventral-to-dorsal striatal transition in the control over habitual cocaine seeking^65,83 is a spiralling pattern of connectivity between the midbrain and the striatum in primates⁸⁴ and in rats⁸⁵ (FIG. 1c). DA neurons of the medial VTA project to the medial NAc shell, from where GABA D1R-expressing MSNs project back to GABA interneurons in the midbrain, which can inhibit more laterally located DA neurons. These DA neurons, in turn, project to the core and eventually to the lateral shell, of the NAc. After several loops, this disinhibitory motif reaches the substantia nigra pars compacta (SNc), which sends its dopaminergic axons to the dorsal striatum. The mPFC exerts potent top-down control on the early loops of the spiral, whereas the OFC controls loops that originate in the SNc, suggesting the sequential involvement of the mPFC and OFC during the emergence of compulsion^84,85.

This model of functional interaction between the ventral striatum and the dorsal striatum is supported by the finding that disconnecting the NAc core and the aDLS (by disabling each structure unilaterally but in opposite hemispheres) disrupts well-established cocaine seeking as effectively as does bilateral blockade of DA receptors in the aDLS⁸⁶. Moreover, DA release in the aDLS, as measured by in vivo voltammetry during conditioned-stimulus-elicited cocaine seeking, was shown to depend on the integrity of the ipsilateral NAc core⁸⁷. Electrophysiological stimulation of the medial ventral striatum was also demonstrated to alter the activity of VTA and SNc neurons projecting to the DLS^88,89. However, the specific synaptic connectivity underlying the operation of this circuitry has yet to be fully investigated.

Thus, although aDLS circuitry becomes dominant in controlling well-established drug seeking, its engagement depends on connectivity with the ventral striatum. This provides a mechanism by which incentive processes, including Pavlovian association and conditioned reinforcement, dependent on activity in the NAc, can influence instrumental behaviours that are controlled by the dorsal striatum^90,91. Recruitment of DA-dependent aDLS control over cocaine seeking depends on the BLA, whereas maintenance of the cocaine-seeking habit depends on the CeA and its DA-dependent functional interaction with the aDLS⁹², paralleling similar CeA–aDLS circuitry involved in habitual responding for food⁹³. The amygdala and aDLS are not directly connected, however, so the circuitry mediating this functional interaction must be polysynaptic. Indeed, this has been demonstrated in vivo: stimulation of the BLA bidirectionally modulated aDLS MSN activity via glutamatergic mechanisms in the NAc core⁹². The pathways functionally linking the CeA to the DLS may involve direct projections from the CeA to the SNc that are also required for Pavlovian conditioned orienting towards food-conditioned stimuli⁹⁴.

An imbalance between goal-directed and habitual drug seeking, with eventual dominance of the habit system, is much less evident in the control over drug taking, where, as we have seen, the NAc and DMS may retain a dominant role. This difference may reflect the relative predictability of the drug outcomes for taking versus seeking: whereas the relationship between drug-taking actions and outcomes is highly predictive of the drug effect, drug seeking less reliably predicts what are often delayed drug experiences⁹⁵.

One cause of the progressive imbalance of systems controlling drug seeking may be that addictive drugs enhance the consolidation of habit learning in the DLS and its associated circuitry^42,81. This is supported by findings that amphetamine, alcohol and nicotine can each accelerate the development of habitual control of seeking of natural rewards in animals^70,96,97 and humans⁹⁸. Moreover, habitual seeking responses for cocaine, alcohol and nicotine become resistant to reinforcer devaluation (implicating the habitual system) more rapidly than does responding for a food reward^70,99–101.

At a neural circuit level, this shift from goal-directed to habitual responding has been suggested to reflect an imbalance between the DMS and the DLS¹⁰². Consistent with this, inhibition of OFC projections to the DMS results in rats being entirely reliant on habit circuitry when goal-directed circuits would usually be dominant, and selective attenuation of glutamatergic transmission at OFC terminals in the DMS prevents the use of goal-directed behaviour^11,68. Cannabinoid receptors (CB1 receptors) play a key gatekeeper role on OFC terminals in the DMS, as their deletion completely prevents the transition from goal-directed to habitual control over instrumental seeking behaviour⁶⁸.

Drug-seeking habits might be a prerequisite sub-strate for compulsive behaviour to manifest, rather than — or in addition to — any bottom-up modulation of goal-directed behaviour by the motivational impact of processes such as sensitization or withdrawal, which may also modulate habit-based control. However, another likely mechanism for habit dysregulation is a loss of top-down executive control.

Loss of goal and control in compulsive drug seeking.

Impairments to top-down control over drug-seeking habits and impairments to the goal-directed system are emergent dysfunctional factors that, we hypothesize, jointly result in compulsive drug seeking^65,68. Evidence of frontal lobe involvement in such processes is available from studies in humans⁴¹ and in experimental animal models¹⁰³. For example, chronic stimulant selfadministration in rhesus monkeys causes substantial reductions in cortical glucose metabolism¹⁰⁴, and amphetamine self-administration reduces spine density in the OFC of rats¹⁰⁵. In humans addicted to cocaine, there is grey matter loss in widespread regions of the anterior cortex — including the OFC, inferior frontal cortex, cingulate, temporal gyrus and insula — as a function of stimulant use⁴¹, as well as hypometabolism in the frontal and cingulate cortices⁴³.

Although the cellular correlates remain to be determined, such anatomically diverse frontal deficits can produce various distinct impairments in executive control. These include loss of goal representations following ventromedial PFC impairments (which may competitively advantage habits) and impairment of inhibitory control over both drug-seeking and drug-taking behaviour as a consequence of right inferior frontal cortex dysfunctional involvement, regardless of whether such behaviour is predominantly goal-directed or habitua^l45,106.

These impairments in executive control probably contribute to major decision-making deficits^107,108, which may in turn exacerbate the drive to compulsive drug seeking and addiction. The loss of inhibitory control over habits may be especially important in this regard. Recent evidence¹⁰⁹ from motor control theory has shown that, for simple actions, competing habitual and goal-directed responses occur in parallel, but that habits are automatically prepared at short latency. In the normal situation, a prepared habit can be held in check, to allow the slower, more reflective, goal-directed process to override it and occur instead. However, if this top-down inhibitory control is lost, the habitual tendency would ‘win’ the control over response output.

One challenge for investigating the neural circuit basis of compulsive drug seeking in animal models is that only a relatively small proportion of individuals develop this behaviour and, even then, only after an extended history of drug taking¹¹⁰. This strongly suggests that compulsivity is also a consequence of the effects of drugs administered over the long term on neural circuits that only manifests itself in vulnerable individuals^{47,51,111–113}. Consistent with this view are findings from a study¹¹⁴ in which the function of frontal structures implicated in the top-down control over drug seeking was experimentally impaired before initial drug self-administration. Lesions of the OFC or the anterior cingulate, prelimbic, infralimbic or anterior insular cortex did not result in either more rapid development of compulsive drug seeking or larger numbers of individuals displaying compulsion compared with non-lesioned controls. This strongly suggests that any impairments in cortical top-down control are emergent, as is also evident in human imaging studies⁴¹. Functional impairment of the lateral habenula was also found to have no effect on compulsive drug seeking¹¹⁵, despite its purported role in processing negative feedback¹¹⁶.

The most direct experimental evidence demonstrating that impaired cortical function can be a result of long-term drug exposure and causal in the development of compulsive drug seeking comes from studies in rats deploying a cocaine seeking–taking chained procedure with probabilistic punishment of seeking responses¹¹⁷. In confirming the original observation¹¹⁸ that compulsivity develops only in a subgroup of rats, it was shown that prelimbic neurons become markedly hypoactive after a history of long-term cocaine taking. Moreover, optogenetic stimulation of the quiescent prelimbic cortex in compulsive, punishment-resistant rats reduced their compulsive cocaine seeking, whereas optogenetic inhibition of the prelimbic cortex in non-compulsive rats increased seeking responses in the face of punishment¹¹⁷. However, the prelimbic cortex is implicated in pain perception¹¹⁹, and changes in sensitivity to footshock following optogenetic manipulation of the prelimbic cortex were not assessed or controlled for in this study; thus, it is possible that such changes could have contributed to punishment-resistant responding.

These data strongly suggest that hypoactivity in the prelimbic cortex at a time when cocaine seeking is insensitive to reinforcer devaluation (and is therefore habitual and aDLS dependent⁸⁰) is causally involved in compulsive drug seeking. Moreover, in rats, seeking under punishment in the seeking–taking task was shown to depend specifically on a discrete zone of the aDLS¹²⁰. Prelimbic neurons project to several subcortical sites, including the NAc core and the DMS, and so it is plausible that decreasing compulsive cocaine seeking by optogenetic stimulation of the prelimbic cortex is associated with re-establishment of goal-directed control over seeking behaviour, whereas optogenetic inhibition of this structure impairs the goal-directed system, leaving the habit system to dominate seeking behaviour. Supporting this view, long-term intermittent ethanol exposure and withdrawal was shown to disrupt top-down control over goal-directed action-selection processes, producing habits¹²¹, and to lead to selectively reduced OFC output to the direct output pathway in the DMS. By contrast, increasing the activity of OFC circuits restored goal-directed behaviour in these animals.

An alternative hypothesis to the model in which changes in prelimbic deficits promote compulsive drug seeking is that, by analogy with the models of compulsive drug taking outlined above, a ‘gain of function’ in the OFC leads to a revaluation of the reinforcing value of the drug versus the shock, in favour of the drug, leading to compulsive drug seeking. Preliminary supporting evidence comes not from drug seeking but from a modification of the oDASS mouse model²⁵ in which seeking responses in a seeking–taking chain were punished¹²². OFC–dorsal striatum synapses were potentiated in compulsive mice (that is, mice whose seeking responses were resistant to punishment) compared with OFC–dorsal striatum synapses in non-compulsive or naive mice. By contrast, mPFC–medial dorsal striatum synapses in the compulsive mice remained unchanged. These observations suggest a pathological gain of function in parts of the PFC — including the OFC — contrasting with the results involving long-term drug self-administration summarized above. This gain-of-function model may represent an early phase of drug use as it does not capture the end state of stimulant addiction, which is characterized by hypometabolism of the OFC⁴³, loss of grey matter throughout the PFC⁴¹ and presumed loss of PFC function, as exemplified by profound deficits in cognitive and executive functioning^107,108,123.

The hypothesis that compulsive drug seeking is associated with an impaired ability to relinquish, rather than to engage, aDLS control over the seeking of alcohol was tested in a seeking–taking procedure with probabilistic punishment⁸² (FIG. 4a,b). In most rats, seeking but not taking behaviours became sensitive to (that is, were reduced by) DA receptor blockade in the aDLS over time. However, only a small proportion of rats — those in which the aDLS controlled seeking responses — were compulsive, persisting in alcohol seeking in the face of punishment. The non-compulsive individuals, which could inhibit alcohol seeking under punishment, were also able to disengage aDLS control over responding. By contrast, compulsive rats were unable to disengage the aDLS strongly. These results suggest that engaging the aDLS habit system is required for compulsive alcohol seeking to develop, and that an inability to disengage the aDLS might define the maladaptive nature of drug-seeking habits in addiction^81,82,124. A key hypothesis to be tested is that disengagement of the aDLS in individuals resistant to developing compulsive alcohol seeking is accompanied by re-engagement of the DMS-dependent goal-directed system^72,82.

Fig. 4. Two examples of the emergence of compulsion in rodents.

a | Failure to disengage the dorsolateral striatum (DLS) reflecting compulsive alcohol consumption. Rats are trained to seek alcohol under a seeking–taking chained schedule of reinforcement (similar to that in FIG. 2b but in which, during training, pressing the taking lever leads to presentation of ethanol from a drinking port). Alcohol reinforcement depends on dopamine (DA) in the nucleus accumbens (NAc), and the acquisition of seeking responses depends on DA in the dorsomedial striatum (DMS) (both shaded blue in the striatal schematic on the left). The maintenance of well-established alcohol seeking loses its dependence on the DMS and becomes dependent on dopaminergic transmission in the anterior DLS (aDLS) (shaded dark blue in the striatal schematic on the right). b | A vulnerable subgroup of rats persists in seeking alcohol even when the seeking lever is probabilistically punished; that is, they are compulsive. Rats that seek alcohol compulsively are unable to disengage the aDLS; in these rats, seeking remains sensitive to DA receptor blockade in the aDLS. By contrast, rats that refrain from seeking lose their sensitivity to aDLS DA receptor blockade; that is, their seeking responses are no longer decreased by this manipulation as was the case earlier in training, when drug seeking was goal-directed. The development of compulsive alcohol seeking is therefore predicted both by engagement of the aDLS and by an inability to disengage it when compulsive, suggesting the maladaptive nature of compulsive alcohol-seeking habits. c | Optogenetic DA neuron self-stimulation (oDASS) results in the emergence of a bimodal distribution of compulsive seeking behaviour. Light stimulation leads to strong activation of the mesolimbic pathway and saturating DA transients in the NAc. Here, a seeking–taking chained schedule is illustrated during which medial prefrontal cortex (mPFC) control of the NAc undergoes synaptic plasticity through DA modulation of glutamatergic afferents. d | During the test phase, pressing the seeking lever is punished, which results in some animals renouncing pressing the taking lever, whereas others persist in pressing it. Strengthened orbitofrontal cortex (OFC)-to-DMS projections are the neural correlate of compulsive oDASS. DAT, dopamine transporter. Schematic in part b inspired by REF.⁸².

Addressing possible critiques

Although the notion that habitual processes have a role in compulsive drug seeking is consonant with neurobehavioural evidence in animals and many anecdotal accounts of drug addiction ‘habits’, there are important critiques of this position — some arising from misunderstandings of the original proposal and other, alternative, perspectives — that have to be taken into account.

The most basic misunderstanding is that drug seeking is incontrovertibly compulsive at the individual level in all situations and contexts. Although both constitutional factors and exposure to drugs may bias behaviour towards habits rather than top-down goal-directed control, such control is not always absent, and behaviour can be ameliorated by environmental contingencies. Thus, providing alternative goals or reinforcers and engaging choice processes is likely to reduce compulsive seeking, although we suggest it would hardly eradicate it. Some evidence in rats¹²⁵ indicates that drug choices can be combated by making palatable food available as an alternative reward in a discrete trial, concurrent choice procedure. However, although it is certainly true that most rats may prefer the food reinforcer, several studies have shown that a small proportion of animals nevertheless prefer, for example, cocaine¹²⁶ or alcohol⁵¹ — and we argue that this corresponds to the (probably small) proportion of animals destined to exhibit compulsive-seeking tendencies characteristic of addiction. In general, this is congruent with our own evidence that a relatively small proportion of cocaine abusers become addicted⁴¹. A similar point may be made with respect to so-called contingency management of addicted individuals^127,128, which has shown some success. However, these points do not represent evidence against habitual compulsive drug seeking; habitual behaviour would be predicted to be combated by arranging a broader choice of goals.

Alternative accounts of compulsive drug seeking consider a radically distinct notion; namely, that it results from enhanced goal-directed tendencies associated specifically with drugs. This position is perhaps most compatible with the ‘gain-of-function’ oDASS model also reviewed here (FIG. 4c,d). This hypothesis was recently arrived at by Hogarth et al.¹²⁹, mainly on the basis of the suggestion that individuals who use multiple drugs, including individuals who are dependent on both alcohol and nicotine, show control-like goal-directed behaviour, including that for drugs. It is difficult to test this hypothesis adequately in individuals diagnosed with substance use disorders, as shown by the recent study¹²⁹ in which a heterogeneous group of individuals each abusing multiple drugs exhibited reduced motivation for water and food reinforcers, and yet showed normal devaluation when satiated. One possible, parsimonious interpretation of this result is that goal-directed motivation is impaired in these individuals, perhaps as a consequence of a narrowed choice of goals, consistent with behavioural and neural findings of apparent deficits in goal-directed behaviour and enhanced appetitive habitual behaviour in addiction³⁷. Nevertheless, the theoretical possibility that goal-directed behaviour is heightened specifically for drug reinforcers remains to be tested experimentally. Enhanced goal-directed behaviours towards drugs could also represent a transition between drug abuse and addiction, before compulsive drug seeking assumes its habitual qualities.

Another expression of the goal-directed nature of compulsive drug seeking has been suggested to be its apparent flexibility. Achieving some variability in the expression of instrumental behaviour could be taken as evidence against the autonomous (often mistaken for automatic) nature of habits. However, mere variability in motoric expression does not necessarily provide evidence against perseverative tendencies; impaired reversal learning, for example, may involve overdue focus on cues formerly related to reward, regardless of their location. Furthermore, it is clear that ‘motor habits’ can be complex sequences of behaviour, as ‘slips of action’, such as the well-known dramatic example of arriving in one’s office at work rather than another, intended destination, owing to one’s habitual tendency. Moreover, new developments in learning theory postulate so-called successor representations that are intermediate between representations of action–outcome and stimulus– response associations, and that may reflect ‘cognitive habits’¹³⁰ relevant to addiction. ‘Cognitive habits’ would not necessarily require invariant motor outputs and may even be internalized (for example, as obsessive thoughts about drugs).

One study reported an apparent invalidation of the notion that habits are central to compulsive drug seeking using a combination of the seeking–taking paradigm and the three-criteria addiction model in rats. However, this study tested the somewhat different proposition that habits are necessary for ‘the development of addiction-like behaviour in rats’¹³¹. In an elegant puzzle-solving seeking–taking task, rats learned to perform three distinct seeking–taking chains (involving different, heterogeneous sequences of lever pressing, nose poking and wheel turning) to self-administer cocaine. Under these conditions, seeking remained goal-directed (as previously shown^132–134) and was not reduced by DLS DA receptor blockade. Indeed, such DA receptor blockade, rather than reducing the motivation to seek cocaine, actually increased seeking, thereby indicating that the aDLS had in fact been engaged even in this situation, perhaps by removing competition between the habit and goal-directed systems. However, as this study did not go on to test the effects of probabilistic punishment of one or more of the seeking responses in the three chains, it remains unclear whether cocaine seeking can become compulsive when it is goal-directed⁸². Instead, the rats were trained to perform three instrumental chains over a long period, and then the resistance to punishment of a taking response was assessed in the three-criteria addiction model⁴⁷. Although some individuals showed resistance to punishment, the nose-poke taking response tested was completely different to the previously trained instrumental chains. The authors of the study concluded that habits are not required for addiction-like behaviour, although only as assessed in the three-criteria model, which is most relevant to drug taking — consistent with the literature reviewed above suggesting that cocaine-taking responses may persist as goal-directed. However, they failed to test the hypothesis concerning the habitual qualities of compulsive drug seeking.

Contingency management.

A behavioural modification intervention that reinforces desired behaviour (such as abstinence) with incentives.

Reversal learning.

Learning of a reversal of the reward contingencies of two options, reflecting behavioural adaptation to environmental change.

Concluding summary

We have identified neurobehavioural mechanisms that contribute to the transition to compulsive behaviour in addiction to several classes of drugs of abuse, including stimulants such as cocaine and alcohol. We distinguish between compulsive drug-seeking and compulsive drug-taking behaviour, which are frequently, perhaps mistakenly, conflated in the published literature. Both may contribute to the key symptoms of addiction specified in the DSM-5, although drug-seeking behaviour is obviously a defining element. We have further made a distinction between controlled and compulsive drug seeking: the former being in the early stages of drug abuse and goal-directed, and the latter, during the course of addiction and involving progressive habitual control. Compulsive drug seeking hypothetically depends both on increasing involvement of the dorsal striatum and also on the progressive loss of top-down, executive control resulting from a loss of PFC and cingulate cortex function. This transition consists not only of a loss of goal representations, which leads to preferential habitual control, but also of a loss of inhibitory control over habits, complementing a possible strengthening of their bottom-up striatal mediation.

We have taken the opportunity in this Review to incorporate the latest data obtained using the novel approach of direct optogenetic stimulation of VTA DA neurons (oDASS), which, through two different procedures, can result in compulsive seeking and taking behaviours. Such compulsive responding is associated with a gain of function in OFC neurons that drive activity in the dorsal striatum. At first sight, this apparently contradicts the main hypothesis presented here of distinct mechanisms underlying drug seeking and drug use (see also BOX 4 for open questions). For human addiction⁹ and animal models of drug addiction, it is currently unclear how this gain of function relates to the reduction of striatal DA activity^34,135,136 and OFC activity^{41,103,105,137} repeatedly demonstrated in earlier studies of animals and humans — and therefore to the loss of top-down control of striatal function exhibited that results from long-term drug taking. However, there are two possible reasons for the apparent discrepancy. First, it may be that the oDASS procedure reflects an early stage in the transition from compulsive use to drug addiction, after which OFC–striatal gain of function precedes OFC–striatal loss of function. Future studies with addictive drugs will be able to test directly whether a phase of gain of function also occurs in the transition to compulsive drug seeking. Second, drugs (such as amphetamine) might exert less-specific neurochemical actions and even neurotoxic effects at diverse neural loci compared with the highly cell-specific optogenetic stimulation.

Box 4. Outstanding questions.

Although the application of learning theory to our understanding of drug addiction, along with the latest generation of circuit investigations, has advanced our understanding of the emergence of compulsion in addiction, many questions remain unanswered. Here we list some that we feel are within reach of being experimentally addressed soon.

• Is activation of the mesolimbic dopamine system a necessary and sufficient mechanism of reinforcement for all addictive drugs, and what is the role of the dopamine system in the addicted state?

• Can optogenetic self- stimulation of mesolimbic dopamine neurons serve as a tool to reveal the mechanisms of addiction shared by all drugs of abuse?

• How can the subjective experience of drug consumption in humans and its underlying neural circuitry best be studied using operational measures in animals while avoiding anthropomorphic inferences?

• Is there a transition from goal- directed to habitual drug taking in addiction, or does drug- taking behaviour persist as goal- directed? Is there a transition from a gain of function of orbitofrontal cortex to dorsomedial striatum projections to a loss of function between the pre- addicted state and end- stage addiction?

• Does the transition to compulsive drug seeking involve a loss of ‘top- down’ control from prefrontal circuits to striatal circuits? Alternatively, is compulsive behaviour a manifestation of a goal- directed choice of a hypervalued drug over negative consequences?

• What is the utility of concurrent choice procedures in models and the treatment of addiction and in understanding the hypothetical narrowing of goals that occurs in the addiction process?

The latter important point is crucial for evaluating how optogenetic methods may be best utilized in addiction research. They may be especially important for testing predictions based on neuropsychological models (such as in REF.⁶⁵) about likely neural circuitry that can be mechanistically characterized using this approach48. The use of optogenetics and similar circuit interrogation methods may also pave the way to possible new treatments. For example, as drug seeking occurs by definition before obtaining a drug and in order to obtain a drug, it is a key element of the process of relapse after abstinence, as well as of the addicted state. Treatments designed to reduce the drug-seeking phase, rather than drug use or taking, therefore have great potential. A circuit model is emerging that explains addiction as altered strength of synaptic connections, opening the possibility for refining circuit therapies such as deep brain stimulation¹³⁸ or transcranial magnetic stimulation¹³⁹ currently used for other indications to be used to reduce drug-seeking behaviour and enhance clinical success.