Cues conditioned to withdrawal and negative reinforcement: Neglected but key motivational elements driving opioid addiction

Cues associated with negative reinforcement can stimulate relapse and contribute to opioid use disorder.

Abstract

Opioid use disorder (OUD) is a debilitating disorder that affects millions of people. Neutral cues can acquire motivational properties when paired with the positive emotional effects of drug intoxication to stimulate relapse. However, much less research has been devoted to cues that become conditioned to the aversive effects of opioid withdrawal. We argue that environmental stimuli promote motivation for opioids when cues are paired with withdrawal (conditioned withdrawal) and generate opioid consumption to terminate conditioned withdrawal (conditioned negative reinforcement). We review evidence that cues associated with pain drive opioid consumption, as patients with chronic pain may misuse opioids to escape physical and emotional pain. We highlight sex differences in withdrawal-induced stress reactivity and withdrawal cue processing and discuss neurocircuitry that may underlie withdrawal cue processing in dependent individuals. These studies highlight the importance of studying cues associated with withdrawal in dependent individuals and point to areas for exploration in OUD research.

INTRODUCTION

Opioid use disorder (OUD) is a chronic disorder that is characterized by cycles of binge/intoxication, withdrawal, and relapse. A recent study reported that more than 47,000 Americans died in 2017 from overdosing on opioids, including oxycodone, heroin, and fentanyl (1). Chronic opioid use produces somatic signs of withdrawal, such as insomnia, pain/hyperalgesia, diarrhea, nausea, vomiting, restlessness, and sweating, and affective symptoms (termed hyperkatifeia), such as dysphoria, anxiety, and irritability. The presence of these symptoms is hypothesized to motivate drug seeking and taking in both patients with OUD and laboratory animals (2–4). The somatic symptoms of withdrawal typically dissipate within days without treatment or can be ameliorated by pharmacological treatment with clonidine or lofexetine that stimulate α₂-adrenergic receptors. However, affective signs of withdrawal are long-lasting (5, 6). Substitution therapy with buprenorphine or methadone reduces both somatic and affective signs of withdrawal by maintaining opioid dependence. Therefore, a better understanding of the neurobiological mechanisms of environmental stimuli and internal states associated with opioid withdrawal and the alleviation of withdrawal may help identify previously unidentified strategies to decrease relapse propensity in individuals who have been abstinent for weeks, months, or even years.

Cues that are conditioned to drug taking can acquire motivational properties that powerfully stimulate drug seeking and taking through positive reinforcement. Positive reinforcement is defined as an increase in the probability of a response (e.g., drug taking) when the presentation of a reward or reward-associated stimulus occurs as a consequence of that behavior. Cues that are conditioned to the hedonic-like effects of opioids can evoke motivation for the “high” that is associated with drug taking.

However, negative reinforcement is defined as increase in the probability of a response as a result of the removal of a stimulus, such as the aversive effects of drug withdrawal in addiction. Here, cues that are associated with withdrawal in OUD can also acquire motivational properties through conditioning. Cues that predict different aspects of the cycle of drug withdrawal and subsequent opioid consumption to remove withdrawal are hypothesized to have unique motivational contributions that drive continued opioid use.

In the present review, we consider two main roles for environmental stimuli and internal states that are conditioned to opioid withdrawal and its removal by drug consumption. First, in conditioned withdrawal, cues that predict opioid withdrawal can come to cause an opioid withdrawal–like state with their presentation (e.g., exposure to an environment where opioid withdrawal was previously experienced). It is well established that cues paired with opioid withdrawal can produce conditioned withdrawal (7). In a seminal study, methadone-maintained patients who were recovering from OUD received injections of the opioid receptor antagonist naloxone, which precipitates withdrawal, with tone and odor cues. The presentation of these conditioned cues without naloxone eventually produced symptoms of opioid withdrawal (7). Second, in conditioned negative reinforcement, the removal of stimuli paired with an aversive event increases the probability of a response (e.g., termination of environmental stimuli that predict aversion during opioid withdrawal). Despite data from human studies that support a role for environmental stimuli and internal states associated with withdrawal in addiction (8–11), they have largely been neglected in both clinical and preclinical research on OUD.

We recently proposed that negative emotional states (hyperkatifeia) intensify as OUD progresses and drive opioid seeking/taking (5). Here, we review evidence that cues surrounding drug withdrawal and use become increasingly associated with these negative emotional states to drive drug taking in opioid dependence. In this way, continued drug taking shifts from positive reinforcement to negative reinforcement to alleviate the negative affective states associated with withdrawal. In reviewing the relevant literature, we seek to identify gaps in our understanding of negative reinforcement processes in addiction. We also discuss ways in which aversive physical and emotional states may promote learning opioid-related cues in chronic pain, in which cues predictive of pain could drive opioid seeking through conditioned negative reinforcement. We argue that opioid withdrawal cue learning is strengthened in the dependent state as withdrawal symptoms intensify, and these learning processes are distinct from cue learning in the nondependent state. In addition, opioid withdrawal cue learning continues to be relevant in OUD into protracted abstinence, as conditioned withdrawal–evoked aversive affective states can drive drug taking through negative reinforcement and promote opioid relapse. However, few preclinical studies have investigated the neurobiological bases of opioid-cue learning in dependent individuals, which has left important motivational processes in OUD understudied. This knowledge gap may explain some discrepancies in the data from patients with OUD and rodent models.

Although there is limited research on sex differences in humans with OUD, women are more likely to fatally overdose than men (12) and to consume drugs as self-medication for stress or depression (13). Similarly, opioid paraphernalia elicits stronger self-reported craving in abstinent women than in men (14, 15). However, sex differences have generally been neglected in opioid seeking and reinstatement studies in preclinical models, and many opioid studies in rodents thus far have not studied opioid-related cue learning in dependent individuals. Here, we also review studies of sex differences in OUD and highlight the importance of studying opioid-related cue learning in dependent male and female individuals that may better reflect the neuroadaptations that occur in human patients with OUD.

We then discuss brain stress systems that may be implicated in aversive emotional learning and underlie relapse specifically in dependent individuals. In particular, the extended amygdala, which includes the central nucleus of the amygdala (CeA), bed nucleus of the stria terminalis (BNST), and nucleus accumbens (NAc) shell, has been implicated in aversion, opioid-cue learning, and opioid dependence. Although less well studied in addiction research, the periaqueductal gray (PAG) is an output from the extended amygdala that is known to be involved in pain, opioid-induced analgesia, and conditioned fear that may be important for driving adaptive, survival responses in response to stress and aversive stimuli (16).

Collectively, we argue that conditioned withdrawal and consequent conditioned negative reinforcement can drive opioid seeking and relapse in dependent individuals, even after somatic signs of withdrawal have subsided. Research on this subject may identify previously unknown treatments for opioid addiction.

DRUG EXPERIENCE AND THE EMERGENCE OF AVERSIVE AFFECT

The transition from recreational to compulsive drug use has been hypothesized to involve a neuroadaptive process that shifts motivational processing from seeking positive reinforcement to avoiding aversive effects of withdrawal (17). Withdrawal from opioids leads to an intense dysphoric state, termed hyperkatifeia, that includes such symptoms as dysphoria, anxiety, irritability, anhedonia, insomnia, and pain when the drug is no longer available. Both the anticipation of somatic withdrawal and aversive emotional states experienced in withdrawal contribute to preoccupation with obtaining the drug (5). Therefore, motivated drug seeking to alleviate unpleasant emotions, or negative reinforcement, is hypothesized to be a key part of the opioid addiction process (18). Patients with OUD may be particularly attentive to opioid withdrawal–related cues that predict and elicit this aversive emotional state (conditioned withdrawal) and at the same time motivate opioid seeking and consumption to alleviate this state (conditioned negative reinforcement).

The pattern of drug taking or amount of drug that is consumed can influence the degree of opioid seeking and taking and is related to the degree of hyperkatifeia generated during withdrawal (3, 19, 20). Rats that were allowed extended access (6 to 23 hours; dependent) to opioids escalated their intake over a few weeks, which was not apparent in animals that were given short (1 hour; nondependent) access (20, 21). Rodent models have shown that drug experience produces an aversive emotion-like state that co-occurs with an increase in drug taking. For example, rodents that were allowed extended access to opioids were more resistant to extinction and exhibited increases in depression- and anxiety-like behaviors, mechanical pain sensitivity, intracranial self-stimulation thresholds (i.e., decrease in reward function), and the expression of somatic signs of withdrawal compared with rodents that were allowed short access to opioids (3, 4, 20–23). These behavioral differences between dependent and nondependent subjects provide a framework to study the neurobiological mechanisms that underlie the transition of opioid intake between recreational opioid users and individuals with OUD.

CUES ASSOCIATED WITH NEGATIVE REINFORCEMENT LEARNING

Although both pleasant and aversive affective states can co-occur during opioid seeking/taking, neurobiological research on drug-cue processing has largely been considered to reflect the disruption of reward-cue processing. The development of opioid dependence and withdrawal symptoms may alter the opioid-cue relationship, such that cues and contexts become linked with both reward and aversive emotional/autonomic processes that are associated with withdrawal. In OUD, negative reinforcement is hypothesized to increase drug seeking to remove the aversive effects of opioid withdrawal, and some of these aversive effects can persist long into protracted abstinence, such as pain (6). Here, we highlight evidence that shows that environmental stimuli and internal states associated with the emergence of withdrawal and escape/avoidance from aversive withdrawal states can drive learning processes with long-lasting consequences in OUD.

Conditioned withdrawal

Studies with patients with OUD have shown that cues that are conditioned to unpleasant somatic and emotional states during acute withdrawal can acquire aversive properties that signal stress and promote relapse. Wikler (24) first observed that the opioid receptor agonist/antagonist nalorphine that was given after opioid receptor agonist administration precipitated acute withdrawal symptoms. Patients with OUD received multiple daily doses of morphine or methadone followed by nalorphine to induce somatic symptoms of withdrawal. Eventually, saline injections that were substituted for nalorphine could evoke withdrawal symptoms, indicating that these symptoms could be conditioned (25, 26). Building on these studies, previously neutral stimuli could be conditioned to naloxone-precipitated withdrawal (7). In one experiment, men who were in recovery from heroin addiction and maintained on daily methadone were given an injection of naloxone (0.1 mg) that was paired with a 700-Hz acoustic tone and peppermint odor for the duration of somatic withdrawal symptoms. After fifteen 1-hour conditioning sessions, the presentation of the peppermint odor and tone without naloxone was sufficient to elicit both somatic and affective signs of withdrawal (Table 1).

Table 1. The presentation of cues that were conditioned to opioid withdrawal was sufficient to produce somatic and subjective signs of opioid withdrawal in men who were maintained on methadone for OUD.

UR, unconditioned response. Modified from (7), with permission.

Variable	Unconditioned response	Conditioned response
Respiration	Increase	Increase
Skin temperature	Decrease	Decrease
Heart rate	Increase	Increase
Motor responses	Increase	Increase
Pupil diameter	Increase	Changes (nonsignificant)
Subjective reaction	Simulates opioid withdrawal	Indistinguishable from UR at this dose (0.1 mg)

Conditioned withdrawal has been studied in animal models using conditioned place aversion. When contextual cues were paired with injections of naloxone at a dose that was lower than doses that induced somatic withdrawal symptoms, opioid-dependent rodents developed an aversion to the chamber that was previously paired with naloxone (27–29). Conditioned place aversion persisted for up to 16 weeks (30). Thus, the aversive memory of the withdrawal-associated context endured far beyond the period of acute withdrawal. Precipitated withdrawal may enhance the learning of cues that are associated with withdrawal-evoked aversion and improve memory consolidation. Rats that were maintained on morphine with osmotic minipumps and injected with naltrexone immediately before novel object exploration exhibited an increase in object recognition memory 3 days later compared with morphine-naive and vehicle-treated animals (31).

Laboratory animals that are trained to operantly respond for opioids similarly learn to associate neutral stimuli with aversive precipitated withdrawal symptoms. Rhesus monkeys that were trained to self-administer morphine increased their responding when they received injections of the mixed opioid receptor agonist/antagonist nalorphine that were paired with a light cue. This increase in responding persisted when the light cue was presented alone (32). Moreover, rats that were given extended access to heroin and repeatedly treated with low-dose naloxone that was paired with cues increased their heroin self-administration and had elevations of intracranial self-stimulation thresholds with presentation of the cues alone (4). These findings indicate that previously neutral cues paired with precipitated withdrawal decreased reward function and increased the motivation for heroin (3, 4). Furthermore, these cues reinstated lever-pressing following extinction after 2 weeks of forced abstinence (3), demonstrating that the cue memory persisted beyond acute withdrawal (Fig. 1). These studies highlight that cues that are conditioned to aversive emotional states during withdrawal can motivate opioid seeking and taking and that aversive memories that are established during dependence can persist beyond the transient period of somatic withdrawal.

Fig. 1. Conditioned withdrawal cues reinstated heroin seeking after 2 weeks of abstinence and extinction.

An odor cue that was previously paired with naloxone-precipitated heroin withdrawal reinstated lever-pressing following extinction. After 2 weeks of forced abstinence, lever-pressing was extinguished in a 1-day protocol. Rats with a history of short-access (ShA; 1 hour) and long-access (LgA; 12 hour) self-administration sessions increased their lever-pressing for the naloxone-paired (Nx) cue compared with the saline-paired (Sal) cue. The data are expressed as means ± SEM. *P < 0.05. Modified from (3), with permission.

Conditioned negative reinforcement

Several studies have reported in human patients with OUD that avoiding the onset of withdrawal symptoms is a major motivation for continuing opioid use (i.e., conditioned negative reinforcement) (2, 33). For patients with OUD undergoing opioid substitution treatment with buprenorphine or methadone, withdrawal discomfort and greater pain were major concerns when choosing to taper off treatment (34–36). In one study of patients with OUD enrolled in buprenorphine maintenance therapy, 89.9% of the participants reported that concerns about withdrawal were a primary reason for continuing treatment (37). In addition, the degree of concern about stopping buprenorphine maintenance was positively correlated with self-reported desire to avoid physical discomfort (35). Therefore, in human patients with OUD, fear of anticipated withdrawal symptoms and the desire to escape somatic and emotional discomfort are major motivational factors that drive continued opioid use.

Preclinical studies have demonstrated that the induction of dependence and the emergence of withdrawal symptoms allow enhanced opioid seeking in withdrawal. Rats that were implanted with morphine pellets (dependent) exhibited a similar morphine-induced conditioned place preference to rats that were implanted with placebo pellets (nondependent rats) following place conditioning with morphine injections. However, when the rats were re-conditioned 6 to 8 weeks after pellet removal (i.e., in protracted abstinence), dependent rats exhibited an increase in preference for the morphine-paired chamber compared to nondependent rats (38). Similarly, heroin-dependent rats increased their drug seeking during withdrawal but only after they were made dependent through passive injections of heroin and were subsequently allowed to self-administer heroin during withdrawal (39). These studies demonstrate that dependent animals will seek opioids to a greater extent than nondependent animals.

One potential explanation for the greater intensity of drug seeking and taking during withdrawal is that opioids have greater reinforcing properties during withdrawal (39). It is difficult to parse the relative role of positive and negative reinforcement under these conditions, as these processes likely occur in concert. However, there is evidence in humans to suggest that a dysphoric emotional state (hyperkatifeia) emerges during drug withdrawal, and drug taking to alleviate this unpleasant state may be an important motivation for continued drug use in drug dependence. Consequently, negative reinforcement is hypothesized to contribute substantially to OUD because the drive to avoid and seek relief from negative affective states provides additional motivation for continued opioid use (9, 40).

In one elegant study, the threshold for brain stimulation reward using intracranial self-stimulation was measured during the development of heroin dependence in rats that were given short-access (1 hour) or long-access (23 hours) sessions of intravenous heroin self-administration (4). Rats that were allowed long-access heroin self-administration sessions exhibited an elevation in intracranial self-stimulation thresholds compared with their predependence baseline. Thus, the emergence of drug dependence paralleled the emergence of a dysphoria-like state during drug withdrawal. A plausible explanation for this finding is that rats self-administered increasing amounts of drug over long-access sessions in an attempt to maintain a hedonic state, but, in the process, generated increasingly elevated brain reward thresholds during withdrawal. Although rats self-administered more heroin with each successive session, their thresholds did not decrease to their previous baseline levels or to the level of nondependent rats. Thus, if heroin reward efficacy increased in dependence, then it would have driven brain stimulation thresholds to baseline levels or lower, reflecting sensitization of the reward pathway. This interpretation is not supported by the data, and changes in brain reward thresholds with extended access to oxycodone suggest that this also occurs in dependence for other opioids (41).

Thus, the responses that allow “escape” from, or avoidance of, these unpleasant states reflect negative reinforcement. It follows that the successful escape or avoidance of negative physical and emotional effects of opioid withdrawal should result in the suppression of brain stress circuitry and activation of reward circuitry to generate a pleasant, “relief” experience (see below). Such an experience would be expected to imbue cues associated with successful escape or avoidance of negative emotional states with motivational properties.

Chronic pain

The increase in prescription opioid use in patients with acute and chronic pain laid the foundation for an increased number of cases of opioid misuse (42). In 2015, the National Survey on Drug Use and Health found that nearly 92 million adults aged 12 and older in the United States reported using prescription opioids. Among them, 11.5 million had engaged in opioid misuse. The primary reason respondents gave for opioid misuse was physical pain relief (43). Aversive affective states, including depression and anhedonia, often accompany chronic pain, and a high degree of aversive affect increases the risk for misuse of prescription opioids (44). Although individuals may misuse opioids to alleviate both physical and emotional pain (45), physical and emotional pain symptoms are often exacerbated with chronic opioid use (46). As shown in Fig. 2, during acute withdrawal from heroin, individuals had lower pain thresholds compared with individuals during protracted abstinence from heroin and healthy volunteers (referred to as ex-users and non-users, respectively). Although they showed some degree of recovery, ex-users with an average of 2.5 years of protracted abstinence still exhibited greater sensitivity to pain compared with non-users (6). As a result, patients with pain may initially take opioids to attempt to cope with the aversive somatic and emotional consequences of pain, yet chronic opioid use can actually produce pain during withdrawal (hyperalgesia) that can perpetuate the chronic pain state (46, 47).

Fig. 2. Greater pain sensitivity in individuals in withdrawal from heroin.

Individuals during withdrawal from heroin exhibited a decrease in ischemic pain thresholds (withdrawal group; heightened sensitivity) in a submaximal effort tourniquet procedure compared with volunteers who were abstinent for 2.5 years (ex-users) and healthy volunteers (non-users). Participants reported when pain was first felt and when pain became intolerable, and the time (in seconds) was recorded. *P < 0.05. Modified (6), with permission.

Supporting this hypothesis, patients with chronic pain report that they are more likely to consume larger quantities of opioids when feelings of pain or stress/aversion precede drug taking, particularly individuals at high risk for opioid misuse (48). However, these larger doses yielded smaller reported reductions in pain and negative affect as tolerance developed compared to patients at lower risk for opioid misuse. Individuals with heightened pain sensitivity also self-reported greater drug craving when exposed to opioid-related cues, and individual pain sensitivity predicted the degree of opioid craving (49). Similarly, the degree of pain intensity and negative affect correlated with increased opioid craving and rates of opioid misuse in patients with chronic musculoskeletal pain (50). Consistent with these findings, patients with chronic pain who misused opioids reported stronger craving when images of physical pain preceded drug cues, indicating that pain cues can contribute to the drive for drug seeking (51). From these results, the authors concluded that a pain-related prime acts as a second-order conditioned stimulus to augment responses to opioid cues (52). Consequently, one may hypothesize that drug cues trigger drug seeking not only through conditioned positive reinforcement but also through their associated ability to signal the alleviation of somatic and emotional suffering in chronic pain states (i.e., conditioned negative reinforcement).

Despite compelling clinical evidence, few studies have directly investigated the intersection between pain relief and opioid-related cue processing in animal models. Data show that physical pain induces a rightward shift of the dose-response function for opioids, such that animals require more drug to achieve pain relief (53, 54). Animals in an inflammatory pain state induced by complete Freund’s adjuvant exhibited an increase in heroin consumption at a high drug dose, but a decrease in intake and motivation at a lower dose, compared with control rats, which was linked to pain-induced decreased μ-opioid receptor activation of the mesolimbic dopamine system (52) (see below). Rodents with spinal nerve injury required higher doses of morphine to develop morphine conditioned place preference than sham rats (55). As tolerance develops, both the rewarding and analgesic properties of opioids decrease (56), which may contribute to the elevated opioid consumption in rodent pain models following repeated drug use.

Rodent pain models have highlighted that cues associated with relief from physical pain can be highly motivating (57). Animals in acute or chronic pain preferred contexts that were associated with pain alleviation, indicating that pain relief can drive learning and motivation (58, 59).

Sex differences in cue reactivity

Women appear to be particularly vulnerable to stress- and drug cue–evoked relapse to opioid use, but females have been historically underrepresented in both clinical and preclinical studies. Women reported greater aversive subjective effects following acute opioid administration than men (60). As shown in Fig. 3, female patients with OUD who were undergoing methadone or buprenorphine maintenance reported greater opioid craving in response to daily life stress than men (61). Although not tested in OUD, women who were recovering from alcohol use disorder were more likely to attribute relapse to aversive emotional states (62) and self-reported painful internal states, such as feelings of stress/aversion, following relapse (63).

Fig. 3. Women with OUD had greater opioid craving with higher stress severity.

Ecological momentary assessment ratings of craving (1 = no craving) for opioids are shown across ratings of stress. Craving as a function of stress was greater in women than in men. P < 0.0001. Modified from (61), with permission.

Differences in cue reactivity between male and female patients with OUD may result from differential sex-specific responses to stress/aversion. Value-based actions of females favor minimizing negative outcomes over maximizing positive outcomes (64), which may make them more vulnerable to cue-evoked relapse to avoid aversive somatic and affective withdrawal states. Women have been hypothesized to transition more rapidly from initial use to addiction and experience more aversive symptoms during withdrawal than men (65, 66). Given that cues and stressors produce activation of the hypothalamic-pituitary-adrenal (HPA) axis and can stimulate subjective feelings of anxiety/stress (67, 68), women may attribute greater motivational significance to cues because of their conditioned aversive qualities. Consistent with this hypothesis, sex differences have been observed at multiple levels of HPA axis regulation. Women have higher cortisol levels following stressful life events and greater neuroendocrine responses to corticotropin-releasing factor (CRF) administration (69). Therefore, women with OUD may be particularly sensitive to aversive emotional states that are evoked by withdrawal and withdrawal cues and be more susceptible to relapse to alleviate these emotions.

Few rodent studies to date have examined sex differences in opioid taking or reinstatement. In a study investigating non–addiction-related reinforcement processes, female mice were biased toward negative reinforcement relative to positive reinforcement, in which females responded more to avoid an aversive shock than to receive a sucrose reward and were more sensitive to punishment than males (70). Female rodents more rapidly acquired morphine and heroin self-administration (22, 71, 72), self-administered more intravenous heroin (22) and fentanyl vapor (23) in long-access self-administration sessions, consumed more oral oxycodone during self-administration (73, 74), and had higher motivation for fentanyl (75) than males. However, females showed no difference from males in stress-induced (73) or cue-induced reinstatement for oral oxycodone (74). Opioid-dependent female mice also exhibited similar somatic signs of naloxone-precipitated heroin withdrawal to dependent male mice (22). In addition, both male and female rats given extended access to heroin incubated their drug craving during forced but not voluntary abstinence (76). Thus, while there are clear sex differences in rodents during opioid taking, these differences are not as apparent during opioid seeking in animal studies to date (Table 2).

Table 2. Sex differences in opioid consumption and cue reactivity.

ARSW, Adjective Rating Scale for Withdrawal; COWS, Clinical Opiate Withdrawal Scale; VAS, visual analog scale; M, male; F, female.

Subjects	Behavioral measure	Sex differences
Human	Prevalence of heroin or prescription opioid use	M > F
		F > M increased rate of heroin use
	Rate of increased/decreased heroin or prescription opioid use across years (2007–2014)	F < M decreased rate of prescription opioid use (12)
Human; heroin-dependent, in-patient setting	Self-reported craving and physiological measures after viewing: heroin-related imagery	F > M self-reported cravings and sadness, systolic blood pressure, heart rate
	Heroin paraphernalia	F > M diastolic blood pressure, self-reported decreased joy
		M > F self-reported anger (14)
Human; opioid-dependent, buprenorphine tapering clinical trial	Self- and clinician-reported withdrawal symptoms (ARSW, COWS) and craving (VAS)	F > M withdrawal symptoms (ARSW, COWS) and subjective craving (VAS) (15)
Human; healthy volunteers	Self-reported drug effects questionnaire after intramuscular morphine administration	M > F self-reported positives effects and drug liking
		F > M self-reported negative subjective effects (72)
Human; opioid-dependent, methadone- or buprenorphine-treated	Self-reported craving and mood using ecological momentary assessment	F > M self-reported craving for opioids as a function of stress severity
		F > M self-reported craving for opioids in presence of stress and cues (73)
Human; opioid-dependent, morphine-stabilized, in opioid tapering clinical trial	COWS and self-reported withdrawal ratings after intramuscular naloxone injection	More F in high withdrawal phenotype than low
		M = F in naloxone-precipitated withdrawal scores (78)
Mouse; 1 or 6 hours for intravenous heroin	Self-administration	F > M self-administered heroin
	Somatic signs of withdrawal	F = M somatic withdrawal signs (22)
Mouse; 1 or 12 hours for fentanyl vapor	Fentanyl intake during transition from short (1 hour) to long (12 hours) access	F > M intake in the first three sessions
	Escalation across 10 days of 12-hour access	M > F change in intake (escalation slope) across days
	Naloxone-precipitated withdrawal	F > M somatic signs of withdrawal (23)
Rat; 6-hour session for intravenous heroin	Days to acquisition criteria	F > M rate of acquisition (faster to acquire)
	Infusions per session	F = M total intake during acquisition (83)
Rat; 4-hour sessions for intravenous heroin	Infusions per session	F > M infusions at 1.25 or 3.75 μg per infusion
		F = M infusions at 15–30 μg per infusion (84)
Rat; 1-hour session for oral oxycodone	Intake across doses	F > M mg/kg intake at 1.0 mg/ml
	Naloxone-precipitated intake	F > M intake after naloxone injection
	Progressive ratio	F = M breakpoint
	Stress-primed reinstatement	F = M active lever presses (85)
Rat; 2-hour session for intravenous fentanyl	Intake across doses	F > M infusions at 0.32–1 μg/kg dose
	Motivation (demand curve)	F > M demand/essential value at 10 μg/kg per infusion (not 3.2 μg/kg per infusion)
		F = M baseline consumption (87)
Rat; 6-hour session for intravenous oxycodone	Buprenorphine effect on reinstatement	Buprenorphine reduced reinstatement in F but not in M
	Buprenorphine effect on reacquisition of self-administration	Buprenorphine reduced reacquisition in F and M (89)
Rat; 6-hour session for intravenous heroin	Incubation of heroin craving after forced abstinence	F and M incubated craving (increased responding) between abstinence days 1 and 21
		No incubation of craving between abstinence days 1 and 21 in F or M (88)
Mouse; 3-hour session for oral oxycodone	Intake across doses	F > M mg/kg intake at the 0.30–1 mg/kg dose
	Cue-induced reinstatement	F = M (86)

One possible reason for the differences between human and rodent studies is that opioid experience might not have produced neuroadaptations in brain stress and anti-reward systems, particularly because most of the rodent studies that assessed reinstatement in rodents allowed short access (nondependent) to opioid self-administration. Extended-access opioid self-administration models provide greater dependence, as measured by both somatic and motivational withdrawal, compared with short access conditions. A study investigating context-induced reinstatement after extended access to opioids observed that female but not male rats reduced their context-induced responding for oxycodone after buprenorphine treatment, highlighting the idea that extended access may reveal sex differences in both stimuli-evoked responding for opioids and treatment options for OUD (77). Future studies should investigate whether dependent males and females differ in opioid withdrawal–induced hyperkatifeia-like responses following extended access. Furthermore, future studies should examine the neurobiology of sex differences, with a particular focus on the ways in which sex differences in stress/aversion contribute to stress- and cue-induced reinstatement of responding for opioids. An additional consideration, as described above, is that in both females and males, cues associated with opioid withdrawal in the dependent state may gain greater motivational significance and promote relapse to a greater extent than cues associated with opioid withdrawal in the nondependent state.

PROPOSED ROLE FOR EXTENDED AMYGDALA CIRCUITRY

Chronic drug use produces neuroadaptations in brain stress systems through repeated allostatic challenges (78). These changes lead to the dysregulation of brain circuits that are involved in stress and aversion, including the HPA axis and extended amygdala. We recently reported that odor cues that were conditioned to naloxone-precipitated withdrawal activated brain stress systems in dependent rats, measured by functional magnetic resonance imaging (3). Withdrawal severity was associated with the activity of a hypothalamic cluster and extended amygdala cluster. In both clusters, the naloxone-paired cue increased activity in heroin-dependent rats, whereas it decreased activity in nondependent rats. Moreover, heroin self-administration during naloxone plus cue conditioning correlated with these two clusters (3). Also recently, an analog of a human neuronal salience network, comprising the anterior insula and anterior cingulate cortex, that facilitates behavioral adaptations to environmental stimuli was identified in rats. Withdrawal-associated cue exposure upregulated activity of the rat neuronal salience network (79). Collectively, these studies indicate that dependence and withdrawal induce long-lasting neuroadaptations that may indicate the dysregulation of emotional learning neurocircuitry.

Few studies have examined whether and the extent to which a history of dependence and the experience of withdrawal alter the neural representation and processing of drug delivery–related cues. Some studies have shown that dependence can influence the impact of pharmacological manipulations on motivation and consumption (80). Our laboratory and many others have shown that drug-experienced, nondependent rats can reinstate drug seeking in response to drug-associated cues, although to a lesser extent than dependent rats [e.g., (3)]. These findings indicate that nondependent rats are also motivated to consume opioids, likely because of their positively rewarding effects. For example, substantial evidence exists to support the involvement of the mesolimbic dopamine system, particularly ventral tegmental area (VTA) dopamine inputs to ventral striatum, amygdala, and cortex in opioid-related cue processing and reward in addiction (81–84). The disruption of dopaminergic signaling has been shown to affect cue-associated drug seeking (85, 86). Other neuromodulatory systems have also been implicated in the rewarding effects of opioids, including the μ-opioid and cannabinoid receptor systems (86). However, the brain regions that are activated and the degree of activation likely differ from dependent animals (87).

Therefore, further examination of the neurobiology of relapse in dependent individuals is warranted to determine whether the same or different processes underlie opioid-related cue processing in the dependent versus nondependent state. Below, we briefly describe key brain regions that are implicated in aversive emotional learning and dependence that may underlie drug-related cue processing in individuals with a history of opioid dependence. We focus on regions that comprise the extended amygdala, including the NAc shell, BNST, and CeA, and output projections to the PAG.

Nucleus accumbens shell

The NAc can be divided into core and shell subregions that play differential roles in motivation and addiction. The NAc shell mediates behavioral responses to both rewarding and aversive/anxiogenic stimuli (88) and is preferentially involved in drug reward (89) and the cue-induced reinstatement of opioid seeking (90, 91). Dopamine signaling in the NAc shell has classically been implicated in driving drug delivery cue-dependent motivated behaviors (92, 93) and addiction (94, 95). For example, chronic injections of opioids potentiated excitatory transmission in D₁ receptor–expressing NAc shell neurons (96). Dopamine influx also increased during heroin context-induced reinstatement, and intra–NAc shell infusions of the D₁ receptor antagonist SCH39166 attenuated reinstatement (97).

Although the NAc shell plays a well-established role in positive reinforcement for drugs of abuse, other studies have shown that this region can mediate aversion/stress. The NAc shell is a transition zone between the striatum and other regions of the extended amygdala that process aversive emotions (98, 99). NAc shell neurons mediate opioid withdrawal and withdrawal cue aversion. In an early study, the NAc was identified as the most sensitive brain structure for the ability of the quaternary opioid antagonist methylnaloxonium to precipitate withdrawal in opioid-dependent rats, measured by the suppression of lever-pressing on a fixed-ratio schedule of reinforcement (100). Place conditioning studies showed that the NAc is a highly sensitive site for producing a methylnaloxonium-induced place aversion in opioid-dependent rats (101). Acute opioid withdrawal is associated with a decrease in firing of the mesolimbic dopamine system and a decrease in dopamine release (102). Spontaneous somatic withdrawal from morphine increased excitatory transmission in D₂ receptor–expressing NAc shell neurons, whereas excitatory transmission increased in D₁ receptor–expressing cells during abstinence (103). Moreover, cues that were conditioned to aversive stimuli excited dopamine terminals in the ventral medial NAc shell, whereas dopamine terminals in other NAc subregions were persistently depressed (104), indicating that dopamine signaling in the NAc shell may also be important for processing aversive cues.

Although identifying the neurobiological actions of opioids on reward circuitry during conditioned relief avoidance is challenging because opioids have reward circuitry–activating properties, research using purely aversive stimuli supports the hypothesis of conditioned negative reinforcement. Rats trained to lever-press during a warning cue to avoid footshock exhibited increased phasic dopamine responses in the NAc during the warning cue, as measured by fast-scan cyclic voltammetry (105). Dopamine release also increased in the trials when the footshock would have been delivered, as typically observed during reward delivery. However, if rats failed to make the avoidance response, they could lever-press to initiate an escape response after footshocks commenced, and dopamine release decreased in these escape trials during the warning cue. Similarly, when cues were conditioned to inescapable footshock, dopamine release decreased during cue presentation. Thus, increased dopamine release predicted successful punishment avoidance, whereas decreased release was associated with aversive outcomes (41).

A key question is whether the same or different neurocircuits mediate conditioned positive versus negative reinforcement, and it is likely that different neurocircuits contribute to each type of reinforcement. Calcium transients in subpopulations of glutamate only, γ-aminobutyric acid (GABA) only, and glutamate and GABA coexpressing neurons in the VTA were monitored using selective expression of a calcium indicator during sucrose or footshock delivery and in response to cues predicting their deliveries (52). While the activity of these subpopulations shared some features, each had unique responses to the different stimulus conditions (52). VTA glutamate neurons increased their activity in response to cues predicting reward or aversion. VTA GABA neurons showed increased activity in response to cues that predicted the absence of reward or the delivery of an aversive stimulus. Glutamate-GABA cotransmitting neurons increased activity in response to errors in the predicted delivery of reward. These response patterns were also unique when compared to the response pattern of VTA dopamine cells. Together, these data suggest that neurocircuits other than dopamine play unique roles in responding to appetitive and aversive stimuli (106).

More relevant to conditioning, another recent study showed a role in negative reinforcement for a population of a genetically identified striosomal population of medium spinal neurons in the dorsomedial striatum, which express the Teashirt family zinc finger 1 (Tshz1) gene and belong to the direct pathway output of the striatum (107). Mice were trained to associate an acoustic warning cue with an aversive air puff, and the air puff could be avoided by running after cue presentation. Genetically encoding calcium imaging revealed that these cells were activated by running to avoid the aversive air puff (negative reinforcement) as well as by the conditioned warning cue. However, significantly fewer Tshz1-expressing cells were excited by the presentation of rewarding stimuli (water) or cues that predicted the rewarding stimuli, and inhibiting the Tshz1-expressing neurons impaired punishment-based learning without affecting reward learning or movement. In summary, canonical reward circuity appeared to be involved in the delivery of appetitive rewards and the avoidance of aversive outcomes.

Although it is difficult to separate the concomitant positive reinforcing and withdrawal-relieving effects of abused drugs, opioid-dependent subjects are motivated to avoid or escape the aversive emotional states experienced during withdrawal, and it is reasonable to suggest that the activity of reward and stress neurocircuits play a role in incentive salience attribution during negative reinforcement learning, as discussed below. We believe that this is an understudied area in neuroscience that warrants further attention.

κ-Opioid receptor signaling in the NAc shell drives negative affect–like responses through dynorphin-expressing neurons, which are known to mediate aversion, pain, and stress (51, 108). The κ-opioid receptor–mediated negative affective state may contribute to the role of the NAc shell in addiction. Prodynorphin mRNA levels increased in the NAc shell in heroin-dependent rats (109) and during withdrawal from chronic morphine administration (110). Intra–NAc shell injections of the κ-opioid receptor antagonist nor-binaltorphimine suppressed both the escalation of heroin self-administration during extended access and the motivation for heroin on a progressive-ratio schedule of reinforcement in dependent rats (109). Nor-binaltorphimine treatment also significantly decreased conditioned place aversion behavior during withdrawal (111). These findings suggest a functional role for dynorphin and κ-opioid receptors in the escalation of heroin self-administration and opioid withdrawal–induced malaise-like behavior.

Several studies have shown that κ-opioid receptor signaling contributes to the aversive affective states evoked by pain and pain-conditioned cues. Systemic or intra–NAc shell blockade of κ-opioid receptors with nor-binaltorphimine reduced inflammatory pain-induced decreases in morphine conditioned place preference (112). Moreover, local infusions of the κ-opioid receptor antagonist nor-binaltorphimine in the ventral NAc shell blocked inflammation-induced conditioned place aversion and lowered the motivation to self-administer sucrose (51). Furthermore, systemic and intra-NAc nor-binaltorphimine decreased heroin self-administration and the motivation to obtain heroin in heroin-dependent rats (109). Although intriguing, much remains to be discovered about the specific neurocircuitry of negative reinforcement learning in opioid addiction and how chronic pain and chronic opioid use interact to promote relief learning.

Other studies have outlined a molecular mechanism by which postsynaptic glutamatergic inputs to NAc α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptors are responsible for the negative affect–like responses produced by opioid withdrawal (8, 113). An increase in the surface/intracellular ratio of NAc GluA1 but not GluA2 increased with morphine treatment, suggesting the postsynaptic insertion of GluA2-lacking AMPA receptors (8). Also, the blockade of AMPA receptors in the NAc shell reduced naloxone-induced conditioned place aversion in morphine-dependent rats (8). Together, these results suggested that chronic opioid administration increases the synaptic availability of GluA1-containing AMPA receptors in the NAc, and this increase in synaptic availability is necessary for triggering negative affective states in response to naloxone (8).

Bed nucleus of the stria terminalis

The BNST is a region within the extended amygdala with an established role in anxiety-like responses in animal models. The BNST has been implicated in opioid withdrawal, drug-induced negative affect (114–117), and the stress-induced reinstatement of drug seeking (118). BNST neurons are recruited during withdrawal to signal stress/aversion (119), measured by heightened Fos expression (120–122) and alterations of neuronal excitability during chronic exposure to and withdrawal from opioids (123–125). Opioid withdrawal–induced Fos immunoreactivity in the extended amygdala parallels the development of opioid-induced place aversion (122). Male and female mice also exhibited sex differences in BNST neuron firing during naloxone-precipitated withdrawal from opioids (126), which may underlie sex differences in withdrawal-induced stress/aversion. The BNST contains neurons that express CRF and its receptors, and CRF receptor signaling in the BNST mediates stress and drug seeking. Local infusions of CRF antagonists in the BNST reduced morphine seeking (114, 118, 127), whereas CRF infusions in the BNST were anxiogenic (128).

Some studies have shown that BNST neurons also respond to cues that predict aversive outcomes and may regulate aversive emotional learning. The BNST has been implicated in posttraumatic stress disorder, and its neurons respond to aversion-predictive cues (129–131). BNST neurons receive inputs from cortical and allocortical regions, including the prefrontal and insular cortices, CeA, and hippocampus, and project to motivation-associated regions, such as the VTA and hypothalamus (132, 133). Several studies have implicated BNST neurons in conditioned drug-seeking behaviors, drug delivery cue-induced reinstatement, and conditioned place preference and aversion (114, 115, 134). The BNST may respond to cues that are associated with aversive events, including opioid withdrawal, to trigger motivated opioid seeking and taking.

Central nucleus of the amygdala

The CeA is a region of the extended amygdala with predominantly GABA neurons that receives inputs from the basolateral complex of the amygdala, insular cortex, hypothalamus, and parabrachial nucleus (135). The CeA also reciprocally interacts with the BNST (136, 137) and sensitizes to glucocorticoid feedback following chronic stimulation of the HPA axis (138, 139). The CeA promotes aversive affective states, including fear-like responses (140) and anxiety-like responses (141). The CeA has been implicated in affective learning, and recently, a reciprocal insular cortex–CeA circuit was shown to link conditioned stimuli valence with interoceptive states to drive conditioned responding (142).

The CeA mediates pain and pain memory processing in rodent models (143). Excitotoxic lesions of the CeA decreased conditioned place aversion that was induced by formalin and acetic acid injections (144). Intra-CeA infusion of a μ-opioid receptor antagonist decreased complete Freund’s adjuvant–induced conditioned place aversion (145). In contrast, intra-CeA morphine microinjections induced conditioned place preference rats with neuropathic pain, and this effect was blocked by microinjections of the μ-opioid antagonist β-funaltrexamine into the anterior cingulate cortex, a region implicated in aversive aspects of pain (146), indicating that the μ-opioid receptor anterior cingulate–CeA circuit mediates conditioned pain–induced aversive affect–like responses. CRF signaling in the CeA may also be important for pain processing (147, 148). For example, CRF₁ receptor blockade in the CeA reversed pain-induced decreases in hindlimb withdrawal thresholds (148), whereas intra-CeA CRF infusions decreased paw withdrawal thresholds in uninjured animals (147).

The CeA contributes to drug-induced aversion, and the drug-induced reinstatement of drug seeking. Spontaneous and conditioned opioid withdrawal upregulated Fos expression in CeA neurons (149–151). The blockade of CRF signaling with α-helical CRF_(9-41) reversed morphine-induced conditioned place aversion and operant responding for conditioned withdrawal–associated cues in dependent animals (149). The CeA is involved in stress-induced reinstatement, in which inhibiting the CeA activity decreased the footshock-induced reinstatement of heroin seeking (152). Inhibiting the CeA with baclofen-muscimol attenuated the cue-induced reinstatement of heroin seeking (134).

Periaqueductal gray

The PAG is a major output of the extended amygdala in the brainstem that promotes adaptive behaviors in response to aversive stimuli, including activation of the HPA axis and “fight-or-flight” responses (16). The dorsal subregion of the PAG innervates the sympathetic nervous system, and the ventral subregion stimulates the parasympathetic nervous system (153, 154). The CeA innervates the PAG through μ-opioid receptor signaling, and this circuit has been linked to stress-induced analgesia (155, 156). The PAG also receives input from the insular cortex that is implicated in subjective feelings of pain, anxiety, and depression (157).

In addition, the PAG is sensitive to opioid administration (158), and Fos expression in the PAG is upregulated during opioid withdrawal (149). The PAG is part of the descending pain modulatory pathway that promotes opioid-induced analgesia and tolerance (159–161). Blocking microglial activation in the PAG attenuated both opioid-induced analgesia and the development of opioid tolerance (162). Naloxone-precipitated morphine withdrawal upregulated proinflammatory cytokines in the PAG, and the inhibition of these factors reduced somatic withdrawal symptoms (163). Therefore, the PAG may convey information about aversive events from and to the extended amygdala to drive autonomic processes that promote relapse.

FINAL CONSIDERATIONS

Although evidence strongly suggests that cues associated with both positive reinforcement (i.e., achieving opioid “high”) and negative reinforcement (i.e., escaping or avoiding opioid withdrawal) can powerfully motivate opioid seeking and taking, the vast majority of studies have focused on positive reinforcing cues. Withdrawal cue memories persist beyond the period of opioid intoxication and acute opioid withdrawal to promote drug taking and relapse in individuals during protracted abstinence. Cues associated with successful escape or avoidance of drug withdrawal (e.g., drug paraphernalia) may also become imbued with uniquely salient properties (i.e., relief salience) via conditioned negative reinforcement as a result of this association to motivate opioid seeking in formerly opioid-dependent individuals. The presence of physical and emotional pain can contribute to opioid misuse and increase the severity of dependence. Conditioned somatic and emotional pain can drive motivated behavior and may stimulate relapse in abstinent individuals. In addition, sex differences are apparent in many aspects of OUD, including the effects of opioids, cue and stress reactivity, pain sensitivity, and HPA axis engagement. However, most studies to date have been conducted in male subjects. Last, chronic opioid administration to the point of dependence clearly alters behavioral and neuroanatomical responses to opioids and presumably strengthens withdrawal-induced conditioning that contributes to negative reinforcement, but nondependent subjects have largely been used to study the neurobiological mechanisms that underlie drug seeking, taking, and relapse in OUD. Therefore, we propose that elucidating the neural mechanisms that underlie the motivational effects of cues associated with various aspects of negative reinforcement learning in opioid-dependent individuals of both sexes and determining the ways in which chronic pain intersects with these processes will provide critical information to further our understanding of OUD.

Last, we propose that the safest and most practical approach to preventing the pathology of OUD such as overdose is to target stimuli paired with drug withdrawal, thereby preventing conditioned withdrawal and the motivational state that drives drug taking and relapse, in formerly opioid-dependent individuals. This could be achieved by behavioral therapies such as cue exposure, cognitive behavioral therapy and mindfulness (164), and pharmacological approaches (165).