Modeling the Development of Drug Addiction in Male and Female Animals

Abstract

An increasing emphasis has been placed on the development and use of animal models of addiction that capture defining features of human drug addiction, including escalation/binge drug use, enhanced motivation for the drug, preference for the drug over other reward options, use despite negative consequences, and enhanced drug-seeking/relapse vulnerability. The need to examine behavior in both males and females has also become apparent given evidence demonstrating that the addiction process occurs differently in males and females. This review discusses the procedures that are used to model features of addiction in animals, as well as factors that influence their development. Individual differences are also discussed, with a particular focus on sex differences. While no one procedure consistently produces all characteristics, different models have been developed to focus on certain characteristics. A history of escalating/binge patterns of use appears to be critical for producing other features characteristic of addiction, including an enhanced motivation for the drug, enhanced drug seeking, and use despite negative consequences. These characteristics tend to emerge over abstinence, and appear to increase rather than decrease in magnitude over time. In females, these characteristics develop sooner during abstinence and/or following less drug exposure as compared to males, and for psychostimulant addiction, may require estradiol. Although preference for the drug over other reward options has been demonstrated in non-human primates, it has been more difficult to establish in rats. Future research is needed to define the parameters that optimally induce each of these features of addiction in the majority of animals. Such models are essential for advancing our understanding of human drug addiction and its treatment in men and women.

1. Introduction

The vast majority of the preclinical data on drug addiction are based on studies conducted in male animals self-administering drugs under short access self-administration conditions (e.g., 1-2 hr/day, fixed ratio 1 schedule). Drug intake under these conditions is stable from day-to-day, and intake is relatively low (Lynch and Carroll 2001). These studies have been incredibly useful for determining the neurobiological basis for drug reinforcement and have helped identify a number a factors that predict a vulnerability to initial drug use (Campbell and Carroll 2000; Deminiere et al. 1989; Wise and Bozarth 1981; Wise and Koob 2014). Short access conditions, however, by virtue of their stability, may not capture critical features of addiction in humans, and as such, the behavioral and neurobiological principles defined by these studies may be restricted to drug reinforcement and initial vulnerability, but not characteristic of “addiction”. Specifically, while the reinforcing effects of drugs are critically involved addiction, particularly during early stages, these effects may diminish over time as the disease progresses (Koob and Volkow 2016). Other factors also appear to be critical to maintaining drug use, particularly once addiction has developed, such as loss of control over drug use and the resulting excessive use of the drug, and the negative reinforcing effects of drugs (i.e., use to alleviate withdrawal or craving; Koob and Mason 2016).

In humans, addiction, or substance use disorders, has been defined in the DSM-5 as meeting two or more of 11 diagnostic criteria (American Psychiatric Association 2013). These criteria focus on evidence of impaired control over drug use, such as an increased time and energy spent seeking and using the drug, intense craving and urge to use the drug, and an inability to reduce or abstain from drug use, and social impairment and compulsive drug use, such as drug use to the exclusion of other activities and despite negative consequences. While no one animal model captures all of the behavioral, pharmacological, and social aspects of addiction, numerous procedures have been developed to focus on one of more of the critical features. Five of the more commonly modeled features of addiction include: 1) escalation/binge patterns of drug use, 2) enhanced motivation for the drug, 3) preference for the drug over other reward options, 4) use despite negative consequences, and 5) enhanced drug-seeking/relapse vulnerability. A greater emphasis is being placed on capturing these features of addiction in animal models given accumulating evidence showing that the neurobiological mechanisms underlying drug taking and seeking behavior change with the development of addiction-like behaviors (for review see Koob 2014; Wolf 2016; Wolf and Tseng 2012; also see Ben-Shahar et al. 2007; Briand et al. 2008; Cohen et al. 2015; Conrad et al. 2008; Doyle et al. 2014; Fischer et al. 2013; Fischer-Smith et al. 2012; George et al. 2008; Greenwell et al. 2009; Hao et al. 2010; Imperio and Grigson 2015; Le Cozannet et al. 2013; Mateo et al. 2005; Ramôa et al. 2014; Recinto et al. 2012; Zorrilla et al. 2012).

There is also an increased emphasis placed on the inclusion of females in studies of addiction particularly in light of mounting evidence from humans and animals demonstrating that the addiction process occurs differently in males and females (Becker and Koob 2016; Bobzean et al. 2014; Carroll et al. 2004; Carroll and Lynch 2016; Fattore et al. 2008; Lynch et al. 2010). In humans, although men are more likely than women to use drugs and have addictive disorders, women are more vulnerable than men on certain aspects of drug addiction (Elton and Kilts 2010; Greenfield et al. 2010; Zilberman et al. 2003). One of the most striking examples is “the telescoping effect” where following initial drug use, women meet criteria for substance abuse disorders and seek treatment after fewer years of drug use as compared to men (Anglin et al. 1987; Brady and Randall 1999; Griffin et al. 1989; Haas and Peters 2000; Hernandez-Avila 2004; McCance-Katz 1999; Westermeyer and Boedicker 2000). Women also report longer periods of use following abstinence (Gallop et al. 2007), and are more likely than men to attribute relapse to drug use to reasons of depression and negative affect (McKay et al. 1996). Similar findings have also been reported in preclinical studies with results showing that female animals self-administer more drug under extended access conditions, and develop certain characteristics of addiction faster and/or following less drug exposure than male animals (Anker and Carroll 2011; Becker and Koob 2016; Lynch 2006; Lynch and Taylor 2004). Cumulative evidence from both humans and animals suggest that the ovarian hormones estradiol and progesterone modulate vulnerability in females (Anker and Carroll 2011; Flores et al. 2016; Ford et al. 2002; 2004; Kucerova et al. 2009; Larson et al. 2007; Lynch et al. 2001; Lynch and Taylor 2005; Lynch and Sofouglu 2010; Wetherill et al. 2016), with results from animals further suggesting for that estradiol may be necessary for the development of features of psychostimulant addiction (Kerstetter et al. 2012; Ramôa et al. 2013; 2014). These preclinical and clinical data show good correspondence indicating that sex differences in drug addiction are biologically based. These findings also suggest that sex and hormonal status are major determinants of drug addiction, and highlight the need to include both males and females in studies of addiction.

There have been several recent reviews on animal models of particular characteristics of addiction such as escalation of drug intake (Ahmed 2009; Edwards and Koob 2013), drug seeking (Mantsch et al. 2016; Marchant et al. 2013; Venniro et al. 2016), incubation of drug seeking/relapse vulnerability (Li et al. 2015; Wolf 2016), enhanced motivation for drug (Allain et al. 2015; Oleson and Roberts 2008), as well as reviews on methods used to induce one or more features of addiction in certain populations of animals (Ahmed 2012; Belin-Rauscent et al. 2016; Deroche-Gamonet and Piazza 2014; Waters et al. 2014). There have also been several recent reviews detailing sex differences in the behavioral and neurobiological mechanisms of addiction as a function of stage of the addiction process (Becker et al. 2017; Becker and Koob 2016; Bobzean et al. 2014; Carroll and Lynch 2016). In this review, the focus is on the procedures that have been used to study the five commonly modeled features of addiction with the goal being to provide a better understanding of the conditions needed to optimize their development in the majority of animals. Sex differences, and other individual differences, are also discussed for each feature since a better understanding of these differences is essential for our understanding of sex and individual differences in the development and treatment of addiction in humans.

2. Animal Models of Escalation/Binge Patterns of Use

A loss of control over use and the excessive use of the drug, two defining features of human drug addiction, have been modeled in animals using several different extended access drug self-administration procedures (Ahmed and Koob 1998; Allain et al. 2015; Balster and Woolverton 1982; Fitch and Roberts 1993; Lynch and Carroll 2001). Early studies in non-human primates and rats showed that these characteristics are readily observed when animals are allowed continuous 24-hr/day access (fixed ratio 1 schedule) to intravenous infusions of cocaine, methamphetamine, heroin, morphine, and phencyclidine infusions (Balster and Woolverton 1982; Bozarth and Wise 1985; Deneau et al. 1969; Johanson et al. 1976). Animals self-administering psychostimulants, such as cocaine, demonstrate periods of erratic and rapid drug intake interspersed with periods of self-imposed abstinence (Bozarth and Wise 1985; Deneau et al. 1969) Animals self-administering opioids, such as heroin, progressively increase their drug intake over time to high levels (Bozarth and Wise 1985; Balster and Woolverton 1982). However, toxicity can develop rapidly under these conditions with these types of drugs, thus necessitating the use of conditions that restrict access in some way. Other drugs, such as nicotine and ethanol, can be available under unlimited-access conditions with limited toxicity (Balster and Woolverton 1982; Valentine et al. 1997; Wolffgramm and Heyne 1995).

Numerous methods have been developed to balance levels of intake and toxicity, particularly for psychostimulant and opioid drug self-administration (for review see Ahmed 2009; 2012; Allain et al. 2015; Edwards and Koob 2013; Roberts et al. 2007). For example, high levels of drug intake can be maintained with low levels of toxicity under continuous 24-hr/day access conditions when low drug doses are available (Carroll and Lac 1997), when the total number of days is limited (3 days, Tornatzky and Miczek 2000), or when the drug is self-administered orally rather than intravenously (Alexander et al. 1981; Barros and Miczek 1996; Meisch 2001). High levels of drug intake with limited toxicity can also be maintained under extended access conditions that restrict the total number of infusions available each day (Henry and Howell 2009; Peoples et al. 1997), the total number of hours of access each day (6-12 hr/day; Ahmed and Koob 1998; Edwards and Koob 2013; Mandt et al. 2015; Panlilio and Goldberg 2007), that use a higher work requirement (e.g., fixed ratio 16; Carroll et al. 2005), or that include a time-out after each delivery (e.g. 15-min; Hutsell et al. 2016a, b).

The most well established extended access drug self-administration procedure is the long access procedure developed by Ahmed and Koob (1998). With this procedure, animals are given continuous access to the drug for 6- to 12-hr/day. Under these conditions, animals self-administer high levels of the drug with few signs of toxicity. This procedure has also been shown to induce escalation of drug intake, or a progressive increase rates and levels of drug intake over time, which is not observed in control animals given short access to the drug (1-2-hr access/day). Drug use escalation has been observed for rats self-administering numerous drugs of abuse including cocaine, methamphetamine, synthetic cathinones or “bath salts”, heroin, oxycodone, and fentanyl (Edwards and Koob 2013; Nguyen et al. 2017; Wade et al. 2015). Drug use escalation has also been observed in non-human primates for oral phencyclidine self-administration (Carroll et al. 2005), and in mice for intravenous oxycodone self-administration using modified access conditions (i.e., 4-hr/day access; Zhang et al. 2014). Escalation of alcohol and nicotine use has also been observed, although its occurrence may depend on the use of cyclic access conditions that alternate between periods of continuous access (12 to 24-hr) and withdrawal (Carnicella et al. 2014; Cohen et al. 2012), or vulnerable populations of animals (i.e. alcohol preferring animals, Becker and Ron 2014). Importantly, a history of escalating drug self-administration has been shown to lead to the development of other core characteristics of addiction including enhanced drug-seeking and its incubation over abstinence (see section 6.0). The effect of escalation on subsequent motivation for drug, however, has been more variable (see section 3.0).

Another method that induces high levels of drug intake with limited toxicity is the discrete trial procedure developed by Fitch and Roberts (1993). With this procedure, animals are given 24-hr access to drug infusions that are available in discrete 10-min trials. Most of the work with this procedure has focused on cocaine self-administration under a 4-trial/hour condition, which allows rats access to a drug infusion every 15 min around the clock for between for 1 and 4 weeks (Dobrin and Roberts 2012). Under these conditions, animals self-administer high levels of the drug in binge/abstinent patterns taking nearly every infusion available for the first 1-2 days, followed by periods of self-imposed abstinence that are interspersed with periods of active drug use (Roberts et al. 2002). This method has also been used for heroin and the combination of heroin and cocaine (speedball) self-administration (Martin et al. 2006; Ward et al. 2006). An advantage of this model is that, unlike the long access procedure, it consistently induces an enhanced motivation for the drug, as well as enhanced drug seeking when responding is assessed following abstinence (see sections 3.0 and 6.0). Unlike the long access procedure, however, intake under the discrete trial procedure does not escalate over time (Roberts et al. 2002). In fact, while intake remains erratic throughout an extended testing period (7-30 days), it typically peaks during the first several days of access.

A newer model, the intermittent access procedure, was recently developed by Zimmer et al. (2012) to address limitations with the discrete trial and long access procedures with the goal being to induce both dysregulated/binge patterns of intake as well as dose escalation over time. With this procedure, animals are given access to drug infusions in 5-min trials that are initiated either every 30 to 60 min for a total of 6-24 hours/day. Drug infusions are continuously available during the 5-min trials under either a fixed ratio 1 schedule or using a hold-down procedure that allows the animals control over the length/dose of each infusion. This is different from the discrete trial procedure where only one infusion of a high drug dose was available (Fitch and Roberts 1993). This difference appears to be critical, as it allows the animals to choose their preferred dose, which starts relatively low, but escalates over time (Zimmer et al. 2012). Intake also escalates over time, and is characterized by binge-abstinence patterns wherein animals experience repeated spiking drug levels. The authors argued that this pattern of use more closely appropriates patterns observed in humans where, as addiction develops, users begin self-administering larger doses that are separated by longer intervals (Zimmer et al. 2012). They further argued that users do not maintain constant blood levels of drug, but rather, experience repeated cycles of intense spiking drug levels that are preceded by substantial reductions in drug levels.

Numerous studies have now used the intermittent access procedure for cocaine or methylphenidate self-administration in rats (Allain et al. 2017; Calipari and Jones 2014; Calipari et al. 2013; 2014a; b; 2015; Kawa et al. 2016; Pitchers et al. 2017), with results showing that it induces a marked increase in both motivation for the drug and drug seeking, and may induce these changes even without the use of protracted abstinence (see sections 3.0 and 6.0). These findings are notable given that intake under the intermittent access procedure has been reported to be markedly lower than levels observed under other extended access procedures (Kawa et al. 2016). These findings add to a growing body of work indicating that the pattern of use is more important than the total level of use with regard to inducing features of addiction (Allain et al. 2015).

In addition to pattern of intake, other relevant factors that influence escalation and binge drug use include access conditions, drug dose, route of delivery, and the drug self-administered (Kitamura et al. 2006; Lynch and Carroll 2001; Mantsch et al. 2004; Roberts et al. 2002). The speed of drug delivery is also crucial, with faster deliveries being associated with greater escalation/binge drug use (Allain et al. 2017; Wakabayashi et al. 2010). These factors can also interact with environmental factors. For example, while enriched versus impoverished environmental conditions markedly reduce oral opioid self-administration (Alexander et al. 1981), such interventions are much less robust (or not apparent) under intravenous drug self-administration conditions (e.g. Bozarth et al. 1989; Gipson et al. 2011). Each of these factors may also affect the development of other features of addiction.

2.1. Individual Differences in Escalation/Binge Patterns of Use

Numerous studies have reported sex differences in levels of intake and patterns of use under extended access conditions. For example, several studies have shown that under the long access procedure female rats and monkeys self-administer higher levels of cocaine, methamphetamine, nicotine, and phencyclidine, and show a greater escalation of drug use over time as compared to males (Carroll et al. 2005; Reichel et al. 2012; Roth and Carroll 2004; Sanchez et al. 2014; Smith et al. 2011). Similar findings have also been observed for cocaine self-administration under the discrete trial procedure with results showing that females self-administer higher levels of the drug, binge for longer initial periods of time, and show a greater disruption in the diurnal control over intake as compared to males (Doyle et al. 2014; Lynch and Taylor 2004; Lynch and Taylor 2005; Lynch et al. 2005; Peterson et al. 2014 b; Ramôa et al. 2013, 2014). These findings indicate an apparent vulnerability in females as compared to males during the transition from controlled, stable drug use to escalation/binge drug use. This sex difference appears to be reliable, as it has been observed for several species, and for several drugs of abuse. Its occurrence also does not appear to differ between extended access procedures, although, to our knowledge, no studies have yet examined sex differences under the newly developed intermittent access procedure.

While the mechanisms underlying these sex differences are surprisingly unknown, they likely include the influence of ovarian hormones. Ovariectomy has been reported to decrease cocaine intake as assessed under both the long access and the discrete trial procedure (Lynch and Taylor 2005; Larson et al. 2007; Martinez et al. 2016; Ramôa et al. 2013, 2014). Notably, the effects of ovariectomy on escalation/binge cocaine use can be restored by estradiol treatment (Lynch and Taylor 2005; Larson et al. 2007; Martinez et al. 2016; Ramôa et al. 2013; 2014). Similar effects of ovariectomy and estradiol replacement have also been reported for nicotine and alcohol self-administration under extended access conditions (23-hr/day; Flores et al. 2016; Ford et al. 2002; 2004). The effects of hormonal manipulations (i.e., gonadectomy and estradiol treatment) have been more variable for heroin self-administration (Roth et al. 2002; Stewart et al. 1996), and do not appear to impact drug taking in males (Jackson 2006) although this work has focused on earlier phases of the addiction process, such as acquisition and maintenance intake. In contrast, progesterone treatment has been reported to block escalation of cocaine intake in both intact females and ovariectomized females with estradiol (Larson et al. 2007). While the effects of progesterone on escalation/binge drug use in males have not yet been examined, recent evidence demonstrating its ability to decrease other features of addiction in both sexes (i.e., drug seeking; Zlebnik et al. 2014), suggests that it may influence escalation/binge use in male as well as females. These findings add to a growing body of evidence from preclinical and clinical studies indicating that estradiol enhances vulnerability in females, and progesterone decreases vulnerability in both sexes (Anker and Carroll 2011; Becker and Koob 2016; Becker et al. 2017; Fattore et al. 2010; Kucerova et al. 2009; Lynch and Sofouglu 2010; Mello et al. 2011). However, given that most of the work in this area has focused on stimulants and alcohol, future research is needed to determine whether this relationship extends to other drugs. Such studies are particularly needed for opioids given recent evidence suggesting that the relationship between ovarian hormones and heroin use/seeking following short access self-administration may be different as compared to stimulants (Lacy et al. 2016; Sedki et al. 2015).

Other individual differences have also been reported to influence escalation/binge drug use such as the initial response to a drug where animals that have a greater initial positive response, as measured under the conditioned place preference test, show greater escalation of drug self-administration (Ettenberg et al. 2015). An initial positive response to the drug is also believed to facilitate repeated drug use in humans (de Wit and Phillips 2012). Factors related to decision-making and reactivity to stress have also been suggested as factors that may predict the development of addiction in humans (de Wit and Phillips 2012). Many of these factors have also been identified in animals as predictive for escalation of drug self-administration. For example, a high level of impulsivity, as assessed by delayed discounting of reward or response inhibition, has been shown to be predictive of escalation of cocaine self-administration as well as subsequent relapse vulnerability (Anker et al. 2009; Dalley et al. 2011). This trait, however, does not appear to predict escalation of heroin self-administration (McNamara et al. 2010). Numerous studies have reported that stressful events enhance escalation of drug self-administration (Koob and Kreek 2007), and recent work indicates that individual differences in the adaptive response to repeated stress (i.e., coping mechanism) predicts the degree of escalation (Burke and Miczek 2015). Response to novelty and sensation seeking also predict escalation of drug self-administration with high responders showing greater escalation of cocaine and alcohol self-administration (Parkitna et al. 2013; Mantsch et al. 2001). Other individual difference factors that predict vulnerability to escalation/binge drug use include genetic strain (Lopez et al. 2017; Picetti et al. 2012), high sweet preference (Holtz and Carroll 2013), and young age (adolescent versus adult; Anker et al. 2012).

It should be noted that features of addiction also emerge in some animals following short access self-administration (1-2 hr/day) as well as following prolonged access self-administration (i.e., 1– 2-h sessions, 30 or more days). The vulnerability factors listed above, such as female sex and high levels of impulsivity, appear to also predict vulnerability to developing features of addiction following short self-administration (Belin and Deroche-Gamonet 2012; Kerstetter et al. 2012; Perry et al. 2013; 2015). However, these animals do not seem to differ from the others on total levels of intake, but perhaps patterns or other use characteristics may help differentiate. A better understanding of how these vulnerable animals differ from the others may provide critical insight into not only vulnerability factors, but also neurobiological mechanisms of addiction.

3. Enhanced Motivation for the Drug

A history of escalating/binge patterns of use appears to be critical for inducing other features characteristic of addiction, including an enhanced motivation for the drug. Numerous studies have used an enhanced motivation for the drug, or sensitization to the reinforcing effects of the drug, to define the development of an addicted phenotype in animals. Changes in motivation are typically assessed using a progressive ratio schedule, with increases determined relative to baseline (prior to extended access self-administration; Lynch and Taylor et al. 2014) or to short access controls (Doyle et al. 2014; Ramôa et al. 2013, 2014). With this schedule, the work effort, or number of responses required to obtain each drug delivery, progressively increases within a session. The final ratio completed, or breakpoint, is then used to define the level of motivation for the drug (Arnold and Roberts 1997). The breakpoint is also believed to be a sensitive and linear measure of reinforcing efficacy, where an increase in the dose of the drug corresponds to an increase in breakpoint. Another method that has been used in several recent studies to examine changes in motivation for drug is the within-session threshold procedure (Oleson and Roberts 2012). With this procedure, measures of demand for the drug are determined within sessions by varying the price (response requirement) and the value (dose) of the drug. The level of motivation is then determined by calculating Pmax, the maximal price paid for the drug, using behavioral-economic demand curve analysis (Bentzley et al. 2013). One advantage of this method over the progressive ratio schedule is that motivation for the drug can be determined independent of an animal's preferred level of drug intake.

Thus far, the results obtained from studies using the progressive ratio schedule and the threshold procedure have revealed similar findings. For example, both procedures have shown differential effects of short versus extended access self-administration on motivation for cocaine (Doyle et al. 2014; Kawa et al. 2016; Ramôa et al. 2013, 2014; Zimmer et al. 2012). Following short access self-administration, motivation for the drug is stable and does not increase over time or following abstinence (Doyle et al. 2014; Ramôa et al. 2013, 2014; Roberts et al. 2007). In contrast, motivation for drug is markedly enhanced following access to the drug under certain extended access conditions. For example, numerous studies have shown that motivation for drug is enhanced following extended access self-administration under the discrete trial procedure (Doyle et al. 2014; Liu et al. 2005; Lynch and Taylor 2004; Morgan et al. 2002; 2005; Ramôa et al. 2013, 2014). This work has focused on cocaine self-administration, and the results have shown that the use of extended access conditions coupled with an abstinence period of seven days or more is critical for inducing this phenotype (for review see Morgan and Roberts 2004). When motivation is assessed earlier during abstinence, results show that it is either decreased or not different from control levels (Lynch and Taylor 2005; Morgan et al. 2002). This delay in the observation of an enhanced motivation may reflect an incubation of motivation over abstinence similar to the incubation effect that has been reported for drug seeking (Grimm et al. 2001). Notably, once this motivational enhancement occurs, it appears to represents a lasting shift in sensitivity to the reinforcing effects of the drug (Roberts et al. 2007).

While these changes in motivation are both robust and persistent, the conditions needed to induce them appear to be fairly specific. For example, the length of both the access and abstinence period appear to be critical with results suggesting that longer periods of abstinence may be required for revealing an enhanced motivation following short versus long periods of extended access self-administration (e.g., Lynch and Taylor 2004; Doyle et al. 2014). The procedure used for extended access self-administration may also be critical. For example, as mentioned earlier, although an enhanced motivation for drug has been reported following escalation using the long access procedure, effects appear to be variable with some studies showing enhanced motivation (Ahmed et al. 2000; Ducret et al. 2016; Paterson and Markou 2003; Wade et al. 2015; Wee et al. 2008; Whitfield et al. 2015), but others findings no change or decreased motivation for the drug (Allen 2014; Anker et al. 2009; Crawford et al. 2013; Larson et al. 2007; Liu et al. 2015; Oleson and Roberts 2007; 2008; Roberts et al. 2007; Whitfield et al. 2015), and evidence for tolerance to its reinforcing effects (Calipari et al. 2013; 2014a; Oleson and Roberts 2009). Although many of the escalation studies have examined levels of motivation within a day or two of the extended access self-administration period, others have examined effects following protracted abstinence and have not observed enhancements (Oleson and Roberts 2009; Roberts et al. 2007). In contrast, an enhanced motivation for drug has been reported to occur under multiple conditions following access under the intermittent access procedure, and as soon as 18-hr after intermittent self-administration (Allain et al. 2017; Calipari et al. 2015; Kawa et al. 2016; Zimmer et al. 2012). This finding is intriguing, and suggests that the time-course for development of an enhanced motivation for the drug depends on the conditions of access (or pattern of drug intake).

3.1. An Enhanced Motivation for the Drug: Individual Differences

Several studies have reported sex and hormonal differences in the development of an enhanced motivation for drug (Carroll and Lynch 2016). Most of the work in this area has focused on motivation for cocaine following extended access self-administration under the discrete trial procedure. In females, motivation to obtain cocaine is increased following 7 days of extended access self-administration and 10 days abstinence (Lynch and Taylor 2014), whereas, in males, a longer period of drug access or a longer abstinence period may be required to induce this motivational shift (14 days; Doyle et al. 2014; Ramôa et al. 2013; 2014). These findings are consistent with clinical results demonstrating a faster course to addiction in women (Greenfield et al. 2010), and suggest that the “telescoping effect” is biologically based. The ovarian hormone estradiol may be required for development of an enhanced motivation for cocaine in females, in that females without estradiol (i.e., ovariectomized females) do not develop an enhanced motivation for cocaine even under conditions optimized for its development (14 days of extended access self-administration plus 14 days of abstinence; Ramôa et al. 2013; 2014).

It is notable that once the shift in motivation occurs, the magnitude of the change does not differ between males and females and both sexes maintain a similar high level of motivation for cocaine (Doyle et al. 2014; Ramôa et al. 2014). For example, under optimized extended access and abstinence conditions, both males and females developed an enhanced motivation for cocaine, with 9 of 11 males and 8 of 10 females showing a greater than 15% increase PR responding for cocaine as compared to short access controls (Ramôa et al. 2014). These findings suggest that while females may have a more rapid transition to developing features of addiction, they may be similar to males once addiction has developed. This hypothesis parallels findings in humans showing that although women develop a substance use disorder faster than males, once addicted, they report similar levels of drug use as compared to males (Haas and Peters 2000; McCance-Katz et al. 1989). To date, no studies have examined sex differences in motivation for cocaine following abstinence from long access or intermittent access self-administration (but see section 6.0 for effects of drug seeking/relapse vulnerability).

Other individual differences include baseline levels of impulsivity and anxiety, which have been reported to be positively associated with the development of an enhanced motivation to obtain alcohol following abstinence from extended access consumption (Radwanska and Kaczmarek 2012). In contrast to findings under short access conditions, sign tracking, which measures an individual's sensitivity to drug-associated cues, does not predict the development of an enhanced motivation for cocaine following extended self-administration (Kawa et al. 2016). As mentioned earlier, there is also a subpopulation of animals that develop features of addiction, including an enhanced motivation for the drug, following short access self-administration (e.g., Belin and Deroche-Gamonet 2012; Belin et al. 2011; Deroche-Gamonet et al. 2004; Perry et al. 2013; 2015).

4. Preference for the Drug over Other Reward Options

One of the defining characteristics of human drug addiction is the compulsive use of the drug to the exclusion of other obligations or rewarding activities. Choice procedures in animals are believed to model this characteristic since choice for drug over food (or other reward options) can be measured over time, and as a function of drug history (Moeller and Stoops 2015). These types of procedures are also believed to mimic the human situation where drug users allocate time and resources to obtain and use drugs rather than to obtain other rewards or engage in other activities. Choice procedures are also frequently used to evaluate drug-taking behavior in humans with substance use disorder (Comer et al. 2008; Jones and Comer 2013).

Choice procedures typically use discrete trial, concurrent, or concurrent chain schedules, which allow animals to choose between two (or more) options by responding on one of two (or more) levers (Banks and Negus 2012; Banks et al. 2015; Lynch and Hemby 2010; Moeller and Stoops 2015). Sessions typically begin with a sampling period, wherein the subject can respond to obtain each of the available reinforcer options (e.g., drug versus sweet reward, drug versus food, a low versus a high dose of drug). The sampling period is then followed by a series of trials or concurrent schedules during which the animals complete the schedule requirement in order to obtain a reinforcer delivery. Response allocation, or the percent of total responding for each reinforcer, provides a measure of reinforcing strength.

While a preference for the drug over the non-drug reward in this task is believed to reflect an enhanced sensitivity to the reinforcing effects of the drug, the development of an addictive phenotype is believed to be reflected by exclusive choice (or near exclusive, greater than 90%) for the drug versus another reward option (for review see, Banks and Negus 2012). This characteristic was very compellingly captured in one of the first studies of addiction using an animal model. This study, conducted by Spragg (1940), showed that chimpanzees that were chronically exposed to morphine developed a strong preference for morphine over food, and when morphine-deprived, showed an exclusive choice for it over food. Many studies have subsequently been conducted in non-human primates with results demonstrating exclusive preference for drug over food under both short and extended access conditions when high doses of the drug are tested (e.g. Aigner and Balster 1978; Negus 2003; Paronis et al. 2002).

While most of the work with choice procedures has been conducted in non-human primates, their utility has recently been extended to studies with rats (e.g. Thomsen et al. 2008; 2012; Tunstall et al. 2014; note: one exception is that numerous studies have examined choice between ethanol and water in rats; e.g. Carnicella et al. 2014; McBride et al. 2014). However, in contrast to non-human primates, in rats, it is more difficult to establish a preference for high doses over lower ones or a preference for drug versus non-drug reinforcer options (Caprioli et al. 2015a; b; Huynh et al. 2017; Lenoir et al. 2007; 2013a; b; Paronis 2013; Tunstall et al. 2014). In fact, results thus far show that only a minority of rats develop a preference for drug over food or other reinforcer options (for review see Ahmed et al. 2013). The percentage of rats that display a drug preference varies between drugs (Tunstall et al. 2014; Vandaele et al. 2016), as a function of drug dose (Kerstetter et al. 2012), and alternate reward option (e.g., Lenoir et al. 2007; Kerstetter et al. 2012), but extended access exposure does not appear to increase the likelihood of its occurrence at least for stimulants. For example, with cocaine choice procedures, only about 10-15% of animals show a drug versus sweet preference, and this percentage does not change following prolonged or extended access self-administration (Cantin et al. 2010; Lenoir et al. 2007; 2013 b). Even lower levels of preference have been reported in similar choice studies with nicotine and methamphetamine, with recent findings showing that rats strongly prefer the sweet/palatable food option to the either drug even after extended access drug self-administration and even when that same palatable food is freely available in the home cage prior to the choice session (Caprioli et al. 2015a; b; Huynh et al. 2016).

Some have argued that because the low probability of preference in rats maps well onto human addiction, where only a minority of users become addicted, choice procedures could represent an objective method of selection of addicted animals (Ahmed et al. 2013). Others are taking advantage of this apparent preference for a sweet/palatable food over drug options and using choice to obtain the sweet/palatable food as a means to model voluntary abstinence (Caprioli et al. 2015a; b; Huynh et al. 2016). However, it is also possible that the conditions used in choice studies with rats are not optimal for inducing an addicted phenotype, and hence a preference for the drug over food. In fact, several of these studies have been designed to investigate individual differences in the development of drug preferences, and to do so, have used threshold or short access conditions (Tunstall and Kearns 2015; Tunstall et al. 2014). Other studies have used highly palatable foods as the competing choice, which may prevent the development of an addicted phenotype (Caprioli et al. 2015a; b; Huynh et al. 2016; Madsen and Ahmed 2015). Additionally, given that specific conditions are needed to induce other features of addiction (i.e. an enhanced motivation for drug), it is possible that the conditions needed to induce an exclusive choice for drug also need to be specifically designed to maximize the likelihood of its occurrence. In support of this idea, results from a recent study showed that under high heroin dose conditions, preference for the drug over food increased following extended versus short access self-administration, with 51% of animals showing a marked preference (Lenoir et al. 2013a). Further research is necessary to determine whether these low levels of preference in rats represent the maximum level or if they are inherent to the access conditions tested.

It should be noted that in humans drug use to the exclusion of other obligations or rewarding activities is typically defined by social impairment (American Psychiatric Association 2013), whereas, most of the work in animals has focused on choice between drug versus a sweet/palatable food. As such, it may be of interest to develop animal models that incorporate social factors, such as the presence of a drug versus non-drug using peer (Smith and Strickland 2017; Smith et al. 2016; Strickland and Smith 2015), into choice procedures.

4.1. Preference for the Drug over Other Reward Options: Individual Differences

Several studies have examined sex differences in cocaine versus non-drug choice behavior following short access self-administration (Kerstetter and Kippin 2011; Kerstetter et al. 2012; Perry et al. 2013; 2015). The results from these studies show that females have higher preferences for cocaine versus food, and are more likely to develop a cocaine preference over time as compared to males (Kerstetter et al. 2012; Perry et al. 2013; 2015). These differences are robust with results showing that under low dose conditions (0.4 mg/kg/infusion), none of the males tested showed a preference for cocaine over food (>50%), whereas, 37% of the females did (Kerstetter et al. 2012). This study further showed that under high dose conditions (1.0 mg/kg/infusion), a majority of females (∼82%), versus a minority of males (∼37%), showed a preference for cocaine over food (Kerstetter et al. 2012). These findings are notable considering that this behavioral phenotype developed following short access self-administration (maximum of 25 infusions/day or 3-hr sessions), indicating that females are suspectible to developing this feature addiction even under short access conditions. In further support of this idea, Perry et al. (2013) showed that a preference for cocaine over food was more likely to develop in females, and was associated with other features of addiction including an enhanced motivation for cocaine, and higher levels of cocaine seeking. As with the development of an enhanced motivation for cocaine, a preference for cocaine over food can be abolished by ovariectomy and restored by estradiol replacement (Kerstetter et al. 2012). These findings suggest that, like an enhanced motivation for drug, an enhanced choice for drug over other reward options develops more readily in females as compared to males, and in females, estradiol may be required for its development. Further research is necessary to determine whether these sex and hormone effects extend to other drugs of abuse, and to determine whether they are different following extended versus short access self-administration.

Findings in non-human primates show that individual differences in response to environmental stressors and enrichment can also influence choice for drug over food. For example, Czoty and Nader (2012) showed that the dose needed establish an exclusive choice for drug over food was lower in animals that were responsive to the environmental stressor, and conversely, that a higher dose was needed to maintain exclusive preference in the animals that were responsive to environmental enrichment. This group has also shown that individual differences in social status can influence the development of an exclusive choice for drug over food (Gould et al. 2017).

5. Use Despite Negative Consequences

Numerous studies have incorporated punishment into drug self-administration procedures in animals in an attempt to model the “drug use despite negative consequences” characteristic of human drug addiction. One of the first studies to demonstrate this feature in animals was conducted by Johanson (1977). This study examined the effects of shock in rhesus monkeys self-administering cocaine under a choice procedure in which both choices were an intravenous infusion of cocaine, but one was paired with electronic shock. She found that while shock suppressed responding for cocaine, the effect could be surmounted by increasing the dose of cocaine that was paired with shock until it was preferred to the cocaine only option. This finding, together with a later finding showing that rhesus monkeys continue to self-administer cocaine that are paired with intermediate intensity shock levels (Bergman and Johanson 1981), demonstrated that drug-taking behavior in animals can become resistant to punishment. This characteristic, resistance to punishment, has subsequently been used to define the development of an addicted phenotype in preclinical studies (Nader and Banks 2014; Vanderschuren and Ahmed 2013). It should be noted that a sustained suppression of drug self-administration can be maintained under lower drug dose conditions (Johanson 1977), and when higher intensity levels of shock are used (Bergman and Johanson 1981). Recent studies have taken advantage of the ability of shock and other punishers to suppress drug self-administration, and like choice studies with palatable food choices, are using punishment as a means to maintain voluntary abstinence (for review see (Venniro et al. 2016). This type of positive punishment approach to maintain abstinence is similar to the use disulfiram (Antabuse), an FDA-approved treatment for alcohol use disorder, which produces an immediate negative reaction to alcohol (e.g., nausea, vomiting; Suh et al. 2006).

Although most of the initial work on resistance to punishment was conducted in non-human primates (Nader and Banks 2014), recent work has examined this phenotype in rats (e.g. Gancarz-Kausch et al. 2014; Jonkman et al. 2012), and mice (Pascoli et al. 2015). One of the first studies to demonstrate this characteristic in rats was conducted by Deroche-Gamonet et al. (2004). This study used several measures to define the development of a cocaine-addicted phenotype including resistant to punishment, increased drug-seeking, and enhanced motivation to obtain the drug. Under the short access conditions that were used (3, 40-min sessions/day), they showed that a subpopulation of animals developed resistance to punishment over the 3-month cocaine self-administration period, and 17% of the sample were positive on each of the three defining measures (i.e., in the 66th to 99th percentile of the distribution). They also showed that the three addictionlike behaviors loaded equally on one factor, suggesting that each measure reflects the development of an addicted phenotype. This is important because it suggests that if one of the measures is not apparent, the conditions used may not have been sufficient for inducing an addicted phenotype (but see Waters et al. 2014).

Resistance to punishment has been observed under both short and extended access conditions; however, it becomes more apparent over time with prolonged access and is more robust after extended versus short access self-administration (Deroche-Gamonet et al. 2004; Pelloux et al. 2015). Additionally, like motivation and drug-seeking, resistance to punishment increases over a period of abstinence. For example, Gancarz-Kausch et al. (2014) examined the effect of punishment on cocaine self-administration following extended-access self-administration and a short (1 day) versus a protracted period of abstinence (30 days), and found that it was less effective at reducing cocaine self-administration following protracted abstinence. Some animals also maintain a resistance to punishment even when an alternative reinforcer is introduced to help facilitate a punishment-induced suppression of drug taking (Pelloux et al. 2015).

Although shock is the most common form of punisher used in these studies, histamine (e.g., Gancarz-Kausch et al. 2014), and the kappa opioid agonist salvinorin A (Freeman et al. 2014) have also been used for intravenous drug self-administration. For oral drug self-administration, the bitter tastant quinine has been commonly used to demonstrate use despite negative consequences. For example, Wolfgramm and colleagues showed in a series of studies that alcohol consumption becomes insensitive to the effects of quinine following excessive alcohol consumption and a protracted period of abstinence (Wolffgramm and Heyne 1991; 1995). Such effects were observed following voluntary consumption under extended access conditions, but not following short access consumption or following forced consumption. They also showed similar effects in this quinine model for oral consumption of etonitazene (Heyne 1996), amphetamine (Galli and Wolffgramm 2004; Heyne and Wolffgramm 1988), and nicotine (Galli and Wolffgramm 2011). Another example of continued use despite negative consequences is that during extended access heroin self-administration, a large proportion of rats develop self-injurious behaviors (Lenoir and Ahmed 2007), and despite these behaviors, they continue to self-administer heroin.

5.1 Use Despite Negative Consequences: Individual Differences

A number of individual differences have been identified that influence the effect of punishment on cocaine self-administration behavior under short access conditions including sex and saccharin preference (Carroll and Smethells 2016). For example, females and low saccharin preferring rats show a slower return to baseline levels of self-administration after punishment is discontinued as compared to males and high saccharin preferring rats (Holtz et al. 2013). Adolescent rats show less of a decrease in response to punishment as compared to adults, but both groups recovered to baseline at the same rate (Holtz and Carroll 2015). However, under these short access conditions, none of the groups displayed resistance to punishment, and all groups showed a marked suppression in levels of drug intake during the punishment phase. Very little is known regarding sex and other individual differences in the effects of punishment on drug taking at later stages of the addiction process (i.e. following extended access self-administration).

6. Enhanced Drug-Seeking/Relapse Vulnerability

Vulnerability to relapse following abstinence is a major challenge for treating drug addiction. Various types of stimuli can precipitate craving in humans such as exposure to the cues associated with drug use (e.g. people, places, drug paraphilia, smells), and re-exposure to small doses of the drug itself (Bossert et al. 2013; O'Brien et al. 1998). Stress is also a critical factor that can lead to craving and relapse (Sinha et al. 2011). Animal models of drug-seeking and relapse have been used extensively over the past several decades, and have provided new potential targets for relapse intervention (Yahyavi-Firouz-Abadi and See 2009). In animals, drug-seeking and relapse vulnerability are typically studied using an extinction/reinstatement paradigm (Bossert et al. 2013; Venniro et al. 2016). With this procedure, animals are first given either short or extended access to the drug, and once the period of drug access is complete, drug infusions are discontinued. Drug seeking is then assessed by examining levels of responding on the formerly active lever under non-reinforced conditions (i.e., drug is no longer available). After responding reaches some criterion of unresponsiveness, or extinguishes, the ability of various stimuli to reinstate drug-seeking is then determined also under non-reinforced conditions. A stimulus is said to reinstate responding if it causes an increase in responding on the lever that was formerly reinforced by the drug. The results from studies using the reinstatement model have revealed that the conditions that reinstate drug-seeking in animals are similar to those that trigger craving and relapse in humans, including small doses of the drug itself, cues associated with the drug, the context associated with drug use, and stress (e.g. de Wit and Stewart 1981; 1983; Katz and Higgins 2003), thereby demonstrating the predictive validity of this model.

While preclinical studies on drug-seeking and relapse have traditionally been conducted following short access self-administration, emphasis is now being placed on examining changes following extended access self-administration. These newer studies also emphasize changes in drug-seeking behavior following a protracted period of abstinence (Doyle et al. 2014; Wolf et al. 2016). Most notable is the demonstration that following abstinence from extended access drug self-administration levels of cue-induced drug-seeking progressively increase, or incubate, over time (Grimm et al. 2001). This incubation effect is a robust and consistent phenomenon that has been observed in humans and animals, and for a variety of different drugs of abuse including cocaine, methamphetamine, heroin, nicotine, and alcohol (for review see Pickens et al. 2001).

Numerous studies have used levels of drug-seeking and its incubation over abstinence as a means to define the development of an addicted phenotype, with increases determined relative to short access controls (e.g. Doyle et al. 2014), or to earlier time-points during abstinence (e.g. Miller et al. 2017). For example, several studies have shown that levels of cocaine, methamphetamine, and heroin-seeking are enhanced following extended versus short access self-administration (Ahmed et al. 2000; Doyle et al. 2014; Fischer et al. 2013; Fischer-Smith et al. 2012; Imperio and Grigson 2015; Kippin et al. 2006; Lenoir and Ahmed 2007; Rogers et al. 2008; Wolf 2016). Additionally, while the incubation effect has been observed for following short access self-administration, it is more robust and prolonged following extended access self-administration (e.g. Fischer et al. 2013; Pacchioni et al. 2011). There is also evidence showing that the molecular changes that underlie the incubation effect only occur following extended access self-administration (e.g., increase an calcium-permeable AMPA receptors in the nucleus accumbens; Wolf 2016). Another measure of drug-seeking that has been used to define the development of an addicted phenotype is the persistence of responding during a period of signaled drug unavailability. This measure is typically used as part of a composite addiction score, which typically includes two other measures, motivation for the drug and resistance to punishment (e.g. Belin and Deroche-Gamonet 2012).

Recent studies have also examined the potential for different environmental and pharmacological interventions to decrease drug-seeking and the incubation effect following extended access self-administration. For example, several studies have shown that exercise during abstinence can prevent the development of the incubation of cocaine, nicotine, and methamphetamine seeking (Beiter et al. 2016; Lynch et al. 2013; Peterson et al. 2014 a; b; Sanchez et al. 2013; Sobieraj et al. 2016; Zlebnik and Carroll 2015). Extinction training during abstinence can also prevent the later increase in drug-seeking (Kelamangalath and Wagner 2010), with findings showing that the effects of extinction training can be enhanced by providing an enriched environment during abstinence (Thiel et al. 2011). Recent finding also suggest that palatable food self-administration during abstinence, as a means of maintaining voluntary abstinence, may also block the development of the incubation of heroin, but not methamphetamine seeking (Venniro et al. 2016). The results from these studies have also revealed a number of potential targets for preventing drug relapse such as orexin receptors (Schmeichel et al. 2015), brain-derived neurotrophic factor (Koskela et al. 2017), dopamine D3 receptors (Sokoloff and Le Foll 2017), metabotropic glutamate receptors (Pomierny-Chamioło et al. 2014), and pathways involved in epigenetic regulation and synaptic plasticity (Hitchcock and Lattal 2014; Wolf 2016).

6.1 Enhanced Drug-Seeking/Relapse Vulnerability: Individual Differences

While a number of factors have been identified that influence drug-seeking following short access self-administration (Arenas et al. 2016; Gardner 2000; Reichel and Bevins 2009; Shalev et al. 2002), much less is known about the factors affecting these behaviors following extended access self-administration (except perhaps for alcohol consumption and relapse, Lê and Shaham 2002). For sex differences, following short access self-administration, females have been reported to have markedly higher levels of methamphetamine seeking (Ruda-Kucerova et al. 2015), and a prolonged time-course for the incubation of cocaine-seeking (up to 6 months in females, Kerstetter et al. 2008). In contrast, following extended access self-administration and abstinence, levels of cocaine, methamphetamine, and heroin seeking are similar between males and females (Doyle et al. 2014; Lynch et al. 2005; Peterson et al. 2014 b; Sanchez et al. 2014; Venniro et al. 2016, but see Reichel et al. 2012), again suggesting that once addiction develops, males and females may be similar at a behavioral level. Notably, levels of drug seeking appear to be highest in females during estrus as compared to other phases, and this difference is observed following both short and extended access self-administration (Peterson et al. 2014 b; Kerstetter et al. 2008).

There is also evidence suggesting that risky decision-making, as assessed using a rat gambling task, predicts vulnerability to developing features of addiction, including enhanced drug seeking (Ferland and Winstanley 2017). The results from this study also indicate that drug use in risk-prone animals can further acerbate risky decision-making. Avoidance of a drug-paired saccharin cue has also been shown to predict enhanced drug-seeking following extended access self-administration (Imperio and Grigson 2015).

7. Conclusions and Future Research

The shift in addiction research from animal models of drugs reinforcement to models that capture features critical for human drug addiction is likely to advance the understanding of the disease. The emphasis on including both males and females also marks a significant advance that will address the sex bias in preclinical studies on addiction (∼80% of the research is conducted in males; Zakiniaeiz et al. 2016), and help to inform our understanding of addiction in both sexes. In humans, only a small percentage of the total number of users develop a substance use disorder indicating that mere drug use does not inevitably lead to addiction. This also appears to be true for preclinical studies, with results suggesting that the experimental conditions needed to induce features of addiction in the majority of animals are fairly specific, and depend on a number of pharmacological and environmental factors. In order to understand sex differences in addiction, as well as other individual differences, it is also important to identify both the threshold conditions that induce features of addiction in some, but not all animals, and the optimal conditions that induce features of addiction in the majority of the animals. The development and use of such models will be essential for our understanding of neurobiology of drug addiction and for the development of behavioral and pharmacological treatments in men and women.

Highlights.

• Procedures used to induce features of addiction in animals are reviewed.

• Extended access drug self-administration is critical for modeling addiction.

• Features emerge over abstinence and increase in magnitude over time.

• In females, features develop sooner/after less drug, and may require estradiol.

• Future research is needed to optimize conditions that induce addiction features.