PHARMACOTHERAPIES FOR DECREASING MALADPATIVE CHOICE IN DRUG ADDICTION: TARGETING THE BEHAVIOR AND THE DRUG

Abstract

Drug addiction can be conceptualized as a disorder of maladaptive decision making in which drugs are chosen at the expense of pro-social, nondrug alternatives. The study of decision making in drug addiction has focused largely on the role of impulsivity as a facilitator of addiction, in particular the tendency for drug abusers to choose small, immediate gains over larger but delayed outcomes (i.e., delay discounting). A parallel line of work, also focused on decision making in drug addiction, has focused on identifying the determinants underlying the choice to take drugs over nondrug alternatives (i.e., drug vs. nondrug choice). Both tracks of research have been valuable tools in the development of pharmacotherapies for treating maladaptive decision making in drug addiction, and a number of common drugs have been studied in both designs. However, we have observed that there is little uniformity in the administration regimens of potential treatments between the designs, which hinders congruence in the development of single treatment strategies to reduce both impulsive behavior and drug choice. The current review provides an overview of the drugs that have been tested in both delay-discounting and drug-choice designs, and focuses on drugs that reduced the maladaptive choice in both designs. Suggestions to enhance congruence between the findings in future studies are provided. Finally, we propose the use of a hybridized, experimental approach that may enable researchers to test the effectiveness of therapeutics at decreasing impulsive and drug choice in a single design.

1. Introduction

Drug addiction can be conceptualized as a disorder of maladaptive decision making in which drugs are chosen at the expense of nondrug alternatives that are key to maintaining good health and supportive social connections (e.g., steady employment; saving money; relationships with family). The decision to take a drug is influenced by a variety of factors that involve an interplay between the organism, the drug, and the environment. Characterizing the ways in which these elements interact to determine drug choice has been a rich area of study for many years, and this work has illuminated our understanding of the underpinnings of behaviors that lead to and perpetuate drug addiction (Banks and Negus, 2012; Lamb et al., 2016). This work has also provided an array of targets for pharmacological intervention, and many drugs across a variety of classes have been developed and tested for their effectiveness at decreasing maladaptive decision making in drug addiction (Banks and Negus, 2012; Bickel et al., 2016; Winstanley, 2011).

In the current review, we submit that the study of decision making in drug addiction has focused primarily on two decision scenarios of divergent emphasis, and these differences in focus have led to separate tracks of treatment development that can benefit from an integration of the findings. The first area of study is impulsive choice in the addiction process, which has largely been studied in the context of delay discounting. Delay discounting refers to a decrease in a reinforcer’s value as a function of its delay to receipt. In the delay discounting context, impulsivity is operationally defined as a tendency to choose smaller, more immediate reinforcers at the expense of larger, more delayed reinforcers (Green and Myerson, 2004). Evidence indicates that impulsive decision making facilitates drug abuse and relapse to drug taking after abstinence (Leung et al., 2017). The target effect for putative treatments for impulsive choice is a reduction in the tendency to choose small, immediate gains over larger, delayed ones with the expectation that decreasing impulsive decision making will also decrease the impulsive choice to take drugs. In the study of impulsive choice, the rationale for pharmacotherapy design is focused primarily on behavior, or more specifically the brain substrates that modulate impulsive decision-making behavior.

The second area of study is the determinants of drug choice, which has largely focused on the effectiveness of treatments at shifting an organism’s choice behavior away from drugs and towards nondrug alternatives. Potential pharmacotherapies in this scenario are typically selected based on their ability to block, mimic, or obfuscate the pharmacological effects of the target drug of abuse with the aim of decreasing the value of the drug relative to nondrug alternatives in the environment. Thus, when using drug-choice designs to develop pharmacotherapies, the primary target is the drug and its pharmacological effects.

Medications development using models of impulsivity and drug choice have been thoroughly reviewed (see Banks and Negus, 2012, 2017; Bickel et al., 2016; Perry and Carroll, 2008; Winstanley, 2011). The current review will focus only on drugs that have been tested for therapeutic potential in both delay-discounting and drug-choice designs with the aim of highlighting treatments that show efficacy in decreasing the “maladaptive” choice in both designs. Although other approaches have been used to investigate drug reinforcement (e.g., progressive-ratio schedules of reinforcement; behavioral economic demand procedures; Hursh and Silberberg, 2008; Richardson and Roberts, 1996) and impulsivity (e.g., Go/No-Go task; Stop Signal Task; Winstanley, 2011), we are limiting our review to delay-discounting and drug-choice studies because these procedures are congruent in their focus on a critical component of decision making in addiction: choices between alternatives. Similarities and differences in the treatment regimens and other aspects of experimental design will be discussed with the aim of enhancing congruence between future delay-discounting and drug-choice studies. Finally, recent studies that have hybridized measures of delay discounting and drug choice into single designs will be discussed as a future direction for testing pharmacotherapies for maladaptive decision making in addiction.

2. Overview of Delay Discounting and Drug vs. Nondrug Choice

2.1 Delay Discounting

As stated above, delay discounting refers to the devaluation of a reinforcer as a function of its delay to receipt. The relationship between a reinforcer’s value and delay is well described in humans and animals by a hyperbolic equation, and this mathematical description has been successfully applied to discounting of both drug and nondrug reinforcers (Green and Myerson, 2004; Huskinson et al., 2015, 2016; Maguire et al., 2016; Woolverton et al., 2007). While all people discount the value of delayed reinforcers, having a heightened propensity to choose smaller, more immediate reinforcers in lieu of larger delayed ones (i.e., steep delay discounting) is thought to be a feature of impulsive behavior (Green and Myerson, 2004).

A large body of research indicates that drug abusers discount the value of delayed outcomes more steeply than non-abusers (for reviews see Bickel et al, 2012; Perry and Carroll, 2008). Furthermore, it is well established that drugs of abuse increase delay discounting in humans and animals (Perry and Carroll, 2008). As such, origins of steep discounting in drug abusers may be innate to the person and/or derived from a drug history. A tendency for steep discounting may perpetuate drug abuse by increasing the likelihood of choosing drugs and their immediate effects in lieu of saving resources for the optimization of future, nondrug goals (e.g., paying rent; accumulating savings). Alternatively, steep discounting can also lead to maladaptive choices when the avoidance of negative but delayed consequences of drug addiction (e.g., sickness, loss of social support) is undervalued in the decision-making process.

Brain substrates associated with steep discounting and other forms of impulsivity have been identified (see Winstead, 2011). These studies have provided potential targets for decreasing impulsive decision making through pharmacological intervention. Accepting that devaluation of delayed outcomes facilitates the decision to take drugs, treatments that increase the value and selection of delayed nondrug outcomes would be expected to decrease maladaptive decision making in drug addiction.

2.2 Drug vs. Nondrug Choice

Drug self-administration techniques have been used for decades in humans and animals to characterize the determinants of drug taking and addiction and to develop pharmacological and behavioral interventions for drug abuse (Jones and Comer, 2013; Panlilio and Goldberg, 2007). In a typical self-administration test, an organism operates a single manipulandum (e.g., pressing a lever) to initiate the delivery of a drug, thus giving the organism control over drug intake. Drugs with robust abuse liability (e.g., cocaine, heroin) engender higher rates of injections than do drugs with no apparent abuse potential (e.g., atropine) or drug vehicles (e.g., saline). This relatively straight-forward design enables researchers to study drugs as reinforcers under a range of interventional and observational scenarios, and it is the cornerstone upon which the study of the determinants of drug-taking has been built (see Huskinson et al., 2014). However, there are limits to the conclusions that can be derived from the results of studies using a single-manipulandum design. For example, an effective treatment for drug abuse will by definition decrease drug taking. However, when a treatment reduces drug taking in a single-manipulandum design, it is difficult to discern between a targeted decrease in the drug’s reinforcing effect or a generalized decrease in behavior due to non-specific effects of the treatment (e.g., sedative effects; motor-disrupting effects).

One way to address this concern is to use drug-choice procedures in which the organism can choose between a drug and a nondrug alternative (e.g., food pellets, money) on separate manipulanda. Under this arrangement, the reinforcing effect of the drug is measured in terms of relative allocation of choice behavior among the two alternatives. This metric is fairly resistant to non-specific, rate-altering effects because percentage choice between the alternatives does not co-vary with response rate (see Banks and Negus, 2012, 2017). Thus, an effective treatment in a drug-choice procedure is operationalized by a decrease in the percentage of drug choice relative to nondrug choice.

In the study of decision-making in drug addiction, drug-choice procedures are highly translational because they model the real-life choice faced by drug abusers; i.e., whether or not to take drugs at the expense of nondrug alternatives. It is well established that drug-taking behavior is altered by the concurrent availability of nondrug alternatives (Carroll et al., 1989, Nader & Woolverton, 1991, 1992; Stitzer et al., 1980; 1983). Accepting that there are always nondrug alternatives in the natural ecology of human drug taking, it is important that the effectiveness of drugs as reinforcers be studied in relation to concurrently-available, nondrug alternatives so that treatments can be evaluated based on their by their ability to diminish the subjective value of drugs relative to nondrug options in the environment. Treatments that selectively decrease the subjective value of drugs could be highly effective at decreasing maladaptive decision making in drug addiction.

In the following, we will review the effects of drugs that have been tested for their ability to decrease delay discounting and drug choice. Drugs are listed in Table 1 and are distinguished by their protein targets (i.e., monoamine transporters or neurotransmitter receptors). Treatment regimens are distinguished as either acute (< 3 consecutive days of treatment at a dose) or chronic (≥ 3 consecutive days of treatment at a dose), and the treatment effects are described as decreasing, increasing, or having no effect on drug choice and impulsive choice. The discussion will focus only on drugs that decreased both impulsive choice and drug choice in at least one study for each design.

Table 1.

Tests of pharmacological treatments on delay discounting and drug choice

Target / Treatment	Delay Discounting				Drug Choice
	A/C	Effect	Species	Reference	A/C	Target Drug	Effect	Species	Reference
Monoamine Transporters
d-Amphetamine	A		Mouse	Oberlin et al., 2010	C	Cocaine		NHP	Banks et al., 2013; Negus, 2003
	A		Rat	Broos et al., 2012b, Floresco et al., 2008; van Gaalan et al., 2006; Wade et al., 2000; Winstanley et al., 2003, 2005	C	Cocaine		Human	Greenwald et al., 2010
	A		Human	de Wit et al., 2002	Both	Cocaine		Rat	Thomsen et al., 2013
	A	--	Rat	Barbelivien et al., 2008; Krebs & Anderson, 2012; Stanis et al., 2008b	A	Heroin (withdrawal)	--	NHP	Negus & Rice, 2009
	Both	--	Rat	Huskinson et al., 2012	C	Cocaine	--	NHP	Hutsell et al., 2016
	A		Mouse	Isles et al., 2003	C	Methamphetamine	--	NHP	Schwienteck & Banks, 2015
	A		Rat	Cardinal et al., 2000; Maguire et al., 2014; Orsini et al., 2017; Tanno et al., 2014	C	Hydromorphone	--	Human	Greenwald et al., 2010
	A	--	Rat	Eubig et al., 2014; Koffarnus et al., 2011; Wooters & Bardo, 2011	C	Cocaine + Hydromorphone	--	Human	Greenwald et al., 2010
	A	--	Human	Acheson & de Wit, 2008; Reed & Evans, 2016
	Both	--	Rat	Stanis et al., 2008a
	A		Mouse	Helms et al., 2006
	A		Rat	Evenden & Ryan, 1996; Krebs et al., 2016; Slezak & Anderson, 2009; Zeeb et al., 2016
	C		Rat	Gipson & Bardo, 2009
Methylphenidate	A		Rat	Pitts & Mckinney, 2005; Slezak et al., 2014; van Gaalan et al., 2006	C	Cocaine		Human	Collins et al., 2006
	A		NHP	Rajala et al., 2015	C	Methamphetamine	--	NHP	Schwienteck & Banks, 2015
	C		Rat	Broos et al., 2012a; Juarez & Guerrero-Alvarez, 2015
	Both		Rat	Slezak & Anderson, 2011
	A		Human	Pietres et al., 2003
	A	--	Human	Shiels et al., 2009
	A		Rat	Tanno et al., 2014
	A	--	Rat	Yates et al., 2014
	A	--	Rat	Wooters & Bardo, 2011
Bupropion	C	--	Human	Ashare & McKee, 2012	A	Cocaine		Human	Stoops et al., 2012
	A	--	Human	Acheson & de Wit, 2008	C	Methamphetamine	--	NHP	Banks & Blough, 2015
Modafinil	A		Human	Schmaal et al., 2014	C	Cocaine		Human	Foltin et al., 2016; Hart et al., 2008
	C	--	Human	Joos et al., 2013	A	Methamphetamine	--	Human	De La Garza et al., 2010
					C	Cocaine	--	Human	Verrico et al., 2014
Desipramine	A	--	Rat	van Gaalen et al., 2006	A	Cocaine	--	NHP	Gasior et al., 2003
Dopamine Receptors
Buspirone	C	--	Human	Bolin et al., 2016	A	Cocaine	--	NHP	Czoty & Nader, 2015
	Both		Rat	Liu et al., 2004	C	Methamphetamine	--	NHP	John et al., 2015
	A	--	Human	Strickland et al., 2017	C	Cocaine	--	Human	Bolin et al., 2016
	A		Rat	Bizot et al., 1999	C	Cocaine		NHP	John et al., 2015
Eticlopride	A	--	Rat	van Gaalen et al., 2006	A	Cocaine	--	NHP	Czoty & Nader, 2013
Flupenthixol	A	--	Rat	Eubig et al., 2014	C	Cocaine	--	NHP	Negus, 2003
	A		Rat	Cardinal et al., 2000; Floresco et al., 2008; Wade et al., 2000
Haloperidol	A		Rat	Boomhower & Rasmussen, 2014	C	Cocaine		NHP	Woolverton & Balster, 1981
Norepinephrine Receptors
Propranolol	A		Rat	Schwager et al., 2014	A	Cocaine		Rat	Perry et al., 2015
Clonidine	A		Rat	van Gaalen et al., 2006	A	Heroin (withdrawal)	--	NHP	Negus & Rice, 2009
Serotonin Receptors
8-OH-DPAT	A		Rat	Bizot et al., 1999	A	Cocaine		NHP	Czoty et al., 2005
	A	--	Rat	Evenden & Ryan, 1999
	A		Rat	Stanis et al., 2008b; Winstanley et al., 2005
Lorcaserin	A	--	Rat	Higgins et al., 2016	C	Cocaine	--	NHP	Banks & Negus, 2016
Mu Opioid Receptors
Methadone	C	--	Human	Robles et al., 2011	C	Heroin		NHP	Griffiths et al., 1975
					C	Heroin	--	NHP	Negus, 2006
					C	Cocaine	--	NHP	Negus & Mello, 2004
Morphine	A	--	Rat	Slezak et al., 2014	A	Heroin (withdrawal)		NHP	Negus & Rice, 2009
	A		Rat	Kieres et al., 2004; Pattij et al., 2009; Pitts & McKinney, 2005; Tanno et al., 2014
	Both		NHP	Maguire et al., 2012
Naloxone	A	--	Rat	Pattij et al., 2009	A	Heroin		Rat	Gerber et al., 1985
	A	--	Rat	Harvey-Lewis et al., 2015	A	Heroin		Human	Comer et al., 2005
					C	Heroin		NHP	Negus, 2006
					A	Heroin		NHP	Griffiths et al., 1975, 1981
Naltrexone	A		Human	Weber et al., 2016	C	Heroin		Human	Sullivan et al., 2006
	A	--	Mouse	Oberlin et al., 2010
	A	--	Rat	Kieres et al., 2004
	A	--	Human	Mitchell et al., 2007
Glutamate Receptors
Ketamine	A	--	Rat	Yates et al., 2017	A	Cocaine		Human	Dakwar et al., 2017
	A		Rat	Floresco et al., 2008
	A		NHP	Maguire et al., 2012
Memantine	A	--	Mouse	Oberlin et al., 2010	A	Cocaine	--	Human	Collins et al., 2007
	A	--	Rat	Yates et al., 2017	C	Cocaine	--	Human	Collins et al., 1998, 2006
GABA Receptors
Diazepam	A		Rat	Evenden & Ryan, 1996	Both	Cocaine		Rat	Augier et al., 2012
	Both	--	Rat	Huskinson & Anderson, 2012
	A	--	NHP	Maguire et al., 2012
	A	--	Human	Acheson et al., 2006; Reynolds et al., 2004
	A		Rat	Theibot et al., 1985
Acetylcholine Receptors
Varenecline	C	--	Human	Asahre & McKee, 2012	A	Nicotine		Rat	Panlilio et al. 2015
CRF Receptors
Antalarmin	A		Rat	Gondre-Lewis et al., 2016	C	Heroin (withdrawal)	--	NHP	Negus & Rice, 2009

A = Acute; C = Chronic; = Increase in delay discounting or drug choice; = Decrease in delay discounting or drug choice; -- = No change; NHP = Nonhuman Primate

3. Psychostimulant Medications

3.1 d-Amphetamine

Dextro-amphetamine (D-AMPH; most widely prescribed as Adderall®), is a monoamine neurotransmitter releaser. When bound to monoamine transporters, D-AMPH preferentially releases the catecholamine neurotransmitters, dopamine (DA) and norepinephrine (NE), and to a lesser degree serotonin (5-HT), from presynaptic terminals, resulting in transient increases in extracellular concentrations of these neurotransmitters (Heal et al., 2013). A large body evidence indicates that abnormalities in brain dopaminergic function contribute to impulsive decision making in drug addiction (Ballard et al., 2015; Joutsa et al., 2015), and D-AMPH’s dopaminergic actions are thought to be a key component of its efficacy in the treatment of Attention-Deficit/Hyperactivity Disorder (ADHD) and other disorders of impulsivity (Briars and Todd, 2016; Winstanley, 2011).

It is therefore not surprising that D-AMPH has received the most experimental attention of any drug as a treatment for decreasing impulsive choice in delay discounting. The majority of studies, which have been conducted in both humans and animals, have demonstrated that D-AMPH treatment can decrease delay discounting, but null and opposite effects have also been reported (see Table 1). Most of the studies within species are reasonably similar in terms of dose and route of D-AMPH administration, but other procedural differences may account for the differences in D-AMPH’s effects across studies. For example, reversing the order in which the delays are presented within a session (i.e., starting with high delays and decreasing to lower delays instead of the more typical low-to-high arrangement) can negate or reverse D-AMPH’s treatment effects on discounting (Maguire et al., 2014; Orsini et al., 2017; Tanno et al., 2014).

Differences in D-AMPH’s effectiveness may also be due to variation in baseline discounting rates between and within studies. Accumulating evidence indicates that the effectiveness of D-AMPH at decreasing impulsive choice depends on the degree to which the subjects discount delayed reinforcement at baseline (i.e., before treatment). Subjects with higher baseline rates (i.e., more impulsive subjects) are more likely to show a decrease in discounting rates after D-AMPH treatment than subjects with more shallow (i.e., less impulsive) discounting baselines (Barbelivien et al., 2008; Huskinson et al., 2012; Krebs and Anderson, 2012; Winstanley, 2003). A recent meta-analysis addressed this issue and concluded that the effects of variation in baseline rates on responsivity to pharmacological treatment are ubiquitous in the literature and largely unreported (Bickel et al., 2016). Designing future studies to account for or incorporate baseline discounting differences into the experimental design will likely render more consistent results with D-AMPH (and other treatments), and may lead to a more optimized treatment approach that is designed to consider baseline impulsivity in patients.

Interestingly, almost all of the delay discounting studies in which D-AMPH was tested used an acute treatment regimen (Table 1), a rather surprising observation given that D-AMPH is typically prescribed as a chronic treatment for ADHD and other disorders of impulsivity (Briars and Todd, 2016). Two studies that incorporated chronic D-AMPH administration found opposite results in rats. Huskinson et al. (2012) reported that chronic, experimenter-administered D-AMPH decreased impulsive choice across a select series of delays in Fischer 344 rats. However, using an D-AMPH self-administration procedure, Gipson and Bardo (2009) found that relatively long sessions of D-AMPH self-administration occurring in the same day as delay discounting tests increased discounting rates over days, an effect that reverted back to baseline when D-AMPH self-administration was terminated. Notably, the total daily intake of D-AMPH in the self-administration study was relatively high (5–10 mg/kg/day) compared to Huskinson et al. (2012; ~1.0 mg/kg/day). Thus, the opposite effects of chronic D-AMPH in these studies could be due to a number of factors including dose, route of administration, and contingency of administration. More work using chronic treatment regimens is clearly needed to determine the optimal parameters under which D-AMPH can reduce impulsive discounting.

In studies of drug choice, D-AMPH was also the most extensively studied (see Table 1). Most of the studies examined D-AMPH’s effects on choice for commonly-abused stimulants (cocaine or methamphetamine), although two studies tested D-AMPH’s effectiveness on reducing choice for a mu opioid agonist and found no effect in humans or NHPs (Greenwald et al., 2010; Negus and Rice, 2009). Using D-AMPH to decrease stimulant choice can be considered a form of agonist-replacement therapy (see Rush and Stoops, 2012; Stoops and Rush, 2013). In four of the five studies that tested D-AMPH on cocaine choice, four reported it to be effective at decreasing cocaine choice (Table 1). The effectiveness of D-AMPH at decreasing cocaine choice is consistent with relatively positive findings in clinical trials with cocaine abusers (Stoops and Rush, 2013).

We know of one study in which D-AMPH was tested on methamphetamine choice. In that report, chronic administration of D-AMPH failed to reduce methamphetamine choice in NHPs choosing between methamphetamine and food (Schwientech and Banks, 2015). This is consistent with clinical studies in humans in which D-AMPH treatment did not reduce methamphetamine use (see Stoops and Rush, 2013). That D-AMPH is more effective at reducing cocaine choice than methamphetamine choice may be related to the differences in pharmacology between cocaine and methamphetamine. Cocaine is a monoamine transporter blocker, and methamphetamine is a monoamine releaser (McCreary et al., 2015). The effectiveness of D-AMPH on decreasing stimulant choice may therefore depend on an interaction between D-AMPH’s releasing effects and cocaine’s blocking effects, an arrangement not present in the pairing of the releasers, D-AMPH and methamphetamine.

The treatment regimen for D-AMPH appears to be a critical factor in its effectiveness at decreasing cocaine choice. All studies in which D-AMPH reduced cocaine choice used a chronic administration regimen (Table 1). Notably, one study compared the effects of chronic and acute D-AMPH treatment on cocaine vs. food choice in rats and found that chronic D-AMPH decreased cocaine choice (Thomsen et al., 2013), consistent with reports in humans (Greewald et al., 2010) and NHPs (Banks and Negus, 2013; Negus, 2003). However, when D-AMPH was given acutely in rats, cocaine choice increased (Thomsen et al., 2013). As such, a chronic treatment regimen appears to be an important element of D-AMPH’s effectiveness at decreasing cocaine’s value in relation to nondrug reinforcers.

When comparing the experimental designs and treatment methodologies between the delay-discounting and drug-choice studies in Table 1, it is apparent that there is little “common ground” in the designs, a theme that recurs for most of the drugs listed in Table 1. First, as stated above, the vast majority of delay discounting studies used an acute-treatment approach while the majority of drug-choice studies administered D-AMPH chronically. Second, of the thirty-two studies that tested D-AMPH’s effects in delay discounting, twenty-six were in rats. Alternatively, of the seven total studies in which D-AMPH was tested on drug choice, five were in NHPs and only one study used rats (the remaining study was in conducted in humans).

Ideally, a single treatment strategy will decrease both impulsive choice and drug choice in drug abusers, but identifying such a strategy will require more congruence between delay-discounting and drug-choice designs. One avenue towards this goal would be to test chronic D-AMPH treatment more thoroughly in delay discounting studies to match the treatment regimens used in drug choice. Chronic administration is also a more translational approach because D-AMPH is typically prescribed as a daily treatment (Briars and Todd, 2016). A second avenue towards enhancing congruence will be to do more within-species work in both delay discounting and drug choice designs. Historically, NHPs have not been widely used in the study of delay discounting. However, work in this model has increased in recent years (Freeman et al., 2009, 2012; Huskinson et al., 2015, 2016; Maguire et al., 2012, 2016; Rajala et al., 2015; Woolverton, 2007), which has made the study of medication effects on delay discounting more feasible. On the other hand, rodents have not been widely used in the study of drug vs. nondrug choice. However, Thomsen et al. (2013) and others (Augier et al., 2012; Panlilio et al., 2015) have shown that rats can be used effectively to demonstrate decreases in drug choice with therapeutics.

3.2 Methylphenidate

Methylphenidate (Ritalin®) is a stimulant medication that, along with D-AMPH, forms the frontline treatment strategy for ADHD (Briars and Todd, 2016). Methylphenidate is similar to cocaine in that it blocks the reuptake of monoamines and produces subjective and reinforcing effects that are similar to cocaine (Dursteler et al., 2015). However, methylphenidate’s stimulant effects are relatively mild when taken as prescribed, and its long duration of effect relative to cocaine has attracted attention for its potential use as an agonist-replacement therapy for cocaine dependence (Dursteler et al., 2015).

Methylphenidate had a high success rate in decreasing impulsive choice in delay discounting, and it was effective in rats, NHPs, and humans (see Table 1). There were only two studies that found an increase in discounting rates with methylphenidate, and the effect in one of those studies may be accounted for by an atypical procedural variation (reversing the order of delays; Tanno et al., 2014). Interestingly, methylphenidate decreased delay discounting with actual reinforcers (money) in children but had no effect with hypothetical reinforcers, suggesting that, at least in children, discounting with hypothetical reinforcers may not be an optimal design for detecting treatment effects (Shiels et al., 2009). There are also indications that methylphenidate exposure in childhood produces enduring reductions in impulsivity that last into adulthood (Adriani et al., 2012). Overall, methylphenidate’s effects in delay discounting are consistent with its reported effectiveness in treating disorders of impulsivity like ADHD (Briars and Todd, 2016).

Surprisingly, there has been little work investigating the effects of methylphenidate on drug choice (Table 1), despite the large amount of clinical testing it has received as an agonist replacement therapy for stimulant dependence (Dursteler et al., 2015; Stoops and Rush, 2013). Methylphenidate has been reported to decrease cocaine choice in humans (Collins et al., 2006), but it was ineffective as an agonist replacement therapy at decreasing cocaine taking in a clinical trial (oral administration; Grabowski et al., 1997) and has generally been inconsistent in other clinical studies as a treatment for cocaine addiction (for review see Dursteler et al., 2015). To our knowledge, methylphenidate has not been tested as a treatment for decreasing cocaine choice in an animal model.

Clinical trial data with methylphenidate have been encouraging for D-AMPH use disorder (for review see Stoops and Rush, 2013), suggesting that under some parameters, methylphenidate may decrease methamphetamine taking. However, methylphenidate was ineffective at decreasing methamphetamine choice in NHPs (Shwientck et al., 2015). This is the only study that we know of that has examined the effects of methylphenidate on choice of any kind of drug in an animal model. Clearly, more choice work with amphetamine-type drugs is needed before firm conclusions can be made regarding methylphenidate’s potential as a treatment for abuse of this class of drugs.

In terms of congruence, the differences in treatment regimens used in delay discounting and drug-choice studies with methylphenidate are similar to those described above for D-AMPH. As we suggested above, efforts to include more studies with chronic treatment in delay discounting and rodent studies in drug choice will do much to enhance comparability in treatment approaches.

4. Other Drugs Targeting Monoamine Transporters

4.1 Bupropion

Bupropion, which is clinically prescribed as Wellbutrin® or Zyban®, is a catecholamine reuptake inhibitor that is used clinically for depression, obesity, and smoking cessation (Patel et al., 2016; Stahl et al., 2004). We know of two studies that have tested its effects in delay discounting, both in humans. Bupropion decreased delay discounting in nicotine-deprived male smokers, but did not do so in females (Ashare & McKee, 2012). Notably, discounting rates were higher in males in the placebo condition, and this more impulsive baseline may have been more amenable to change from treatment, as has been suggested (see Bickel et al., 2016). Acheson and de Wit (2008) reported that bupropion had no effect in humans on a delay/probability discounting task, but D-AMPH had no effect in this study either, suggesting that the population or test parameters may have lacked the sensitivity to detect therapeutic effects. Administration regimen may have also been a factor in the differences, as Ashare and McKee (2012) administered bupropion as a chronic treatment while Acheson and de Wit (2008) gave it acutely before testing.

We know of two drug-choice studies in which bupropion was tested, one in humans and one in NHPs. Stoops et al. (2012) reported that acute treatment with bupropion decreased cocaine choice in humans. The other study tested the effects of chronic bupropion on methamphetamine choice in NHPs and found no effect (Banks and Blough, 2015). We know of no animal work that has tested bupropion on cocaine choice or human work that has tested bupropion on methamphetamine choice. Opportunities for further testing are apparent.

4.2 Modafinil

Modafinil (Provigil®) is used clinically to treat narcolepsy and promote wakefulness. It has been shown to have weak effects at catecholamine transporters where it causes modest extracellular increases in these neurotransmitters (Minzenberg and Carter, 2008). Two studies in humans have examined the effects of modafinil in delay discounting. When given acutely, modafinil decreased impulsive choice (Schmaal et al., 2014), but had no effect when given chronically in another study (Joos et al., 2013). Drug-choice studies that have tested modafinil have all used human subjects as well. When given chronically, modafinil decreased cocaine choice in two studies (Foltin et al., 2016; Hart et al., 2008) and had no effect in another (Verrico et al., 2014). Another study examined acute administration of modafinil on methamphetamine choice and found no effect (De La Garza et al., 2010). Notably, we know of no studies that have studied the effects of modafinil on delay discounting and drug choice in an animal model. It has been suggested that modafinil’s effects may depend on the particular drug histories of subjects in clinical trials (Stoops and Rush, 2013). Animal work using delay-discounting and drug-choice designs can do much to experimentally address the effects of drug history on modafinil’s effectiveness at decreasing impulsive and drug choice.

5. Drugs Targeting Neurotransmitter Receptors

5.1 Buspirone

Buspirone (Buspar®), which is used clinically to treat anxiety, is a partial 5-HT_1A agonist and an antagonist at D₂-like receptors. When given acutely, buspirone did not alter performance in delay discounting in rats using food (Liu et al., 2004) or in humans in a design modified to capture sexual risk (Strickland et al., 2017). However, when administered chronically, it decreased delay discounting in rats for food (Liu et al., 2004) and sexual discounting in humans (Bolin et al., 2016), suggesting that chronic administration is an important factor for buspirone’s effectiveness at decreaseing impulsive choice.

Buspirone’s effects on drug choice are mixed. When given chronically in humans, buspirone had no effect on cocaine choice. When administered chronically in NHPs, buspirone increased cocaine choice but had no effect on methamphetamine choice (John et al., 2015a). However, when given acutely, buspirone decreased cocaine choice in dominant monkeys and had the opposite effect in co-housed subordinate monkeys (Czoty & Nader, 2015), suggesting that conditions of higher enrichment may facilitate buspirone’s treatment efficacy for cocaine abuse. Notably, there were no encouraging findings with chronic administration on drug choice, suggesting that buspirone may not be effective as a maintenance medication for decreasing drug choice.

5.2 Naltrexone

Naltrexone is an opioid receptor antagonist used in the treatment of alcohol and opioid use disorders (Aboujaoude and Salame, 2016). Naltrexone has also shown positive effects in the treatment of gambling disorder, suggesting its utility in the treatment of disorders of impulsivity (Leeman et al., 2014). We know of four studies that tested the effects of naltrexone on delay discounting (see Table 1). Only one of the studies, which was conducted in humans, found a decrease in delay discounting with naltrexone (administered acutely; Weber et al., 2016). Naltrexone has also been shown to block morphine-induced increases in delay discounting (Kieres et al., 2004). Notably, none of the studies tested naltrexone as a chronic regimen. This work will be important for translating findings in delay discounting to clinical situations in which naltrexone is prescribed chronically (Aboujaoude and Salame, 2016).

We know of only one study that has tested naltrexone on drug choice, a surprising finding given its high utility in the treatment of opioid dependence. Sullivan et al. (2006) found that naltrexone, when administered chronically as a depot implant, reduced heroin choice in humans. Similar findings have been reported with the shorter-acting opioid antagonist, naloxone, in rats (Gerber et al, 1985), NHPs (Negus, 2006), and humans (when combined with buprenorphine; Comer et al., 2005). It should be noted, however, that for opioid abusers, the effects of mu-opioid antagonists can vary widely depending on the current state of the patient on a dependence continuum. Negus (2006) demonstrated that while naloxone decreases heroin choice in nondependent NHPs, subjects given naloxone in a dependent state increase their choices for heroin, presumably due to precipitated withdrawal induced by the opioid antagonist. Therefore, opioid antagonists are likely to be most beneficial in post-dependent patients as a chronic regimen (see Negus and Banks, 2013 for a review of this issue).

5.3 Diazepam

Diazepam is a GABA agonist in the family of benzodiazepine drugs (Weintraub SJ, 2017). Two studies in rats reported that acute and chronic treatment with diazepam reduced delay discounting, although one of these reports found an increase in discounting at a larger dose (Evenden and Ryan, 1996; Huskinson et al., 2012). Another study in rats also found an increase in discounting with acute diazepam treatment (Theibot et al., 1985). Three studies, one in NHPs (Maguire et al., 2012) and two in humans (Acheson et al., 2006; Reynolds et al., 2004) reported no effect of acute diazepam on delay discounting. We found only one study that tested the effects of diazepam on drug choice (Augier et al., 2012). Rats given acute and chronic administration of diazepam reduced their choices for cocaine in a cocaine vs. saccharin design.

5.4 Varenicline

Varenicline (Chantix®) is a partial nicotinic agonist used clinically to promote smoking cessation (Mihalak et al., 2006). We know of only one study each in which varenicline was tested on delay discounting and drug choice. Varenecline, when administered chronically, was found to decrease delay discounting in human male but not female smokers undergoing abstinence from nicotine (Ashare and McKee, 2012). In the drug-choice study, varenicline decreased nicotine choice in rats (Panlilio et al., 2015). Although studies are minimal for varenecline, the fact that both studies found positive results suggests that more should be done, particularly within a species.

6. Convergence of Designs

We have reviewed the effects of potential therapeutics tested in delay-discounting and drug-choice designs with a focus on identifying drugs that decreased the maladaptive choice in both procedures. We also suggested ways in which to enhance congruence in the treatment strategies between the designs in future work. Looking forward, the efficiency of testing the effects of therapeutics on delay discounting and drug choice might be increased if the tests could be run in a single design. We and others have recently conducted studies that incorporate delay discounting and drug vs. nondrug choice into single designs, and these procedures present new opportunities for testing therapeutics (Bickel et al., 2011; Huskinson et al., 2015, 2016; Maguire et al., 2013). The approach combines allomorphic choice (i.e., choice between qualitatively different reinforcers) with intertemporal choice (i.e., choice between immediate and delayed reinforcers). The experimental arrangements are similar to drug vs. nondrug choice studies with the exception that operating for one of the options results in immediate reinforcement while operation for the other option results in delayed reinforcement (e.g., see Huskinson et al., 2015).

We recently compared hyperbolic discounting of delayed food pellets in NHPs when the immediate alternative was either food or an intravenous injection of cocaine (Huskinson et al, 2015; see Figure 1). As expected, increasing the delay for a relatively large quantity of food decreased choice for that option in a delay-dependent manner when cocaine or food were the immediate options. However, delayed food was discounted much more steeply by the immediate delivery of cocaine than with immediate food, suggesting that immediate drug effects discount delayed nondrug outcomes to a greater degree than immediate nondrug options. Notably, doubling the amount of delayed food pellets did not change the discounting rate for delayed food when cocaine was the immediate option.

Figure 1.

Average normalized values of delayed food pellets as a function of the delay to food delivery for three monkeys that were tested in all three conditions. The symbols represent the predictions of the average discounting function for each condition. From Huskinson et al., 2015; Printed with permission from Elsevier.

An effective treatment in this design should not only decrease drug choice, it should also increase the relative value of the delayed, nondrug alternative. Graphically, this would be illustrated by a decrease in the steepness of the discounting function for delayed food when drug is the immediate option. To have such an effect, the treatment must counteract the degree to which the delay for the delivery of the nondrug increases the reinforcing potency of the immediate drug. This interactive process is illustrated in idealized data in which subjects choose between cocaine and food (see Figure 2). When there is no delay or pharmacological manipulation applied to the choice situation, choice for cocaine increases from low to high in a dose-dependent manner relative to a fixed amount of food. When a delay is imposed on the delivery of food, cocaine’s potency as a reinforcer increases in a delay-dependent manner, which is represented in the graph as leftward shifts in the dose-response function (see Huskinson et al., 2015 for an actual demonstration). Alternatively, treating the subjects chronically with D-AMPH decreases cocaine’s potency as a reinforcer, and this is depicted by rightward shifts in the dose-response function (see Negus, 2003 for an actual demonstration).

Figure 2.

The graph depicts an idealized choice study in which subjects choose between a range of doses of cocaine and a fixed amount of a nondrug alternative (represented by food here). Cocaine’s potency as a reinforcer is increased with delay to food delivery and decreased with amphetamine treatment. Amph = Amphetamine

We have yet to see a demonstration of what a pharmacological treatment will do to the discounting rate of a delayed, nondrug outcome when a drug is the immediate option. Doing so will require an approach that can quantify changes in hyperbolic discounting because testing the effects of a treatment on a single dose of a drug can not separate the treatment’s effects on potency vs. rate of discounting. To better understand this distinction, we will walk through the analysis. We begin by applying linear regression to the dose-response functions like those depicted in Figure 2. The resulting ED₅₀ values serve to quantify the potency of cocaine under the various conditions. As the dose-response relations move leftward with increasing delay to food delivery, the diminishing value of delayed food is quantified and scaled according to the ED₅₀ value for immediate cocaine at each delay. In this way, we can quantify the subjective value of food at each delay in terms of cocaine dose. When these subjective values are plotted as a function of delay, they form a hyperbolic function like those seen in Figure 1. When the data are fit with a hyperbolic equation, they render a constant, k, that quantifies discounting rate (see Green and Myerson, 2004).

Referring again to our idealized data, Figure 2 depicts what could be considered competing effects between delay for a nondrug outcome and D-AMPH treatment on cocaine’s potency as a reinforcer. When both manipulations are applied, D-AMPH treatment would be expected to offset decreases in cocaine’s reinforcing potency when food is delayed. However, a mere antagonistic effect would not be sufficient to decrease the discounting rate of delayed food as higher doses of cocaine could be used to overcome that effect at the zero-delay condition (as seen in the right-shifting functions), and the proportion of discounting per unit increase in delay may not change (note here that this effect would not be detected if we used a non-hyperbolic procedure that tested only a single dose of cocaine). Rather, for the treatment to change discounting rate, it must reduce the magnitudes of the leftward shifts caused by delayed food in relation to the non-delayed condition. These reductions in the “spaces” between the leftward shifts would translate to subjective value points that diminish at a lower rate with each increasing delay to food presentation, and when plotted as a function of delay, the treatment effect would be evident by a more shallow hyperbolic function relative to the untreated condition.

To summarize, an ideal medication should shift the dose-response function for immediate drug rightward, similar to what has been reported in drug-choice studies (Banks et al., 2013; Negus, 2003). Furthermore, it should also mitigate the effects of delaying food on cocaine’s immediate potency as a reinforcer, which would appear as a reduction in discounting rates for delayed food. From this single design, one could conclude that D-AMPH decreased cocaine’s reinforcing effects and increased the value of and choice for an immediate and delayed, nondrug alternative. Alternatively, one may find that a treatment does one and not the other, which would suggest that the treatment does not have dual efficacy in decreasing impulsive and drug choice. However, until studies like these are conducted, the utility of the design and possible results remain theoretical.

7. Conclusion

Accepting that impulsive choice and the tendency to choose drugs over nondrug alternatives are factors in addiction, treatments that mitigate both of these tendencies may improve outcomes for recovering addicts. Several drugs have been tested for their ability to decrease delay discounting and drug choice. However, the stimulant medications for ADHD, D-AMPH and methylphenidate, have by far received the most experimental attention. The relatively high number of studies with these drugs made some divergent trends in experimental design apparent between the delay-discounting and drug-choice studies. First was the overwhelming tendency to use acute administration of treatments in delay discounting studies, while the drug-choice studies used mostly chronic treatment. The second difference was that the delay discounting studies were mostly conducted with rats, while rat models were rarely used in drug choice. We suggest that future delay discounting studies include chronic treatment in their designs to increase congruence with the treatment regimens used in drug choice and to increase the translational value of the work given that medications for substance use disorders will likely be prescribed for daily, long-term use. Also, studies in NHPs will be useful in delay discounting to increase comparability with the relatively large amount of NHP work in drug choice. We also suggest that drug-choice studies include more rat work, which has been shown to be a feasible model for testing treatments (Augier, 2012; Panlilio et al., 2015; Thomsen et al., 2013). Human work could incorporate relatively inexpensive delay discounting tests with hypothetical reinforcers into their drug-choice studies so that the treatment regimens under study can be applied to both questions simultaneously (e.g., see Bolin et al., 2016). Finally, in the context of each of these suggestions, the role of sex as a biological variable should be studied.

Highlights.

• Much of our understanding of maladaptive decision making in drug addiction has been derived from studies of delay discounting and drug vs. nondrug choice

• Divergent methodologies have been used to develop pharmacological treatments for decreasing delay discounting and drug choice

• The current review suggests novel ways to increase congruence between delay discounting and drug choice studies in the approaches used to test putative therapeutics for decreasing maladaptive decision making in both designs