Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

Abstract

Rationale

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

Objectives

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

Results

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

Conclusions

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Introduction

Inter-temporal choice refers to selecting between outcomes that differ across several dimensions, most notably delay and magnitude (Bradshaw 2015; Lowenstein and Thaler 1989). The term “impulsive choice” is often used in behavioral pharmacology experiments and can be defined as consistently choosing a small, immediate reinforcer over a large, delayed reinforcer (Ainslie 1975). Similar to impulsive choice, the term delay discounting is commonly used; furthermore, this term is often used interchangeably with impulsive choice in behavioral pharmacology experiments (e.g., Anderson and Woolverton 2005; Boomhower and Rasmussen 2014; Darna et al. 2015; Eubig et al. 2014; Gipson and Bardo 2009; Helms et al. 2006; Holcomb et al. 2013; Kayir et al. 2014; Marusich and Bardo 2009; Mejia-Toiber et al. 2014; Mitchell et al. 2014; Orsini et al. 2017; Simon et al. 2007; Smethells and Carroll 2015; Smethells et al. 2016; Stanis et al. 2008; Winstanley et al. 2003; Yates et al. 2014; Yates et al. 2017a; Yates et al. 2017b). However, there are important distinctions between impulsive choice and delay discounting; impulsive choice is a behavioral pattern, whereas delay discounting is a theoretical construct that refers to the subjective devaluation of a reinforcer as the delay to its delivery increases (Bradshaw 2015). Changes in impulsive choice do not always correlate with changes in delay discounting; for example, a decrease in responding for a large, delayed reinforcer (including when both alternatives are made available) results in increased impulsive choice, although the rate of discounting is unchanged (see the Analyzing and Interpreting Drug Effects in Delay Discounting section for an example). Additionally, there is some argument that delay discounting procedures do not adequately measure impulsive choice (see Blanchard et al. 2013). To avoid confusion, this review will primarily use the terms delay discounting (or discounting rates) and sensitivity to delayed reinforcement (or sensitivity to delay/delay sensitivity) when discussing the effects of drugs on performance in discounting paradigms, as opposed to the term impulsive choice.

Increased sensitivity to delayed reinforcement is associated with attention-deficit/hyperactivity disorder (ADHD; Antrop et al. 2006; Bitsakou et al. 2009; Kuntsi et al. 2001; Marco et al. 2009; Scheres et al. 2010; Scheres et al. 2013; Schweitzer and Sulzer-Azaroff, 1995; Solanto et al. 2001; Sonuga-Barke et al. 1992; Tripp and Alsop 2001; Vloet et al. 2010), schizophrenia (Ahn et al. 2011; Heerey et al. 2007; Weller et al. 2014), bipolar disorder (Ahn et al. 2011), Parkinson’s disease (Al-Khaled et al. 2015; Evens et al. 2015), pathological gambling (Albein-Urios et al. 2014; Cosenza and Nigro 2015; Petry 2001b; Wiehler et al. 2015), and substance use disorders (Bickel et al. 1999; Coffey et al. 2003; Field et al. 2007; Heil et al. 2006; Hoffman et al. 2006; Kirby et al. 1999; Kollins 2003; Madden et al. 1997; Mitchell 1999; Ohmura et al. 2005; Petry 2001a; Reynolds et al. 2004; Vuchinich and Simpson 1998). Because increased delay sensitivity is linked to various psychiatric disorders, there is an emphasis on understanding the neural mechanisms of impulsive choice. Reviewing the contribution of each neuroanatomical structure or neurotransmitter system is not the goal of the current literature review, as there are several reviews that have already been published on this topic (Body et al. 2017; Cardinal 2006; Dalley et al. 2008; Winstanley 2011). Instead, the review aims to discuss the different procedures and statistical analyses that are applied to discounting data, which can alter how we interpret drug effects on choice between small, immediate and large, delayed reinforcers. Because the focus is on procedural/analytic approaches to delay discounting, this review will not discuss factors such as sex, housing condition, species/strain, and basal levels of impulsivity (although see the Discussion section on how the principles detailed in this review can be applied to studies comparing different groups of animals). To understand how procedural/analytic approaches can alter drug effects in delay discounting or the interpretation of such effects, this review will first provide an overview of the procedures and the statistical analyses that are used to measure sensitivity to delayed reinforcement. Even though there is a heavy focus on glutamatergic compounds in this review (Figs. 1, 2, 3, and 4), the issues described below can be applied to any drug class/neurotransmitter system.

Figure 1.

Effects of memantine on delay discounting. (a) Mean (±SEM) raw proportion of responses for the large magnitude reinforcer as a function of delay. (b) Mean (±SEM) A and k parameter estimates derived from the hyperbolic discounting function. (c) Mean (±SEM) AUC values. *p < .05, relative to vehicle. These graphs were derived from data previously published (Yates et al. 2017a). Note, for simplicity, only the dose that produced statistically significant differences in performance is included.

Figure 2.

Effects of MK-801 on delay discounting. (a) Mean (±SEM) raw proportion of responses for the large magnitude reinforcer as a function of delay. (b and c) Mean (±SEM) A and k parameter estimates derived from the hyperbolic discounting function. In panel b, the parameter estimates were derived and then subjected to a secondary analysis; in panel c, the parameter estimates were derived using nonlinear mixed effects modeling. (d) Mean (±SEM) AUC values. # indicates a main effect of dose only (p < .05). *p < .05, relative to vehicle. These graphs were derived from data previously published (Yates et al. 2015). Note, for simplicity, only the dose that produced statistically significant differences in performance is included.

Figure 3.

Effects of D-cycloserine on delay discounting. (a) Mean (±SEM) raw proportion of responses for the large magnitude reinforcer as a function of delay. (b) Mean (±SEM) A and k parameter estimates derived from the hyperbolic discounting function. (c) Mean (±SEM) AUC values. *p < .05, relative to vehicle. These graphs were derived from data previously published (Yates et al. 2017a). Note, for simplicity, only the dose that produced statistically significant differences in performance is included.

Figure 4.

Effects of CGS 19755 on delay discounting. (a) Mean (±SEM) raw proportion of responses for the large magnitude reinforcer as a function of delay. (b and c) Mean (±SEM) A and k parameter estimates derived from the hyperbolic discounting function and exponential discounting function, respectively. (d) Mean (±SEM) AUC values. *p < .05, relative to vehicle. These graphs were derived from data previously published (Yates et al. 2017a).

Measuring Delay Discounting in Animals

In animals, most delay discounting procedures are conducted using operant conditioning paradigms. Although delay discounting can be tested with a T-maze (Bizot et al. 1999; Poulos et al. 1995; Rudebeck et al. 2006), most behavioral pharmacology experiments use fully automated procedures that are conducted in operant conditioning chambers. As such, the focus of this review will be on procedures that utilize automated operant chambers as opposed to T-mazes. Most of the following procedures are described elsewhere (see Madden and Johnson 2010), but they will be discussed briefly as follows.

Adjusting Amount Procedure

Animals are able to respond for a reinforcer delivered after a fixed delay and a smaller reinforcer delivered immediately (Richards et al. 1997). Each session is composed of free-choice trials, in which animals are able to respond on either manipulandum, as well as forced-choice trials, in which only one manipulandum is available. The forced-choice trials ensure each animal is exposed to both contingencies of reinforcement. A response for the fixed alternative results in a percentage increase in the amount of reinforcement delivered for the alternative, whereas a response on the adjusting-amount alternative leads to a decrease in the amount delivered for that reinforcer. Across sessions, the delay to the fixed alternative is adjusted, such that multiple indifference points can be calculated (see Fig. 1 of Helms et al. 2006 for an example). Plotting indifference points as a function of the delay to the fixed alternative allows one to determine the rate at which animals devalue the adjusting alternative (e.g., Calvert et al. 2010; Green et al. 2007; Helms et al. 2006; Kieres et al. 2004; Mitchell and Rosenthal 2003; Richards et al. 1997; Stein et al. 2012; Wilhelm and Mitchell 2009; Wilhelm and Mitchell 2010; Wilhelm and Mitchell 2012). Because each drug dose has to be administered multiple times (across each delay for the fixed alternative), behavioral pharmacology experiments using this procedure test each dose at one delay to the fixed alternative only (Helms et al. 2006; Kieres et al. 2004; Moschak and Mitchell 2013; Richards et al. 1999a; Wade et al. 2000; Wilhelm and Mitchell 2012). Because indifference points are being calculated for one delay to the fixed alternative, determining how drugs affect the rate at which animals devalue the large reinforcer is not feasible (due to generation of a single data point).

Adjusting Delay Procedure

In the adjusting delay procedure, animals are trained to make choices between a large magnitude reinforcer delivered after an adjusting delay and a small magnitude reinforcer delivered after a fixed delay (Mazur 1987). Similar to the adjusting amount procedure, animals complete blocks of trials, which consist of forced-choice trials followed by free-choice trials. If an animal chooses the smaller reinforcer during each free-choice trial, the delay to the larger reinforcer is decreased. Conversely, if an animal chooses the larger reinforcer on each free-choice trial, the delay to the larger reinforcer is increased. The delay to the larger reinforcer is not altered if the animal chooses each reinforcer during a block of trials. This procedure is repeated across several fixed delays to the smaller reinforcer, and adjusting delays are plotted as a function of the fixed delay to the small magnitude reinforcer. One is able to determine the rate at which animals discount the large magnitude reinforcer (e.g., Craig et al. 2014; da Costa Araújo et al. 2009; Green et al. 2007; Mazur 1987; Mazur 2007; Mazur and Biondi 2009; Mobini et al. 2000; Torres et al. 2011; Valencia-Torres et al. 2012). Like the adjusting amount procedure, the adjusting delay procedure is not as amenable to testing drug effects on delay discounting (due to having to administer each dose of the drug multiple times).

Modified adjusting delay procedure

The adjusting delay procedure was modified, such that an animal can titrate the delay to the large magnitude reinforcer across trials in a single session (Perry et al. 2005). In a typical modified adjusting delay procedure, the session consists of multiple trial blocks, which are similar to those described for the adjusting delay procedure. During free-choice trials, a response on the manipulandum associated with the small, immediate reinforcer decreases the delay to the large, delayed reinforcer. Conversely, responses for the large reinforcer increase the delay to that alternative. Unlike the adjusting delay procedure, the modified adjusting delay procedure does not alter the delay to the fixed alternative. Instead, a single data point (mean adjusting delay [MAD] score) is generated from each session. Like the adjusting amount procedure, the modified adjusting delay paradigm is not extensively used in behavioral pharmacology experiments (Blasio et al. 2012; Cottone et al. 2013; Gipson and Bardo 2009; Moschak and Mitchell 2013; Perry et al. 2008b; Smethells and Carroll 2015; Wooters and Bardo 2011; Yates et al. 2014), although this task is somewhat popular for assessing the relationship between impulsive choice and drug abuse vulnerability (Anker et al. 2009; Marusich and Bardo 2009; Perry et al. 2005; Perry et al. 2008a; Yates et al. 2012).

One major limitation of the modified adjusting delay procedure is the generation of a single dependent variable (MAD scores). Because delay discounting is influenced by sensitivity to delayed reinforcement and sensitivity to reinforcer magnitude (Ho et al. 1999), the modified adjusting delay procedure does not allow one to dissociate which behavioral mechanism is controlling choice of the small magnitude and large magnitude reinforcers. For example, if a drug decreases MAD scores, one cannot determine if the decrease is due to increased sensitivity to delay or if it is due to decreased sensitivity to reinforcer magnitude.

Another potential limitation of the adjusting delay/modified adjusting delay procedures is that they do not always produce consistent choice across sessions. For example, in an adjusting delay procedure, even after extensive training, rats fail to show stable patterns of choice between the adjusting alternative and the fixed alternative (Cardinal et al. 2002). Specific to the modified adjusting delay procedure, MAD scores across each baseline session are rarely reported, but there are a couple of studies that show highly variable MAD scores across training. One report shows that, across 28 sessions of baseline training, MAD scores follow an oscillating pattern, and these scores appear to show an upward trend even after the final baseline session (see Fig. 1a of Marusich and Bardo 2009). This oscillating pattern of MAD scores can also be observed in rats given short access to amphetamine for 21 days (see Fig. 1b of Gipson and Bardo 2009). The unstable MAD scores observed in Marusich and Bardo (2009) and Gipson and Bardo (2009) could be an artifact of altering the lever associated with the large, delayed reinforcer between the left lever and the right lever each session. Regardless, the oscillating pattern of adjusting delay scores observed with this procedure can make interpreting drug effects difficult. For example, a drug that decreases MAD scores may not be selectively altering delay discounting; instead, the altered MAD scores may be attributed to the fluctuations in this variable that are unrelated to pharmacological manipulations.

Procedures that Systematically Increase/Decrease Delay to Large Reinforcer

The Evenden and Ryan (1996) procedure incorporates blocks of trials, in which the delay to obtaining the large reinforcer increases within a single session, although the delay to obtaining this alternative can be manipulated between sessions (Mobini et al. 2000). As the delay of obtaining the large reinforcer increases, animals typically switch their preference to the small, immediate reinforcer (Evenden and Ryan, 1996). Importantly, the decreased responding for the large magnitude reinforcer observed across blocks of trials is not due to satiation, as animals consistently choose a large reinforcer over a small reinforcer when delivery of the large reinforcer is not delayed (e.g., Evenden and Ryan, 1996). Cardinal et al. (2000) use a modified version of the Evenden and Ryan (1996) procedure, which includes forced-choice trials and requires animals to initiate the extension of each manipulandum. Cardinal et al. (2000) argue that this latter alteration is important because it ensures animals are equidistant from each manipulandum (e.g., lever) at the beginning of each trial. After a choice is made, an inter-trial interval is imposed, such that animals cannot earn more reinforcers by consistently choosing the small, immediate reinforcer.

The major advantage of this procedure is that determining sensitivity to delayed reinforcement can be generated within a single session, thus allowing one to determine how pharmacological manipulations alter delay discounting. Performance in the Evenden and Ryan (1996) and adjusting delay procedures are positively correlated (Craig et al. 2014), suggesting these two tasks measure the same construct. As such, Craig et al. (2014) argue that the Evenden and Ryan (1996) procedure is a more appropriate method to use in behavioral pharmacology experiments compared to adjusting delay procedures. Not surprisingly, the Evenden and Ryan (1996)/Cardinal et al. (2000) paradigm is the most extensively used procedure to determine the neurochemical underpinnings of delay discounting (Adriani et al. 2004; Baarendse and Vanderschuren 2012; Barbelivien et al. 2008; Boomhower and Rasmussen 2014; Broos et al. 2012; Eppolito et al. 2011; Eubig et al. 2014; Evenden and Ryan 1999; Floresco et al. 2008; Garciá-Lecumberri et al. 2011; Hand et al. 2009; Hellemans et al. 2005; Hernandez et al. 2014; Higgins et al. 2016; Huskinson et al. 2012; Isherwood et al. 2015; Isherwood et al. 2017; Kayir et al. 2014; Koffarnus et al. 2011; Korte et al. 2017; Krebs and Anderson 2012; Krebs et al. 2016; Liu et al. 2004; Liu et al. 2017; Loos et al. 2010; Madden et al. 2010; Maguire et al. 2014; Mendez et al. 2010; Mendez et al. 2012; Ozga and Anderson 2018; Paine et al. 2003; Pardey et al. 2012; Paterson et al. 2012; Pitts and McKinney 2005; Robinson et al. 2008; Schippers et al. 2016; Schneider et al. 2011; Simon et al. 2007; Slezak and Anderson 2009; Slezak and Anderson 2011; Slezak et al. 2012; Slezak et al. 2014; Sukhotina et al. 2008; Sun et al. 2012; Stanis et al. 2008; Talpos et al. 2006; Tanno et al. 2014; van den Bergh et al. 2006; van Gaalen et al. 2006; Winstanley et al. 2003; Winstanley et al. 2005; Winstanley et al. 2007; Wiskerke et al. 2011; Yates et al. 2015; Yates et al. 2017a; Yates et al. 2017b; Zeeb et al. 2010). As such, a majority of this review will focus on this procedure.

The disadvantages of the Evenden and Ryan (1996) procedure are detailed elsewhere (Madden and Johnson 2010), but a couple will be discussed here briefly. First, responses during the first block of trials seem to influence responses for the large magnitude reinforcer in subsequent trial blocks. Although the delay to the large magnitude reinforcer typically increases across blocks of trials, some studies decrease the delay across the session (Fox et al. 2008; Maguire et al. 2014; Slezak and Anderson 2009; Tanno et al. 2014; Yates et al. 2017b). Comparing ascending and descending schedules, some reports show differential basal levels of delay sensitivity across schedule type (Fox et al. 2008; Yates et al. 2017b; but see Slezak and Anderson 2009). This issue will be discussed in more detail later in this review (see Procedural Differences Within the Evenden and Ryan (1996) Paradigm section).

Related to the first point, responses during the first block of trials may be influenced by responses during the final block of trials during the preceding session. For example, responses for the large magnitude reinforcer during the 0-s delay block may be relatively low (less than 60%), which may be due, at least in part, to the low responding that occurs for this reinforcer at the highest delay during the previous session. With my experience with the Evenden and Ryan (1996) procedure, some rats, even after extensive training, will respond less than 60% of the time for the large magnitude reinforcer at the 0-s delay on random sessions, which may reflect imperfect stimulus control.

Another limitation of the Evenden and Ryan (1996) procedure is that discounting curves are not always graded. That is, some rats will show exclusive preference for the large magnitude reinforcer when its delivery is immediate but will show exclusive preference for the small magnitude reinforcer when a delay to the large reinforcer is imposed. This is problematic if one wants to determine if a drug increases delay sensitivity, as further decreases in responding for this alternative cannot be observed. Conversely, some rats will show exclusive preference for the large magnitude reinforcer across each delay; thus, determining if a drug decreases sensitivity to delayed reinforcement is impossible. In the latter case, some studies will exclude rats that show exclusive preference for the large magnitude reinforcer across each delay (e.g., Baarendse and Vanderschuren 2012; Madden et al. 2010; Robinson et al. 2008; Simon et al. 2007; Winstanley et al. 2006), but this is also problematic as real data are being excluded, which lowers sample size (thus, decreasing statistical power) and prevents one from determining if the drug alters other measures that can be obtained with these procedures (e.g., response latencies, trial omissions, etc.). To circumvent this limitation, a concurrent-chains procedure can be used to assess delay discounting (see below).

Concurrent-Chains Procedure

In a concurrent-chains procedure (Aparicio et al. 2013; Aparicio et al. 2015; Aparicio et al. in press; Beeby and White 2013; Grace 2002; Johnson et al. 2013; Oliveira et al. 2014; Ong and White 2004; Orduña 2015; Orduña and Mercado 2017; Orduña et al. 2013; Pitts and Febbo 2004), animals do not make a choice between rewards directly; instead, they make a choice between sources of reinforcement. Specifically, each trial is composed of an initial link and a terminal link. During the initial link, responses on one manipulandum (typically a random interval [RI] or variable interval [VI] schedule of reinforcement) are recorded but do not lead to reinforcement directly. Instead, reinforcement is made available during the terminal link. In this procedure, choice is measured by calculating the relative response rates for each alternative during the initial link. The major advantage of the concurrent-chains procedure is that it can be designed to prevent exclusive preference for one alternative over the other. Therefore, increases in preference for the large magnitude reinforcer can be observed at the 0-s delay, and decreases in responding for this alternative can be observed at the highest delays. Although the concurrent-chains procedure provides this critical advantage, this procedure is rarely used to assess the neurochemical underpinnings of delay discounting (but see Johnson et al. 2013; Pitts and Febbo 2004).

Analyzing and Interpreting Drug Effects in Delay Discounting

Due to its widespread use and the fact that several analyses can be applied to data collected from this procedure, this section will focus on the Evenden and Ryan (1996) paradigm, although the generalized matching law will be discussed in context to concurrent-chains procedures.

Raw Proportion of Responses for the Large, Delayed Reinforcer

The proportion of responses for the large magnitude reinforcer is plotted as a function of delay. These responses are analyzed using a two-way ANOVA, with delay as a within-subjects factor and dose as either a within-subjects factor (if each subject received each dose) or a between-subjects factor (if different subjects received one dose). Additional factors can be included in the analysis, such as delay presentation order, housing condition, and sex. This analysis is commonly used in studies assessing drug effects on delay discounting (Adriani et al. 2004; Baarendse and Vanderschuren 2012; Barbelivien et al. 2008; Boomhower and Rasmussen 2014; Broos et al. 2012; Cardinal et al. 2000; Eubig et al. 2014; Evenden and Ryan 1996; Evenden and Ryan 1999; Floresco et al. 2008; Hand et al. 2009; Koffarnus et al. 2011; Liu et al. 2004; Liu et al. 2017; Mendez et al. 2012; Paterson et al. 2012; Robinson et al. 2008; Schippers et al. 2016; Sukhotina et al. 2008; Sun et al. 2012; Stanis et al. 2008; Talpos et al. 2006; van den Bergh et al. 2006; van Gaalen et al. 2006; Winstanley et al. 2003; Winstanley et al. 2005; Wiskerke et al. 2011).

There are several limitations to using ANOVA analyses on the raw proportion of choices for the large magnitude reinforcer. First, there are instances in which a drug causes a decrease in responding for the large reinforcer at each delay, including the 0-s delay. For example, pramipexole (0.32 mg/kg), a dopamine D₃ agonist, decreases responding for the large magnitude reinforcer at each delay (Koffarnus et al. 2011). When an ANOVA is applied to these data, there are main effects of drug and delay, as well as a drug × delay interaction, suggesting that the drug increases sensitivity to delayed reinforcement. As Koffarnus et al. (2011) argue, because pramipexole decreases choice for the large reinforcer across each delay, some other behavioral process (e.g., ability to discriminate reinforcer magnitudes) is being altered instead. Koffarnus et al. (2011) observe similar findings with amphetamine (1.0 mg/kg), the dopamine non-selective agonist apomorphine (0.32 mg/kg), the D₁-like agonist SKF 81297 (0.32 and 1.0 mg/kg), the D₁-like antagonist SCH 23390 (0.032 mg/kg), the D₂-like antagonists haloperidol (0.1 mg/kg) and L-741,626 (1.0 and 3.2 mg/kg), and the D₃ antagonist PG01037 (56.0 mg/kg).

Another limitation to using ANOVA is that significant interactions have to be probed by additional ANOVAs and/or paired-samples t tests (see Young et al. 2009). In order to control for familywise error, the alpha level needs to be adjusted. For example, when five comparisons are being made (one for each delay when five delays are used), a Bonferroni adjustment would require an individual to divide the alpha level (.05) by five, resulting in an adjusted alpha value of .01. This may result in cases where there is a significant interaction, but none of the pairwise comparisons are statistically significant (an example of increased Type II error). This issue will become more apparent in the next section (see Quantitative Methods of Delay Discounting).

Third, listwise deletion of an entire subject’s dataset occurs if missing data are included. Consider the following example: rats are being treated with several doses of a drug, and one rat does not respond during one block of trials after being treated with the highest drug dose. Because the subject did not respond during a block of trials, the entire dataset for that particular subject will be excluded when ANOVA is used. When missing data are present, one can use imputation to create a value for the subject; yet, there are flaws in using imputation, such as not being able to differentiate subjects that provided a complete data set from subjects that had missing data (see Young 2017).

Finally, from a theoretical standpoint, using ANOVA to assess delay discounting may not be appropriate, as this analysis says nothing about the rate at which subjects discount the large magnitude reinforcer as the delay to its delivery increases. For example, my laboratory (Yates et al. 2017a) has shown that the drug memantine, used therapeutically to treat Alzheimer’s disease, decreases responding for the large magnitude reinforcer at the 0-s and 10-s delays (significant main effects of delay and dose, as well as a significant interaction; see Fig. 1a). At first glance, the argument may be that memantine increases sensitivity to delayed reinforcement, which has been argued in the past using a modified adjusting delay procedure (Cottone et al. 2013), but this does not appear to be the case. The rate at which responding decreases for the large magnitude reinforcer across delays is similar for rats when treated with vehicle and when treated with memantine (Fig. 1b; see subsection below for more details).

Quantitative Models of Delay Discounting

Instead of analyzing the raw proportion of responses for the large magnitude reinforcer with ANOVAs, a mathematical model can be applied to the data to derive one (or more) parameter estimate(s). Detailed discussions of the various quantitative models of discounting already exist (Green and Myerson 2004; Ho et al. 1999; Killeen 2009; Killeen 2011; Madden and Johnson 2010); thus, the current review will briefly describe a few models. The exponential model borrows from microeconomic theory (Samuelson 1937) and is expressed by the equation V = Ae^−kD, where V is the subjective value of the large magnitude reinforcer, A is reinforcer amount, k is the slope of the function, and D is the delay to the large reinforcer. Because the exponential model does not account for preference reversals (Ainslie 1974; Strotz 1956), the hyperbolic function (Ainslie 1975; Mazur 1987) is often used, which is defined as V = A/(1 + kD). The hyperbolic discounting function was originally applied to adjusting delay procedures to show the rate at which indifference points decrease as a function of the fixed delay to the small magnitude reinforcer. In the original discounting functions, k is the only free parameter. The hyperbolic function can be modified by raising the denominator to a power (s): V = A/(1 + kD)^s (Green et al. 1994); here, s represents the nonlinear scaling of amount and/or time.

Some studies that utilize quantitative models typically derive the k parameter estimate and then analyze this parameter with a secondary test, such as a t test or ANOVA (Bickel et al. 1999; Coffey et al. 2003; Kayir et al. 2014; Madden et al. 1997; Mitchell 1999; Richards et al. 1999b; Voon et al. 2010; Yates et al. 2015). Because k parameters are rarely normally distributed, parametric tests cannot be applied unless a transformation is applied to the data. To this end, most studies analyze log-transformed k estimates. Using two-stage approaches to analyzing discounting parameters is problematic for several reasons (see Jonsson et al. 2000; Young 2017 for full discussions). One major disadvantage of using a secondary analysis is that listwise deletion of an entire subject’s dataset occurs if missing data are included (similar to what can occur when the raw proportion of responses for the large, delayed reinforcer is analyzed; see previous paragraph).

Instead of using a two-stage approach to analyzing parameter estimates derived from hyperbolic functions, nonlinear mixed effects (NLME) modeling can be used to simultaneously estimate each subject’s parameters and the group’s parameters. NLME offers several advantages compared to ANOVAs and two-stage analyses (see Pinherio and Bates 2004; Young 2017; Young et al. 2009 for full discussions). Briefly, NLME accounts for missing data, better controls for Type I error, and has increased power relative to ANOVA. By using NLME analyses, one does not have to conduct post hoc tests at each delay; instead, changes in discounting rate can be determined using discounting functions. When applying discounting functions to data generated from the Evenden and Ryan (1996) procedure, the k parameter still refers to the slope of the function. Instead of merely reflecting reinforcer amount, the A parameter can now provide a measure of how much an animal responds for the large magnitude reinforcer when its delivery is immediate. When conducting NLME analyses, A and k are both free parameters. Allowing A to be a free parameter is important, as this value can vary greatly following pharmacological manipulations. Going back to the memantine example described above, NLME analyses show that memantine significantly decreases the A parameter without altering the k parameter (see Fig. 1b). Thus, memantine appears to affect some other behavioral process without altering sensitivity to delayed reinforcement (see the following paragraph for a discussion of how changes in A can be interpreted). Despite the advantages of NLME, this analysis is rarely used in behavioral pharmacology studies assessing the effects of drugs on delay discounting (but see Yates et al. 2017a; Yates et al. 2017b).

The following scenario shows how using two-stage analyses can lead to different conclusions relative to NLME when determining the effects of pharmacological manipulations on delay discounting. Figure 2 shows the effects of MK-801 (0.03 mg/kg), a glutamate N-methyl-D-aspartate (NMDA) receptor uncompetitive antagonist, on performance in a delay discounting procedure (Yates et al. 2015). When an ANOVA is used to determine how MK-801 affects responding for the large magnitude reinforcer, there is a main effect of dose only (Fig. 2a). When k parameter estimates are derived using the hyperbolic discounting function and then subjected to a repeated measures ANOVA, results suggest that MK-801 decreases sensitivity to delayed reinforcement, as evidenced by decreased k estimates (Fig. 2b), thus corroborating the findings presented in Figure 2a. One issue with generating k parameter estimates for each individual rat using a program such as Excel or GraphPad Prism is that extreme values can be observed in some subjects. For example, if a subject shows exclusive preference for the large magnitude reinforcer at the 0-s delay but shows exclusive preference for the small, immediate reinforcer at each subsequent delay, Excel generates a k parameter estimate of 347.635. If the proportion of responses during the first delay increases from 0 to 0.05, the k parameter estimate decreases to 5.174. Even though the proportion of responses barely changes in this example (5% increase at the first imposed delay), the corresponding change in the k estimate is substantial (98.512% decrease). Related to the MK-801 results, some rats had high k estimates following vehicle treatment (hence, the large variability depicted in Fig. 2b). Thus, slight increases in the proportion of responses for the large magnitude reinforcer can lead to large decreases in k. Even though the two-stage analysis shows that MK-801 decreases sensitivity to delayed reinforcement, NLME analysis shows no significant differences in k parameter, or A parameter, estimates following MK-801 administration (Fig. 2c). In NLME, group estimates and individual estimates are simultaneously affected by one another. The result is that outliers have a minimal effect on the group k estimate, but the outlier’s estimate will be more affected by the estimates of the other subjects (see Young 2017 for an excellent discussion on this “shrinkage” effect).

Although NLME offers several advantages, there are a couple of drawbacks to this analysis. First, when using NLME, the individual needs to determine which function best applies to the data, although this should not be much of an issue in the discounting field, as most data can be described by hyperbolic functions. Second, for individuals that are not familiar with curve fitting, starting values have to be estimated before running the analysis; if an individual fails to use appropriate starting values, the discounting function will not converge (see Young et al. 2009). Third, interpreting what the A parameter estimate means is somewhat difficult. This parameter is sometimes referred to as a measure of sensitivity to reinforcer magnitude (Yates et al. 2015; Yates et al. 2017a; Yates et al. 2017b). Because we (e.g., Yates et al. 2015; Yates et al. 2017a; Yates et al. 2017b) do not systematically adjust the magnitudes of the reinforcers or run control groups where the large and small magnitude reinforcers are both made available immediately across the entire session, this may not be the most appropriate interpretation of this parameter. Another potential interpretation is that this parameter provides a measure of the discriminability of the two reward alternatives. For example, Slezak and Anderson (2009) show that amphetamine (≥ 0.3 mg/kg) decreases responding for the large magnitude reinforcer at the 0-s delay (similar to the results reported by Koffarnus et al. 2011; see previous paragraph). Slezak and Anderson (2009) acknowledge that interpreting these results is difficult because the decreased responding at the 0-s delay may be interpreted as impaired discrimination of the reinforcer amounts as opposed to a selective change in sensitivity to delayed reinforcement.

Area Under the Curve (AUC)

Due to the numerous models that can be applied to delay discounting data, Myerson et al. (2001) propose calculating AUC as a “theoretically neutral” way of measuring choice between reinforcer alternatives. The delay and subjective value for each data point are first normalized. Delay is expressed as a proportion of the maximum delay, and the subjective value is expressed as a proportion of the nominal amount. These normalized values are used as x coordinates and y coordinates, respectively, to construct a graph of the data. Vertical lines are drawn from each data point to the x-axis, subdividing the graph into a series of trapezoids (note: one does not actually need to draw vertical lines on the graph to calculate AUCs; see Fig. 3 of Myerson et al. 2001). The area of each trapezoid is equal to (x₂ − x₁)[(y₁ + y₂)/2], where x₁ and x₂ are successive delays, and y₁ and y₂ are the subjective values associated with these delays. The area under the discounting function is equal to the sum of the areas of these trapezoids. AUC values range from 0 to 1, with values closer to 0 indicating increased sensitivity to delayed reinforcement and values closer to 1 representing decreased delay sensitivity. The atheoretical nature of AUC is advantageous because one does not need to specify the functional form of the data (e.g., hyperbolic or exponential). Although Myerson et al. (2001) show that AUCs are normally distributed, other reports show that these values are skewed (Mitchell et al. 2015; Yoon et al. 2017; note: although Mitchell et al. 2015 report skewed AUCs, they report that the values were within ranges appropriate for parametric tests).

Despite the growing popularity of this analysis (Boomhower and Rasmussen 2014; Eubig et al. 2014; Higgins et al. 2016; Huskinson et al. 2012; Korte et al. 2017; Krebs and Anderson 2012; Krebs et al. 2016; Slezak and Anderson 2009; Slezak and Anderson 2011; Slezak et al. 2012; Slezak et al. 2014; Tanno et al. 2014; van den Bergh et al. 2006), AUCs have the same limitation as MAD scores generated from the modified adjusting delay procedure: this variable does not allow one to determine if a drug selectively alters sensitivity to delayed reinforcement or alters discriminability of the alternatives. Specifically, decreases in AUCs could be driven by decreases in responding at the 0-s delay, but not at any other delay. Borges et al. (2016) provide another limitation of using AUC analyses. Because time intervals between delays become larger as the delays increase, longer delays disproportionately affect AUC values relative to shorter delays. To circumvent this limitation, Borges et al. (2016) suggest using base-10 logarithmic or ordinal scaling transformations of delays.

Generalized Matching Law

The matching law states that the relative rate of responses in a schedule matches the relative rate of reinforcement provided by that schedule (Herrnstein 1961). The matching law is typically applied to concurrent variable interval (VI) schedules of reinforcement and is defined by the equation B₁/(B₁+B₂) = r₁/(r₁+r₂), where B represents the absolute response rate for each alternative (denoted by 1 and 2), and r represents the absolute reinforcement rate. The matching law can be altered to reflect that other parameters, such as relative magnitude of each alternative and relative immediacy (reciprocal of delay), affect relative responses rates (Baum 1974; Baum and Rachlin 1969). This generalized matching law can be described with the equation B₁/(B₁+B₂) = b(r₁/r₂)^s, where b represents bias (e.g., subjects show preference for one manipulandum across the session, regardless of what contingencies of reinforcement are in place) and s represents the sensitivity of the behavior ratio to the ratio of the values of the activities (in this case, b is a measure of sensitivity to delayed reinforcement). One advantage of using the generalized matching law is that the equation described previously can be presented logarithmically as log(B₁/B₂) = s×log(r₁/r₂) + logb, which allows one to estimate the parameters s and b using simple linear regression models. Aparicio et al. (2015) show positive correlations between k parameter estimates derived from hyperbolic discounting and s parameters derived from the generalized matching law, suggesting these parameters measure similar processes. One note is that the generalized matching law is typically applied to discounting paradigms using a concurrent-chains procedure (e.g., Aparicio et al. 2013; Aparicio et al. 2015; Aparicio et al. in press; Orduña 2015; Pitts and Flebbo 2004; Pope et al. 2015).

Although the generalized matching law has been used to describe how reinforcer magnitude affects sensitivity to delayed reinforcement (Orduña et al. 2013) and to compare discounting between strains of rats and mice (Aparicio et al. 2013; Aparicio et al. 2015; Aparicio et al. in press; Orduña 2015; Pope et al. 2015), few studies have used the generalized matching law to determine how drugs affect bias and sensitivity to delayed reinforcement. Pitts and Febbo (2004) used the generalized matching law to describe how methamphetamine affects delay sensitivity in pigeons; the results indicated that methamphetamine typically reduced sensitivity to delayed reinforcement, but for some animals, methamphetamine decreased sensitivity to reinforcer amount as well. Using this approach to analyzing drug effects on delay discounting is beneficial because one does not have to specify which mathematical function best describes choice (similar to the advantage provided by AUCs), but it also allows one to derive two parameters that can be influenced by pharmacological manipulations (unlike AUCs).

How Procedural Differences Can Alter Interpretation of Drug Effects

Few studies directly compare delay discounting across different paradigms, but results show similar performance in adjusting amount and adjusting delay procedures (Green et al. 2007), as well as in adjusting delay/modified adjusting delay and the Evenden and Ryan (1996) procedures (Craig et al. 2014; Smethells and Carroll 2015). Specific to behavioral pharmacology experiments, drugs can differentially alter behavior across discounting procedures. For example, cocaine (2, 5, and 15 mg/kg) decreases preference for the large, delayed reinforcer when the Evenden and Ryan (1996) procedure is used but does not alter MAD scores in the modified adjusting delay procedure (Smethells and Carroll 2015). The discrepant results may be attributed to the finding that cocaine decreases preference for the large reinforcer, even at the 0-s delay, suggesting that some other behavioral process independent of sensitivity to delayed reinforcement is affected. Because the modified adjusting delay generates a single data point to assess delay discounting, detecting alterations in the discriminability of the two reinforcers may be difficult in this task. These results are similar to findings obtained with other drugs. 8-OH-DPAT (serotonin 5-HT_1A agonist; 0.3 and 1.0 mg/kg) decreases sensitivity to delayed reinforcement when the modified adjusting delay procedure is used (Blasio et al. 2012), but 8-OH-DPAT (0.3 mg/kg) decreases responding for the large magnitude reinforcer even when its delivery is immediate (Evenden and Ryan 1999). A similar finding is observed following antagonism of NMDA receptors. When the modified adjusting delay procedure is used, the NMDA receptor uncompetitive antagonists ketamine (10.0 and 20.0 mg/kg) and memantine (5.0 and 10.0 mg/kg) decrease MAD scores, suggesting increased delay sensitivity (Cottone et al. 2013). However, using the Evenden and Ryan (1996) procedure, other reports show a decrease in responding for the large magnitude reinforcer at the 0-s delay following ketamine (10.0 mg/kg) and memantine (5.0 mg/kg) administration (Yates et al. 2015; Yates et al. 2017a). Finally, when using an adjusting amount procedure, blocking dopamine D₂ receptors increases sensitivity to delayed reinforcement (Wade et al. 2000) but decreases responding for the large magnitude reinforcer at the 0-s delay in the Evenden and Ryan (1996) procedure (Koffarnus et al. 2011).

Other evidence shows disparate drug effects in one discounting paradigm relative to another. When using the modified adjusting delay procedure, no change in MAD scores is observed following methylphenidate (1.0–10.0 mg/kg) administration in outbred strains of rats (Perry et al. 2008b; Wooters and Bardo 2011); however, methylphenidate (1.0–4.0 mg/kg) decreases sensitivity to delay when the Evenden and Ryan (1996) procedure is used (Paterson et al. 2012; van Gaalen et al. 2006). Similarly, the dopamine D₁ receptor antagonist SCH 23390 (5.0–20.0 μg/kg) does not alter sensitivity to delayed reinforcement when an adjusting amount procedure is used (Wade et al. 2000), but SCH 23390 (0.01 and 0.02 mg/kg) increases delay sensitivity in the Evenden and Ryan (1996) procedure (Koffarnus et al. 2011; van Gaalen et al. 2006).

Although drugs can produce differential effects in one discounting procedure relative to another, some caution needs to be taken into account because there are other factors that can account for the differential results observed across paradigms. For example, when comparing the Blasio et al. (2012) and Cottone et al. (2013) studies to the Evenden and Ryan (1999) and Yates et al. (2017a) studies described previously, the aforementioned studies used Wistar rats and non-caloric reinforcers, whereas the latter studies used Sprague Dawley rats and caloric reinforcers. Similar studies as the one conducted by Smethells and Carroll (2015) are important for directly comparing drug effects in various discounting paradigms.

Procedural Differences Within the Evenden and Ryan (1996) Paradigm

The Evenden and Ryan (1996) procedure can be modified numerous ways. For example, the delays to the large magnitude reinforcer can be increased, decreased, or randomized across the session (Fox et al. 2008; Maguire et al. 2014; Slezak and Anderson 2009; Tanno et al. 2014; Yates et al. 2017b); the delay to the large magnitude reinforcer can be signaled or unsignaled (Cardinal et al. 2000; Slezak and Anderson 2009); the range of delays can be narrow (e.g., 0.4–6.5 s; Floresco et al. 2008) or wide (e.g., 0–100 s; Yates et al. 2017a); and the magnitude differences between the small, immediate reinforcer and large, delayed reinforcer can be adjusted (e.g., 1 vs. 3 [Boomhower and Rasmussen 2014; Hand et al. 2009; Isherwood et al. 2015; Isherwood et al. 2017; Koffarnus et al. 2011; Mendez et al. 2012; Paterson et al. 2012; Slezak and Anderson 2011; Tanno et al. 2014], 1 vs. 4 [Baarendse and Vanderschuren 2012; Cardinal et al. 2000; Eubig et al. 2014; Higgins et al. 2016; Korte et al. 2017; Liu et al. 2004; Liu et al. 2017; Robinson et al. 2008; Schippers et al. 2016; Stanis et al. 2008; Sukhotina et al. 2008; Sun et al. 2012; Talpos et al. 2006; van den Bergh et al. 2006; van Gaalen et al. 2006; Winstanley et al. 2003; Wiskerke et al. 2011; Yates et al. 2015; Yates et al. 2017a; Yates et al. 2017b], 1 vs. 5 [Adriani et al. 2004; Barbelivien et al. 2008; Evenden and Ryan 1996; Evenden and Ryan 1999; Hellemans et al. 2005; Pardey et al. 2012], or 2 vs. 4 [Floresco et al. 2008]). The few studies that systematically examine these variables are described in the following section.

Increasing or decreasing the delay across blocks of trials

Concerning drug effects in discounting procedures using ascending/descending schedules, amphetamine (1.0 and 1.7 mg/kg) decreases AUC values in rats trained on both ascending and descending schedules, although rats trained on the descending schedule show a greater percentage decrease in AUCs (Slezak and Anderson 2009). An important consideration is that Slezak and Anderson (2009) report decreased responding for the large magnitude reinforcer at the 0-s delay in both conditions, suggesting that amphetamine may have impaired discriminability of the reward alternatives, not necessarily sensitivity to delayed reinforcement.

Although Slezak and Anderson (2009) report similar changes in delay discounting following amphetamine administration in ascending and descending schedules, amphetamine (1.0 and 1.78 mg/kg; Maguire et al. 2014; Tanno et al. 2014) and methylphenidate (10.0 and 17.8 mg/kg; Tanno et al. 2014) increase responding for the large reinforcer when an ascending schedule is used but decreases responding when a descending schedule is used. Furthermore, my lab (Yates et al., 2017b) shows that the metabotropic glutamate receptor (mGluR) 1 antagonist JNJ 16259685 (1.0 mg/kg) increases delay sensitivity in all rats, regardless of which schedule is used; however, JNJ 16259685 (1.0 mg/kg) decreases responding for the large magnitude reinforcer at the 0-s delay when the descending schedule is used only. My laboratory has recently collected data examining the effects of the NMDA NR2B subunit antagonist Ro 63-1908 on delay discounting in ascending and descending schedules. Results of this experiment show that Ro 63-1908 (0.3 and 1.0 mg/kg) increases sensitivity to delayed reinforcement, but only when the delays decrease across the session (Yates et al. under review). Overall, there is growing evidence to suggest that the order in which delays are presented can alter how drugs affect delay discounting.

Signaling the delay to the large magnitude reinforcer

When examining baseline discounting, there are some inconsistencies in the literature concerning if the use of a cue during the delay alters delay sensitivity. For example, Cardinal et al. (2000) show that baseline levels of discounting are similar in rats exposed to a cue during the delay to reinforcement relative to rats that were not exposed to a cue. However, Zeeb et al. (2010) report that inserting a cue increases responding for the large magnitude reinforcer, but this effect is more pronounced in rats that have a low baseline level of delay sensitivity. The null effects observed in the Cardinal et al. (2000) study may be partially explained by the fact that they did not examine the contribution of basal levels of delay sensitivity in their study.

Using a cue to signal the delay to delivery of the reinforcer can modulate drug effects in the Evenden and Ryan (1996) procedure. For example, a low dose of amphetamine (0.3 mg/kg) decreases sensitivity to delayed reinforcement when the delay to delivery of the large magnitude reinforcer is cued, but higher doses (1.0 and 1.6 mg/kg) increase delay sensitivity when no signal is provided during the delay (Cardinal et al. 2000). Cardinal et al. (2000) suggest the discrepant findings are due to amphetamine’s ability to enhance the value of conditioned reinforcers (i.e., the cue to reinforcement; Robbins et al. 1983). A visual inspection of the data (see Figs. 3b–3c of Cardinal et al. 2000) shows that amphetamine (1.6 mg/kg) causes a marked reduction in responding for the large reinforcer at the 0-s delay in both the cue (~80% following vehicle treatment vs. ~50% following amphetamine treatment) and no cue (~80% following vehicle treatment vs. ~40% following amphetamine treatment) conditions. The lowest dose of amphetamine (0.3 mg/kg) appears to cause an increase in responding for the large reinforcer at the 0-s delay in the cue condition (~80% following vehicle treatment vs. ~95% following amphetamine treatment). Thus, one cannot rule out the possibility that amphetamine impairs stimulus control instead of altering sensitivity to delayed reinforcement per se. Somewhat related to this point, signaling the delay to reinforcement does not modulate amphetamine’s effects on delay discounting; instead, amphetamine decreases responding at the 0-s delay for both conditions (Slezak and Anderson 2009), which is similar to the findings reported by Cardinal et al. (2000). Future studies can further increase our understanding of how cues affect delay discounting by incorporating quantitative analyses (e.g., fitting hyperbolic functions or using the generalized matching law).

There is some evidence that cues can selectively modulate drug effects in sensitivity to delayed reinforcement. Direct administration of the DA D₁ or D₂ antagonists SCH 23990 and eticlopride into orbitofrontal cortex (OFC) do not alter delay sensitivity when a cue is absent. However, when the delay to delivery of the large reinforcer is signaled, these drugs tend to increase sensitivity to delayed reinforcement (Zeeb et al. 2010). The decreased responding for the large magnitude reinforcer is not apparent at the 0-s delay, suggesting that blocking these receptors selectively increases sensitivity to delayed reinforcement as opposed to decreasing discriminability of the reward alternatives. When these data are taken into consideration with the data reported by Cardinal et al. (2000) and Slezak and Anderson (2009), the decreases in responding at the 0-s delay may be due to amphetamine’s actions outside of the OFC.

Schedule of reinforcement

Most studies assessing delay discounting use a fixed ratio (FR) 1 schedule of reinforcement, in which a single response on the manipulandum leads to reinforcement. However, a higher response requirement (FR 10) is reported in the literature (Huskinson and Anderson 2013; Yates et al. 2017a, 2017b). Interestingly, Huskinson and Anderson (2013) show that higher FR requirements promote responding for the large magnitude reinforcer, whereas my laboratory uses an FR 10 because rats trained on the FR 1 schedule do not discount the large magnitude reinforcer (see supplemental Fig. 1 of Yates et al. 2017a). Despite these differences, no studies (to my knowledge) systematically determine if the schedule of reinforcement modulates drug effects in delay discounting. So far, only indirect evidence suggests that the FR requirement can alter drug effects. For example, MK-801 (0.03 and 0.06 mg/kg) decreases sensitivity to delayed reinforcement when an FR 1 schedule of reinforcement is used (Higgins et al. 2016; Yates et al. 2015) but does not alter delay discounting when an FR 10 schedule of reinforcement is used (Yates et al. 2017a). Similarly, when an FR 1 schedule of reinforcement is used, antagonism of mGluR₁ receptors decreases delay sensitivity (Sukhotina et al. 2008); however, blocking these receptors increases sensitivity to delay when an FR 10 is used (Yates et al. 2017b). Finally, Ro 63-1908 (1.0 mg/kg) decreases sensitivity to delay when an FR 1 is used (Higgins et al. 2016), but our lab was unable to replicate this finding using an FR 10 schedule (Yates et al. under review).

Reinforcer magnitude

Recent evidence suggests that reinforcer magnitude can modulate drug effects in the Evenden and Ryan (1996) procedure. Specifically, rats show increased preference for the small magnitude reinforcer when 1 and 3 pellets are used for the small magnitude and large magnitude rewards, respectively, relative to when they are allowed to choose between a 2-pellet alternative and 6-pellet alternative (Krebs et al. 2016). Furthermore, amphetamine (1.0 and 1.8 mg/kg) increases delay sensitivity, but only in the large magnitude (2 vs. 6 pellets) condition (note: in the small magnitude condition, amphetamine appears to decrease responding at the 0-s delay, but this analysis was not included).

Range of delays to the large magnitude reinforcer

Although the range of delays typically goes from 0–40 s (Broos et al. 2012; Eubig et al. 2014; van den Bergh et al. 2006) or 0–60 s (Baarendse and Vanderschuren 2012; Barbelivien et al. 2008; Cardinal et al. 2000; Evenden and Ryan 1996; Koffarnus et al. 2011; Krebs et al. 2016; Liu et al. 2017; Stanis et al. 2008; van Gaalen et al. 2006; Winstanley et al. 2003), there are some studies that use shorter or longer ranges. However, to date, studies rarely consider how the range of delays can modulate drug effects in the Evenden and Ryan (1996) procedure. However, in a study measuring delay discounting in Lewis and Fischer 344 rats, Huskinson et al. (2012) show that amphetamine (0.3, 1.0, and 1.7 mg/kg) decreases sensitivity to delayed reinforcement in Lewis rats trained on a discounting procedure in which the maximum delay is set to 16 s; however, only the highest dose of amphetamine (1.7 mg/kg) decreases sensitivity to delayed reinforcement in these rats when the maximum delay is set to 40 s.

How Different Statistical Analyses Can Alter Interpretation of Drug Effects

Because the Evenden and Ryan (1996) procedure can be analyzed several ways, this section will focus on this paradigm. There are several cases in which using one analysis (e.g., ANOVA) can lead to different conclusions about a drug’s effects on delay discounting compared to another analysis (e.g., NLME). Two examples of how different analyses can alter drug effects have already been provided. When ANOVA analyses are used to determine the effects of memantine on delay discounting, results suggest that this drug increases preference for the small, immediate reinforcer (Fig. 1a). However, when a hyperbolic function is applied to the data, results show no changes in the rate of discounting; instead, the intercept of the function decreases, which suggests alterations in some other behavioral process (e.g., discriminability of reinforcer alternatives), as opposed to shifts in preference for one alternative relative to the other (Fig. 1b). Additionally, when AUCs are analyzed (Fig. 1c), the analyses show a decrease in AUC values following memantine administration, suggesting increased sensitivity to delayed reinforcement. In this case, two analyses (raw proportion of responses and AUC) lead to the same conclusion, whereas NLME leads to different interpretations of memantine’s effects on delay discounting.

As discussed previously, MK-801 (0.03 mg/kg) increases responding for the large magnitude reinforcer (Fig. 2a; Yates et al. 2015). The effects of MK-801 on k parameter estimates are dependent on how they are analyzed (i.e., MK-801 decreases k estimates when a two-stage analysis is used, but has no effect on k when NLME is used; Figs. 2b and 2c, respectively). Finally, MK-801 significantly increases AUCs (Fig. 2d). In this case, three analyses (raw proportion of responses, two-stage analyses of k parameter estimates, and AUC) are interpreted as decreases in sensitivity to delayed reinforcement; however, when k parameter estimates are derived via NLME, MK-801 does not affect delay discounting, which is what our laboratory reported in a follow-up study (Yates et al. 2017a).

Figure 3 shows the effects of D-cycloserine, a partial agonist at the glycine site of NMDA receptors, on delay discounting (Yates et al. 2017a). ANOVA analyses show that D-cycloserine (15.0 mg/kg) significantly increases responses for the large magnitude reinforcer at the 10-s delay only (significant dose × delay interaction only; Fig. 3a). However, when NLME, as well as AUC, analyses are applied to the data, D-cycloserine does not affect delay discounting (Figs. 3b and 3c, respectively). The null effects observed following NLME and AUC analyses are more consistent with previous research showing no effects of D-cycloserine on delay discounting (van den Bergh et al. 2006). The significant effect obtained following ANOVA may be an example of increased Type I error (see Young et al. 2009). In this case, using k estimates derived via NLME and AUCs leads to similar conclusions: D-cycloserine has little effect on sensitivity to delayed reinforcement.

Figure 4 shows the effects of CGS 19755 (5.0 mg/kg), a competitive antagonist at NMDA receptors, on delay discounting (Yates et al. 2017a). When ANOVA analyses are applied, CGS 19755 (5.0 mg/kg) significantly decreases responding for the large magnitude reinforcer, suggesting increased sensitivity to delayed reinforcement (Fig. 4a). Concerning NLME analyses, the results are dependent on which function is applied to the data. When the hyperbolic function is applied to the data (as is the case for this review), CGS 19755 tends to increase sensitivity to delayed reinforcement (p = .064; Fig. 4b). When an exponential function is used (as in the case of Yates et al. 2017a), CGS 19755 significantly increases k parameter estimates (p = .009; Fig. 4c). The discrepant findings observed between using k parameter estimates derived from the hyperbolic and exponential functions exemplify another limitation of using parameter estimates derived from nonlinear functions; that is, using one function (in this case, exponential) can yield parameter estimates that are statistically different (when compared to vehicle), whereas using a different function (in this case, hyperbolic) can lead to no differences following vehicle and drug treatments. Because the hyperbolic discounting function, but not exponential function, accounts for preference reversals (see Green and Myerson 2004 for a review), this model provides a better theoretical measure of delay discounting. One important consideration is that the non-significant effects observed following use of the hyperbolic function may be attributed to the increased variance in k parameter estimates (compare Figs. 4b and 4c). Finally, there are no differences in AUC values between vehicle and CGS 19755 (Fig. 4d). Here, two analyses (raw proportion of responses and parameter estimates derived from the exponential model) lead to the conclusion that CGS 19755 increases sensitivity to delayed reinforcement, whereas two analyses (parameter estimates derived from the hyperbolic model and AUCs) lead to the conclusion that CGS 19755 does not alter delay discounting.

The examples above highlight just a few ways in which statistical analyses can alter interpretation of drug effects in the Evenden and Ryan (1996) procedure. One observation worth highlighting is that detecting changes in AUCs appears to be difficult when a drug affects responses for the large magnitude reinforcer at one delay only. For example, memantine, which decreases responding for the large reinforcer at the 0-s and 10-s delays, causes a significant decrease in AUCs (Fig. 1c). A similar finding is observed following ifenprodil (NR2B subunit antagonist) administration (Yates et al. 2017a). However, when a drug alters responding at one delay only, especially at a short delay, changes in AUC become more difficult, as is the case following D-cycloserine and CGS 19755 administration. Also, the null effects observed for AUCs following D-cycloserine and CSG 19755 may be due to the fact that short delays contribute less to AUC values relative to long delays (see Borges et al. 2016). By log-transforming each delay as suggested by Borges et al. (2016), statistically significant difference between vehicle and CGS 19755 can be observed. Furthermore, a significant main effect of dose is observed for AUCs following D-cycloserine administration when the method proposed by Borges et al. (2016) is used, although post hoc tests do not show a significant difference between vehicle and any of the D-cycloserine doses.

Discussion

Although the primary focus of this review was to discuss how procedural/analytic approaches can alter how one interprets drug effects in discounting paradigms, the issues discussed previously can be applied to experiments in which delay sensitivity is being compared between two or more groups. For example, when comparing young rats to aged rats, Simon et al. (2010) argue that younger rats show greater delay sensitivity; however, this group of rats responds less for the large magnitude reinforcer when its delivery is immediate. Because discounting functions were not applied to these data, stating that young rats show greater sensitivity to delayed reinforcement may not be entirely accurate. Furthermore, when examining group differences across different discounting paradigms, discordant findings can be observed. Specifically, sex differences are typically not observed in the Evenden and Ryan (1996) procedure (Eubig et al. 2014; Smethells et al. 2016), but females show increased preference for a small, immediate reinforcer in a modified adjusting delay procedure (Perry et al. 2007). If the goal is to determine if certain factors such as sex, animal strain, and housing condition alter delay discounting, using procedures that manipulate delay across sessions (e.g., adjusting amount/adjusting delay) may be more informative, as sensitivity to reinforcer magnitude and/or sensitivity to delayed reinforcement can be assessed. For example, studies that assess how excitotoxic lesions alter sensitivity to reinforcer magnitude/sensitivity to delayed reinforcement often manipulate the delay to one alternative across sessions (e.g., Acheson et al. 2006; Bezzina et al. 2007; Bezzina et al. 2009; Kheramin et al. 2002; Mobini et al. 2002).

After reviewing the literature, there are a few recommendations for assessing drug effects in delay discounting. First, when possible, use at least two different paradigms in the same study (e.g., Smethells and Carrol 2015). Comparing two different behavioral measures of impulsive behavior is common (e.g., impulsive choice paradigms vs. motor impulsivity tasks [e.g., five-choice serial reaction time task]; Baarendse and Vanderschuren 2012; Broos et al. 2012; Hellemans et al. 2005; Higgins et al. 2016; Isherwood et al. 2015; Isherwood et al. 2017; Korte et al. 2017; Liu et al. 2017; Mendez et al. 2012; Paine et al. 2003; Paterson et al. 2012; Robinson et al. 2008; Schippers et al. 2016; Schneider et al. 2011; Sukhotina et al. 2008; Sun et al. 2012; Talpos et al. 2006; Winstanley et al. 2004; Winstanley et al. 2007; Wiskerke et al. 2011), but studies assessing two delay discounting paradigms are rare. As such, determining if the differential drug effects observed across discounting procedures are due to differences in the paradigm itself or are due to idiosyncrasies across studies (e.g., number of forced/free choice trials; the magnitudes of the reinforcers, the delays to reinforcement) is difficult when making these comparisons across studies as opposed to within a single study.

From a practical standpoint, testing drug effects in two (or more) paradigms in the same experiment may not always be feasible. If testing animals in a single discounting procedure, one recommendation is to use a concurrent-chains procedure (e.g., Aparicio et al. 2013) instead of the Evenden and Ryan (1996) procedure. This procedure offers one major advantage to the Evenden and Ryan (1996) procedure; it prevents animals from showing exclusive preference for one reward alternative, thus allowing one to observe upward/downward responses for the large magnitude reinforcer at each delay. The concurrent-chains procedure can also be used to determine how a drug affects sensitivity to delayed reinforcement and sensitivity to reinforcer amount (see Pitts and Febbo 2004). The use of concurrent-chains procedures to measure delay discounting has become more common in recent years (Aparicio et al. 2013; Aparicio et al. 2015; Aparicio et al. in press; Beeby and White 2013; Grace 2002; Johnson et al. 2013; Oliveira et al. 2014; Ong and White 2004; Orduña 2015; Orduña and Mercado 2017; Orduña et al. 2013; Pitts and Febbo 2004), although pharmacological manipulations have rarely been included in this paradigm (but see Johnson et al. 2013; Pitts and Febbo 2004). If using a concurrent-chains procedure, it may be best to use the generalized matching law to describe how drugs alter delay discounting, as this analysis allows one to determine if pharmacological manipulations selectively alter sensitivity to delayed reinforcement or alter sensitivity to reinforcer magnitude (see Fig. 1 of Pitts and Febbo 2004 for an excellent illustration of how drugs can differentially affect these parameters), and one does not need to make inferences about which discounting function best represents the data.

If using the Evenden and Ryan (1996) procedure, researchers should be encouraged to avoid analyzing discounting data by conducting ANOVAs on the raw proportion of choices for the large reinforcer, as it has theoretical and practical limitations (e.g., decreased power, provides no information about the rate of discounting, etc.). Instead, it is recommended to calculate AUCs and/or derive A and k parameter estimates from the hyperbolic function via NLME. Because AUC is an atheoretical measure of delay discounting, one does not need to specify which mathematical function best describes the data; however, if a drug decreases responding at the 0-s delay, using hyperbolic functions can allow one to determine if a drug selectively alters sensitivity to delayed reinforcement. Although the A parameter estimate does not provide much information about an animal’s sensitivity to reinforcer magnitude per se, changes in this parameter indicate that some other behavioral process (e.g., ability to discriminate the large and small magnitude reinforcer) is being altered by the drug. This is critically important for determining if a drug is a potential pharmacotherapy for impulse-control disorders. For example, Figure 5 shows hypothetical data for a potential ADHD medication. This figure shows that k parameter estimates are lower for the drug relative to vehicle, suggesting this drug is effective in reducing sensitivity to delayed reinforcement. However, the drug also decreases responding at the 0-s delay, resulting in decreased A parameter estimates. The decreased intercept suggests that the potential pharmacotherapy is altering the animal’s ability to effectively discriminate the two reward alternatives. Because the observed effects are not isolated to sensitivity to delayed reinforcement, this drug may not be an effective treatment for disorders characterized by impulse-control deficits.

Figure 5.

Hypothetical discounting data showing how a potential pharmacotherapy (dashed line) for impulse-control disorders can increase responding for a large, delayed reinforcer across multiple delays but can decrease responding for the large reinforcer when its delivery is immediate relative to vehicle (solid line). At first glance, these data seem to support the idea that the drug is a potential treatment for disorders characterized by increased delay sensitivity. However, because the drug also decreases responses for the large alternative at the 0-s delay (decrease in A parameter), the drug may be causing a general disruption in task performance. The curves were generated using GraphPad Prism (version 5.0).

In conclusion, the major goal of this literature review was to highlight how procedural/analytic differences may account for some discrepant findings observed across behavioral pharmacology experiments. Another goal was to discuss some of the potential limitations associated with commonly used procedures/analyses and to offer some suggestions for future studies. Hopefully by utilizing some of the suggestions detailed in this review, we may better elucidate the neural underpinnings of delay discounting.