Changing Delay Discounting and Impulsive Choice: Implications for Addictions, Prevention, and Human Health

Abstract

Delay discounting describes the tendency to devalue delayed consequences or future prospects. The degree to which an individual discounts delayed events appears trait-like in that it is stable over time and across functionally similar situations. Steeply discounting delayed rewards is correlated with most substance-use disorders, the severity of these disorders, rates of relapse to drug use, and a host of other maladaptive decisions affecting human health. Longitudinal data suggest steep delay discounting and high levels of impulsive choice are predictive of subsequent drug taking, which suggests (though does not establish) that reducing delay discounting could have a preventive health-promoting effect. Experimental manipulations that produce momentary or long-lasting reductions in delay discounting or impulsive choice are reviewed, and behavioral mechanisms that may underlie these effects are discussed. Shortcomings of each manipulation technique are discussed and areas for future research are identified. Although much work remains, it is clear that impulsive decision making can be reduced, despite its otherwise trait-like qualities. Such findings invite technique refinement, translational research, and hope.

In trying to solve the terrifying problems that face us in the world today, we naturally turn to the things we do best. We play from strength, and our strength is science. . . . B. F. Skinner (1971, p. 3)

With these ominous, yet optimistic words, Skinner opens his book, Beyond Freedom and Dignity. When one reads Skinner’s opening chapter in 2019, there is the continued sense that we face a range of “terrifying problems.” Yet there is also cause for optimism in the precise direction Skinner specifies. Some of the seemingly intractable problems facing the world in the 1970s (e.g., the Malthusian prediction that worldwide food shortages and starvation would inevitably accompany an overpopulated Earth; Ehrlich, 1968) were overcome by playing to our strengths in science (e.g., agricultural research and technology that dramatically increased crop yields).

One truth that spans the centuries is that these terrifying problems have, at their root, the decisions that humans make on a daily basis—small decisions borne of convenience or momentary pleasure that sum to warm the climate, weaken family/community ties, and decrement the health of self and others. Consider the small decision to drive short distances rather than walking or riding a bicycle. Driving allows us to get what we want now at minimal effort relative to walking; however, driving contributes to global warming and negatively affects personal health (obesity, bone density, blood pressure, heart disease, etc.). So many of the maladaptive decisions that we repeatedly make, and later regret, have in common a conflict between our preference for immediacy (convenience, low effort) and our preference for better outcomes in the long run. When immediacy trumps better long-run outcomes (i.e., a larger or better quality reinforcer) we refer to the choice as “impulsive”; the opposite situation—when better long-run outcomes are favored over lesser, but immediately available reinforcers—is referred to as a “self-controlled” choice. These everyday terms have the baggage of implying reified explanations; we will attempt to avoid such logical circularities here.

Problem drug use has long been conceptualized as an impulsive choice because the larger–later benefits of drug abstinence (or responsible drug use) are forfeited for the smaller–sooner benefits of another episode of excessive drug taking (Ainslie, 1975; Herrnstein & Prelec, 1992). The utility in this conceptual analysis resides in the 50+ years of behavioral research on the nature of impulsive choice. For an objectively larger but delayed reward to be foregone in favor of a smaller–sooner one requires that the subjective value of the former fall below that of the latter. Chung and Herrnstein (1967) provided an initial quantitative model of this process, one that would be expanded upon and put to further empirical tests by Mazur (1987) and others (Ainslie & Herrnstein, 1981; Kirby & Herrnstein, 1995; Loewenstein & Prelec, 1992; Madden, Bickel, & Jacobs, 1999; Navarick & Fantino, 1976; Rachlin, 1989; Rachlin & Green, 1972).

These decades of cross-species research reveal that the value of a commodity declines according to a hyperbolic (or hyperboloid) discounting function like the ones shown in Figure 1 (Green & Myerson, 2004; Mazur, 1987; Rachlin, Raineri, & Cross, 1991).1 When faced with a choice between a smaller–sooner reward (SSR; e.g., getting high) and a larger–later reward (LLR; future abstinence-related improvements in health/social/vocational outcomes), the individual that steeply discounts the future (solid curve) will make an impulsive choice (choosing the SSR) because the discounted value of the LLR (open circle) falls well below the undiscounted value of the SSR. For the individual who, by nature, experience, or some combination of the two, discounts less steeply (dashed curve) the value of the LLR at the time of the choice (open square) is greater than the SSR and the impulsive choice (e.g., drug use) is avoided.

Fig. 1.

Discounted values of a larger-later reward (LLR) and smaller–sooner reward (SSR). The heights of the filled bars reflect the objective (undiscounted) value of each reward. The steep discounting function (solid curve) reflects how the LLR is discounted in value such that, from the temporal vantage point of the stick-figure decision maker, the value of the SSR exceeds that of the LLR (open circle). The shallower discounting function (dashed curve) illustrates that from the same temporal vantage point, the discounted value of the LLR (open square) exceeds that of the undiscounted SSR

Consistent with this conceptual analysis, individual differences in how steeply LLRs are discounted robustly correlates with the incidence and severity of problem substance use (Bickel, Odum, & Madden, 1999; Madden, Petry, Badger, & Bickel, 1997; Heil, Johnson, Higgins, & Bickel, 2006; Mitchell, Fields, D’Esposito, & Boettiger, 2005; see meta-analyses by Amlung, Vedelago, Acker, Balodis, & MacKillop, 2017; MacKillop et al., 2011) and a host of other maladaptive decisions affecting human health (Daugherty & Brase, 2010; Fields, Sabet, Peal, & Reynolds, 2011; Jarmolowicz et al., 2014; Odum, Madden, Badger, & Bickel, 2000; Snider, DeHart, Epstein, & Bickel, 2019). Longitudinal studies suggest that steep delay discounting precedes and predicts human drug use (Audrain-McGovern et al., 2009; Barlow, McKee, Reeves, Galea, & Stuckler, 2016; Fernie et al., 2013; Khurana et al., 2013; Kim-Spoon, Farley, Holmes, Longo, & McCullough, 2014), lower rates of drug abstinence, and greater relapse during treatment (Coughlin, Tegge, Sheffer, & Bickel, 2018; Harvanko, Strickland, Slone, Shelton, & Reynolds, 2019; Loree, Lundahl, & Ledgerwood, 2015; MacKillop & Kahler, 2009; Sheffer et al., 2014). Indeed, when Coughlin et al. (2018) used machine learning to predict short- and long-term smoking cessation during cognitive behavioral therapy (CBT), delay discounting proved to be “the single best predictor of group CBT treatment response” (p. 1). Pretreatment discount rates correctly predicted the smoking status of 69.5% of the participants in the training cohort (i.e., the portion of the data used by the machine-learning algorithm to decide how best to predict smoking outcomes) and 76.4% of a validation cohort. Other measures (e.g., memory, locus of control, scores on the Fagerström test of nicotine dependence) either improved these predictions very little (training cohort), or not at all (validation cohort).

The hypothesis that steeply discounting future outcomes precedes and predicts drug-taking and relapse has also been explored in nonhuman research. Male and female rats that predominantly choose smaller–sooner food rewards are more likely to acquire low-dose cocaine self-administration than rats that prefer to wait for a larger food reward (Perry, Larson, German, Madden, & Carroll, 2005; Perry, Nelson, & Carroll, 2008; Zlebnik & Carroll, 2015). This finding has been inconsistently replicated, however, with other drugs of abuse (Poulos, Le, & Parker, 1995; Schippers, Binnekade, Schoffelmeer, Pattij, & De Vries, 2012; see Stein & Madden, 2013 for review). During post-acquisition phases of the nonhuman drug-taking timeline, some evidence suggests impulsive choice is predictive of higher cocaine or methylphenidate intake (Anker, Perry, Gliddon, & Carroll, 2009; Marusich & Bardo, 2009; Perry et al., 2008), more robust cocaine-seeking when price increases are encountered (Koffarnus & Woods, 2013), and greater reinstatement of drug seeking (Perry et al., 2008).

The hyperbolic shape of the discounting function also helps to explain the previously mentioned violations of the stationarity axiom in economics (Strotz, 1955); i.e., the tendency to prefer LLRs when SSRs are not readily available, and to reverse this preference when the SSR is immediately accessible. This has been documented empirically many times in human and nonhuman subjects (e.g., Kirby & Herrnstein, 1995; Rachlin & Green, 1972) and its relation to hyperbolic discounting is illustrated in Figure 2. When both the SSR and LLR are temporally distal at T1, the discounted value of the LLR (solid curve) exceeds that of the SSR (dashed curve), and the LLR is preferred. However, as the decision maker moves through time to T2, the curves cross and an impulsive choice is made when the discounted value of the LLR falls below the undiscounted value of the SSR. From the temporal perspective provided at T1, the decision maker can clearly see the benefits of drug abstinence, adhering to a diet, safe driving. But as one approaches T2 (e.g., when a cigarette is offered, a meal is about to be ordered, or a text is received while driving), this perspective is replaced with temporal myopia and an impulsive choice. This apparent “loss of control” is ubiquitous, predicted by hyperbolic discounting, and is at the heart of our shared, periodic sense of self-loathing.

Fig. 2.

Discounted values of the larger–later reward (LLR) and smaller–sooner reward (SSR); values indicated by the solid and dashed curves, respectively. The heights of the filled bars reflect the objective (undiscounted) value of each reward. At time T1 both the SSR and LLR are delayed and the discounted value of the LLR exceeds that of the SSR. At time T2 the value of the immediately available SSR exceeds the discounted value of the LLR.

Discounting Trait and State

Within personality research, traits are defined as behavioral and cognitive patterns that are relatively stable over time and functionally similar situations (e.g., Roberts, 2009). There is strong evidence that the degree to which an individual discounts delayed monetary rewards is relatively stable over time (Odum, 2011b). Although the extent to which delayed rewards are discounted decreases over the lifespan (Green, Fry, & Myerson, 1994), the rank-order correlation of these discounting rates is comparable to other psychological traits across the 3-month to 4-year range of test–retest intervals explored thus far (Anokhin, Golosheykin, & Mulligan, 2015; Audrain-McGovern et al., 2009; Fernie et al., 2013; Hulka et al., 2015; Kirby, 2009; Ohmura, Takahashi, & Kitamura, 2005; Peters & Büchel, 2009). That is, those individuals who demonstrate steep discounting at the initial assessment continue to be ranked at the steep end of the participant pool at the retest.

With respect to the consistency of discounting across situations, Odum (2011b) reported from archival data (five studies) that although discounting rates varied across commodities (e.g., steeper discounting of delayed food relative to delayed money), those who steeply discounted one commodity tended also to steeply discount the other one; in general, data published since support this conclusion (Bickel, Jarmolowicz, Mueller, Franck et al., 2012; Friedel, DeHart, Madden, & Odum, 2014). This finding is core to the contention that steeply discounting future outcomes (all future outcomes) renders delay discounting a causal behavioral process in a variety of addictive disorders; what Bickel, Jarmolowicz, Mueller, Koffarnus, and Gatchalian (2012) have called a “trans-disease process.” Although this hypothesis has justifiably garnered a good deal of attention, it is important to note that the evidence for a relation between delay discounting and addictions remains correlational; third-variable accounts remain tenable (e.g., Levin, Haeger, Ong, & Twohig, 2018). A causal relation can be established only by experimentally manipulating delay discounting, monitoring potential mediators, and evaluating the effects of this manipulation on subsequent addictive behavior.

With respect to the latter, the trait status of delay discounting should not be read as implying that discounting cannot be changed. Psychological traits might best be viewed as starting points. Ample evidence reveals that experimental manipulations can reduce delay discounting and impulsive choice (for reviews, see Koffarnus, Jarmolowicz, Mueller, & Bickel, 2013; Rung & Madden, 2018a) or increase it (e.g., Van den Bergh, Dewitte, & Warlop, 2008). If delay discounting proves to be a trans-disease process influencing disorders of choice, then a future research priority should be the further development and refinement of experimental methods designed to reduce delay discounting. In what follows of this article, we highlight some of the more important findings from prior reviews of the experimental manipulations of delay discounting literature, and include several recent studies published since. We categorize these experimental manipulations based on their effects, producing either momentary or lasting reductions in delay discounting. Said another way, these manipulations produce either state changes in decision making or trait alterations that span time and, perhaps, situation. Despite hundreds of papers already having been published in this area, the work is still in its infancy.

Momentary Manipulations

The majority of the experimental manipulations of delay discounting in humans have focused on brief in-lab manipulations, with effects typically examined in a single session. Here we focus on three of these manipulations—framing, episodic future thinking, and nature exposure—because they have produced the largest and most consistent momentary reductions in delay discounting.

Framing

Framing manipulations consist of altering the manner in which the choice alternatives are described, with the constraint that they remain economically equivalent. For example, the typical framing of choice alternatives in a delay–discounting task might ask the participant to choose between $50 now and $100 in 1 year. One approach to reframing these alternatives makes explicit the implicit outcomes of choosing one alternative or the other. This is accomplished by asking the participant to choose between $50 now and $0 in 1 year, and $0 now and $100 in 1 year (italics indicate the implicit outcome made explicit in this frame). When this explicit-zero framing is used, the extent to which the LLR is discounted is significantly reduced (Magen, Dweck, & Gross, 2008; Radu, Yi, Bickel, Gross, & McClure, 2011; Wu & He, 2012) with medium to large effect sizes (see Rung & Madden, 2018a for meta-analysis). The effect is driven by drawing participant attention to the opportunity costs of choosing the SSR (Read, Olivola, & Hardisty, 2017; Wu & He, 2012); i.e., the receipt of $0 in the future.

A second effective framing manipulation specifies the date on which the LLR will be provided. That is, rather than describing the LLR as $100 in 1 year it would be described as $100 in [insert date 1 year from today]. This date-framing manipulation typically produces medium to large reductions in delay discounting (DeHart & Odum, 2015; Klapproth, 2012; LeBoeuf, 2006; Read, Frederick, Orsel, & Rahman, 2005; see Rung & Madden, 2018a for meta-analysis). The date-framing effect might occur by drawing participant attention to when the LLR will be provided, rather than to how long one must wait (LeBoeuf, 2006; Read et al., 2005).

Episodic Future Thinking

Where framing manipulations direct participant attention to different components of the choice alternatives, episodic future thinking (EFT) manipulations guide participants through a session in which they imagine a personal future event in detail (Atance & O’Neill, 2001; Szpunar, Spreng, & Schacter, 2014). For example, the participant might be asked to imagine an event happening in 1 year. If the participant imagines, for example, going on vacation in a year, they will then be prompted to describe concrete particulars of the vacation; e.g., what will you do while on vacation? what will you see and who will accompany you? When these participants subsequently complete a discounting task and are prompted to imagine their future events, the extent to which they discount delayed rewards is reduced relative to a control group (Daniel, Stanton, & Epstein, 2013; Kwan et al., 2015; Lin & Epstein, 2014; O’Donnell, Daniel, & Epstein, 2017; Stein et al., 2017). Evidence that EFT can also positively affect health decision-making is provided by laboratory reductions in cigarette smoking (hypothetical and real; Stein, Tegge, Turner, & Bickel, 2018; Stein et al., 2016), hypothetical alcohol consumption (Bulley & Gullo, 2017; Snider, LaConte, & Bickel, 2016), and real calorie consumption in lab and field settings (Daniel et al., 2013; Daniel, Said, Stanton, & Epstein, 2015; O’Neill, Daniel, & Epstein, 2016; Sze, Daniel, Kilanowski, Collins, & Epstein, 2015).

Rung and Madden (2018b) raised the concern that these beneficial effects of EFT might be due to demand characteristics inherent in the experimental sessions (Orne, 1962). In particular, prompting participants to think about future events (e.g., going on vacation in 1 year) while choosing between money now and money in 1 year may, because of the correspondence between the delays (1 year), reveal the purpose of the study to the participants, who may play the “good subject” role by choosing the LLR; i.e., faking a lower rate of delay discounting. In support of this hypothesis, Rung and Madden reported that participants who read a fictional description of an EFT experiment were able to correctly deduce that the fictional experimenter expected the manipulation to reduce the fictional participant’s delay discounting and junk-food consumption.

Two subsequent studies, however, do not support this demand–characteristics hypothesis (Rung & Madden, 2019; Stein et al., 2018). In one of these, Rung and Madden (2019) experimentally manipulated procedures thought to induce demand characteristics: in particular, components of the cues used to prompt engaging in EFT while completing the discounting task. When cue components hypothesized to maximize demand characteristics were removed (i.e., participants were cued to engage in EFT without a time interval corresponding to the delay to the LLR), EFT remained effective in reducing delay discounting. When cue components hypothesized to prompt the active ingredient of EFT (thinking episodically) were removed, the EFT effect was no longer observed. That said, when participants were not cued to engage in EFT during the discounting task, delay discounting was unaffected by having previously engaged in EFT. These findings suggest EFT can reduce delay discounting independent of demand characteristics but also that training participants to deploy their EFT skills at appropriate times is an important future research direction.

Nature Exposure

Several studies have now demonstrated that brief exposure to nature cues (e.g., images depicting nature scenes such as forests, mountains, and streams; or spending time in nature itself) produces medium to large reductions in delay discounting (Berry et al., 2015; Berry, Sweeney, Morath, Odum, & Jordan, 2014; van der Wal et al., 2013; see Rung & Madden, 2018a for meta-analysis). The mechanism of this effect is unclear, but inconsistent or null effects of mechanisms such as time perception and affect changes suggest that nature cues either operate via an expanded sense of space (i.e., feeling less boxed in; Repke et al., 2018) or through evolutionary mechanisms (i.e., viewing pastoral scenes signal safety and abundance, conditions in which waiting is adaptive because the threat of predation is low and resources are plentiful; van der Wal et al., 2013). Repeated exposure to nature has known health benefits (e.g., Li et al., 2008) and Repke et al. (2018) reported that these benefits are indirectly linked to changes in delay discounting. That is, long-term nature exposure coincides with reduced delay discounting and this reduction coincides with decision making that improves health.

From State to Trait

The Repke et al. (2018) findings suggest that momentary manipulations need not be momentary and that the effects of such manipulations on decision making may, over time/repeated exposure, transition from state to trait. That is, those individuals who have daily access to nature appear to experience a long-lasting change in delay discounting that influences their health decision making in a variety of settings. The same could be true of framing and EFT, if these interventions were expanded with an eye toward teaching individuals to either reframe their options in a way that promotes the self-control choice (e.g., explicitly recognizing that choosing the SSR means there will be no LLR) or to engage in momentary EFT prior to making a choice (see, e.g., Sze et al., 2015). Such changes in the choice repertoire could be taught in a variety of settings such as elementary schools or traditional talk-therapy sessions, with opportunities to deploy these skills in increasingly realistic choice contexts.

Lasting Reductions in Delay Discounting

The repertoire-altering effects just alluded to have been the explicit focus of several lines of animal research. We consider just three of these research lines—timing interventions, delay fading, and delay exposure—as the replicability of these effects have been most thoroughly explored.

Timing Interventions

The empirical observation that imprecision in interval timing is correlated with impulsive choice (Darcheville, Rivière, & Wearden, 1992; Marshall, Smith, & Kirkpatrick, 2014) and delay discounting (McClure, Podos, & Richardson, 2014) led to experimental manipulations of timing precision and subsequent evaluations of the effects of this change on impulsive choice. Timing precision refers to unsystematic error in the timing of an interval and is typically measured using either a peak procedure (McClure et al., 2014) or a temporal bisection task (Marshall et al., 2014).

Smith, Marshall, and Kirkpatrick (2015) reported that providing rats with extended training under time-based schedules of reinforcement (e.g., fixed- or variable-interval schedules) increased preference for the LLR. These effects have proven replicable (Bailey, Peterson, Schnegelsiepen, Stuebing, & Kirkpatrick, 2018; Fox, Visser, & Nicholson, 2018; Peterson & Kirkpatrick, 2016; Stuebing, Marshall, Triplett, & Kirkpatrick, 2018). The effect size across studies is moderate in magnitude (Rung & Madden, 2018a), the effect lasts at least 9 months following training (Bailey et al., 2018), is observed in male and female rats (Stuebing et al., 2018), and in aged male rats (Peterson & Kirkpatrick, 2016).

Although these effects are clear, the mechanism of change is less so. When McClure et al. (2014) reported a significant relation between timing precision and delay discounting in rats, the complete profile of their data led them to conclude that this relation was due to a third variable correlated with these two dependent measures. Although Smith et al. (2015) reported that training rats with interval schedules of reinforcement improved timing precision, other studies have not replicated this effect (Peterson & Kirkpatrick, 2016; Stuebing et al., 2018; Fox et al., 2018). Subsequent studies suggest the relation between timing precision and impulsive choice is less clear: several studies have found no significant relation between these variables as they naturally occur (Peterson & Kirkpatrick, 2016; Rung, Buhusi, & Madden, 2018), and postintervention the significance of their relation is mixed (e.g., Peterson & Kirkpatrick, 2016; cf. Fox et al., 2018).

Future research in this domain should include mediation analyses to statistically reveal the role of timing mechanisms on impulsive choice. Identifying the mechanism of change is important prior to conducting translational research designed to reduce delay discounting in humans. If a different mechanism is at work than the one hypothesized, identifying the correct mechanism may suggest a more effective or expedient training regimen.

Delay Fading

One training procedure that has produced long-lasting reductions in impulsive choice, and has proven effective in human applied research is delay fading. In the first demonstration of the efficacy of this procedure, Mazur and Logue (1978) began by establishing pigeons’ preference for a large over a small food reward when both rewards were delayed. Over the course of many subsequent sessions the delay to the smaller reward was very gradually reduced (making it a slightly more attractive reward), while being careful to maintain a strong preference for the LLR. The delay to the SSR eventually was completely faded out (i.e., 0-s to food) and these pigeons continued to prefer the LLR far more than a control group not given this delay–fading training. The group difference proved robust when retested 11 months later (Logue & Mazur, 1981).

Several human studies have arranged training based on the Mazur and Logue (1978) procedures, although they have “faded in” the delay to the LLR rather than “fading out” the delay to the SSR. For example, Schweitzer and Sulzer-Azaroff (1988) reported large reductions in six preschoolers’ impulsive choices following this “fading-in” training and this effect has been replicated several times (e.g., Dixon et al., 1998; Fisher, Thompson, Hagopian, Bowman, & Krug, 2000) with several studies suggesting larger reductions in impulsivity may be realized by providing signals or distractors during the delay to the LLR (e.g., Newquist, Dozier, & Neidert, 2012; Vessells, Sy, Wilson, & Green, 2018).

That delay fading can reduce impulsive choice in pigeons and humans is moderately well established (see below for further discussion); what is not well understood is how it works. Future research should evaluate if these reductions are mediated by improvements in interval timing, reductions in delay aversion, or some other learning-based mechanism.

Delay–Exposure Training

Several years ago, our lab conducted an unpublished experiment designed to evaluate the effects of delay fading on impulsive choice in rats. Although all of the rats maintained their preference for the LLR as the delay to the SSR was gradually faded out, almost none of the rats continued to choose the LLR when the lever assignments were reversed. That is, the rats failed to track the new location of the LLR, instead continuing to press the same lever, which then produced the SSR. A follow-up study was designed to reduce this protracted side bias by switching the location of these rewards midway through the session. Under this arrangement, very few rats maintained preference for the LLR.

At the same time, we completed an experiment demonstrating that lasting reductions in rats’ impulsive choice could be achieved by providing them with “bundling training” (Stein, Smits, Johnson, Liston, & Madden, 2013). The specifics of that training are irrelevant to this discussion, save that a postexperiment analysis of the hypothesized mechanism of behavior change revealed that the efficacy of reward bundling during training did not predict posttraining reductions in impulsive choice. Instead, what correlated with reductions in impulsive choice was simply exposure to delays, i.e., rats making the fewest impulsive choices were those that had been exposed to the largest number of response–reinforcer delays during training. Our lab had previously reported this effect but it was regarded as a nuisance outcome at the time (Madden, Francisco, Brewer, & Stein, 2011).

Recognizing that simple exposure to delays to food reinforcers might also account for the delay–fading effect in pigeons and humans, Stein, Johnson et al. (2013) conducted an experiment designed to evaluate if exposure to delayed reinforcement contingencies would reduce impulsive choice in male rats. Rats randomly assigned to the delay–exposure training group completed 120 sessions in which they pressed the center lever once and then waited during a signaled 17.5-s delay (lever retracted and cue light illuminated), after which two food pellets were delivered. Rats assigned to the no-delay group received the same number of sessions (and trials) but pellets were provided immediately after the lever press. When rats subsequently chose between one pellet immediately and three pellets following a 15-s delay, delay–exposure rats made far fewer impulsive choices than did rats in the no-delay group. The effect was maintained when retested approximately 2 months later.

Replication studies revealed (a) delay-exposure training significantly reduced impulsive choice when the LLR was delayed by 30 s (Stein, Renda, Hinnenkamp, & Madden, 2015), (b) the effect was not due to increased impulsive choice in the no-delay comparison group (Renda, Rung, Hinnenkamp, Lenzini, & Madden, 2018), (c) the impulsivity-reducing effect of delay–exposure training could be obtained in a different strain of rats (Renda & Madden, 2016), (d) the training duration could be cut in half while maintaining significant reductions in impulsive choice (Renda, 2018), and (e) the effect was maintained at a 4-month follow-up (Renda & Madden, 2016; Renda, 2018). The median effect size across these studies is very large: Hedges g_s = 2.00. A shortcoming of these replications is that they have all been conducted in the same lab and only with male rats. Whether these effects could be replicated in other labs, with other species, and in females is unknown.

Delay–exposure training: Mechanisms of change

Although delay–exposure training reliably and robustly reduces impulsive choice in male rats, the behavioral mechanism(s) responsible for this effect has only begun to be explored. Building on the previously reviewed evidence that improved precision of interval timing may reduce impulsive choice (Smith et al., 2015), Rung et al. (2018) evaluated if delay–exposure training reduces impulsive choice by improving interval timing in a temporal bisection task. Although delay–exposure training significantly reduced impulsive choice, these rats were no more precise in their timing than the more impulsive comparison groups; indeed, timing precision was not correlated with impulsive choice.

A recently completed study in our lab evaluated an alternative hypothesis—that delay–exposure training reduces impulsive choice by reducing aversion to delay–signaling stimuli (Peck, Rung, & Madden, forthcoming). Following delay–exposure training (and a significant reduction in impulsive choice) rats were given the opportunity to temporarily terminate a long-delay signaling stimulus (see Brown & Flory, 1972 for a similar procedure). When these escape opportunities were provided during forced-choice trails within the impulsive-choice assessment, delay–exposure rats made fewer temporary escapes from these stimuli than comparison groups given either no-delay training or no training at all. These findings provide preliminary evidence supporting the hypothesis that delay–exposure training increases preference for the LLR by mitigating aversion to delays and/or delay–signaling stimuli.

The above finding is of interest in light of the observation that delay aversion plays a role in human impulsive choice, particularly in children diagnosed with attention deficit hyperactivity disorder (ADHD; Marco et al., 2009; Sonuga-Barke, Taylor, Sembi, & Smith, 1992). These children report experiencing a more negative affect during delays than controls (Lemiere et al., 2012) and ADHD children demonstrate an attentional bias for delay–signaling stimuli in much the same way that anxious children are biased toward threat-signaling stimuli (Sonuga-Barke, De Houwer, De Ruiter, Ajzenstzen, & Holland, 2004). Whether delay–exposure training protects against a comparable attentional bias is unknown, but extended exposure to delays ending in rewarding outcomes might be expected, either through habituation or Pavlovian processes, to reduce negative emotional responses. Evidence supporting or refuting these suppositions must await future research.

A third possible behavioral mechanism of delay–exposure training is that it improves rats’ ability to solve what artificial intelligence researchers refer to as the “assignment of credit problem” (Niv, Joel, Meilijson, & Ruppin, 2002; Sutton & Barto, 1990). That is, when a delayed reinforcer is obtained, to what does the organism attribute the food reward? Was it produced by pressing the LLR lever, by approaching the feeder during the delay, or was it provided noncontingently? Recall that delay–exposure trained rats complete many sessions in which they press a lever and, following a delay in which no further lever pressing is possible, receive a food reward. This training may improve their ability to learn contingent relations between temporally distant events. If so, this could increase preference for the LLR for the simple reason that these rats have learned to assign credit for the LLR when it occurs to having pressed the LLR lever prior to the delay (i.e., they have learned the operant contingency between response and delayed reinforcer). By contrast, control rats, having considerably less experience with delayed reinforcement contingencies, may inaccurately assign credit for the LLR, leaving the contingency on the SSR lever as the only one they have correctly learned (Killeen, 2011). Ongoing research in our lab is exploring this possibility.

Translational Prevention Research

Interventions focused on . . . strengthening the ability to delay gratification (improve self-control, reduce delay-discounting rates . . .) could help better prevent a wide range of negative behavioral health outcomes. . . Several evidence-based behavioral . . . interventions to strengthen various components of the “self-control” network could be incorporated into resiliency-building programs. (Volkow & Baler, 2015)

The resiliency-building programs referred to by Nora Volkow (director of the National Institute on Drug Abuse) and her colleague, Rubén Baler, constitute the end product of the translational research needed in the coming years. We have reviewed some of the evidence suggesting, though not definitively demonstrating, that steep delay discounting plays a role in the origin, development, and continuation of addictive behavior. The prevention research called for by Volkow and Baler (2015), if it succeeds in producing large and lasting reductions in delay discounting, will simultaneously provide the data needed to evaluate the hypothesis that discounting and addictions are causally related. That is, successful resiliency-building programs that reduce delay discounting should produce clinically significant reductions in addictive behavior, when compared with a no-training control group. If they do not, the search for the third variable underlying steep delay discounting and addictions will continue (e.g., Watts, Duncan, & Quan, 2018).

Because this research may be decades in the making, this hypothesis might be more expediently evaluated in the nonhuman laboratory. Delay–exposure training offers one procedure for producing large and lasting reductions in impulsive choice in male rats, timing training offers another. If training-produced reductions in impulsive choice are subsequently accompanied by, for example, reduced prevalence in acquiring low-dose cocaine self-administration, then this would provide one piece of experimental evidence supporting the hypothesis that delay discounting plays a causal role in one form of addictive behavior.

Translating Interventions with Lasting Effect

The prevention research advocated by Volkow and Baler (2015) might best be conducted in schools with young at-risk children. To our knowledge, of the experimental manipulations reviewed above, only delay fading has been evaluated and proven effective in young children (Schweitzer & Sulzer-Azaroff, 1988) with several systematic replications in children diagnosed with autism or other intellectual or developmental disabilities (Dixon et al., 1998; Dixon & Cummings, 2001; Gokey, Wilder, Welch, Collier, & Mathisen, 2013; Hanley, Jin, Vanselow, & Hanratty, 2014; Newquist et al., 2012; Passage, Tincani, & Hantula, 2012; Vessells et al., 2018).

If delay fading, or any of the other manipulations producing lasting reductions in delay discounting are to affect human decision-making, at least three challenges must be addressed. First, given the replication crisis within psychology (Klein et al., 2018; Open Science Collaboration, 2015) these impulsivity-reducing effects must be replicated in preregistered experiments with sample sizes that will satisfy the diversity of researchers in the psychological and social sciences. Second, the post-training duration of these effects need to be evaluated. Laboratory research with nonhumans has evaluated the duration of these effects (e.g., Bailey et al., 2018; Logue & Mazur, 1981; Renda & Madden, 2016) but long-term outcomes are infrequently assessed in humans. Third, the translational studies that will form the research basis for effective prevention work will need to evaluate the generalizability of the effect to other settings and tasks in which impulsive choices are possible (Dunkel-Jackson, Dixon, & Szekely, 2016; Luczynski, Hanley, & Rodriguez, 2014). Failures of generalization are common and should be planned for in the design of any widely used intervention (Lovaas, Koegel, Simmons, & Long, 1973; Stokes & Baer, 1977).

A prevention program that has begun to address some of these challenges is the delay-tolerance training component of the Preschool Life Skills program (Hanley, Heal, Tiger, & Ingvarsson, 2007). Preschool children in the Hanley et al. study were given several prompted (e.g., “please wait”) opportunities to wait for a desired outcome during normal activities in the preschool classroom. After establishing that children almost always failed to wait quietly for 30 s when prompted to do so, they were taught, through instructions and modeling, to repeat out loud (and eventually quietly to themselves), a phrase designed to reiterate the waiting strategy and the consequence of waiting, “When I wait quietly, I get what I want.” Successful waiting behavior was descriptively praised (e.g., “I like how you waited so quietly until the end”) whereas failures led to additional instructions and opportunities to demonstrate the skill. After four days in which delay-tolerance training (~13 trials per child) was incorporated into the normal schedule, successful waiting occurred on more than 80% of test trials, and problem behavior during the delay decreased by 74%. Similar, increases in waiting have been reported in replication studies using between-groups statistical designs (Luczynski & Hanley, 2013) and more modest improvements in waiting have been demonstrated in Head Start preschools for low-income at-risk children (Hanley, Fahmie, & Heal, 2014; for review see Luczynski & Fahmie, 2017). Future evaluations of this program should examine the long-term efficacy of training and if improvements in waiting are correlated with changes in interval timing or reductions in the aversiveness of delay-signaling stimuli.

Translating Interventions with Momentary Effects

Translational research in the momentary manipulations category is well underway but, to our knowledge, has yet to be evaluated in young children (although see Nisan [1974] for a manipulation that bears formal similarity to EFT). Daniel et al. (2015) demonstrated that EFT could reduce delay discounting and energy intake in 9–14 year-old children in the lab and a small pilot study suggested that web-based delivery of EFT could reduce children’s (11 years old, on average) energy intake at home (Sze et al., 2015). Unknown is the lower range of ages at which EFT can have a beneficial impact on delay discounting and health-related behavior. Also unknown is if nature exposure can reduce rates of delay discounting in young children, or if these children can learn to reframe choice alternatives in a way that promotes self-control choice. As above, these studies should be preregistered, adequately powered, and should assess the duration and generalizability of these effects. Given the importance of demonstrating self-control when real delays and rewards are at stake, this research should expand beyond the typical discounting tasks that arrange hypothetical rewards and delays. A comprehensive approach to developing a prevention program might usefully integrate one or more of these momentary manipulations with interventions that have demonstrated lasting effects in the nonhuman lab.

Summary and Conclusions

In trying to solve the terrifying problems that face the world in 2019 (e.g., global climate change, an epidemic of obesity, an opioid crisis), we recognize the role of human behavior in general, and impulsive choice in particular. Reducing the extent to which individuals discount the value of delayed outcomes holds promise in influencing the many small choices that contribute to these crises. The technology of behavior advocated by Skinner (1971), as applied to impulsive choice, remains in its infancy. Much of the early work in delay discounting was correlational but that is changing. Today there is less interest in replicating the finding that steep delay discounting is correlated with addictions, and there is greater interest in longitudinal research evaluating discounting as a precursor to addictions, and experimental work designed to reduce delay discounting and impulsive choice. This article has provided an incomplete list of such techniques and has loosely mapped out a line of translational research aimed at developing a prevention program that could reduce addictive behaviors. Whether therapeutic manipulations with momentary effects can be meaningfully incorporated into existing addictions therapies must await empirical evaluation. It should be clear from this review that these techniques fall short of a “technology of behavior.” There remains much to be learned by those who study basic behavioral processes, and we urge such researchers to keep one eye on the translational potential of their discoveries. Now more than ever, it is important to play to our strengths—by investing our time, effort, and resources in experimental behavioral science and translational research.

Funding

The last author’s time was supported by NIH grants R21 DA042174-01 and R03 DA044927-01.