Experimental Manipulations of Delay Discounting & Related Processes: An Introduction to the Special Issue

Behavioral economics has and continues to play an important role in the analysis of behavior. This current issue of JEAB, entitled Experimental Manipulations of Delay Discounting & Related Processes, constitutes the third special issue addressing behavioral economics that has been published by this journal. The first was in 1995 and edited by Bickel, Green, and Vuchinich, while the second was published in 2013 and was edited by Hackenberg. Where the previous special issues had broad scopes (e.g., demand curves, budget constraints, expansion paths, change-detection learning) the present issue is focused primarily on engendering change in delay discounting and related processes.

We see two reasons to narrow the focus for the current special issue. First, and as illustrated in Figure 1, over the last decade or so there has been a substantial increase in the number of studies that have examined the phenomena of delay discounting. One reason for this increased research activity is that steeply discounting the value of delayed outcomes is robustly correlated with problem substance use (MacKillop et al., 2011; Yi, Mitchell, & Bickel, 2010). That is, if the value of a future consequence is steeply devalued (e.g., vocational success after earning a college degree) such that it cannot compete with the undiscounted value of a smaller–sooner reward (e.g., the high produced by consuming drugs) then drug taking may ensue. Consistent with this account, longitudinal data suggest that steeper delay discounting is predictive of subsequent drug use in adolescence (e.g., Audrain-McGovern et al., 2009; Fernie et al., 2013; Kim-Spoon, McCullough, Bickel, Farley, & Longo, 2014; King, Fleming, Monahan, & Catalano, 2011), and a similar sequential relation is observed in laboratory studies of rodents' delay discounting and subsequent cocaine self-administration (e.g., Anker, Perry, Gliddon, & Carroll, 2009; Koffarnus & Woods, 2013; Perry, Larson, German, Madden, & Carroll, 2005; Perry, Nelson, & Carroll, 2008). Strengthening the apparent relation between delay discounting and drug use are reports that steeper delay discounting is correlated with the severity of drug use (e.g., Amlung & MacKillop, 2014a), the number of substances concurrently used (e.g., García-Rodriguez, Secades-Villa, Weidberg, & Yoon 2013), and poor outcomes in drug treatment (e.g., Sheffer et al., 2012, 2014).

Fig. 1.

Number of delay discounting articles published per year. Source: PubMed.

In recent years, discounting of delayed reinforcers has been suggested as a transdisease process that plays a role in various forms of addiction, obesity, problem gambling, risky sexual behavioral, and a variety of other indicators of poor health (Bickel, Jarmolowicz, Mueller, Koffarnus, & Gatachalian, 2012; Bickel, Koffarnus, Moody &Wilson, 2014; Story, Vlaev, Seymour, Darzi, & Dolan, 2014). If steep delay discounting is a behavioral marker of addiction and health behavior, then this relation would appear to be more than correlational. Hence, the second reason for narrowing the focus of the current JEAB special issue—if a technology of behavior is developed for experimentally manipulating delay discounting, then not only might it impact delay discounting, but it may reduce the impulsive choices that comprise behavioral patterns of addiction and poor health-decision-making (Koffarnus, Jarmolowicz, Mueller, & Bickel, 2013). Such research could profitably be conducted in nonhuman subjects and in basic, preclinical, and clinical human research settings (e.g., Hanley, Heal, Tiger, & Ingvarsson, 2007; Mazur & Logue, 1978; Schweitzer & Sulzer-Azaroff, 1988)

The special issue's focus on seeking variables that can be used to change delay discounting may seem to contradict convincing arguments that delay discounting is a trait-like phenomenon. This contradiction, however, is more apparent than real. Delay discounting is subject to both robust state (environmental) as well as trait effects (Odum, 2011). On one hand, delay discounting is stable over time, predictive of subsequent behavior, correlated across different reinforcers and magnitudes, and is associated with certain genetic alleles (Baker, Johnson, & Bickel 2003; Bickel, Landes et al., 2011; Friedel, DeHart, Madden, & Odum, 2014; Kirby, 2009; MacKillop, 2013). On the other hand, ample evidence suggests that choices made in a discounting context can be positively influenced by framing effects (e.g., Radu, Yi, Bickel, Gross, & McClure, 2011) and a better understanding of these effects, and their limits, could be used to aid humans in making important decisions; the Save More Tomorrow program (Thaler & Benartzi, 2004) comes to mind as an example of using the technology of delay discounting to positively influence a single choice made by an employee starting a new job, but a choice that has a lasting effect on his or her ability to save for retirement. This program may not produce a trait-like change in delay discounting, but it encourages a momentary future orientation that has lasting effects on health and wellbeing.

On the other hand still further, other experimental manipulations of delay discounting are focused on producing lasting trait-like effects that will generalize to a wide variety of intertemporal choices. For example, taking a lead from the neuroscience of delay discounting and dual-systems theory, Bickel, Yi et al. (2011) provided treatment-seeking stimulant users with working memory training and, relative to a sham-trained control group, decreased the degree to which they discounted delayed monetary rewards.

The ostensible contradiction between discounting as a trait and experimental manipulations of delay discounting may result from allegiance to old conceptual categories. Consider how the nature versus nurture debate has been dissolved by epigenetics. Perhaps we should consider the stability of delay discounting as similar to operant behavior interacting with an ongoing schedule of reinforcement. As this journal has amply demonstrated, subjects may bring individual differences to the chamber, and these may alter the parameters necessary to produce behavior change, but altering the schedule engenders change, and training for generalization often yields the desired effect. When these effects prove replicable across individuals and across labs, they provide the building blocks of a translational behavioral science that can contribute positively to public health.

To that end, this issue of JEAB includes five manuscripts that directly build on previously published work, extending and expanding the current understanding of experimental variables that can influence delay discounting. One of these manuscripts is by DeHart and Odum, which builds on a framing effect noted by Read, Frederick, Orsel, & Rahman (2005). The latter researchers reported a “date/delay effect,” in which lower rates of discounting were observed when the delay to the larger–later outcome was identified with a date description (e.g., January 1) rather than delay from the present (e.g., 1 month from now). DeHart and Odum replicated this effect and report on a novel framing effect in which delayed rewards are discounted more steeply when the delay is described as the number of days to reward fruition; one might call this a days/delay effect. Further, the DeHart and Odum study addresses a procedural limitation of the Read et al. study, allowing for a comparison of model fits to the delay-discounting data. The results support the hyperboloid model across the variously framed discounting tasks. The authors speculate on the possible mechanisms of these framing effects, an important step toward deducing the principles of framing which might be used for translational gain.

The Lahav, Shavit, and Benzion article also builds on past findings, this time the extensive literature examining delay discounting from a developmental perspective (the role of age, e.g., Green et al., 1994). In addition, Lahav et al. consider the context in which intertemporal decisions are made. On the former, the study compared discounting by middle and high school students; on the latter, the study examined discounting when decisions were framed as immediate/delayed rewards versus earned wages. Consistent with the established literature, younger students showed higher rates of delay discounting in a standard scenario (sans the wage context). No age difference was observed in the wage scenario, though greater delay discounting was observed across groups in the wage scenario relative to the standard reward scenario. From a theoretical perspective, this may align with previous research showing greater discounting when an option appears to delay consumption (Weber et al., 2007). In contrast, the results may be superficially inconsistent with research showing less discounting for money owed relative to money won (Weatherly, Terrell, & Derenne, 2010).

The article by Stein, Renda, Hinnenkamp, and Madden builds on previous work by the same group that found that rats with previous exposure to delayed rewards subsequently are willing to wait for a larger, delayed reward in an impulsive-choice task (Stein et al., 2013). These results were replicated, with rats pre-exposed to 17.5s delayed access to food more likely to wait for a larger–later amount of food than rats pre-exposed to immediate food access. Perhaps most importantly, the training effect was maintained at a 5-week follow-up suggesting a traitlike change in delay discounting. The novel inclusion of an open-field test indicated that differential stress exposure is an unlikely explanation for this effect. This body of research now provides early but compelling evidence that training on reinforcement schedules with delayed rewards can result in subsequent self-control. The fact that posttraining differences in impulsive choice did not result in differences in alcohol consumption indicates that we should practice caution when assuming that the correlational relations frequently observed between delay discounting and drug use is causal.

The article by Renda, Stein, and Madden takes a different tack to replication, seeking to examine cross-species generality of the effect of working memory training on delay discounting that was reported by Bickel, Yi, et al. (2011) in humans. Renda and colleagues exposed two groups of rats, matched on pretraining delay discounting, to 140 sessions of working memory or sham training. In contrast to the previous research with humans, no changes were found in delay discounting resulting from working memory training, though large improvements in working memory were observed. Clearly, there are notable differences in the methods between the original study in humans and this investigation using rodents. The nature of the working-memory tasks employed in training, for example, were different, as was the delay discounting task, one involving verbally described prospective delays to monetary rewards, the other nonverbally experienced delays to food. Nonetheless, the lack of interspecies generality suggests the interrelationships between working memory and delay discounting are complex and direct and systematic replication is called for.

A second study revealing results that are inconsistent with previous findings is the manuscript by Roewer, Wiehler, and Peters reporting no effect of nicotine deprivation on monetary delay discounting. The absence of an effect of withdrawal is in contrast to several previous studies (Field, Santarcangelo, Sumnall, Goudie, & Cole, 2006; Mitchell, 2004; Yi & Landes, 2012), although some inconsistencies are present in those studies too. Notably, there are substantial differences in the delay discounting task parameters (format and number of items), the commodities under consideration (money and cigarettes), the magnitude of the rewards, and whether the tasks are consequated (providing an actual outcome or the opportunity for an actual outcome) or not. The Roewer et al. study uses heavy smokers, 24 hr deprivation periods, and elegant Bayesian analyses that are among the most compatible with the tradition of JEAB, with a focus on the behavior of individuals. The lack of a deprivation effect on delay discounting suggests, most narrowly, that either the previous findings were false positives or, more broadly and more likely in our opinion, that influences of withdrawal may be most potent under certain methodological conditions. Like Renda et al., this study suggests that more work is needed to clarify robustness of the effects of nicotine withdrawal on delay discounting and identify which conditions (e.g., real vs. hypothetical rewards and delays) bring these relationships into sharpest relief.

In contrast to the preceding studies that focus on delay discounting, the study reported by Owens, Ray, and MacKillop extends the existing literature using behavioral economics to improve the measurement of acute motivation for alcohol. Traditionally, acute motivation in laboratory studies is assessed using subjective desire/craving, which may be subject to a number of measurement biases (MacKillop & Monti, 2007). The alternative strategy pursued in this line of work is using behavioral economic purchase tasks to characterize the state-level demand for the drug. Previously, MacKillop et al. (2010) found that the presence of alcohol cues dramatically increased several indices of alcohol demand, and tobacco cues and experimentally induced withdrawal do the same (Acker & MacKillop, 2013; MacKillop et al., 2012). Most recently, Amlung and MacKillop (2014b) found that stress increased the relative reinforcing efficacy of alcohol in heavy drinkers using the Trier Social Tress Test (Kudielka & Wüst, 2010). Using a different stress-induction task, the present manuscript replicated the previous findings of increased alcohol demand as a function of increased stress. By including additional measures, the study further found that the increase in alcohol demand was moderated by income, as well as the CRH-BP genotype. These moderation analyses expand our understanding of the individual variation observed when examining the effects of stress on drug motivation.

We consider these above mentioned studies to be important extensions of existing lines of research in the service of a truly programmatic approach to understanding the nature of delay discounting, be they replications or failures to replicate. The need for confidenceinpublished empirical research is particularly important currently (Ioannidis, 2005) and multiple research groups assiduously pursuing these questions systematically is essential to arrive at a true and valid understanding of these phenomena.

In contrast to the aforementioned, several studies in the special issue do not take specific previous studies as their point of departure, extending the broader line of inquiry about dynamic effects on behavioral economic decision-making. The article by Johnson et al. takes up the question of species-level differences in delay discounting by examining the influence of pursuing alternative reinforcement during a discounting task (typical of human paradigms) and entirely restricting activity during the delay (typical of animal paradigms). They found that levels of impulsive discounting scaled to the availability of alternative reinforcers (or the opportunity costs of waiting), suggesting that the differences between humans and nonhumans are not exclusively based on biology.

The manuscript by Rung and Young returns to the question of reducing impulsive choice but in a very novel way; here, a video game-based fading procedure was investigated as a strategy for increasing willingness to wait in an accumulating-rewards task. Over the course of the training condition the time across which the participant had to withhold a response (pulling the trigger to shoot a virtual enemy) to maximize the reward (doing damage to the enemy) was gradually increased. Of interest was the reward dimension that increased with progressively longer wait times. In one group the magnitude of the reward increased, whereas for the other group the probability of a reward increased (expected value was held constant across groups). The training procedures increased wait times overall, but less so when reward magnitudes escalated than when probabilities increased. This paper is of considerable interest as the field continues to explore ways in which to experimentally reduce impulsivity. Not only may reward probability prove more potent in improving delay-of-gratification behavior but the video-game environment may prove a particularly palatable way to provide interactive learning designed to improve this component of impulsivity. Although further work is clearly necessary, we believe this strategy has considerable applicability and promise.

The Dennhardt et al. article further extends these questions into a clinical context. In this study, 97 young adult heavy drinkers underwent an evidence-based brief intervention, combined with a supplemental behavioral economic component or an educational control component. Prior to and following these interventions, participants were assessed for delay discounting preferences, alcohol demand, and proportionate alcohol-related reinforcement. Very interestingly, the brief intervention (irrespective of additional components) significantly reduced alcohol demand acutely. Specifically, three indices of demand were affected by the intervention, intensity (consumption at minimal price), Omax (maximum expenditure), and elasticity (proportionate price sensitivity). It is worth noting that this can be broadly seen as reciprocal findings to those reported by Owens et al. In that study, stress drove alcohol demand up, and in Dennhardt et al., the intervention drove it down, illustrating the increases and decreases in motivation that concurrently operate on an individual.

In addition to this finding, Dennhardt et al. also found that both the baseline value and the changes in alcohol demand were predictive of drinking outcomes, most commonly predicting reductions in binge drinking and alcohol-related problems. For the proportionate reinforcement index, changes postintervention were predictive of alcohol-related problems at 6 months, although not the other outcomes. Monetary delay discounting was largely irrelevant in this study, as it was not affected by the intervention, correlated with drinking behavior cross-sectionally, or predictive of outcomes. In what is probably the most applied investigation in this special issue, Dennhardt et al. illustrate how changes in behavioral economic processes (although not discounting) over larger time scales, and as consequences of an intervention, appear to be important mechanisms of behavior change. From a theoretical perspective, this prediction can clearly be made but much of the work in this area remains experimental and/or proof-of-concept, making this clinical study of particular importance, in our opinion.

The next four papers illustrate that the manner in which choices are arranged can have large impacts on the degree of delay discounting. Intertemporal choices, which involve tradeoffs between the amount of a reward and the delay to the reward, often involve differences beyond simply delay and amount. For example, a choice between dessert right now (a smaller–sooner reward) and improved health in the future (a larger–later reward) also involves differences in effort. Fortes, Basconcelos, and Machado explore the effect of effort on choice in an adjusting-delay preparation. They report with pigeons that, when delay is not confounded with effort, requiring more effort (more pecks) increases preference for the larger-later food reward (i.e., a reduction is delay discounting). In other words, adding effort to the impulsive option increased self-control. By extension, people wishing to achieve or maintain a healthy body weight could increase the effort associated with the unhealthy option.

Another difference in self-control situations is how often a choice can be made. In some situations, only one opportunity to choose presents itself in a given time frame, whereas in others, choices may be made repeatedly in that same time frame. In laboratory models of self-control choice, organisms may be able to maximize, that is, to increase overall reward rate, by choosing the smaller-sooner outcome repeatedly, when trial spacing is not controlled. In this case, it would be a misnomer to call such choices “impulsive”. Given the complexities in the previous literature in both procedure and findings, Smethells and Reilly examined the effects of the time between each choice trial, the intertrial interval, on the proportion of choices for the delayed and immediate food rewards. For both the pigeon and rat subjects, with trial spacing controlled, more time in between choice trials resulted in more choices of the larger-later outcome. These experiments highlight an important possible influence on decision making in self-control situations. How often choices can be made, in addition to the delays to the rewards, can influence the decision.

Test–retest reliability is an important aspect of delay discounting for both theoretical and practical reasons. Alternate-form test–retest reliability examines performance on two different versions of a task purported to measure the same construct, whereas same-form test–retest reliability examines performance on the same task across time. Few studies have addressed this topic in nonhumans prior to the Peterson, Hill, and Kirkpatrick paper, which appears in this special issue. They examined rats' choices for delayed and immediate food rewards using three procedures. In the two systematic procedures, the delay to the larger–later reward changed in a predictable fashion, either after several sessions of training or within experimental sessions. In the adjusting procedure, the delay changed after each block of trials within an experimental session, based on the larger–later choices in that block. All of the procedures produced good test–retest reliability, showing that performance at one time is predictive of performance at another time. Only the two systematic procedures, however, showed alternate-form test–retest reliability, suggesting that they are measuring the same construct. Measures of temporal control within choice trials were stronger with the systematic procedures. Peterson and colleagues conclude that the systematic procedures provide better estimates of impulsive behavior in rats. These data also provide evidence in nonhumans that impulsive behavior in one task predicts impulsive behavior in another task, as is already well documented with humans.

Self-control in human participants is often investigated using delay discounting and delayed gratification procedures. In both procedures, choice is betweena smaller–sooner and a larger–later outcome. In delay discounting procedures, once a choice is made, the participant is committed to the chosen reward. In delayed gratification procedures, however, the participant must remain steadfast in the choice of the larger–later reward in the face of the continued immediate availability of the smaller reward. Young and McCoy investigated the degree of larger–later choice in these two procedures using a first-person shooter video game. Participants chose the larger–later reward(a delayed but more powerful blast of their weapon) more so in the delay-discounting version of the game than in the delayed-gratification version that allowed defections to the smaller–sooner reward. Choices were less sensitive to changing incentive structure, however, under the delay-discounting version, which did not allow defections.

The next two papers are theoretically inspired experimental papers. That is, the authors attempt to answer a theoretical question with empirical data. Paglieri and colleagues provide a conceptual reexamination of two types of potential motivational influences in delay discounting procedures. They first review evidence thatinsome experiments with prepotent salient stimuli (e.g., food for nonhuman primates), procedural arrangements could make an impulsive choice look like choice of a larger–later reward. This potential misinterpretation arises because the larger reward is presented concurrently with the smaller reward, and the participant must point to the preferred reward. In this case, the choice of the larger reward could represent failed inhibition of choice for more over less (‘go for more’), which is not an option, rather than a true choice of a larger but later reward over a smaller but sooner reward.

The second point of Paglieri et al.'s paper informs the human literature on differences in the degree of discounting by delay for different reward types. In general, consumable rewards (e.g., food) are discounted more steeply than money. Paglieri and colleagues present new empirical evidence that the motivation to maximize these rewards differs in the absence of delay. Thus, they reveal a new potential underlying principle: Rewards for which we have weak motivation may be discounted more steeply than rewards for which we have strong motivation. This principle would unify the literature on reward type and delay discounting.

The Franck et al. paper complements the Paglieri paper by highlighting the importance of examining underlying assumptions about what laboratory procedures measure and how to best describe the process of delay discounting. Possibly ubiquitously until now, researchers have assumed that there is one process underlying delay discounting. Concurrently we have wondered: Is discounting best described by an exponential model, in which value is decreased by the same proportion per unit time, or a hyperbolic one, in which proportionally more value is lost over shorter delays? Perhaps the hyperbola would better fit delay-discounting data with an added exponent? Should the exponent modify the entire denominator, or just the delay term? Franck and colleagues suggest an entirely different approach. Does one model provide the best fit for all, or could it be the case that for one person, one model might best describe the discounting process, but for another person, another model might best describe the discounting process? They apply Bayesian model selection to identify the model of best fit, out of five widely used models, for each individual. Franck et al. also suggest a common metric to assess the degree of discounting across different models with different numbers of free parameters. Finally, they simulate data from discounting experiments using the known data evaluated in the first section. When data were generated using multiple models, the multiple model approach correctly identified the model used to generate the data. When data were generated using a single model, the multiple model approach fared no worse than the single model approach. These results recommend exploration of multiple models for individual data.

The final three papers are review papers that explore a conceptual viewpoint with respect to delay discounting. These papers are important for synthesizing existing data as well as proposing novel extensions of existing concepts. Ashe and colleagues consider the success of two approaches in treating drug addiction, and how these approaches may interact with the degree of discounting a person experiences. On one hand, mindfulness-based techniques teach people to attend without judgment to their moment-to-moment experience. On the other hand, distraction-based techniques teach people to engage in some activity (either covert or overt) to turn their attention away from drug-related situations and feelings. These two seemingly opposing strategies both provide successful treatment for addiction to a substantial proportion of people. Ashe and colleagues suggest that a person's degree of delay discounting may provide the key for understanding and predicting which treatment is effective for which people under which circumstances.

Current models of delay discounting have in common that the value of the outcome is multiplied by a discount factor as a function of the delay to its receipt. Killeen proposes what is on the surface a very different model. The additive utility model suggests that delay discounting arises when the utility of an outcome is added to the disutility of waiting. This model makes many ofthe same predictions as more familiar models. One area where it differs is that of discount rate. Differences in the degree of discounting (e.g., with different magnitudes or different types of commodities) do not arise from different discount rates. Instead, according to the additive model, they arise from differences in time horizon and/or marginal utility. This intriguing model will surely stimulate empirical work to examine its utility.

The final paper in the special issue uses a delay-discounting framework to conceptualize a puzzling and detrimental behavior, problematic gambling. Rachlin and colleagues note that gambling can be thought of in terms of strings of losses ending with a win of some magnitude. If the value of the string is computed only when the win occurs, and hyperbolic discounting underlies this string-valuation process, then gambling outcomes can have a higher discounted value than nongambling rewards. The authors extend this analysis to suggest a therapeutic technique that may moderate problem gambling.

As the delay discounting literature matures beyond its infancy of identifying differences across groups such as problem gamblers and substance-dependent individuals, we begin to gain a greater understanding of the causal role that delay discounting plays, or does not play, in health-impacting decision-making. The intertemporal choices arranged in discounting experiments intersect the competing push and pull of reward amount and delay. Such environmental factors put our daily decision-making at an intersection of conflict, a point at which a single impulsive choice can impact our health and the health of others (e.g., unprotected sex). Understanding the bio-behavioral variables that influence the degree to which delays are tolerated, or that help to reveal the contingency between present behavior and future consequences is critical if we are to address issues impacting global health and the environment. Can framing effects be employed to encourage investment in sustainable energy? If an accumulating probability of reward impacts delay-of-gratification better than accumulating reward amounts, can this be parlayed into a technology of behavior (Skinner, 1971) that facilitates the discrimination of contingencies involving delayed outcomes? As these findings contribute to the ever- growing and maturing delay-discounting literature, it is our hope that they will inspire new basic research to address the many knowledge gaps that remain and will inspire translation of existing knowledge to improve the human condition and the environment in which we coexist with other organisms.