Excessive discounting of delayed reinforcers as a trans-disease process

Abstract

Delay discounting, or the process by which reinforcers lose value with delay to their receipt, has been identified as a trans-disease process underlying addiction, other disorders, and maladaptive health behaviors. Delay discounting has been identified as an endophenotype for multiple psychiatric disorders including substance use disorder, ADHD, and major depressive disorder, with this endophenotype being linked to deficits in dopaminergic and serotonergic neurotransmission. In addition, neuroanatomical and neurophysiological deficits in areas of the executive and impulsive systems have been associated with both steeper discounting and substance use disorders. Delay discounting constitutes a novel target for interventions to change health behaviors. A new theory, termed reinforcer pathology, has been developed uniting these findings and setting the stage for future research.

Delay discounting (DD), one of the most extensively studied behavioral economic measures, describes the decrease in a reward’s value as a function of the delay to its receipt (see Box 1). All people discount delayed rewards to some extent, depending on the time frames examined. However, individuals with substance use disorder have significantly higher rates of DD [1**,2]. In one study, the delay at which $1,000 had lost 50% of its value (e.g., ED50) for non-alcohol abusing controls was 166 months compared to just 23 months for individuals actively abusing alcohol [3]. DD has been suggested to serve as a behavioral marker of addiction at all stages of the process and has been predictive of treatment outcomes [2,4,5]. For example, among cigarette smokers, abstinence decreased by approximately 40% per one standard deviation increase in DD [4]. Similarly, among alcohol-dependent smokers, risk of relapsing to smoking increased by 40–50% per one standard score increase on DD [6]. A prior review of DD [7] suggested that excessive discounting of delayed reinforcers is a trans-disease process not only in substance use disorder, but also in other disorders (e.g., obesity, pathological gambling, ADHD and schizophrenia) and maladaptive health behaviors (e.g., risky sexual behaviors). Importantly, the concept of trans-disease processes can allow for the insights from one disorder to be applied to another[8]. This short review will update the prior review by summarizing new findings including: the expansion of DD as a within-person determinant of multiple maladaptive health behaviors; the genetic and neural mechanisms undergirding DD; and new potential interventions and treatments based on influencing DD rates.

Box 1. Measuring Delay Discounting.

Delay discounting procedures present choices between smaller, sooner and larger, later reinforcers of varying magnitudes and at varying delays (e.g., $50 now versus $100 in 3 months). DD is generally assumed to be hyperbolic, with the subjective value of a reinforcer declining rapidly at short delays before reaching a relative plateau [9]. Adjusting-amount DD procedures [e.g., 10] present choices between immediate and delayed reinforcers (often monetary, although sometimes other commodities such as health outcomes or drugs [11,12]) to determine points of indifference between these options at each delay. As a result, these methods require a substantial number of trials to obtain indifference points across a number of delays. To more rapidly assess DD, an adjusting-delay DD procedure was developed requiring only five trials and less than a minute to obtain DD rates [see 13 for data and justification of these procedures]. However, the adjusting-delay task is designed to assess DD of a single commodity over varying delays. Most real-world choices are between different reinforcers available following different delays (e.g., fast food now or weight loss later). To address these choices, a cross-commodity discounting procedure was developed to examine the interaction among intertemporal choice and reinforcer type (initially, cocaine or money) [14]. Since the introduction of the first cross-commodity study, several other reinforcer combinations (e.g., sex, fast food, and weight loss) have been assessed using this procedure [15*,16*]. These cross-commodity procedures allow for dissociation of the relative discounting and utility functions for each reinforcer being examined.

Expansion of the role of delay discounting in health behaviors

At the time of the prior review, the only health behaviors that had been specifically identified as correlated with DD were risky sexual behavior and needle sharing [7]. Recent work has expanded the range of health behaviors related to DD [17] to include type II diabetes [18], prediabetes, fast food consumption [19], texting while driving [20], poor medication adherence, seatbelt nonadherence, sunscreen non adherence, hypertension, visiting the dentist, and receiving the influenza vaccine [17].

The above research explored correlations between DD rates and engagement in health behaviors across individuals, but more recently, DD has been shown to predict engagement in multiple health behaviors within individuals. For example, Snider et al., [21]** asked cigarette smokers a wide range of questions regarding various health behaviors including financial, exercise, and eating behaviors. These questions were then grouped together into meaningful latent factors using structural equation modeling. Delay discounting significantly predicted these factors indicating that even within cigarette smokers, DD predicted other maladaptive behaviors. A recent follow-up has found similar results in obese individuals and healthy controls.

In a similar study, Sheffer et al. [22]** found that DD predicted a variety of health behaviors in cancer survivors. Delay discounting predicted weekly alcohol consumption, current cigarette smoking and other tobacco use, tanning booth use, and yearly visits to a primary care physician. These findings are powerful, given that these behaviors are particularly harmful for those with a history of cancer (and can play an etiological role in future cancer development).

Genetic markers of delay discounting

Decades of research has demonstrated similarity of neural mechanisms and high heritability of substance use disorders across drug classes [23]. Despite this, early investigations of candidate genes and genome wide associations have been unable to account for this high level of observed heritability. Given the diagnostic variability in substance use and other psychiatric disorders, investigative efforts have been redirected towards elucidating endophenotypes: heritable, neurobiological/behavioral mechanisms that undergird multiple disorders and provide a more precise biomarker for diagnosis [24].

A growing body of evidence supports DD as an endophenotype. Studies in humans and animals have demonstrated levels of heritability similar to substance use disorders (~50%), with elevated rates of DD seen in non-affected relatives of addicted individuals. These studies have shown that genetic polymorphisms association with altered dopaminergic signaling may underlie this endophenotype. For a review see [25]. In a seminal study, Sanchez-Roiges and colleagues performed a genome-wide association study of DD in over 23,000 individuals of European ancestry [26]*. Delay discounting was significantly associated with a polymorphism in a neuronal membrane protein (GPM6B), previously implicated in deficient serotonin transmission. Additionally, 12% of the variance in DD was accounted for by genotype and this genetic signature showed a positive correlation with ADHD, major depressive disorder, lifetime smoking, and BMI. Taken together, these findings show strong support for DD as an endophenotype for substance use disorders as well as a trans-disease process underlying multiple psychiatric disorders and deleterious behavioral states.

Neural mechanisms underlying delay discounting and addiction

The competing neurobehavioral decisions systems (CNDS) theory posits that healthy decision-making processes rely on a balance between two distinct neural systems termed the impulsive and executive systems [7,27,28]. The impulsive system, embodied in limbic and paralimbic brain structures (e.g., ventral striatum, hippocampus, cingulate cortex), regulates reward and motivation and is involved in the valuation of and decision for immediate reinforcers. The executive system, embodied in prefrontal cortical regions, regulates executive functions (e.g., attention, working memory, cognitive flexibility, and inhibitory control) and is involved in our ability to value future outcomes and delay reinforcement. As previously discussed, patients with substance use disorders demonstrate maladaptive decision-making processes and tend to choose immediate over delayed reinforcers. The CNDS theory posits that this behavioral imbalance stems from a regulatory imbalance between a hyperactive impulsive and a hypoactive executive system. Recent neurophysiological, neuroimaging, and neuromodulatory studies have lent support to this idea.

Electroencephalography (EEG) studies have revealed neurophysiological correlates of DD in populations with and without substance use disorders. In non-clinical populations, steeper DD is associated with lower levels of baseline brain activity in the prefrontal cortex, especially in the high beta frequency range (21.5–30 Hz) [29]. Others have found that non-deprived people with substance use disorders show decreased resting frontal, central, or whole brain delta (1.5–3.5 Hz), theta (4–7.5 Hz), and alpha (8–12.5 Hz) activity, with the lowest levels of activity predicting the highest levels of addictive behavior [30,31]. During DD task performance, the reward positivity signal (a positive-going event-related potential (ERP) that occurs approximately 250–350 ms post-stimulus presentation and is sensitive to gain instead of loss outcomes) is elicited following immediate but not delayed reinforcers, with a larger ERP amplitude seen in steeper discounters [32]. Together, these results indicate that decreased neural synchrony and efficiency of the cortical networks [33]as well as altered reward processing signaling, especially in frontal brain regions, may serve as a neurological indication of altered decision-making properties and propensity for substance use disorders [34].

Structural and functional magnetic resonance imaging (MRI) studies have also helped us understand the neuroanatomical correlates of DD and substance use disorders. Using MRI, researchers have shown that steeper DD is associated with lower cortical [35]and higher subcortical [36]gray matter volume, with similar neuroanatomical findings shown in addicted populations [37]. During resting state fMRI, a large-scale brain network known as the default mode network (DMN) is activated, which is involved in self-referential thinking as well as the ability to remember the past and think about the future. New research has revealed that a greater disorder of activity in the DMN is related to steeper DD [38], and DMN functional connectivity is dysregulated in substance use disorders [39,40]. fMRI studies have revealed that during the DD task, healthy individuals show heightened activation and functional connection in areas of the executive system during choices associated with delayed reinforcers [41,42]. However, individuals with substance use disorders show decreased activity in areas of the executive system and heightened activity in areas of the impulsive system [43,44]. Collectively, these studies suggest that individuals with high levels of DD and/or substance use disorders may show neuroanatomical deficits, as well as differences in functional activation and connectivity at rest and during reward processing in both executive and impulsive systems.

Novel interventions to target delay discounting

Several interventions are now targeting DD as a means to positively affect associated disorders, and we will briefly address the few most promising (neuromodulation and episodic future thinking). However, note that other research has investigated working memory training, mindfulness, transcranial direct current stimulation (TCDS) and others as potential interventions [45,46].

Neuromodulatory methods may shed unique insight into both the neural substrates underlying discounting itself and the role of DD in health and disorders. Specifically, transcranial magnetic stimulation (TMS) is a non-invasive form of neuromodulation, which may be used to either potentiate or decrease activity in cortical regions using electromagnetic fields. By repeated applications of TMS (a procedure known as rTMS), Cho and colleagues [47] were able to experimentally implicate the medial prefrontal cortex (mPFC) in DD. rTMS to the mPFC (compared to the vernex) attenuated both DD rates and striatal dopamine release, supporting a dual-decision model for the emergence of DD from an interaction between executive and impulsive regions [48]. These neuromodulatory approaches have also been applied therapeutically, as a novel intervention. Recently, Sheffer and colleagues [22]** applied rTMS to another prefrontal, executive region implicated in DD, the dorsolateral prefrontal cortex (dlPFC), in a double-blind, randomized controlled trial of 29 smokers. Active rTMS both reduced DD and enhanced the effects of a self-help smoking cessation program, increasing abstinence and reducing relapse at 12 weeks. This treatment had both high acceptability and promising efficacy, suggesting that neuromodulation, like other interventions targeting DD, has promising translational potential in addiction treatment.

Another successful approach to increasing the valuation of the future is episodic future thinking (EFT), derived from the science of prospection. Episodic future thinking entails vividly imagining positive, personal future events, and has been shown to reduce DD rates [49]*. Presentation of these episodic future events has been shown to engage the hippocampus and amygdala and enhance prefrontal-mediotemporal interactions [49]*. Most importantly, EFT has reduced excessive DD in several disorders (e.g., overweight/obesity, smoking, alcohol use disorder, and gambling). New fMRI research has revealed that EFT enhances functional connectivity between the ventromedial PFC and the ventral striatum in both healthy individuals and pathological gamblers, which the authors suggest is a possible mechanism driving the EFT-induced reduction in DD [50]. This manipulation has remarkable potential as a therapeutic intervention.

Discussion

Here, we updated the findings regarding DD as a trans-disease process. Specifically, new evidence supports the expansion of this concept to include multiple health behaviors, both between and within individuals. Moreover, methodological advances permit the rapid assessment of DD perhaps heralding future applications in a wider array of studies. Since the first report, scientific investigations have identified both genetic and neural substrates of DD, clarifying this measure’s role as an endophenotype of multiple behaviors. Not surprisingly, given these results, interventions are being devised to modulate DD (Figure 1).

Figure 1.

Schematic representation of delay discounting (DD) as a trans-disease process. Genetic, neural, environmental and other factors can increase DD rates and therefore modulate the risk of substance use disorder (e.g., AUD), other disorders (e.g., obesity and schizophrenia) as well as a maladaptive health behavior (e.g., poor medication adherence and texting while driving). Importantly, many of the risks modulated by DD (e.g., AUD, and obesity) have the capacity to feed-back and increase the steepness of DD rates. DD’s ability to predict engagement in multiple health behaviors has been shown not only across individuals, but more recently, within individuals as well. Promising novel interventions (e.g., TMS and EFT) are now targeting DD as a means to positively affect associated disorders and behaviors.

Those disorders associated with excessive consumption of brief, intense reinforcers are commonly relapsing. Importantly, the hyperbolic function of DD predicts preference reversals that may undergird this relapse. For example, one might prefer the delayed benefits of weight loss, and would intend to decline dessert. However, when the dessert tray is presented, the preference reverses. More formally, a larger and more delayed reinforcer may initially be preferred when considered from afar; however, as both reinforcers become more proximal, the smaller, sooner reinforcer’s value increases.

One consequence of the manipulation of DD has been the examination of its effects on other measures, in particular the valuation of brief, intense reinforcers (e.g., drugs, food). One contemporary theory of addiction, reinforcer pathology, has emerged from this line of study [51–53]. Specifically, reinforcer pathology proposes that DD measures the temporal window over which reinforcers are integrated, which determines the relative value of substance-related and/or prosocial reinforcers [51,52*,53]. For example, studies have found that increasing or decreasing the temporal window (reducing and increasing DD) has reduced or increased valuation of brief, intense reinforcers, including alcohol [54]*, cigarettes [55], and food [16*,56]. As a result, DD should be further examined as an etiologic process in the development of these disorders.If the results of those explorations are robust, the stage may be set for novel theoretical understanding of these disorders and new interventions to treat them.

Highlights.

• Delay discounting (DD) indicates choice between smaller, sooner and larger, later reinforcers.

• DD modulates the risk of substance use disorder, other disorders, and maladaptive health behaviors.

• DD predicts engagement in multiple health behaviors between and within individuals.

• Novel interventions targeting DD may positively affect associated disorders and behaviors.