Genomic Basis of Delayed Reward Discounting

Abstract

Delayed reward discounting (DRD) is a behavioral economic measure of impulsivity, reflecting how rapidly a reward loses value based on its temporal distance. In humans, more impulsive DRD is associated with susceptibility to a number of psychiatric diseases (e.g., addiction, ADHD), health outcomes (e.g., obesity), and lifetime outcomes (e.g., educational attainment). Although the determinants of DRD are both genetic and environmental, this review focuses on its genetic basis. Both rodent studies using inbred strains and human twin studies indicate that DRD is moderately heritable, a conclusion that was further supported by a recent human genome-wide association study (GWAS) that used single nucleotide polymorphisms (SNP) to estimate heritability. The GWAS of DRD also identified genetic correlations with psychiatric diagnoses, health outcomes, and measures of cognitive performance. Future research priorities include rodent studies probing putative genetic mechanisms of DRD and human GWASs using larger samples and non-European cohorts. Continuing to characterize genomic influences on DRD has the potential to yield important biological insights with implications for a variety of medically and socially important outcomes.

1. Introduction

The preference for smaller immediate rewards relative to larger delayed rewards is a behavioral economic concept that reflects the capacity to delay gratification (Green et al., 1994). Delayed reward discounting (DRD) is used to measure how rapidly a reward loses its value based on its temporal distance. Thus, greater DRD reflects a preference for smaller immediate rewards rather than larger, delayed rewards and is one form of impulsivity. Meta-analyses show consistent associations between greater DRD and adverse psychiatric outcomes including substance use disorders, gambling disorder, and attention-deficit/hyperactivity disorder (ADHD) (Amlung et al., 2016a; Jackson & MacKillop, 2016; MacKillop et al., 2011). In terms of nonpsychiatric health outcomes, greater DRD is positively associated with obesity (Amlung et al., 2016b), and negatively associated with glycemic adherence in type 2 diabetes (Lebeau et al., 2016; Reach et al., 2011), obtaining preventative medical care (e.g., flu shots, breast and prostate exams; (Bradford, 2010)), and seatbelt use (Bradford et al., 2014). Finally, even after attempting to control for parental income and cognitive ability, DRD is negatively associated with lifetime outcomes including educational attainment, income, and employment (Golsteyn et al., 2014). Individuals with high DRD appear to be less thoughtful of their future selves, which leads to increased risks for a multitude of deleterious mental, physical, and social outcomes. As such, DRD has been proposed as a target for treatment (Gray & MacKillop, 2015; Lowe et al., 2018; Sheffer et al., 2018) and is one component of the Research Domain Criteria (RDoC) (Lempert et al., 2018), a National Institute of Mental Health (NIMH) initiative that emphasizes basic dimensions of functioning that span the full range of human behavior from normal to abnormal.

This mini-review will highlight current research relating to the genetic basis of DRD, including data from animal models. We begin with a summary of DRD measurement in humans and nonhuman animals, followed by a review of findings from heritability and genome-wide association studies (GWASs). We conclude our review by identifying promising future research directions. We will not review the many candidate gene studies that have been conducted on this topic, in part because of the consistent difficulty in replicating candidate loci for complex traits (Chabris et al., 2012; Farrell et al., 2015; Hart et al., 2013), and because candidate genes for DRD have been summarized in two of our recent publications (MacKillop et al., 2019; Sanchez-Roige et al., 2018).

Like all psychological traits, DRD is influenced by environmental and genetic factors and presumably also their many interactions. With regard to environmental influences, research indicates that child maltreatment (Oshri et al., 2018a, 2018b), trauma (van den Berk-Clark et al., 2018), and substance use (Mendez et al., 2010; Mitchell et al., 2014; Setlow et al., 2009; Simon et al., 2007) appear to increase levels of DRD. While no-well powered studies have investigated gene-by-environment interactions relevant to DRD, it is likely that certain environmental exposures modulate DRD in a genotype-specific manner. Thus, while this mini-review is focused on the genetic basis of DRD, research seeking to understand environmental and gene-by-environment interactions also represent important lines of inquiry.

2. Delayed Reward Discounting Measurement

DRD is typically assessed by providing organisms with a choice between smaller immediate and larger delayed rewards. In humans, these rewards are usually choices between smaller amounts of money today versus larger amounts of money after a delay, though food and drugs have been in place of money (Green and Lawyer, 2014; Odum and Rainaud, 2003; Robertson and Rasmussen, 2018). For example, one of the most widely-used measures, the Monetary Choice Questionnaire (MCQ), consists of 27 questions such as “Would you rather have $24 today or $35 in 29 days?” (Gray et al., 2016; Kirby et al., 1999). Although the rewards are typically hypothetical rather than real, this does not appear to impact responding (Madden et al., 2003; Matusiewicz et al., 2013; Robertson & Rasmussen, 2018). In animals such as pigeons and rodents, DRD is typically assessed using delayed food or water rewards and the animals always receive the rewards associated with their choice (Isles et al., 2004; Mazur, 1987; Mitchell, 2014; Richards et al., 2013).

In both humans and non-human species, organisms typically devalue delayed rewards in a nonlinear fashion, modeled as a hyperbolic function (Vanderveldt et al., 2016). The extent of DRD can be quantified in several ways (Myerson et al., 2014), such as calculating the slope of the hyperbolic discounting function (k) or model-free methods such as area under the curve and immediate choice ratio (Green & Myerson, 2004; Myerson et al., 2001). Figure 1 shows two prototypic hyperbolic demand curves in humans with differing slopes (more impulsive k = .1, less impulsive k = .01) that exhibit the discounted subjective value of $100 delayed from 1 day to 1 year. For example, at 60 days, $100 is equal in subjective value to $62 today for the less impulsive profile and $14 today for the more impulsive profile.

Figure 1.

Prototypic hyperbolic delayed reward discounting curves. The curves reflect the discounted subjective value of $100 delayed from 1 day to 1 year. The k values refer to the slopes of the two discounting curves. The k values are derived from the Mazur (1987) equation: V = A/(1 + kD), where V is the present value of the delayed reward A at delay D, and k is a free parameter that determines the discount rate. The higher one’s discount rate (k) is, the more they discount larger future rewards.

Although there are many parallels between the DRD models used with humans and laboratory animals, there are also several notable differences that may affect generalizability across species (for an in depth discussion see Vanderveldt et al., 2016). First, in humans there is a well-documented magnitude effect, whereby humans discount small, delayed rewards more steeply than larger delayed rewards. This effect has been shown across reward types including money (Johnson and Bickel, 2002; Madden et al., 2003), food (Odum et al., 2006), and liquid rewards (Jimura et al., 2009). However, the magnitude effect has not been consistently observed in nonhuman animals (e.g., Green et al., 2004; Richards et al., 1997). Second, the time frame of the procedures, and presumably the time frame for self-control, differs in humans and laboratory animals. In the animal procedures the delays are in seconds or minutes whereas in most human procedures the delays are days to months. Moreover, in the animal procedures, the delays are experienced directly and relate to their immediate thirst or hunger, whereas in humans the delays are communicated by instructions and typically involve a secondary reinforcer (money) (de Wit et al., 2018). Nonetheless, both humans and laboratory animals discount delayed rewards in an orderly manner, suggesting a fundamental behavioral homology.

3. Heritability

The heritability of DRD has been examined in both humans and rodents. In humans, studies with monozygotic and dizygotic adolescent twins provide evidence of robust heritability, which tends to increase through development (i.e. 12 years old (yo) [30%] and 14 yo [51%], (Anokhin et al., 2011); 16 yo [35–46%], 17 yo [47–51%], and 18 yo [55–62%] (Anokhin et al., 2015; Isen et al., 2014; Sparks et al., 2014)). The increase in genetic influence on DRD throughout development may reflect the changing importance of competing environmental factors and the maturation of the prefrontal cortex in adolescence (Argyriou et al., 2018), a critical region for DRD (Wesley & Bickel, 2014).

In mice and rats, a significant proportion of the variance in DRD can be attributed to between-strain versus within-strain differences (16–50%), which is analogous to the twin model design (Anderson & Woolverton, 2005; Isles et al., 2004; Madden et al., 2008; Richards et al., 1997; Stein et al., 2012; Wilhelm and Mitchell, 2009). The lowest estimate (16%) came from the only study with mice conducted to date (Isles et al., 2004), whereas estimates of heritability in rats were much higher (40–50%) (Richards et al., 2013; Wilhelm and Mitchell, 2009). However, comparisons across strains of rodents have some limitations. First, strains were sometimes obtained from different vendors and thus genotype and the different environment of each vendor facility are confounded. Second, studies vary with regard to training procedure, type of reinforcer (e.g., condensed milk, water), delay range (e.g., 8 vs. 16 seconds maximum delay), number of sessions, and dependent variable (e.g., ratio of delayed choices, AUC, k). On balance, findings from both humans and rodents suggest that DRD is a moderately heritable trait, although the variability in estimates suggests significant moderators of its heritability.

4. Genome-wide Association Studies

A GWAS is a study of a set of genetic variants sampled across the whole genome to identify polymorphisms associated with a trait (Visscher et al., 2017). The primary goal of GWAS is to better understand the biology of the trait. Because millions of variants are tested, a stringent significance testing threshold must be employed. It is generally accepted that the significance threshold for any single polymorphism is p < 5 × 10⁻⁸. This threshold accounts for an estimated 1 million independent tests, and variants beyond this threshold tend to replicate (McCarthy et al., 2008; Visscher et al., 2017). Over the past decade, it has become clear that for virtually all common traits, associations tend to be numerous small-effect variants spread across most of the genome, in or near genes that have no obvious biological connection to the trait (e.g., Boyle et al., 2017). Nonetheless, GWASs are thought to yield new insights into the biology of complex traits (Visscher et al., 2017) and ultimately facilitate the discovery of novel treatments (Cook et al., 2014; Nelson et al., 2015).

To date, two GWASs have been conducted on DRD. The first was conducted in collaboration with the genetics company 23andMe, Inc., and included 23,217 adults of European ancestry (Sanchez-Roige et al., 2018). This study found single nucleotide polymorphism (SNP)-based heritability of DRD of 12.2%. This SNP-based heritability is lower than heritability estimates obtained using human twins and rodent inbred strains for a number of reasons, including that the SNP-based heritability is an underestimation due to the absence of rare variants (Marouli et al., 2017; Yang et al., 2015), and that pedigree estimates are inflated due to shared environmental and non-additive genetic effects (Polderman et al., 2015). In Sanchez-Roige et al., (2018), one SNP, rs6528024, which is located in an intron of the gene GPM6B (Neuronal Membrane Glycoprotein M6B), reached genome-wide significance (p = 2.40 × 10⁻⁸). This association was supported by an independent cohort of 928 participants (meta-analysis p = 1.44 × 10⁻⁸).

GPM6B encodes a protein that is involved in the internalization of the serotonin transporter and has been implicated in prepulse inhibition and altered response to the 5-HT2A/C agonist DOI in mice (Dere et al., 2015; Fjorback et al., 2009). A large body of research has explored the relationship between serotonergic functioning and DRD; the findings are inconsistent and have primarily relied on rodent models. For example, there is some evidence that serotonin may be more related to increased confidence in reward delivery than to increased capacity to wait for a delayed reward (Dalley and Ersche, 2019; Miyazaki et al., 2018). In humans, GPM6B expression is downregulated in the brains of depressed suicide victims (Fuchsova et al., 2015) and DRD has been linked to suicide attempts with a pooled odds ratio = 3.14 (95% confidence interval: 1.48–6.67) (Liu et al., 2017). The link between DRD and suicidality is further supported by genetic correlations identified in the study by Sanchez-Roige et al (2018), which found positive genetic correlations between DRD and major depression and neuroticism as well as smoking behaviors, ADHD, BMI, and negative associations with years of education and childhood IQ.

The second DRD GWAS used a sample of 986 healthy young adults of European ancestry (MacKillop et al., 2019). That study identified a genome-wide significant variant (p = 2.8 × 10⁻⁸), rs13395777, on chromosome 2, an association that was not observed in the 23andMe cohort (p = .45). There are two most likely explanations for this failure to replicate. The finding may have been a false positive, which would explain why it was not detected in a cohort that was ~25x larger. Alternatively, the smaller study was comprised of young adults and required low levels of substance use, whereas the larger study included a wider age range, resulting in substantially higher mean age and income, and allowed for psychopathology.

5. Future Directions

DRD is a moderately heritable phenotype that is both phenotypically and genetically associated with an array of negative psychological, cognitive, and health outcomes. The largest GWAS to date identified a single locus that was associated with DRD and showed that genetic predisposition to high DRD is positively genetically correlated with many of the negative outcomes that have been previously associated with higher DRD. Future studies will be required to further define the genetic basis of DRD. We are currently using rodents with mutations in GPM6B to examine DRD and related behavioral traits. We are also continuing to increase the sample size for future GWASs of DRD, which may allow us to identify additional loci (Marouli et al., 2017; Visscher et al., 2017). Another future direction may be to study diverse ancestral groups, expanding current data from individuals of European ancestry (Duncan et al., 2018; Locke et al., 2015). Additionally, it will be important to further parse causality between DRD and associated outcomes (e.g., addiction, years of education) using methods such as longitudinal designs and Mendelian randomization (Burgess et al., 2015; Grant and Chamberlain, 2014). Finally, DRD is only one element of impulsivity, which is a broader construct that appears to comprise three broad and generally independent domains (MacKillop et al., 2016). Thus, understanding the genetics of impulsivity will also require exploration of other measures of impulsivity (e.g., response inhibition and impulsive personality traits; Gray et al., 2018; Sanchez-Roige et al., 2019; Weafer et al., 2017).

6. Conclusion

DRD is a heritable trait that can be assessed quickly and reliably both in person and over the internet (Koffarnus & Bickel, 2014; Sanchez-Roige et al., 2018; MacKillop et al., 2018), and influences a variety of health-related outcomes. Although at an early stage, GWASs have begun to identify loci and genes that influence variability in DRD, setting the stage for a deeper understanding of its molecular, cellular and circuit-level bases, and perhaps ultimately informing the treatment of psychiatric disorders and other conditions to which it confers risk.

Highlights.

• Delayed reward discounting (DRD) is a measure of capacity to delay gratification.

• DRD is moderately heritable and associated with mental, physical, and social outcomes.

• DRD is a component of Research Domain Criteria and a putative target for treatment.

• The largest GWAS to date yielded a SNP heritability of 12% and one significant SNP.

• Future priorities include GWAS with larger samples and non-European cohorts.