Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Aug 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2023 Jan 26;31(4):786–792. doi: 10.1037/pha0000623

Utility of the 5-trial adjusting delay task in screening for high delay discounting rates: A cohort study of individuals reporting harmful alcohol and tobacco use

Candice L Dwyer 1,2,*, William H Craft 1,3,*, Allison N Tegge 1,4, Yeh Yu-Hua 1, Warren K Bickel 1
PMCID: PMC10368788  NIHMSID: NIHMS1887047  PMID: 36701520

Abstract

High delay discounting (DD) rates are associated with several health conditions, including addiction. Investigators interested in the modulation of DD rates may be interested in screening for individuals with high DD rates within substance using samples. The adjusting delay discounting task (ADT) is a brief DD rate measure suitable for screening purposes, but how performance on this task is related to more granular DD measures (e.g., adjusting amount task; AAT) in individuals who use substances is unknown. This study investigated the relationship between DD rates measured by the ADT and AAT to assess the utility of the ADT in screening for high discounting rates in individuals who use tobacco and alcohol. Participants (N = 488) were screened for high discounting rates (ln(k) >= −4.3118) using the ADT. Subsequently, participants completed the AAT. Linear regressions were conducted to understand the relationship between the ADT and AAT DD rates. Separate analyses between the total, high-quality and low-quality samples were conducted. Significant relationships between the ADT and AAT were found for the total (p<0.001), high-quality (p<0.001), and low-quality samples (p<0.001). Although the ADT overestimated DD rates on the AAT by 2.49, 2.29, and 2.97 units in the total, high-quality, and low-quality samples, the ADT accurately identified individuals with high discounting rates. Specifically, 71% of the total sample, 80% of the high-quality sample, and 66% of the low-quality sample maintained a high DD rate on the AAT. These results demonstrate that ADT can be used as a quick, accurate screening tool to identify high discounting rates in individuals who use substances.

Keywords: alcohol, delay discounting, adjusting delay task, adjusting amount task, screening

Utility of the 5-trial adjusting delay task in screening for high delay discounting rates in substance use research

The preference for immediate reinforcement at the expense of greater reinforcement in the future, often referred to as delay discounting, is a trans-disease process (Bickel et al., 2012, 2019). High rates of delay discounting are broadly associated with an array of psychiatric disorders and maladaptive health behaviors including but not limited to: obesity (Amlung et al., 2016) depression (Amlung et al., 2019), gambling (Weinsztok et al., 2021), and substance use (Bickel et al., 2019; MacKillop et al., 2011; Odum et al., 2020; Strickland et al., 2021). Regarding substance use specifically, high rates of delay discounting relate to important aspects including: differentiating individuals dependent on substances from healthy controls (Businelle et al., 2010; Finn & Hall, 2004; Madden et al., 1997), substance use initiation (Audrain-McGovern et al., 2009), severity of use (Albein-Urios et al., 2012; MacKillop et al., 2010; Reynolds et al., 2004; Vuchinich & Simpson, 1998), and substance use treatment outcomes (Dallery & Raiff, 2007; Washio et al., 2011; Yoon et al., 2007). These findings are consistent across different substances of abuse, including alcohol (Amlung et al., 2017; MacKillop et al., 2010; Moody et al., 2017; Phung et al., 2019) and tobacco (Amlung et al., 2017; Barlow et al., 2017; Bickel et al., 2008; DeHart et al., 2020).

Recent investigations on the modulation of delay discounting rates demonstrate concomitant changes in both delay discounting rates and maladaptive behaviors including unhealthy reinforcers (Athamneh et al., 2021; Bulley & Gullo, 2017; Mellis et al., 2018; Snider et al., 2020, 2021; Stein et al., 2016; Sze et al., 2017) and pain severity (Craft et al., 2020). Importantly, rate-dependence (i.e., an orderly relationship between a baseline response level and that response level following an intervention) has been observed for delay discounting rates in individuals following substance use treatment, such that individuals with higher discounting rates at baseline saw the greatest decrease in discounting post-treatment (Bickel et al., 2014). Similar rate-dependent effects were observed for interventions (i.e., episodic future thinking) which targeted delay discounting directly (Snider et al., 2018) Given the relationship between high delay discounting rates, poor health outcomes, and evidence of rate-dependence, investigators may be interested in screening for and selectively recruiting individuals with higher discounting rates within cohorts of individuals using substances.

While many tasks to measure delay discounting are available (Tang et al., 2019), one of the most commonly used in addiction research is the adjusting amount task (AAT) (MacKillop et al., 2011). However, the AAT and similar tasks involve a large number of choice trials and take upwards of several minutes to complete. Given potential constraints during eligibility screening (e.g., time and participants’ attention), the adjusting delay task (ADT) may be a particularly amenable replacement. The ADT is a brief, 5-trial delay discounting task which can be completed in less than a minute (Koffarnus & Bickel, 2014). The ADT has been previously validated as a reliable measure by examining the relationship between the ADT and the AAT. However, the validation study sampled college undergraduate students who reported low use of tobacco and/or alcohol and demonstrated relatively low discounting rates. Whether these findings extend to individuals reporting risky substance use and/or those with higher discounting rates is unknown. In light of this gap in knowledge, this study sought to assess the utility of the ADT to screen individuals reporting daily tobacco use and harmful alcohol use for high delay discounting rates by examining the relationship between performance on the ADT and AAT in an online sample.

Method

In this section, we report how our sample size was determined, all data exclusions, all manipulations, and all study measures. The data, study materials, and analysis code are available upon request by emailing the corresponding author. This study was not pre-registered.

Participants

The present study is a secondary analysis from a larger study investigating delay discounting and emotion regulation in individuals who endorsed tobacco use and harmful alcohol use. Therefore, the sample size (N = 488) was determined by the number of individuals who completed this larger study. Participants were recruited via Amazon Mechanical Turk (MTurk; https://www.mturk.com/), an online crowdsourcing platform where individuals can earn monetary compensation for completing Human Intelligence Tasks (HITs). To be included in this study, participants were required to 1) be living in the United States; 2) have an approval rating on prior completed HITs of >= 95% (Athamneh et al., 2019; DeHart et al., 2019; Sze et al., 2017); 3) have an Alcohol Use Disorder Identification Test Score (AUDIT; Saunders et al., 1993) of >=16, indicating harmful alcohol use (Babor et al., 2001); 4) report smoking 11 or more cigarettes per day; and, 5) have a delay discounting rate of >= −4.3118, as measured by the ADT. We selected −4.3118 as the cut-off because the majority of individuals included in the prior study demonstrated discounting rates below this threshold (Koffarnus & Bickel, 2014) and this discounting rate is consistent with substance use disorder treatment-seeking samples and rate-dependent effects of delay discounting (Bickel et al., 2014).

In addition to living in the US and a high HIT approval rating, we included several measures to improve data quality: 1) a reCAPTCHA to prevent automated responding; 2) two attention check questions in the AAT to identify inattentive and/or random responses, and 3) post hoc application of systematic delay discounting criteria (Johnson & Bickel, 2008). In this study, 68 individuals failed one or more attention check questions and 228 displayed non-systematic delay discounting. Individuals that failed these quality checks (n = 296) were labeled “low-quality”, while individuals that passed these quality checks (n = 192) were labeled as “high-quality”. Participants were paid $2.50 for completing the HIT. The Institutional Review Board approved the study at Virginia Polytechnic and State University. Consent was implied through the completion and submission of the HIT.

Delay Discounting Measures

Adjusting Delay Discounting Task:

During the screener, the 5-trial adjusting delay discounting task (ADT; Koffarnus & Bickel, 2014) was used to recruit high discounters (ln(k) >= −4.3118). The task includes 31 potential time delays ordinally ranging from 1-hour to 25-years (see Table 1 in Koffarnus & Bickel, 2014 for a full list of ADT time delays). In the first trial, participants were asked to choose between $1000 available after a 3-week delay or half that amount ($500) available immediately. Based on the participant’s response, the time delay on the next trial is titrated by 8 delays either up (i.e., if the participant chose the delayed choice on the first trial, the time delay presented in the second trial is 2-years) or down (i.e., if the participant chose the immediate choice on the first trial, the time delay presented in the second trial is 1-day). On the subsequent three trials, this pattern continues, with the time delay adjusting by half the previous adjustment (i.e., by 4 delays, then 2 delays, then 1 delay). The ADT measures the Effective Delay 50% (the delay at which a commodity has lost 50% subjective value: ED50) of $1000. The discounting rate, or k value, is calculated as the inverse of the ED50 (Yoon & Higgins, 2008), with larger k values indicating a greater preference for immediate rewards.

Adjusting Amount Delay Discounting Task:

After completing the screener, individuals completed the $1000 adjusting amount delay discounting task (AAT; Du et al., 2002). In this task, participants made repeated choices between a smaller immediately available amount of money or a larger, delayed amount of money across 7 delay conditions (1 day, 1 week, 1 month, 3 months, 1 year, 5 years, and 25 years), which were shown in a randomized order. While the delayed amount was fixed at $1000, the immediate amount on the first trial of each condition was $500. On the successive 5 choice trials of the same condition, the immediate amount is adjusted up (delayed choice) or down (immediate choice), based on the participant’s response to the prior trial. The immediate amount to be used for the 7th trial was taken as an indifference point. The indifference points, representing the value at which the immediate and delayed rewards are approximately equal at each delay, were fit to a hyperbolic model using Mazur’s equation (Mazur, 1987) to calculate the k value. All k values were natural-log transformed (ln(k)) to stabilize the variance and normalize the data.

Demographics

Participants were asked demographic questions about age, sex, race, ethnicity, education, and annual income.

Statistical analysis

Demographic characteristics were summarized using mean (standard deviations) for continuous measures and frequency (percentages) for categorical measures. Differences in demographics between the low- and high-quality groups were measured using t test and Chi-square test. The positive predictive value (PPV) of the ADT was determined by dividing the proportion of individuals who had high discounting rates on both the ADT and AAT by the number of individuals who had high discounting rates on both tasks and the number of individuals who only had a high discounting rate on the ADT (i.e., true positives / true positives + false positives). A linear regression model was used to understand the relationship between the ADT discounting rate measured during screening and the AAT discounting rate measured during the main survey. Additionally, to understand if alcohol and tobacco use affected the relationship between the ADT and AAT discounting rates, we utilized a linear regression model including AUDIT score and cigarettes per day as covariates. Low- and high-quality groups were analyzed separately along with the full sample to investigate the influence of data quality on the findings. All analyses were conducted in R Studio version 4.0.4 (R Core Team, 2021).

Results

Table 1 shows the demographic characteristics of the sample. The total sample was on average 36.17 (±9.3) years old, 32.6% female, 91.8% white, and 74.8% non-Hispanic. When stratifying by data quality group, several statistically significant differences were observed. Relative to the high-quality sample, the low-quality sample included higher proportions of males (p < 0.001) and individuals who identified as Hispanic (p < 0.001), as well as individuals who had higher education (p = 0.049), AUDIT scores (p = 0.001), and delay discounting rates on the ADT (p < 0.001).

Table 1.

Sample characteristics.

Characteristics: mean (SD)/frequency (%) Total (N =488) High-quality (n = 192) Low-quality (n = 296) p valuea
Age - yearsb 36.17 (9.30) 36.55 (10.37) 35.92 (8.55) 0.462
Sexc <0.001
 Female 158 (32.4) 81 (42.2) 77 (26.0)
 Male 327 (67.0) 110 (57.3) 217 (73.3)
 N/A 3 (0.60) 1 (0.5) 2 (0.70)
Racec
 American Indian or Alaskan Native 5 (1.0) 3 (1.6) 2 (0.7)
 Asian 4 (0.8) 0 (0.0) 4 (1.4)
 Black or African American 26 (5.3) 12 (6.3) 14 (4.8)
 Native Hawaiian or Pacific Islander 2 (0.4) 0 (0.0) 2 (0.7)
 White 445 (91.1) 174 (91.0) 271 (91.6)
 Multiple races 2 (0.4) 2 (1.0) 0 (0.0)
 Prefer not to answer 1 (0.2) 0 (0.0) 1 (0.3)
 N/A 3 (0.6) 1 (0.5) 2 (0.7)
Ethnicityc <0.001
 Hispanic 115 (23.6) 29 (15.1) 86 (29.1)
 Non-Hispanic 363 (74.4) 161 (83.9) 202 (68.2)
 Prefer not to answer 7 (1.4) 1 (0.5) 6 (2.0)
 N/A 3 (0.6) 1 (0.5) 2 (0.7)
Educationc 0.049
 Less than High School 1 (0.2) 0 (0.0) 1 (0.3)
 High school diploma or GED 6 (1.2) 5 (2.6) 1 (0.3)
 Associate or Bachelor’s degree 322 (65.9) 137 (71.4) 185 (62.5)
 Master’s degree 155 (31.9) 49 (25.5) 106 (35.8)
 Professional degree (MD, JD, DDS) 2 (0.4) 0 (0.0) 2 (0.7)
 N/A 2 (0.4) 1 (0.5) 1 (0.3)
 Annual Incomec 0.134
 Less than $30,000 43 (8.8) 19 (9.9) 24 (8.1)
 $30,000 to $49,999 90 (18.4) 45 (23.4) 45 (15.2)
 $50,000 to $69,999 83 (17.0) 29 (15.1) 54 (18.2)
 $70,000 to $89,999 138 (28.3) 45 (23.4) 93 (31.5)
 $90,000 to $109,999 80 (16.4) 32 (16.8) 48 (16.2)
 $110,000 to $139,999 26 (5.3) 10 (5.2) 16 (5.4)
 $140,000 or greater 25 (5.2) 11 (5.7) 14 (4.7)
 N/A 3 (0.6) 1 (0.5) 2 (0.7)
AUDIT Scored 22.60 (6.48) 25.40 (6.10) 27.39 (6.61) 0.001
ln(k)e Adjusting Delay Task 0.63 (2.47) −0.61 (2.12) 1.43 (2.36) <0.001
ln(k)e Adjusting Amount Task −2.14 (5.45) −2.50 (2.66) −1.90 (6.66) 0.240
Open in a new tab

Note. GED = General Educational Development.

a

p value represents comparison between included and excluded samples.

b

M (SD).

c

Frequency (%).

d

Alcohol Use Disorder Identification Test (AUDIT) Score.

e

Delay Discounting Rate (k).

To examine the utility of the ADT to screen for high discounting rates, we measured the positive predictive value (PPV) of using the ADT to identify individuals with a high discounting rate on the AAT. Initially, 74% of the sampled cohort (i.e., individuals who smoked 11+ cigarettes per day and scored >= 16 on the AUDIT) screened positive for high discounting using the ADT. Screening with an ADT discounting rate of >=−4.3118 resulted in a PPV of 71% (347 /488) for the total sample, 80% (153 /192) for the high-quality sample, and 66% (194/296) for the low-quality sample.

To further understand the relationship between the ADT and the AAT in this online cohort of individuals that endorse tobacco and harmful alcohol use, a linear regression model was used with the AAT discounting rate as the dependent variable and ADT discounting rate as the independent variable. For the total sample, a significant relationship between tasks was observed (β = 0.556, t (486) = 5.749, p < 0.001), with an intercept significantly different from zero, (β = −2.49, t (486) = −10.083, p < 0.001), suggesting that the ADT overestimated the AAT value by 2.49 units. Similar results were observed for the high-quality sample, a significant relationship between tasks was observed (β = 0.332, p < 0.001, t (190) = 3.78, p < 0.001), with an intercept significantly different from zero, (β = −2.29, t (190) = −11.871, p < 0.001), suggesting that the ADT overestimated the AAT value by 2.29 units. For the low-quality sample, a significant relationship between tasks was observed (β = 0.750, t (294) = 4.729, p < 0.001) with an intercept significantly different from zero (β = −2.97, t (294) = −6.81, p < 0.001), suggesting that the ADT overestimated the AAT value by 2.97 units. To investigate if alcohol or tobacco use severity affected the relationship between the two discounting tasks, we performed linear regression with AAT discounting rate as the dependent variable and ADT discounting rate, AUDIT score, and cigarettes per day as the independent variables. The ADT discounting rate remained a significant predictor of AAT discounting rate in the total sample (β = 0.511, t (484) = 5.157, p < 0.001), high-quality (β = 0.320, t (188) = 3.653, p < 0.001), and low-quality samples (β = 0.695, t (292) = 4.321, p < 0.001), while both AUDIT score and cigarettes per day were not significant predictors of AAT discounting rates in any of the samples (ps > 0.13).

Discussion

This study aimed to assess the utility of the ADT to screen individuals reporting daily tobacco use and harmful alcohol use for high delay discounting rates by examining the relationship between performance on the ADT and AAT in an online sample. Overall, our results support the ADT as a useful tool to screen for high delay discounting rates in substance use research. Specifically, the ADT had a PPV of 71% (i.e., 71% of individuals who were screened for high discounting maintained a high discounting rate on the AAT). The PPV of the ADT increased to 80% after excluding low-quality data from the sample. Further, extending the findings of (Koffarnus & Bickel, 2014), we report a statistically significant relationship between performance on the ADT and AAT.

Consistent with extant literature, in our study, discounting rates as measured by the ADT overestimated discounting rates as measured by the AAT (Koffarnus & Bickel, 2014). However, while Koffarnus & Bickel (2014) found that in healthy controls, discounting rates on the ADT overestimated the AAT by 0.60, our study found that the ADT overestimated the AAT by a larger magnitude (2.49 units in the overall sample and 2.29 units after excluding individuals who failed attention checks and/or provided non-systematic discounting data). The difference between the studies may be explained by the sample characteristics, although further investigation is needed to pinpoint the cause. Note that the ADT task’s design precludes the utilization of non-systematic criteria. Recent studies have shown that individuals reporting tobacco and alcohol use have high proportions of low-quality responding, between 40% and 60% (Craft et al., 2022; Yeh et al., 2022). Thus, the inability to detect low-quality responses in the ADT potentially reduces the strength of the relationship between the ADT and the AAT and precludes the identification of individuals before study participation who might provide poor quality data.

Poor data quality is a concern on MTurk (Chmielewski & Kucker, 2020; Kennedy et al., 2020; Mellis & Bickel, 2020). Recent studies in substance-using samples have provided empirical evidence that excluding individuals who fail attention check questions or provide non-systematic delay discounting data can improve the data quality (Craft et al., 2022; Yeh et al., 2022). In our study, we found that excluding data from individuals who failed attention checks and/or provided non-systematic discounting from the regression model reduced the magnitude of the overestimation of discounting rates measured by the ADT relative to the AAT from 2.97 to 2.29 units. This finding further supports the use of such quality control methods on MTurk.

Several potential limitations of this study exist. This study was conducted online and whether these results would be similar to an in-lab study remains to be tested. However, several studies have found concordance between in-lab and online studies (Crump et al., 2013; Paolacci et al., 2010). Additionally, we note the high proportion of individuals with low quality data. Previous investigations have found a recent increase in poor quality data on MTurk (Chmielewski & Kucker, 2020; Stokel-Walker, 2018). However, empirical methods to safeguard data quality have been identified (Craft et al., 2022; Freitas-Lemos et al., 2022; Yeh et al., 2022). The majority of this sample identified as white, male, and non-Hispanic; thus, how generalizable these results are to more diverse samples is unknown. Similarly, the study was conducted with a tobacco and alcohol using sample; the relevance of these findings to other substance-using samples (e.g., cocaine, methamphetamines, and opiates), other psychiatric conditions (e.g., depression), and maladaptive health behaviors (e.g., obesity) is unknown.

Conclusion

This study investigated the relationship between delay discounting rates measured via two tasks in a sample of individuals using tobacco and alcohol: the ADT and AAT. Replicating and extending the prior findings of (Koffarnus & Bickel, 2014), we found a statistically significant relationship between discounting rates in the two tasks. Together, these results demonstrate that the ADT can be used as a quick, accurate screening tool to identify high discounters within cohorts of individuals reporting substance use. Specifically, depending on the data quality control methods used, researchers can expect 70–80% of individuals screened for high discounting (>= −4.3118) with the ADT to have concordant discounting rates on the AAT. However, when screening substance-using individuals for high discounting using the ADT, researchers should be aware that the ADT may overestimate the discounting rate.

Figure 1.

Figure 1.

Open in a new tab

Correlation between the adjusting delay task and the adjusting amount task for (A) the original validation study (adapted from Koffarnus & Bickel, 2014) and (B) the high-quality sample in the present study.

Public Health Significance.

This study demonstrates the utility of the adjusting delay discounting task as a brief method of screening for high delay discounting rates among a cohort of individuals reporting daily tobacco and harmful alcohol use and can inform methodological considerations in substance use research.

Disclosures and Acknowledgments

This work was supported by National Institutes of Health grants R01AA027381 and R01DA054216 and the Fralin Biomedical Research Institute at VTC. The funders had no role other than financial support.

All authors contributed in a significant way and all authors have read and approved the final manuscript.

Although the following activities/relationships do not create a conflict of interest pertaining to this manuscript, in the interest of full disclosure, Dr. Bickel would like to report the following: W. K. Bickel is a principal of HealthSim, LLC; BEAM Diagnostics, Inc.; and Red 5 Group, LLC. In addition, he serves on the scientific advisory board for Sober Grid, Inc., and Ria Health, is a consultant for Alkermes, Inc., and works on a project supported by Indivior, Inc. Dr. Tegge would like to report work on a project supported by Indivior, Inc. Candice L. Dwyer, William H. Craft, and Yu-Hua Yeh have no disclosures to report.

Footnotes

This work has not been presented at any meeting or conference or otherwise disseminated prior to this submission. The data, study materials, and analysis code are available upon request by emailing the corresponding author. This study was not preregistered.

References

  1. Albein-Urios N, Martinez-González JM, Lozano O, Clark L, & Verdejo-García A (2012). Comparison of impulsivity and working memory in cocaine addiction and pathological gambling: Implications for cocaine-induced neurotoxicity. Drug and Alcohol Dependence, 126(1–2), 1–6. [DOI] [PubMed] [Google Scholar]
  2. Amlung M, Marsden E, Holshausen K, Morris V, Patel H, Vedelago L, Naish KR, Reed DD, & McCabe RE (2019). Delay Discounting as a Transdiagnostic Process in Psychiatric Disorders: A Meta-analysis. JAMA Psychiatry. 10.1001/jamapsychiatry.2019.2102 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Amlung M, Petker T, Jackson J, Balodis I, & MacKillop J (2016). Steep discounting of delayed monetary and food rewards in obesity: a meta-analysis. Psychological Medicine, 46(11), 2423–2434. [DOI] [PubMed] [Google Scholar]
  4. Amlung M, Vedelago L, Acker J, Balodis I, & MacKillop J (2017). Steep delay discounting and addictive behavior: a meta-analysis of continuous associations. Addiction, 112(1), 51–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Athamneh LN, Brown J, Stein JS, Gatchalian KM, LaConte SM, & Bickel WK (2021). Future thinking to decrease real-world drinking in alcohol use disorder: Repairing reinforcer pathology in a randomized proof-of-concept trial. Experimental and Clinical Psychopharmacology. 10.1037/pha0000460 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Athamneh LN, Stein JS, & Bickel WK (2019). Narrative theory III: Evolutionary narratives addressing mating motives change discounting and tobacco valuation. Experimental and Clinical Psychopharmacology. 10.1037/pha0000315 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Audrain-McGovern J, Rodriguez D, Epstein LH, Cuevas J, Rodgers K, & Wileyto EP (2009). Does delay discounting play an etiological role in smoking or is it a consequence of smoking? Drug and Alcohol Dependence, 103(3), 99–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Babor TF, Higgins-Biddle JC, Saunders JB, & Monteiro MG (2001). The Alcohol Use Disorders Identification Test Guidelines for Use in Primary Care, 2nd edition. World Health Organization. http://apps.who.int/iris/bitstream/handle/10665/67205/WHO_MSD_MSB_01.6a.pdf;jsessionid=D6AB692BE0C0BE59F2CE71B98A54C170?sequence=1 [Google Scholar]
  9. Barlow P, McKee M, Reeves A, Galea G, & Stuckler D (2017). Time-discounting and tobacco smoking: a systematic review and network analysis. International Journal of Epidemiology, 46(3), 860–869. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bickel WK, Athamneh LN, Basso JC, Mellis AM, DeHart WB, Craft WH, & Pope D (2019). Excessive discounting of delayed reinforcers as a trans-disease process: Update on the state of the science. Current Opinion in Psychology, 30, 59–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bickel WK, Jarmolowicz DP, Mueller ET, Koffarnus MN, & Gatchalian KM (2012). Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. Pharmacology & Therapeutics, 134(3), 287–297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Bickel WK, Landes RD, Kurth-Nelson Z, & Redish AD (2014). A Quantitative Signature of Self-Control Repair: Rate-Dependent Effects of Successful Addiction Treatment. Clinical Psychological Science, 2(6), 685–695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bickel WK, Yi R, Kowal BP, & Gatchalian KM (2008). Cigarette smokers discount past and future rewards symmetrically and more than controls: is discounting a measure of impulsivity? Drug and Alcohol Dependence, 96(3), 256–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Bulley A, & Gullo MJ (2017). The influence of episodic foresight on delay discounting and demand for alcohol. Addictive Behaviors, 66, 1–6. [DOI] [PubMed] [Google Scholar]
  15. Businelle MS, McVay MA, Kendzor D, & Copeland A (2010). A comparison of delay discounting among smokers, substance abusers, and non-dependent controls. Drug and Alcohol Dependence, 112(3), 247–250. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Chmielewski M, & Kucker SC (2020). An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results. Social Psychological and Personality Science, 11(4), 464–473. [Google Scholar]
  17. Craft WH, Tegge AN, & Bickel WK (2020). Episodic future thinking reduces chronic pain severity: A proof of concept study. Drug and Alcohol Dependence, 215, 108250. [DOI] [PubMed] [Google Scholar]
  18. Craft WH, Tegge AN, Freitas-Lemos R, Tomlinson DC, & Bickel WK (2022). Are poor quality data just random responses?: A crowdsourced study of delay discounting in alcohol use disorder. Experimental and Clinical Psychopharmacology. 10.1037/pha0000549 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Crump MJC, McDonnell JV, & Gureckis TM (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PloS One, 8(3), e57410. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Dallery J, & Raiff BR (2007). Delay discounting predicts cigarette smoking in a laboratory model of abstinence reinforcement. Psychopharmacology, 190(4), 485–496. [DOI] [PubMed] [Google Scholar]
  21. DeHart WB, Friedel JE, Berry M, Frye CCJ, Galizio A, & Odum AL (2020). Comparison of delay discounting of different outcomes in cigarette smokers, smokeless tobacco users, e-cigarette users, and non-tobacco users. Journal of the Experimental Analysis of Behavior, 114(2), 203–215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. DeHart WB, Mellis AM, Kaplan BA, Pope DA, & Bickel WK (2019). The Experimental Tobacco Marketplace: Narratives engage cognitive biases to increase electronic cigarette substitution. Drug and Alcohol Dependence, 197, 203–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Du W, Green L, & Myerson J (2002). Cross-Cultural Comparisons of Discounting Delayed and Probabilistic Rewards. The Psychological Record, 52(4), 479–492. [Google Scholar]
  24. Finn PR, & Hall J (2004). Cognitive ability and risk for alcoholism: short-term memory capacity and intelligence moderate personality risk for alcohol problems. Journal of Abnormal Psychology, 113(4), 569–581. [DOI] [PubMed] [Google Scholar]
  25. Freitas-Lemos R, Tegge AN, Craft WH, Tomlinson DC, Stein JS, & Bickel WK (2022). Understanding data quality: Instructional comprehension as a practical metric in crowdsourced investigations of behavioral economic cigarette demand. Experimental and Clinical Psychopharmacology, 30(4), 415–423. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Johnson MW, & Bickel WK (2008). An algorithm for identifying nonsystematic delay-discounting data. Experimental and Clinical Psychopharmacology, 16(3), 264–274. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kennedy R, Clifford S, Burleigh T, Waggoner PD, Jewell R, & Winter NJG (2020). The shape of and solutions to the MTurk quality crisis. Political Science Research and Methods, 8(4), 614–629. [Google Scholar]
  28. Koffarnus MN, & Bickel WK (2014). A 5-trial adjusting delay discounting task: accurate discount rates in less than one minute. Experimental and Clinical Psychopharmacology, 22(3), 222–228. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, & Munafò MR (2011). Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology, 216(3), 305–321. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. MacKillop J, Miranda R Jr, Monti PM, Ray LA, Murphy JG, Rohsenow DJ, McGeary JE, Swift RM, Tidey JW, & Gwaltney CJ (2010). Alcohol demand, delayed reward discounting, and craving in relation to drinking and alcohol use disorders. Journal of Abnormal Psychology, 119(1), 106–114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Madden GJ, Petry NM, Badger GJ, & Bickel WK (1997). Impulsive and self-control choices in opioid-dependent patients and non-drug-using control participants: drug and monetary rewards. Experimental and Clinical Psychopharmacology, 5(3), 256–262. [DOI] [PubMed] [Google Scholar]
  32. Mazur JE (1987). An adjusting procedure for studying delayed reinforcement. The Effect of Delay and of Intervening Events on Reinforcement Value, 344, 55–73. [Google Scholar]
  33. Mellis AM, Athamneh LN, Stein JS, Sze YY, Epstein LH, & Bickel WK (2018). Less is more: Negative income shock increases immediate preference in cross commodity discounting and food demand. Appetite, 129, 155–161. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Mellis AM, & Bickel WK (2020). Mechanical Turk data collection in addiction research: utility, concerns and best practices. Addiction. 10.1111/add.15032 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Moody LN, Tegge AN, & Bickel WK (2017). Cross-commodity delay discounting of alcohol and money in alcohol users. The Psychological Record, 67(2), 285–292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Odum AL, Becker RJ, Haynes JM, Galizio A, Frye CCJ, Downey H, Friedel JE, & Perez DM (2020). Delay discounting of different outcomes: Review and theory. Journal of the Experimental Analysis of Behavior, 113(3), 657–679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Paolacci G, Chandler J, & Ipeirotis PG (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), 411–419. [Google Scholar]
  38. Phung QH, Snider SE, Tegge AN, & Bickel WK (2019). Willing to work but not to wait: Individuals with greater alcohol use disorder show increased delay discounting across commodities and less effort discounting for alcohol. Alcoholism, Clinical and Experimental Research, 43(5), 927–936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. R Core Team. (2021). R: A Language and Environment for Statistical Computing. https://www.R-project.org/
  40. Reynolds B, Richards JB, Horn K, & Karraker K (2004). Delay discounting and probability discounting as related to cigarette smoking status in adults. Behavioural Processes, 65(1), 35–42. [DOI] [PubMed] [Google Scholar]
  41. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, & Grant M (1993). Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption--II. Addiction, 88(6), 791–804. [DOI] [PubMed] [Google Scholar]
  42. Snider SE, Deshpande HU, Lisinski JM, Koffarnus MN, LaConte SM, & Bickel WK (2018). Working Memory Training Improves Alcohol Users’ Episodic Future Thinking: A Rate-Dependent Analysis. Biological Psychiatry. Cognitive Neuroscience and Neuroimaging, 3(2), 160–167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Snider SE, Mellis AM, Poe LM, Kocher MA, Turner JK, & Bickel WK (2020). Reinforcer pathology: Narrative of hurricane-associated loss increases delay discounting, demand, and consumption of highly palatable snacks in the obese. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors, 34(1), 136–146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Snider SE, Turner JK, McClure SM, & Bickel WK (2021). Reinforcer pathology in cocaine use disorder: Temporal window determines cocaine valuation. Drug and Alcohol Dependence, 225, 108795. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Stein JS, Wilson AG, Koffarnus MN, Daniel TO, Epstein LH, & Bickel WK (2016). Unstuck in time: episodic future thinking reduces delay discounting and cigarette smoking. Psychopharmacology, 233(21–22), 3771–3778. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Stokel-Walker C (2018, August 10). Bots on Amazon’s Mechanical Turk are ruining psychology studies. New Scientist https://www.newscientist.com/article/2176436-bots-on-amazons-mechanical-turk-are-ruining-psychology-studies/
  47. Strickland JC, Lee DC, Vandrey R, & Johnson MW (2021). A systematic review and meta-analysis of delay discounting and cannabis use. Experimental and Clinical Psychopharmacology, 29(6), 696–710. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Sze YY, Stein JS, Bickel WK, Paluch RA, & Epstein LH (2017). Bleak Present, Bright Future: Online Episodic Future Thinking, Scarcity, Delay Discounting, and Food Demand. Clinical Psychological Science, 5(4), 683–697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Tang J, Chrzanowski-Smith OJ, Hutchinson G, Kee F, & Hunter RF (2019). Relationship between monetary delay discounting and obesity: a systematic review and meta-regression. International Journal of Obesity, 43(6), 1135–1146. [DOI] [PubMed] [Google Scholar]
  50. Vuchinich RE, & Simpson CA (1998). Hyperbolic temporal discounting in social drinkers and problem drinkers. Experimental and Clinical Psychopharmacology, 6(3), 292–305. [DOI] [PubMed] [Google Scholar]
  51. Washio Y, Higgins ST, Heil SH, McKerchar TL, Badger GJ, Skelly JM, & Dantona RL (2011). Delay discounting is associated with treatment response among cocaine-dependent outpatients. Experimental and Clinical Psychopharmacology, 19(3), 243–248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Weinsztok S, Brassard S, Balodis I, Martin LE, & Amlung M (2021). Delay Discounting in Established and Proposed Behavioral Addictions: A Systematic Review and Meta-Analysis. Frontiers in Behavioral Neuroscience, 15, 786358. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Yeh Y-H, Freitas-Lemos R, Craft WH, & Bickel WK (2022). The relationship between nonsystematic delay discounting and low-quality survey responses in a sample of smokers: ROC curve analysis. Experimental and Clinical Psychopharmacology. 10.1037/pha0000584 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Yoon JH, & Higgins ST (2008). Turning k on its head: comments on use of an ED50 in delay discounting research. Drug and Alcohol Dependence, 95(1–2), 169–172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Yoon JH, Higgins ST, Heil SH, Sugarbaker RJ, Thomas CS, & Badger GJ (2007). Delay discounting predicts postpartum relapse to cigarette smoking among pregnant women. Experimental and Clinical Psychopharmacology, 15(2), 176–186. [DOI] [PubMed] [Google Scholar]

RESOURCES