Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Dec 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2020 Apr 20;29(6):696–710. doi: 10.1037/pha0000378

A Systematic Review and Meta-Analysis of Delay Discounting and Cannabis Use

Justin C Strickland 1, Dustin C Lee 1, Ryan Vandrey 1, Matthew W Johnson 1
PMCID: PMC8376219  NIHMSID: NIHMS1731610  PMID: 32309968

Abstract

Delay discounting reflects the systematic reduction in the value of a consequence by delay to delivery. Theoretical and empirical work suggests that delay discounting is a key behavioral mechanism underlying substance use disorder. Existing work on cannabis use, however, is mixed with many studies reporting null results. The purpose of this review was to provide an in-depth assessment of the association between delay discounting and cannabis use. We conducted meta-regression analyses to determine the omnibus correlation between delay discounting and cannabis use, and to evaluate task-based and sample-based moderators. Studies included evaluated an association between delay discounting and cannabis quantity-frequency or severity measures in human participants (27 studies, 61 effect sizes, 24,782 participants). A robust variance estimation method was used to account for dependence among effect sizes. A significant, but small, omnibus effect was observed (r = .082) in which greater cannabis use frequency or severity was associated with greater discounting. Incentive structure and outcome type were each significant moderators in a multiple moderator model such that incentivized tasks correlated with severity measures showed stronger associations (r = .234) than hypothetical tasks correlated with quantity-frequency measures (r = .029). Comparisons to historic effect size data supported the hypothesis that, at present, the relationship between cannabis use and delay discounting appears empirically smaller than for other substances. Future work should explore theoretical rationales explaining this modest relationship involving cannabis use and delay discounting, such as reflecting the smaller magnitude of perceived long-term clinical outcomes associated with cannabis compared to other substances.

Keywords: Behavioral Economics, Delayed Reward Discounting, Marijuana, Meta-Regression, Time Preferences

Introduction

Delay discounting describes the systematic devaluation of a consequence as a function of its delay (Ainslie, 1975; Chung & Herrnstein, 1967; Rachlin & Green, 1972). In the context of delayed reward discounting, for example, this concept is best exemplified by an individual’s preference for a smaller, sooner reinforcer (e.g., $500 now) over a larger, but delayed reinforcer (e.g., $1000 in 5 years). A variety of delay discounting tasks (e.g., task-based, questionnaire-based) and analytic techniques (e.g., hyperbolic functions, area under the curve) have been developed to index these preferences and the rate at which an outcome’s value (or utility; Killeen, 2009) is reduced by delay (L. Green & Myerson, 2004; Kirby, Petry, & Bickel, 1999; Myerson, Green, & Warusawitharana, 2001; Rachlin, 2006; Richards, Zhang, Mitchell, & de Wit, 1999). A consistent conclusion across these approaches is that delay discounting plays a central role in addictive behaviors and, more broadly, in behavioral health (see reviews in Amlung, Marsden, et al., 2019; Amlung, Vedelago, Acker, Balodis, & MacKillop, 2017; Bickel, Jarmolowicz, Mueller, Koffarnus, & Gatchalian, 2012; Koffarnus & Kaplan, 2018; MacKillop et al., 2011). Specifically, these studies have found that individuals with a substance use disorder show greater discounting of delayed rewards indicating a steeper devaluation of future consequences. Such findings have been taken to indicate that delay discounting represents a transdiagnostic marker underlying clinical conditions for which undesirable behaviors with immediate benefits are preferred despite serious, but delayed, health consequences (e.g., the decision to smoke tobacco cigarettes now despite later health consequences; Bickel et al., 2012; Bickel, Koffarnus, Moody, & Wilson, 2014).

Cannabis use has been notably less consistent in its relationship to delay discounting with comparably little attention in the delay discounting literature until recent years. Studies attempting to relate cannabis use to delay discounting have been largely equivocal when assessed based on their qualitative outcomes. In one of the formative studies on the relationship between cannabis use and delay discounting, for example, individuals with Cannabis Use Disorder (CUD) did not significantly differ on a task-based delay discounting task from controls or from individuals reporting prior (but not current) cannabis use (effect size at d = 0.20; M. W. Johnson et al., 2010). Similarly, theoretical perspectives describing the role of delay discounting as a relevant behavioral mechanism have, at best, briefly recognized that the pattern of results for cannabis use seem to differ from other substances with limited rationale as to why (Bickel et al., 2012; Bickel et al., 2014; MacKillop, 2013). This is not to say that significant associations between cannabis use and delay discounting have not been described (Aston, Metrik, Amlung, Kahler, & MacKillop, 2016; Heinz, Peters, Boden, & Bonn-Miller, 2013; Lopez-Vergara, Jackson, Meshesha, & Metrik, 2019; Strickland, Lile, & Stoops, 2017). In fact, several of these studies with positive outcomes have identified delay discounting as a specific and unique predictor of cannabis severity measures above and beyond other behavioral economic mechanisms (e.g., behavioral economic demand; Aston et al., 2016; Strickland et al., 2017).

There are several potential reasons for the discrepancies concerning cannabis use in the delay discounting literature. The effect sizes relating cannabis use and delay discounting could be smaller than those for other substances and, therefore, unlikely to be detected without adequate statistical power in large sample size studies. Alternatively, there may be important task-specific or sample-specific variables that moderate the association between cannabis use and delay discounting with substantive variability across studies resulting in discrepant effect size estimates. This idea is supported by studies for which a unique and specific role of delay discounting in predicting cannabis use severity has been observed (e.g., Aston et al., 2016; Strickland et al., 2017). Similarly, differences in motives for use (e.g., use for therapeutic purposes versus recreational purposes) or the products used (e.g., cannabidiol [CBD] dominant products versus THC dominant products) may explain the variations in the associations observed. Finally, it is possible that there is no such discrepancy, and apparent inconsistencies are due to measurement error in small sample size studies as well as a focus on comparisons based on qualitative outcomes rather than effect sizes estimates.

Two meta-analyses have been conducted to date that are particularly germane to this relation between cannabis use and delay discounting. The first of these meta-analyses evaluated differences in delay discounting between individuals reporting substance use and controls (MacKillop et al., 2011). This analysis found a medium omnibus effect size difference (d = .58) with a modestly higher effect when evaluating clinical versus subclinical samples (d = 0.61 versus 0.45). However, only one study with participants reporting cannabis use was included, which precluded evaluation of differences in effect size based on cannabis use compared to other substance use history. The second of these meta-analyses used a similar framework to evaluate continuous associations between delay discounting and quantity-frequency or severity measures (Amlung et al., 2017). This study found a small omnibus effect size correlation (r = .14) with no statistically significant differences across substance use types. Cannabis use variables, specifically, were associated with quantity-frequency and severity measures at a small effect size (r = .10) when evaluated across 9 studies with 12 effect sizes and 2654 unique participants. This still relatively small number of studies and the scope of the meta-analysis, nevertheless, precluded the testing of specific moderators of the discounting-to-cannabis association.

The purpose of the present meta-analysis was to provide an updated and focused assessment on the relationship between delay discounting and cannabis use. The aims of this analysis were to determine an omnibus estimate and evaluate possible task-based and sample-based moderators of this discounting-cannabis relationship.

Method

Study Selection

Empirical studies were initially identified through searches of PubMed and ProQuest Central databases for peer-reviewed journal articles published as of 10 July 2019. Search terms included the following keywords: discounting AND (cannabis OR marijuana). Additional articles were identified through review of previous meta-analyses on delay discounting and substance use (Amlung et al., 2017; MacKillop et al., 2011). We re-reviewed these databases on 14 November 2019 following the publication of a large sample size study on delay discounting and cannabis use during the preparation of this meta-analysis, but following our initial search (Patel, Naish, & Amlung, 2019). Figure 1 provides an overview of the study selection process. This was a systematic review and meta-analysis of existing literature and therefore not considered human subjects research.

Figure 1.

Figure 1.

PRISMA Study Inclusion/Exclusion Diagram.

The primary inclusion criteria were: 1) study included a bivariate association between delay discounting (money or cannabis delay discounting) and cannabis use variables, 2) human participants research, and 3) peer-reviewed publication in an English language journal. Cannabis use variables included quantity-frequency (e.g., days of past month use, grams per week) and severity (e.g., CUD) measures. Studies were also included if they reported outcomes taken at non-contemporaneous timepoints (e.g., naturalistic longitudinal studies) as long as there were no experimental manipulations that occurred between assessments (Janssen et al., 2015; Khurana, Romer, Betancourt, & Hurt, 2017; Kim-Spoon et al., 2019). Only studies on delayed reward discounting were used given that only one article was identified evaluating delayed loss discounting (Mejía-Cruz, Green, Myerson, Morales-Chainé, & Nieto, 2016).

Coding Procedures and Computation of Effect Sizes

Effect sizes were coded from each study as the correlation coefficient indexing the association between delay discounting and cannabis use variables. Studies reporting effect sizes as Cohen’s d were converted to correlation coefficients prior to analysis using available information. All effect sizes were recorded such that positive correlations indicated a relationship between greater delay discounting and more frequent or severe cannabis use (i.e., correlations involving AUC values were reverse coded for directional consistency). Meta-analyses were conducted on the Fisher’s z transformed values to provide for variance-stabilized estimates (Borenstein, Hedges, Higgins, & Rothstein, 2011; Viechtbauer, 2010). All estimates were back-transformed following meta-analysis for presentation as correlation coefficients. Effect sizes were interpreted based on convention (r = .1 [small], .3 [medium], .5 [large]) (Cohen, 1992).

Moderator Variables

Incentive Structure.

Incentive structure was coded as a dichotomous variable of hypothetical (k = 54) versus incentivized (k = 7) tasks. Incentivized choices were typically collected using the Experiential Discounting Task (EDT) or by using random selection of one task choice for reward delivery.

Discounting Measure.

Discounting measures were coded as area under the curve (AUC; k = 15), discounting rate (k = 44), and single indifference point (k = 2). Because only two effect sizes were available with the indifference point measure, moderation analyses focused on comparing AUC and discounting rate measures. All studies that modeled a discounting parameter (as opposed to using non-modeled ones as in the 5-choice procedure) used Mazur’s hyperbolic decay model (Mazur, 1987) which contains a single free parameter.

Task Type.

Task types included the Monetary Choice Questionnaire (MCQ) (Kirby & Maraković, 1996; Kirby et al., 1999), Adjusting-Amount Titration Tasks (M. W. Johnson & Bickel, 2002; Richards et al., 1999), 5-Choice Adjusting-Delay Task (Koffarnus & Bickel, 2014), EDT (Reynolds & Schiffbauer, 2004), and Single Item Indifference Point Estimation (Romer, Duckworth, Sznitman, & Park, 2010). For moderation analysis purposes, tasks were collapsed into questionnaire-based (i.e., those that were not adaptive; k = 11) and titration-based (i.e., those that adaptive options based on responses; k = 48). Single-item indifference point estimation tasks were not included given that they did not fit into these two groups and only included two effect size estimates precluding addition as a third group (i.e., 2 studies with k = 2 effect size estimates).

Cannabis Use Outcome.

Cannabis use outcome was coded dichotomously as quantity-frequency (k = 42) and severity (k = 19). Examples of quantity-frequency variables included past month days of cannabis use and grams of cannabis used per week. Examples of severity variables included the Marijuana Problem Scale (MPS; Stephens, Roffman, & Curtin, 2000) and number of DSM criteria for CUD endorsed (DSM-IV or DSM-5 depending on when the study was conducted).

Large-Later Reward Magnitude.

Magnitude of the larger-later reward was included as a model covariate given evidence that greater magnitude delayed rewards result in less delay discounting (i.e., more choices for the delayed reinforcer) (Baker, Johnson, & Bickel, 2003; L. Green, Myerson, & McFadden, 1997; M. W. Johnson & Bickel, 2002) and may differentially impact observed associations with substance use variables (Mellis, Woodford, Stein, & Bickel, 2017). This variable was treated continuously with the exception that one study with a particularly high delayed value ($40,000) was recoded to the next highest in the analyzed studies ($1,000). Values were mean centered prior to analysis for interpretation of the intercept coefficient. This moderator was selected post-hoc during the revision process.

Discounted Commodity.

Commodity type, including money (k = 51) and cannabis (k = 10), was initially explored as a moderator. However, commodity type could not be evaluated because the limited number of effect sizes for cannabis and studies reporting those effects (n = 4) resulted in unstable estimates with degrees of freedom less than those recommended for the small-sample adjusted RVE model.1

Sample-Based Moderators.

Sample-based moderators included age, sex, and comorbidities in the sample. Age and sex were treated as continuous moderators and were the average age and percentage of female participants reported in the sample, respectively. These continuous moderators were mean centered prior to analysis for valid interpretation of the intercept coefficient. The comorbidities moderator distinguished studies that only targeted healthy normal or cannabis use populations (k = 48) from those that included samples specifically targeted for other psychopathology (k = 13). This moderator was selected post-hoc during the revision process.

Comparisons to Historic Effect Size Data

A supplemental analysis was conducted to provide a comparison of the analyzed cannabis effect sizes to historic effect sizes for other substances. Effects sizes reported in the most recent meta-analysis on substance use and delay discounting were gathered (Amlung et al., 2017) and used to conduct this analysis. Specifically, a meta-regression was conducted comparing effect sizes for cannabis to all other substances included in that historic dataset (i.e., alcohol, cigarettes, opioids, and stimulants). Within-study clustering was considered in the meta-analytic model, as appropriate (see more in Meta-Analytic Methods).

Meta-Analytic Methods

An RVE meta-regression method was used to calculate omnibus effect size estimates and test proposed moderators (Hedges, Tipton, & Johnson, 2010; Tipton, 2015). The RVE method is a recently developed meta-regression method that allows for the incorporation of dependent effect size measures without violation of independence assumptions (i.e., allows for use of multiple effect size measures from the same participants). Briefly, the RVE approach uses robust standard errors based on heteroskedasticity-robust estimates and clustered methods as traditionally applied in general linear models (for further technical description of this approach see Hedges et al., 2010; Tipton, 2015). We used a modified form of the RVE method that applies a small-sample size adjustment for residuals and degrees of freedom based on a Satterthwaite approximation to improve protection of intended type I error rates (Tipton, 2015).

Coefficients in these meta-regression models are interpreted as follows: the intercept reflects the correlation for the reference group (when all parameters are zero) and the moderator coefficients reflects the change in the correlation with a one-unit change in the moderator (or the presence of that variable if it is dichotomous). For example, in a model testing moderation by outcome type with a dummy coded variable of quantity-frequency, the intercept would be the estimated correlation for severity outcomes (reference group) and the intercept plus the model coefficient would be the estimated correlation for quantity-frequency outcomes.

Additional analyses were conducted using a standard approach for handling dependent effect size measures. Specifically, this approach averaged effect size estimates from individual studies with multiple effects and conducted a random-effects meta-regression analysis on those values. Furthermore, because traditional publication bias measures (e.g., Egger’s plot for funnel asymmetry) have not yet been widely adopted or validated for RVE models and are not available in publicly available statistical packages, we conducted tests of publication bias using the averaged effect size model.

All tests were conducted in R statistical language with RVE models evaluated using the robumeta package (Fisher & Tipton, 2015) and traditional meta-analysis models tested using the metafor package (Viechtbauer, 2010). Type I error rates were set at .05 and all tests were two-tailed.2

Results

Overview of Studies

A total of 27 studies were identified, comprising k = 61 individual effect sizes in 24,782 unique participants. Characteristics of individual studies are summarized in Table 1. See Supplemental Materials for more detailed information regarding the individual effect size estimates included in this meta-analysis.

Table 1.

Summary of Included Studies

First N Sample HYP Commodity LL Amount Task Delay Outcome Cannabis Use Outcome

Amlung & MacKillop 2014 918 Daily Tobacco Cigarette Use Yes Money $204 (avg) MCQ sqrt(k) -Cannabis Use Frequency (Never, Monthly, Weekly)
Aston et al. 2016 83 Non-Treatment Seeking Frequent Cannabis Use Yes Money $10 Titration AA AUC -DSM-IV Cannabis Dependence Symptom Count
-% Cannabis Use Days (Past 60 Days)
Bobova et al. 2009 393 AUD, Conduct Disorder, and Controls No Money $50 Titration AA log(k) -Lifetime Cannabis Use Problems (SSAGA-II)
- Cannabis Use Days/Week (Past 3 Months)
Crane et al. 2013 69 Non-Treatment Seeking Frequent Cannabis Use Yes Money $55 (avg) MCQ log(k) -Lifetime Use Total Grams
-Past Year Use Total Grams
-Past Month Use Total Grams
Dennhardt et al. 2015 97 Undergraduates with One or More Past Month Alcohol Binge Day Yes Money $100 MICT log(k) -Past Month Cannabis Use Days
-MPS
Ellingson et al. 2018 1038 Undergraduates No Money $0.30 EDT ln(k) -Number of Joints in Last 6 Months
Finn et al. 2015 542 Community Adults Stratified on Externalizing Pathology No Money $50 Titration AA log(k) -Lifetime Cannabis Use Problems (SSAGA-II)
Heinz et al. 2013 73 Treatment-Seeking Military Veterans with CUD Yes Money $1,000 Titration AA log(k) -Quantity x Frequency (Past 90 days)
-Daily Cannabis Use (Yes/No)
-MPS
Janssen et al. 2015 284 Secondary School Children Yes Money NR Titration AA AUC -Lifetime Cannabis Use (Yes/No)
Johnson et al. 2010 48 CUD and Controls Yes Money $1,000 Titration AA log(k) -Presence of DSM-IV Cannabis Dependence (i.e., Cannabis Dependence group versus control group)
Khurana et al. 2017 290 Community Adolescents (Philadelphia Trajectory Study) Yes Money $100 Single
Item
IP -DSM-5 CUD Symptom Count
Kim-Spoon et al. 2015 106 Community
Adolescents
No Money $55 MCQ log(k) -Cannabis Use Frequency (6-point Likert)
Kim-Spoon et al. 2019 157 Community
Adolescents
Yes Money $100 Titration AA log(k) -Cannabis Use Frequency (6-point Likert)
Lopez-Vergara et al. 2019 104 Non-Treatment Seeking Frequent Cannabis Use Yes Money $10 Titration AA AUC -DSM-IV Cannabis Abuse Symptom Count
-DSM-IV Cannabis Dependence Symptom Count
-MPS
-% Cannabis Use Days (Past 60 Days)
-Times Cannabis Used/Day
Oshri et al. 2018 1011 United States Adults (mTurk) Yes Money $55 (avg) MCQ log(k) -Cannabis Use Frequency (4-point Likert)
Patel et al. 2019 2857/
1069a
United States Adults (mTurk) in States with Legalized Cannabis Yes Money;
Cannabis
$10/$100/
10g
Titration
5C
ln(k) -Cannabis Use Frequency (4-point Likert)
-Cannabis Use Disorder Identification Test
Peters et al. 2013 93 Treatment-Seeking or Referred CUD No Money $0.30 EDT ln(k) -Days Used Cannabis 28 Days Prior to Treatment
Petker et al. 2019 1121 Human Connectome Project Yes Money $200/$40,000 Titration AA AUC -THC Positive Screen
-DSM-IV Cannabis Abuse or Dependence
-Lifetime Cannabis Use
Reynolds & Fields 2012 141 Adolescent Smoking, Experimenter, and Control No Money $10 Titration AA AUC -Past 6 Month Cannabis Use Frequency (6 -point Likert)
Romer et al. 2010 900 NASY National Survey (2005) Yes Money $1,000 Single Item IP -Lifetime Cannabis Use Frequency (6 - point Likert)
Sanchez-Roige et al. 2018 13067/12805b 23andMe Sample (European Ancestry) Yes Money $55 (avg) MCQ log(k) -Days Cannabis Use (Past 30 Days) -Days of Cannabis Use (Heaviest Lifetime 30 Day Period)
Stanger et al. 2012 164/163c T reatment-Interested Adolescents with CUD Yes Money; Cannabis $100/$1,000 Titration AA ln(k) -Past Month Marijuana Use Days
Stea et al. 2011 217 Undergraduates Yes Money $1,000 Titration AA AUC -ASSIST (Cannabis Scale)
Strickland et al. 2017 64 Recent Cannabis Use History (mTurk) Yes Money;
Cannabis
$1,000 Titration
5C
log(k) -Cannabis Grams/Week (Past 30 Days)
-Past Month Cannabis Use Days
-DSM-IV Cannabis Dependence Symptom Count
Strickland et al. 2019 76 Past Year NMPO Use History (mTurk) Yes Money;
Cannabis
$1,000 Titration
5C
log(k) -Cannabis Grams/Week (Past 30 Days)
-Past Month Cannabis Use Days
-DSM-5 CUD Symptom Count
Thamotharan et al. 2017 139 Undergraduates and Community Young Adults Yes Money $10 Titration AA AUC -Past 6 Month Cannabis Use Frequency (6 -point Likert)
VanderBroek et al. 2016 730 United States Adults (mTurk) Yes Money $303 (ava) MCQ log(k) -Past 3 Month Cannabis Use Frequency (5-point Likert)

Note. AUC = area under the curve; HYP = hypothetical; IP = indifference point; mTurk = Mechanical Turk.

EDT = Experiential Discounting Task; MCQ = Monetary Choice Questionnaire; MICT = multiple item choice task; Single Item = measure of single indifference point; Titration 5C = 5-Choice Adjusting Delay Task; Titration AA = Adjusting-Amount Titration Tasks; log(k) = base 10 logarithm transformed; ln(k) = natural logarithm transformed.

a

2857 for money and 1069 for cannabis

b

13067 for past 30-day cannabis use frequency and 12805 for lifetime heaviest 30-day cannabis use frequency

c

164 for money and 163 for cannabis

Omnibus Meta-Analysis of Discounting-Cannabis Association

The omnibus RVA meta-analysis indicated a significant and small effect size correlation between delay discounting and cannabis use variables, r = .082, p < .001. A summary of the included effect sizes in this estimate are plotted in Figure 2. Estimates of heterogeneity indicated substantive variation in the omnibus estimate (I2 = 78.2%). A consistent omnibus estimate was observed when collapsing effect sizes into a single average estimate for each study, r = .082, p < .001 (Table 2 right column).

Figure 2.

Figure 2.

Forest plot for omnibus effect size estimate. Effect sizes are divided into quantity-frequency (QF) measures (top half) and severity measures (bottom half) and sorted by year of publication. Values represent correlation coefficient and 95% confidence intervals. Size of the point estimate reflects weighting based on sample size.

Table 2.

Summary of Meta-Analysis

Robust Variance Estimation (RVE) Meta-Analysis Average Effect Size Meta-Analysis

N 24,782 24,782
K 61 27
Studies 27 27
Effect Size
Omnibus r (95%CI) .082 (.040, .125)*** .082 (.042, .122)***
I 2 78.2% 81.31%
T 2 0.006 0.007
Sensitivity/Publication Bias
Fail-Safe N - 659
Orwin's Fail-Safe N - 27
Egger's Test of Asymmetry - z = 0.408 (p = .68)
Jackknife Estimate Range - r = .072-.093
Trim-Fill r - .082
Task-Based Moderators
Incentive Structure
 Intercept (Incentivized) 0.188 0.188
 Hypothetical −0.141** −0.141***
Discounting Measure
 Intercept (Discounting Parameter) 0.086 0.085
 AUC 0.012 0.011
Task Type
 Intercept (Titration) 0.108 0.107
 Questionnaire −0.058 −0.057
Cannabis Use Outcome Type
 Intercept (Severity) 0.142
 Quantity-Frequency −0.082#
Delayed Magnitude
 Intercept (Mean Magnitude) 0.083
 Quantity-Frequency −0.006
Sample-Based Moderators
Comorbid Clinical
 Intercept (No Sampled Comorbidity) 0.074 0.074
 Mixed Use 0.037 0.038
Age
 Intercept (Mean Age) 0.081 0.084
 Years (Continuous) −0.002 −0.002
Female
 Intercept (Mean % Female) 0.083 0.085
 % Female (Continuous) 0.002 0.002

Note. AUC = area under the curve. Orwin’s Fail-Safe N tested for a half-size reduction in the omnibus effect size estimate. Moderator value represent the meta-regression results and are described by the intercept and moderator tested in each model.

#

p < .10

*

p < .05

**

p < .01

***

p < .001

Publication Bias

Estimates of publication bias are presented in Table 2 (middle section). Egger’s test for funnel plot asymmetry was not statistically significant consistent with a visual inspection of the funnel plot (see Supplemental Materials). Trim and fill methods accordingly showed no change in the effect size estimate. Jackknife analyses suggested some influential cases with minor variation in the range of effect sizes generated using a leave-one-out method. However, this range remained in a reasonably consistent range of small effect sizes centered around the omnibus estimate (range of r values = .072 to .093). The traditional fail-safe N value indicated that a large number of studies would be needed to attenuate the effect to zero. Orwin’s Fail-Safe N indicated a smaller number of studies would be needed to attenuate the omnibus in half, which was consistent with the small omnibus effect size observed.

Moderator Analyses

Results of moderator analyses are presented in Table 2 (bottom section). A significant moderator effect was observed for incentive structure. This effect reflected stronger associations between delay discounting and cannabis use when an incentivized task was used (r = .188 versus r = .047 based on model coefficients). The cannabis use outcome moderator did not reach statistical significance, p = .055, but showed a trend towards stronger associations when severity outcomes were measured compared to quantity-frequency ones (r = .142 versus r = .06 based on model coefficients). When these two moderators were entered into a single multiple moderator model, both effects were uniquely significant, Intercept: b = 0.234; Hypothetical Task: b = −0.134, p = .003; Quantity-Frequency Outcome: b = −0.071, p = .013. Discounting measure, task type, and delayed reward magnitude were not statistically significant moderators, p values > .16. Sample-based moderators of age, sex, and sample composition (i.e., comorbidities sampled) were not statistically significant, p values > .14. An additional moderator test was conducted post-hoc comparing studies recruiting adolescents (less than 18 years old) to those not recruiting these samples and indicated similar results with no significant effect, p = .424. Similar results were observed when using an average effect size analysis (Table 2 bottom-right column).

Comparison to Other Substances

An omnibus RVA meta-analysis was conducted comparing the effect sizes for correlations of delay discounting with cannabis use as reported here to historic meta-analysis effect sizes involving other substances (alcohol, cigarettes, stimulants, and opioids from Amlung et al., 2017). This meta-analysis indicated that the omnibus cannabis effect size was smaller than the omnibus effect sizes pooled across these other substances, Intercept: b = 0.159; Cannabis: b = −0.080, p = .005.

Discussion

Overview of Findings

The purpose of the present meta-analysis was to provide an updated and comprehensive view on the association between delay discounting and cannabis use and to explore task-based and sample-based moderators of this relationship. A small effect size omnibus relationship was observed reflecting a significant, but small relationship with greater delay discounting being associated with more frequent/severe cannabis use behaviors. Incentive structure and outcome type were significant and unique moderators in a multiple moderator test such that tasks using incentivized outcomes with effect estimates involving cannabis severity measures showed stronger associations than hypothetical tasks with effect estimates involving quantity-frequency measures. Comparisons to historic effect size data also suggested that, at present, the omnibus correlation between cannabis use and delay discounting appears smaller than for other common substances (i.e., pooled across alcohol, cigarettes, opioids, and stimulants). These findings provide an up-to-date view on the nature of cannabis in the broader delay discounting literature and, in doing so, identify key gaps that future research should address.

Omnibus Outcomes

The omnibus estimate indicated a small effect size correlation between greater delay discounting and greater cannabis use (r = .082). This estimated effect for cannabis is, for example, over half that of the effect size observed for tobacco cigarettes (r = .17) in a prior meta-analysis evaluating continuous associations between delay discounting and addictive behaviors (Amlung et al., 2017). A supplemental meta-analysis conducted comparing the cannabis effect sizes included in this meta-analysis to those for alcohol, cigarettes, opioids, and stimulants included in this prior meta-analysis indicated a significantly smaller effect size for cannabis use correlations. Although this analysis did not include effect sizes for those other substances that would have been published since the prior meta-analysis was conducted, these findings provide support for the idea that relationship between cannabis use and delay discounting, at present, appears empirically weaker than that observed for other substances considered broadly.

It is difficult to make specific judgements about the clinical utility of an effect size of this magnitude without considering the context in which such associations may be applied. On the one hand, a small effect size may prove clinically useful if it contributes to improved clinical diagnostics. Some clinical work suggests that, for example, baseline delay discounting values may predict cannabis treatment abstinence (e.g., Stanger et al., 2012). However, like the literature on cross-sectional or naturalistic longitudinal behavior, studies evaluating delay discounting as a prognostic indicator are equally mixed (e.g., Heinz et al., 2013; Peters, Petry, Lapaglia, Reynolds, & Carroll, 2013 for null effects) and would benefit from similar meta-analytic aggregation when an appropriate number of studies are available.

On the other hand, a small magnitude effect suggests that delay discounting is unlikely to function as a clear transdiagnostic predictor for all addictive commodities and for all substance-related outcomes. Ideas put forth in reinforcer pathology and related theories have highlighted a compelling set of arguments underlying delay discounting as a behavioral mechanism relevant to substance use disorder (Bickel et al., 2012; Gray & MacKillop, 2015; MacKillop, 2013). Although this suggestion holds for much of the empirical literature, care should be taken to avoid overstating the strength of this evidence, particularly when it comes to cannabis use. The effect size reported here, for example, suggests that, when taken on average, the relation between delay discounting and cannabis use is small, at least when measured using delayed rewards of a monetary nature (see Future Directions section for more details on this issue).

Moderator Outcomes

Recent increases in the number of studies on cannabis use, broadly, and those including delay discounting measures, specifically, provided an opportunity to evaluate task-specific and sample-specific moderators of discounting-cannabis associations with improved statistical power. Although previous meta-analyses have evaluated some of these moderators when collapsing across addictive behaviors, the small number of effect sizes specific to cannabis use (k = 1 in MacKillop et al., 2011 and k = 12 in Amlung et al., 2017) precluded a more fine-grained approach to cannabis relationships. We found consistent with these prior reports that discounting task type was not a significant moderator. We also found that measurement type (AUC versus discounting rate) and sample characteristics were not robust moderators. These findings indicate that the relationship between delay discounting and cannabis use behaviors was generally insensitive to the task selected, the analytic approach applied for discounting measurement, and the overall demographic composition of the sample collected.

Two unique moderators were identified using a multiple moderator test of discounting-cannabis associations – incentive structure and cannabis use measure type. These moderators were uniquely predictive in that each remained statistically significant when combined into a single meta-regression model. In fact, estimates from that model indicated that when a task was incentivized and when a cannabis severity measure was evaluated, the estimated effect size was closer to a moderate effect size (r = .234). The finding that outcomes from incentivized tasks were more closely associated with cannabis use was somewhat surprising given extensive work demonstrating that outcomes from real and hypothetical tasks do not differ (M. W. Johnson & Bickel, 2002; Lawyer, Schoepflin, Green, & Jenks, 2011; Locey, Jones, & Rachlin, 2011; Madden, Begotka, Raiff, & Kastern, 2003; Matusiewicz, Carter, Landes, & Yi, 2013) (but see domain specificity of this in R. M. Green & Lawyer, 2014). It is possible that although this prior work has failed to detect significant differences between task types, that small variations between real and hypothetical tasks are amplified when considered in correlative comparisons with substance use variables. It is also possible that persons reporting cannabis use show a different pattern of effects given that this prior work has not specifically evaluated actual versus hypothetical performance in the context of cannabis use. It is also relevant to consider that some incentivized tasks of an experiential nature capture mechanisms relevant to substance use other than just delay discounting as well as processes within delay discounting that may differ from questionnaire-based, hypothetical tasks. For example, the EDT includes not only components of delay, but also probability discounting. Differences in the experience of opportunity costs associated with reinforcement and the subjective experience of this opportunity cost may also influence the observation of discounting among varied tasks (see discussion in P. S. Johnson, Herrmann, & Johnson, 2015; Paglieri, 2013). Nevertheless, these findings indicate that measurement and correspondence with cannabis use indicators may be improved when using incentivized tasks (M. W. Johnson, 2012; Reynolds & Schiffbauer, 2004).

Delay discounting was also more closely associated with severity measures than quantity-frequency measures. Important to note is that this moderator was outside of statistical significance (p = .055) when considered in an unadjusted model, but did reach statistical significance when considered in a multiple moderator setting. That severity measures showed a likely improved relationship with delay discounting is consistent with evidence from both prior meta-analyses on delay discounting (Amlung et al., 2017; MacKillop et al., 2011) and specific empirical studies on delay discounting and cannabis use (Aston et al., 2016; Strickland et al., 2017). These findings suggest that delay discounting may provide a better measure of clinically relevant use and the consequences associated with that use as compared to the frequency or quantity of consumption alone. This could reflect the fact that delay discounting functionally corresponds to a myopic view of future consequences that underlies many of the consequences of substance use that are captured in severity diagnostics (see similar arguments by Aston et al., 2016). Clinically, these findings indicate that delay discounting may prove better in distinguishing who has (or who will)3 progress to clinically-relevant use as opposed to differentiating those who do (or who will) use cannabis. This outcome is particularly relevant given recent meta-analytic evidence that behavioral economic demand, another behavioral economic measure, is more closely associated with quantity-frequency than severity outcomes for alcohol (Kiselica, Webber, & Bornovalova, 2016) and illicit substances (Strickland, Campbell, Lile, & Stoops, 2020). These findings collectively support reinforcer pathology models that predict unique roles for high reinforcer valuation (i.e., behavioral economic demand) and extreme preferences for immediate reinforcers (i.e., delay discounting) in determining a constellation of substance use behaviors (Bickel, Snider, Quisenberry, & Stein, 2017).

Analytic Contributions

A number of studies included in this meta-analysis contributed multiple effect size estimates from the same participants. Conditions like this present an analytic concern for researchers wishing to maximize the amount of information include while avoiding violating statistical assumptions relevant to dependent observations. Three common options to address these issues have included averaging across estimates from a single group (which averages over relevant information), utilizing complex methods that require knowledge about the covariance structure of the underlying effect sizes (which are rarely readily available), and ignoring this relationship by treating effect sizes as independent (which violates potentially critical assumptions about independence of observations).

We used an alternative approach, the RVA method, to address this statistical dependence. The benefits of this approach included an easy-to-implement method available in standard statistics packages (i.e., R statistical language) that appropriately accounts for the correlated effect size estimates. It is relevant to note that the effects did not substantively differ from an averaged effect size approach suggesting that this within-subject relatedness did not exert a substantive effect on the overall outcomes. Nevertheless, the use of an RVA method allowed us to most appropriately include moderators like outcome measure for which several studies included both quantity-frequency and severity associations. Such RVA methods may provide a benefit for future meta-analyses that include dependent effect sizes (e.g., analysis of drug effects on cognitive performance for which participants provide multiple outcomes, longitudinal/treatment effect meta-analyses with multiple time points).

Future Directions for Cannabis and Discounting

This systematic review of delayed discounting research indicates clear and substantive research gaps relevant to cannabis use. Future work should address these areas to provide a more comprehensive view of the role (or lack thereof) of delay discounting in cannabis use and CUD as well as provide additional insight into the theoretical nature of delay discounting as it relates to addictive behaviors, broadly.

First, the vast majority of studies have focused on money as the discounted commodity with only 4 of 27 studies included in this meta-analysis evaluating delay discounting for cannabis (Patel et al., 2019; Stanger et al., 2012; Strickland et al., 2017; Strickland, Lile, & Stoops, 2019). This is understandable given that monetary delay discounting provides a universal commodity that can be compared between populations with and without the behavioral condition of interest (i.e., in this case cannabis use history). Nevertheless, other research has clearly documented how the use of commodity-relevant discounting measures can improve prediction of substance use behaviors as well as other measures of behavioral health (M. W. Johnson & Bruner, 2012; M. W. Johnson, Herrmann, Sweeney, LeComte, & Johnson, 2017; P. S. Johnson, Sweeney, Herrmann, & Johnson, 2016; Rasmussen, Lawyer, & Reilly, 2010; Strickland et al., 2017; Strickland, Lile, et al., 2019; Tsukayama & Duckworth, 2010). By inference, then, the measurement of delay discounting for cannabis among individuals reporting cannabis use should help improve the prediction of cannabis-relevant outcomes.

Second, existing studies have focused almost exclusively on gains. To date, few studies have evaluated the role of loss or combinations of loss and gain functions. Prior work has shown that delayed loss discounting is similarly sensitive to individual differences in substance use (e.g., Baker et al., 2003). Notably, one of the few studies to evaluate loss discounting found that individuals with cannabis dependence recruited from a residential addiction treatment center discounted a delayed monetary loss more steeply than controls (Mejía-Cruz et al., 2016). Other studies have found that delay interacts in a multiplicative manner with probability to determine the rate of devaluation in other discounted commodities (Cox & Dallery, 2016, 2018; Vanderveldt, Green, & Myerson, 2015). Understanding how cannabis use may interact with these alternative and more complex variations of discounting could prove relevant for revealing a more nuanced and potentially improved view of the discounting-cannabis relationship.

Third, cross-commodity relationships (e.g., Bickel et al., 2011) have received little attention in the discounting literature, broadly, and no published studies have evaluated these relationships with respect to cannabis use, specifically. Cross-commodity discounting procedures present participants with decisions between different commodities that vary in their immediate versus delayed nature (e.g., money now versus cannabis later or cannabis now versus money later). These tasks provide a potentially useful adaptation of the discounting procedure by incorporating more complex environmental contexts that may better match real-world decision-making for choices involving trade-offs between commodities that vary in the delay to their occurrence (e.g., cigarette use now versus better health later as described in the Introduction). To date, successful research has been conducted on cross-commodity discounting with money and drug commodities to include alcohol (Moody, Tegge, & Bickel, 2017), psychomotor stimulants (Bickel et al., 2011; Yoon et al., 2018), e-cigarettes (Pericot-Valverde, Yoon, & Gaalema, 2020), and cigarettes (Mitchell, 2004). Extending such work to cannabis use would also be consistent with other behavioral economic studies that have begun to appreciate the complexity involved in real-world decision making between varied and competing reinforcers available in the environment. For example, work on cross-commodity behavioral economic demand has found that illicit and licit cannabis can serve as asymmetrical substitutes with greater decreases in illicit cannabis demand with licit cannabis availability (Amlung & MacKillop, 2019; Amlung, Reed, et al., 2019). Research on cannabis within a cross-commodity discounting framework may be relevant for assessing similar mechanistic and clinical information (e.g., the decision to use a more readily available illicit product instead of delayed licit product).

Fourth, we identified two likely relevant moderators, but more work is needed to understand other variables that may moderate the association between delay discounting and cannabis use (and substance use generally). This recommendation is made even more salient when considering the substantial heterogeneity that remained even when these two moderators were included in a single meta-regression model (I2 = 61.08%; τ2 = 0.003 in that moderated meta-regression model). This variability is also expected to increase in the coming years within a rapidly evolving landscape surrounding cannabis use. Variations in the regional legal status, medicinal versus recreational use, cannabis product type (e.g. high THC, high CBD, etc.), route of administration, and perceived personal or community norms surrounding cannabis consumption all could influence the association between delay discounting and cannabis consumption. Such features have largely not been captured in existing research either because they were not comprehensively recorded or have only come to recent prominence in use patterns (e.g., product use as more homogenous and/or difficult to standardize and define in prior illicit markets).

Relatedly, few studies have clearly differentiated cannabis use by use patterns such as typical routes of administration and motives for use. This is important because route of administration (e.g., smoked, oral, vaporized) and motives for use (e.g., social, coping) vary widely among individuals who use cannabis and can have important implications for the nature of that use. In the case of administration route, such differences have direct effects on cannabinoid toxicology with corresponding changes in subjective, cognitive, and physiological effects (e.g., Newmeyer, Swortwood, Abulseoud, & Huestis, 2017; Spindle et al., 2018; Vandrey et al., 2017). Similarly, variations in cannabis use motives can influence patterns of cannabis use and use-relevant distress or dependence (e.g., Bonar et al., 2017; Brodbeck, Matter, Page, & Moggi, 2007; Buckner, Zvolensky, & Schmidt, 2012; van der Pol et al., 2013). It is possible that incorporating variations in these variables might better explain when delay discounting is and when delay discounting is not associated with and predictive of cannabis use.

This meta-analysis was limited in other ways besides the directions for future research described above. Although the amount of information was about three times that of existing meta-analyses relevant to delay discounting and cannabis use, there remains a sizable knowledge gap in the study of cannabis in the delay discounting literature. This still limited number of studies did reduce power and prevented the testing of some potentially relevant moderators (i.e., commodity discounted). These analyses also focused on cross-sectional relationships with only a few studies included assessing time-series data of a naturalistic nature. More work is needed evaluating this prospective prediction, particularly a predictive capacity in treatment settings. Existing studies for other substances have suggested that delay discounting likely plays both an etiological role in the progression of substance use as well as changes as a consequence of experience with particular substances (e.g., see discussion in reviews by MacKillop, 2016; Perry & Carroll, 2008). Little of this work has been conducted on cannabis consumption making studies of a longitudinal nature valuable for future work.

The selected moderators captured much of the variability we would expect across discounting procedures to include features like incentive structure, analytic approach, and task type. However, other factors relevant to variability across studies require additional consideration in future work. Comparisons to other substances in the supplemental meta-analysis were limited by heterogeneity in the type of quantity-frequency measures utilized across drug classes. For example, the majority of quantity-frequency measures evaluating nicotine use measured cigarettes per day, whereas the majority of cannabis quantity-frequency measures evaluated use behavior over a larger temporal scope (e.g., past 30 day or past 6-month use). It is possible that variations in the temporal frame of the quantity-frequency measurements rather than the target substance may explain some of the differences observed in the strength of the correlations between these drug classes. Similarly, comorbidity with other substance use or psychopathology was not readily identifiable in many of these studies and our selected moderator likely only captured some of this potential variation. Examining features like comorbidity more clearly will be relevant given prior work demonstrating elevated discounting in individuals with comorbid substance use disorder and other psychopathology (Moody, Franck, & Bickel, 2016) as well as higher discounting among individuals with dual- or tri-substance use than mono-substance use (Moody, Franck, Hatz, & Bickel, 2016). More targeted assessments of how cannabis use may interact with these other forms of psychopathology is therefore warranted.

Conclusion and Theoretical Implications

The current findings provide an updated review on the association between delay discounting and cannabis use behaviors (i.e., quantity-frequency and severity outcomes). The results presented here suggest that, to date, there is a limited relationship between delay discounting and cannabis use that depends on factors like the nature of the discounting task and outcome measured. Despite decades of focused work evaluating the role of delay discounting in substance use disorder, little emphasis has been placed on cannabis use. This is surprising given that cannabis use is the most widely used illicit substance in the world (United Nations Office on Drugs and Crime, 2019). The introduction of medicinal and recreational cannabis laws across the United States and internationally means that the salience of cannabis use in everyday life is only expected to increase in coming years, but also that variance in the types of cannabis products and the lessened stigma associated with using cannabis may impact the relation between cannabis use and delay discounting.

Future work exploring these changes in the marketplace and more specific information about cannabis product use and motives may help to clarify a theoretical rationale for why cannabis use has shown a weaker relationship with cannabis use behaviors. It is possible that the consequences of cannabis use, both those of an immediate and delayed nature, are perceived as minimal relative to other commonly used substances that have shown more robust associations with delay discounting (e.g., tobacco cigarette use). To the extent that selection (i.e., predisposition) plays a role in group differences relevant to delay discounting, this lower perceived harm would help explain the weaker associations observed for cannabis. Similarly, use patterns characterized by strictly therapeutic motives (whether within a legal structure or not) that are not fully captured in samples of participants reporting cannabis use may result in the dilution of an effect that may be stronger among a sample of individuals with CUD without therapeutic intentions. Ultimately, we believe that future work in these understudied areas of the delay discounting literature will be essential for verifying the relative contribution of delay discounting and other discounting mechanisms in cannabis use and, in doing so, shed additional light on the theoretical nature of delay discounting and substance use behaviors, more broadly.

Supplementary Material

Supplemental Material

Public Significance Statement.

How much individuals devalue future consequences (i.e., delay discounting) has been extensively studied for its relevance to substance use disorder. Fewer studies have evaluated associations with cannabis use and those that exist present mixed evidence. This meta-analysis summarizes this literature and finds that the association between delay discounting and cannabis use is, on average, small and is greater when evaluating severity of cannabis use with discounting assessed by incentivized tasks as compared to when evaluating use rates with discounting assessed by hypothetical tasks.

Disclosures and Acknowledgements

This review was supported by the National Institute on Drug Abuse (NIDA) of the National Institutes of Health (T32 DA07209). This funding source had no role in the preparation and submission of the manuscript. Data presented in this meta-analysis have not been presented previously. The authors have no financial conflicts of interest in regard to this research. Data and code to reproduce analyses presented here are available at https://osf.io/4thbk/?view_only=431ed149ce8e4d52aea1b56824977e4b.

Footnotes

1

Simulation studies conducted by Tipton (2015) during development of the small-sample adjusted RVE model indicated that the risk of inflated type I error increases dramatically when Satterthwaite corrected degrees of freedom were less than 4.

2

Code to replicate these analyses is available at https://osf.io/4thbk/?view_only=431ed149ce8e4d52aea1b56824977e4b.

3

If delay discounting acts as a predictor of who will progress to clinically-relevant use rests on time-series, longitudinal studies that are generally lacking in the cannabis literature (see more on this issue in the Future Directions section).

References

  1. Ainslie G (1975). Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin, 82(4), 463–496. doi: 10.1037/h0076860 [DOI] [PubMed] [Google Scholar]
  2. Amlung M, & MacKillop J (2014). Clarifying the relationship between impulsive delay discounting and nicotine dependence. Psychology of Addictive Behaviors, 28(3), 761–768. doi: 10.1037/a0036726 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Amlung M, & MacKillop J (2019). Availability of legalized cannabis reduces demand for illegal cannabis among canadian cannabis users: Evidence from a behavioural economic substitution paradigm. Canadian Journal of Public Health, 110(2), 216–221. doi: 10.17269/s41997-018-0160-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Amlung M, Marsden E, Holshausen K, Morris V, Patel H, Vedelago L, . . . McCabe RE (2019). Delay discounting as a transdiagnostic process in psychiatric disorders: A meta-analysis. JAMA Psychiatry. doi: 10.1001/jamapsychiatry.2019.2102 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Amlung M, Reed DD, Morris V, Aston ER, Metrik J, & MacKillop J (2019). Price elasticity of illegal versus legal cannabis: A behavioral economic substitutability analysis. Addiction, 114(1), 112–118. doi: 10.1111/add.14437 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Amlung M, Vedelago L, Acker J, Balodis I, & MacKillop J (2017). Steep delay discounting and addictive behavior: A meta-analysis of continuous associations. Addiction, 112(1), 51–62. doi: 10.1111/add.13535 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Aston ER, Metrik J, Amlung M, Kahler CW, & MacKillop J (2016). Interrelationships between marijuana demand and discounting of delayed rewards: Convergence in behavioural economic methods. Drug and Alcohol Dependence, 169, 141–147. doi: 10.1016/j.drugalcdep.2016.10.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Baker F, Johnson MW, & Bickel WK (2003). Delay discounting in current and never-before cigarette smokers: Similarities and differences across commodity, sign, and magnitude. Journal of Abnormal Psychology, 112(3), 382–392. doi: 10.1037/0021-843x.112.3.382 [DOI] [PubMed] [Google Scholar]
  9. Bickel WK, Jarmolowicz DP, Mueller ET, Koffarnus MN, & Gatchalian KM (2012). Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: Emerging evidence. Pharmacology & Therapeutics, 134(3), 287–297. doi: 10.1016/j.pharmthera.2012.02.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bickel WK, Koffarnus MN, Moody L, & Wilson AG (2014). The behavioral- and neuro-economic process of temporal discounting: A candidate behavioral marker of addiction. Neuropharmacology, 76, 518–527. doi: 10.1016/j.neuropharm.2013.06.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bickel WK, Landes RD, Christensen DR, Jackson L, Jones BA, Kurth-Nelson Z, & Redish AD (2011). Single- and cross-commodity discounting among cocaine addicts: The commodity and its temporal location determine discounting rate. Psychopharmacology (Berl), 217(2), 177–187. doi: 10.1007/s00213-011-2272-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Bickel WK, Snider SE, Quisenberry AJ, & Stein JS (2017). Reinforcer pathology: The behavioral economics of abuse liability testing. Clinical Pharmacology and Therapeutics, 101(2), 185–187. doi: 10.1002/cpt.443 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bobova L, Finn PR, Rickert ME, & Lucas J (2009). Disinhibitory psychopathology and delay discounting in alcohol dependence: Personality and cognitive correlates. Experimental and Clinical Psychopharmacology, 17(1), 51–61. doi: 10.1037/a0014503 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Bonar EE, Goldstick JE, Collins RL, Cranford JA, Cunningham RM, Chermack ST, . . . Walton MA (2017). Daily associations between cannabis motives and consumption in emerging adults. Drug and Alcohol Dependence, 178, 136–142. doi: 10.1016/j.drugalcdep.2017.05.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Borenstein M, Hedges LV, Higgins JP, & Rothstein HR (2011). Introduction to meta-analysis: John Wiley & Sons. [Google Scholar]
  16. Brodbeck J, Matter M, Page J, & Moggi F (2007). Motives for cannabis use as a moderator variable of distress among young adults. Addictive Behaviors, 32(8), 1537–1545. doi: 10.1016/j.addbeh.2006.11.012 [DOI] [PubMed] [Google Scholar]
  17. Buckner JD, Zvolensky MJ, & Schmidt NB (2012). Cannabis-related impairment and social anxiety: The roles of gender and cannabis use motives. Addictive Behaviors, 37(11), 1294–1297. doi: 10.1016/j.addbeh.2012.06.013 [DOI] [PubMed] [Google Scholar]
  18. Chung SH, & Herrnstein RJ (1967). Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 10(1), 67–74. doi: 10.1901/jeab.1967.10-67 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Cohen J (1992). A power primer. Psychological Bulletin, 112(1), 155–159. doi: 10.1037//0033-2909.112.1.155 [DOI] [PubMed] [Google Scholar]
  20. Cox DJ, & Dallery J (2016). Effects of delay and probability combinations on discounting in humans. Behavioural Processes, 131, 15–23. doi: 10.1016/j.beproc.2016.08.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Cox DJ, & Dallery J (2018). Influence of second outcome on monetary discounting. Behavioral Processes, 153, 84–91. doi: 10.1016/j.beproc.2018.05.012 [DOI] [PubMed] [Google Scholar]
  22. Crane NA, Schuster RM, & Gonzalez R (2013). Preliminary evidence for a sex-specific relationship between amount of cannabis use and neurocognitive performance in young adult cannabis users. Journal of the International Neuropsychological Society, 19(9), 1009–1015. doi: 10.1017/s135561771300088x [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Dennhardt AA, Yurasek AM, & Murphy JG (2015). Change in delay discounting and substance reward value following a brief alcohol and drug use intervention. Journal of the Experimental Analysis of Behavior, 103(1), 125–140. doi: 10.1002/jeab.121 [DOI] [PubMed] [Google Scholar]
  24. Ellingson JM, Potenza MN, & Pearlson GD (2018). Methodological factors as a potential source of discordance between self-report and behavioral measures of impulsivity and related constructs. Addictive Behaviors, 84, 126–130. doi: 10.1016/j.addbeh.2018.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Finn PR, Gunn RL, & Gerst KR (2015). The effects of a working memory load on delay discounting in those with externalizing psychopathology. Clinical Psychological Science, 3(2), 202–214. doi: 10.1177/2167702614542279 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Fisher Z, & Tipton E (2015). Robumeta: An r-package for robust variance estimation in meta-analysis. arXiv Preprint, 02220. [Google Scholar]
  27. Gray JC, & MacKillop J (2015). Impulsive delayed reward discounting as a genetically-influenced target for drug abuse prevention: A critical evaluation. Frontiers in Psychology, 6, e1104. doi: 10.3389/fpsyg.2015.01104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Green L, & Myerson J (2004). A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin, 130(5), 769–792. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Green L, Myerson J, & McFadden E (1997). Rate of temporal discounting decreases with amount of reward. Memory & Cognition, 25(5), 715–723. doi: 10.3758/bf03211314 [DOI] [PubMed] [Google Scholar]
  30. Green RM, & Lawyer SR (2014). Steeper delay and probability discounting of potentially real versus hypothetical cigarettes (but not money) among smokers. Behavioural Processes, 108, 50–56. doi: 10.1016/j.beproc.2014.09.008 [DOI] [PubMed] [Google Scholar]
  31. Hedges LV, Tipton E, & Johnson MC (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. doi: 10.1002/jrsm.5 [DOI] [PubMed] [Google Scholar]
  32. Heinz AJ, Peters EN, Boden MT, & Bonn-Miller MO (2013). A comprehensive examination of delay discounting in a clinical sample of cannabis-dependent military veterans making a self-guided quit attempt. Experimental and Clinical Psychopharmacology, 21(1), 55–65. doi: 10.1037/a0031192 [DOI] [PubMed] [Google Scholar]
  33. Janssen T, Larsen H, Peeters M, Boendermaker WJ, Vollebergh WAM, & Wiers RW (2015). Do online assessed self-report and behavioral measures of impulsivity-related constructs predict onset of substance use in adolescents? Addictive Behaviors Reports, 1, 12–18. doi: 10.1016/j.abrep.2015.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Johnson MW (2012). An efficient operant choice procedure for assessing delay discounting in humans: Initial validation in cocaine-dependent and control individuals. Experimental and Clinical Psychopharmacology, 20(3), 191–204. doi: 10.1037/a0027088 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Johnson MW, & Bickel WK (2002). Within-subject comparison of real and hypothetical money rewards in delay discounting. Journal of the Experimental Analysis of Behavior, 77(2), 129–146. doi: 10.1901/jeab.2002.77-129 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Johnson MW, Bickel WK, Baker F, Moore BA, Badger GJ, & Budney AJ (2010). Delay discounting in current and former marijuana-dependent individuals. Experimental and Clinical Psychopharmacology, 18(1), 99–107. doi: 10.1037/a0018333 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Johnson MW, & Bruner NR (2012). The sexual discounting task: Hiv risk behavior and the discounting of delayed sexual rewards in cocaine dependence. Drug and Alcohol Dependence, 123(1–3), 15–21. doi: 10.1016/j.drugalcdep.2011.09.032 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Johnson MW, Herrmann ES, Sweeney MM, LeComte RS, & Johnson PS (2017). Cocaine administration dose-dependently increases sexual desire and decreases condom use likelihood: The role of delay and probability discounting in connecting cocaine with hiv. Psychopharmacology (Berl), 234(4), 599–612. doi: 10.1007/s00213-016-4493-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Johnson PS, Herrmann ES, & Johnson MW (2015). Opportunity costs of reward delays and the discounting of hypothetical money and cigarettes. Journal of the Experimental Analysis of Behavior, 103(1), 87–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Johnson PS, Sweeney MM, Herrmann ES, & Johnson MW (2016). Alcohol increases delay and probability discounting of condom-protected sex: A novel vector for alcohol-related hiv transmission. Alcoholism: Clinical and Experimental Research, 40(6), 1339–1350. doi: 10.1111/acer.13079 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Khurana A, Romer D, Betancourt LM, & Hurt H (2017). Working memory ability and early drug use progression as predictors of adolescent substance use disorders. Addiction, 112(7), 1220–1228. doi: 10.1111/add.13792 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Killeen PR (2009). An additive-utility model of delay discounting. Psychological Review, 116(3), 602–619. doi: 10.1037/a0016414 [DOI] [PubMed] [Google Scholar]
  43. Kim-Spoon J, Lauharatanahirun N, Peviani K, Brieant A, Deater-Deckard K, Bickel WK, & King-Casas B (2019). Longitudinal pathways linking family risk, neural risk processing, delay discounting, and adolescent substance use. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 60(6), 655–664. doi: 10.1111/jcpp.13015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Kim-Spoon J, McCullough ME, Bickel WK, Farley JP, & Longo GS (2015). Longitudinal associations among religiousness, delay discounting, and substance use initiation in early adolescence. Journal of Research on Adolescence, 25(1), 36–43. doi: 10.1111/jora.12104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Kirby KN, Maraković NN (1996). Delay-discounting probabilistic rewards: Rates decrease as amounts increase. Psychonmic Bulletin & Revew, 3(1), 100–104. [DOI] [PubMed] [Google Scholar]
  46. Kirby KN, Petry NM, & Bickel WK (1999). Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. Journal of Experimental Psychology: General, 128(1), 78–87. [DOI] [PubMed] [Google Scholar]
  47. Kiselica AM, Webber TA, & Bornovalova MA (2016). Validity of the alcohol purchase task: A meta-analysis. Addiction, 111(5), 806–816. doi: 10.1111/add.13254 [DOI] [PubMed] [Google Scholar]
  48. Koffarnus MN, & Bickel WK (2014). A 5-trial adjusting delay discounting task: Accurate discount rates in less than one minute. Experimental and Clinical Psychopharmacology, 22(3), 222–228. doi: 10.1037/a0035973 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Koffarnus MN, & Kaplan BA (2018). Clinical models of decision making in addiction. Pharmacology, Biochemistry, and Behavior, 164, 71–83. doi: 10.1016/j.pbb.2017.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Lawyer SR, Schoepflin F, Green R, & Jenks C (2011). Discounting of hypothetical and potentially real outcomes in nicotine-dependent and nondependent samples. Experimental and Clinical Psychopharmacology, 19(4), 263–274. doi: 10.1037/a0024141 [DOI] [PubMed] [Google Scholar]
  51. Locey ML, Jones BA, & Rachlin H (2011). Real and hypothetical rewards. Judgment and Decision Making, 6(6), 552–564. [PMC free article] [PubMed] [Google Scholar]
  52. Lopez-Vergara HI, Jackson KM, Meshesha LZ, & Metrik J (2019). Dysregulation as a correlate of cannabis use and problem use. Addictive Behaviors, 95, 138–144. doi: 10.1016/j.addbeh.2019.03.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. MacKillop J (2013). Integrating behavioral economics and behavioral genetics: Delayed reward discounting as an endophenotype for addictive disorders. Journal of the Experimental Analysis of Behavior, 99(1), 14–31. doi: 10.1002/jeab.4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. MacKillop J (2016). The behavioral economics and neuroeconomics of alcohol use disorders. Alcoholism: Clinical and Experimental Research, 40(4), 672–685. doi: 10.1111/acer.13004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, & Munafo MR (2011). Delayed reward discounting and addictive behavior: A meta-analysis. Psychopharmacology (Berl), 216(3), 305–321. doi: 10.1007/s00213-011-2229-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Madden GJ, Begotka AM, Raiff BR, & Kastern LL (2003). Delay discounting of real and hypothetical rewards. Experimental and Clinical Psychopharmacology, 11(2), 139–145. doi: 10.1037/1064-1297.11.2.139 [DOI] [PubMed] [Google Scholar]
  57. Matusiewicz AK, Carter AE, Landes RD, & Yi R (2013). Statistical equivalence and test-retest reliability of delay and probability discounting using real and hypothetical rewards. Behavioural Processes, 100, 116–122. doi: 10.1016/j.beproc.2013.07.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Mejía-Cruz D, Green L, Myerson J, Morales-Chainé S, & Nieto J (2016). Delay and probability discounting by drug-dependent cocaine and marijuana users. Psychopharmacology (Berl), 233(14), 2705–2714. doi: 10.1007/s00213-016-4316-8 [DOI] [PubMed] [Google Scholar]
  59. Mellis AM, Woodford AE, Stein JS, & Bickel WK (2017). A second type of magnitude effect: Reinforcer magnitude differentiates delay discounting between substance users and controls. Journal of the Experimental Analysis of Behavior, 107(1), 151–160. doi: 10.1002/jeab.235 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Mitchell SH (2004). Effects of short-term nicotine deprivation on decision-making: Delay, uncertainty and effort discounting. Nicotine & Tobacco Research, 6(5), 819–828. doi: 10.1080/14622200412331296002 [DOI] [PubMed] [Google Scholar]
  61. Moody L, Franck C, & Bickel WK (2016). Comorbid depression, antisocial personality, and substance dependence: Relationship with delay discounting. Drug and Alcohol Dependence, 160, 190–196. doi: 10.1016/j.drugalcdep.2016.01.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Moody L, Franck C, Hatz L, & Bickel WK (2016). Impulsivity and polysubstance use: A systematic comparison of delay discounting in mono-, dual-, and trisubstance use. Experimental and Clinical Psychopharmacology, 24(1), 30–37. doi: 10.1037/pha0000059 [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Moody L, Tegge AN, & Bickel WK (2017). Cross-commodity delay discounting of alcohol and money in alcohol users. Psychological Record, 67(2), 285–292. doi: 10.1007/s40732-017-0245-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Myerson J, Green L, & Warusawitharana M (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76(2), 235–243. doi: 10.1901/jeab.2001.76-235 [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Newmeyer MN, Swortwood MJ, Abulseoud OA, & Huestis MA (2017). Subjective and physiological effects, and expired carbon monoxide concentrations in frequent and occasional cannabis smokers following smoked, vaporized, and oral cannabis administration. Drug and Alcohol Dependence, 175, 67–76. doi: 10.1016/j.drugalcdep.2017.02.003 [DOI] [PubMed] [Google Scholar]
  66. Oshri A, Kogan SM, Kwon JA, Wickrama KAS, Vanderbroek L, Palmer AA, & MacKillop J (2018). Impulsivity as a mechanism linking child abuse and neglect with substance use in adolescence and adulthood. Development and Psychopathology, 30(2), 417–435. doi: 10.1017/s0954579417000943 [DOI] [PubMed] [Google Scholar]
  67. Paglieri F (2013). The costs of delay: Waiting versus postponing in intertemporal choice. Journal of the Experimental Analysis of Behavior, 99(3), 362–377. [DOI] [PubMed] [Google Scholar]
  68. Patel H, Naish KR, & Amlung M (2019). Discounting of delayed monetary and cannabis rewards in a crowdsourced sample of adults. Experimental and Clinical Psychopharmacology, ePub ahead of print. doi: 10.1037/pha0000327 [DOI] [PubMed] [Google Scholar]
  69. Pericot-Valverde I, Yoon JH, & Gaalema DE (2020). Single- and cross-commodity delay discounting of money and e-cigarette liquid in experienced e-cigarette users. Drug and Alcohol Dependence, 206, 107740. doi: 10.1016/j.drugalcdep.2019.107740 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Perry JL, & Carroll ME (2008). The role of impulsive behavior in drug abuse. Psychopharmacology (Berl), 200(1), 1–26. doi: 10.1007/s00213-008-1173-0 [DOI] [PubMed] [Google Scholar]
  71. Peters EN, Petry NM, Lapaglia DM, Reynolds B, & Carroll KM (2013). Delay discounting in adults receiving treatment for marijuana dependence. Experimental and Clinical Psychopharmacology, 21(1), 46–54. doi: 10.1037/a0030943 [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Petker T, Owens MM, Amlung MT, Oshri A, Sweet LH, & MacKillop J (2019). Cannabis involvement and neuropsychological performance: findings from the Human Connectome Project. Journal of Psychiatry & Neuroscience: JPN, 44(6), 414–422. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Rachlin H (2006). Notes on discounting. Journal of the Experimental Analysis of Behavior, 85(3), 425–435. doi: 10.1901/jeab.2006.85-05 [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Rachlin H, & Green L (1972). Commitment, choice and self-control. Journal of the Experimental Analysis of Behavior, 17(1), 15–22. doi: 10.1901/jeab.1972.17-15 [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Rasmussen EB, Lawyer SR, & Reilly W (2010). Percent body fat is related to delay and probability discounting for food in humans. Behavioural Processes, 83(1), 23–30. [DOI] [PubMed] [Google Scholar]
  76. Reynolds B, & Fields S (2012). Delay discounting by adolescents experimenting with cigarette smoking. Addiction, 107(2), 417–424. doi: 10.1111/j.1360-0443.2011.03644.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Reynolds B, & Schiffbauer RM (2004). Measuring state changes in human delay discounting: An experiential discounting task. Behavioural Processes, 67(3), 343–356. [DOI] [PubMed] [Google Scholar]
  78. Richards JB, Zhang L, Mitchell SH, & de Wit H (1999). Delay or probability discounting in a model of impulsive behavior: Effect of alcohol. Journal of the Experimental Analysis of Behavior, 71(2), 121–143. doi: 10.1901/jeab.1999.71-121 [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Romer D, Duckworth AL, Sznitman S, & Park S (2010). Can adolescents learn self-control? Delay of gratification in the development of control over risk taking. Prevention Science, 11(3), 319–330. doi: 10.1007/s11121-010-0171-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Sanchez-Roige S, Fontanillas P, Elson SL, Pandit A, Schmidt EM, Foerster JR, ... & MacKillop J (2018). Genome-wide association study of delay discounting in 23,217 adult research participants of European ancestry. Nature Neuroscience, 21(1), 16–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Spindle TR, Cone EJ, Schlienz NJ, Mitchell JM, Bigelow GE, Flegel R, . . . Vandrey R (2018). Acute effects of smoked and vaporized cannabis in healthy adults who infrequently use cannabis: A crossover trial. JAMA Network Open, 1(7), e184841. doi: 10.1001/jamanetworkopen.2018.4841 [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Stanger C, Ryan SR, Fu H, Landes RD, Jones BA, Bickel WK, & Budney AJ (2012). Delay discounting predicts adolescent substance abuse treatment outcome. Experimental and Clinical Psychopharmacology, 20(3), 205–212. doi: 10.1037/a0026543 [DOI] [PMC free article] [PubMed] [Google Scholar]
  83. Stea JN, Hodgins DC, & Lambert MJ (2011). Relations between delay discounting and low to moderate gambling, cannabis, and alcohol problems among university students. Behavioural Processes, 88(3), 202–205. doi: 10.1016/j.beproc.2011.09.002 [DOI] [PubMed] [Google Scholar]
  84. Stephens RS, Roffman RA, & Curtin L (2000). Comparison of extended versus brief treatments for marijuana use. Journal of Consulting and Clinical Psychology, 68(5), 898–908. [PubMed] [Google Scholar]
  85. Strickland JC, Campbell EM, Lile JA, & Stoops WW (2020). Utilizing the commodity purchase task to evaluate behavioral economic demand for illicit substances: A review and meta-analysis. Addiction, 115(3), 393–406. doi: 10.1111/add.14792 [DOI] [PubMed] [Google Scholar]
  86. Strickland JC, Lile JA, & Stoops WW (2017). Unique prediction of cannabis use severity and behaviors by delay discounting and behavioral economic demand. Behavioural Processes, 140, 33–40. doi: 10.1016/j.beproc.2017.03.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  87. Strickland JC, Lile JA, & Stoops WW (2019). Evaluating non-medical prescription opioid demand using commodity purchase tasks: Test-retest reliability and incremental validity. Psychopharmacology, 236(9), 2641–2652. doi: 10.1007/s00213-019-05234-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  88. Thamotharan S, Hahn H, & Fields S (2017). Drug use status in youth: The role of gender and delay discounting. Substance Use & Misuse, 52(10), 1338–1347. doi: 10.1080/10826084.2017.1280831 [DOI] [PubMed] [Google Scholar]
  89. Tipton E (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20(3), 375–393. doi: 10.1037/met0000011 [DOI] [PubMed] [Google Scholar]
  90. Tsukayama E, & Duckworth AL (2010). Domain-specific temporal discounting and temptation. Judgment and Decision Making, 5(2), 72–82. [Google Scholar]
  91. United Nations Office on Drugs and Crime. (2019). World Drug Report 2019. (Sales No. E.19.XI.8). Vienna, Austria: United Nations [Google Scholar]
  92. van der Pol P, Liebregts N, de Graaf R, Korf DJ, van den Brink W, & van Laar M (2013). Predicting the transition from frequent cannabis use to cannabis dependence: A three-year prospective study. Drug and Alcohol Dependence, 133(2), 352–359. doi: 10.1016/j.drugalcdep.2013.06.009 [DOI] [PubMed] [Google Scholar]
  93. VanderBroek L, Acker J, Palmer AA, de Wit H, & MacKillop J (2016). Interrelationships among parental family history of substance misuse, delay discounting, and personal substance use. Psychopharmacology (Berl), 233(1), 39–48. doi: 10.1007/s00213-015-4074-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  94. Vanderveldt A, Green L, & Myerson J (2015). Discounting of monetary rewards that are both delayed and probabilistic: Delay and probability combine multiplicatively, not additively. Journal of Experimental Psychology. Learning, Memory, and Cognition, 41(1), 148–162. doi: 10.1037/xlm0000029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  95. Vandrey R, Herrmann ES, Mitchell JM, Bigelow GE, Flegel R, LoDico C, & Cone EJ (2017). Pharmacokinetic profile of oral cannabis in humans: Blood and oral fluid disposition and relation to pharmacodynamic outcomes. Journal of Analytical Toxicology, 41(2), 83–99. doi: 10.1093/jat/bkx012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  96. Viechtbauer W (2010). Conducting meta-analyses in r with the metafor package. Journal of Statistical Software, 36(3), 1–48. [Google Scholar]
  97. Yoon JH, Weaver MT, De La Garza R 2nd, Suchting R, Nerumalla CS, Omar Y, . . . Newton TF (2018). Comparison of three measurement models of discounting among individuals with methamphetamine use disorder. The American Journal on Addictions. doi: 10.1111/ajad.12761 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material

RESOURCES