Abstract
Background:
Delay discounting (DD) research has improved our understanding of important behavioral processes associated with tobacco use. Little research has explored DD among e-cigarette users, and these studies have exclusively examined money as the only available commodity. This secondary analysis of a laboratory study explored discounting for money and e-liquid among e-cigarette users using two single-commodity discounting (SCD) tasks and one cross-commodity discounting (CCD) task. A secondary goal was to explore the extent to which results from the SCD and CCD tasks were correlated to each other and with measures of e-cigarette use.
Methods:
E-cigarette users (N = 27) completed two SCD tasks and one CCD task. The SCD tasks assessed choices between various amounts of either money now versus money later (M-M) or e-liquid now versus e-liquid later (mL-mL). The CCD task assessed choices between e-liquid now versus money later (mL-M). Discounting results were compared using logk and AUClog.
Results:
Discounting was greatest in the mL-mL task, followed by the M-M task, and then the mL-M task. AUClog and logk were significantly correlated across all discounting tasks. Attempts to quit vaping was positively associated with logk and negatively associated with AUClog and in both SCD tasks.
Conclusions:
E-cigarette users discount e-liquid more than money in a SCD task. However, when the two commodities, money and e-liquid (CCD), are compared the substance of abuse is discounted to a lesser extent. Interventions that provide alternative reinforcers to compete with the reinforcing effects of nicotine intake may be especially indicated for treating e-cigarette dependence.
Keywords: Delay discounting, Cross-commodity discounting, Single-commodity discounting, E-cigarettes, Impulsivity
1. Introduction
E-cigarettes are an alternative tobacco product that delivers nicotine in an aerosolized form (Breland et al., 2017). Since their introduction to the US market, e-cigarettes have dramatically increased in popularity and usage (Huerta et al., 2017) especially among youth and young adults (Pericot-Valverde et al., 2017), and even among those who have never smoked cigarettes (Johnston et al., 2015; Sawdey et al., 2018; USDHHS, 2016). Recent studies have also demonstrated e-cigarette use is associated with a higher likelihood of smoking uptake of combusted cigarettes (Leventhal et al., 2015; Loukas et al., 2018; Spindle et al., 2017). Altogether, these findings highlight the importance of identifying and understanding the underlying mechanisms associated with e-cigarette use.
Delay discounting (DD) is an operational measure of impulsive decision-making that characterizes the rate at which a reward loses value as the delay to its receipt increases (Bickel and Marsch, 2001; Reynolds, 2006). DD is frequently explored through tasks which ask participants to choose between a smaller, more immediate reward and a larger, delayed one. By varying the delay and magnitude of choices an individual curve that describes the discounting rate can be obtained (Odum, 2011a).
DD has been used to investigate various unhealthy and non-adaptive behaviors (e.g., compulsive eating, needle sharing) including substance use disorders (SUDs)(Bickel et al., 2011a,b; Jarmolowicz et al., 2014; Koffarnus et al., 2016; Mackillop et al., 2011; Odum et al., 2000; Stein et al., 2018a; Steward et al., 2017). These studies have demonstrated that individuals with SUDs have higher levels of DD relative to matched controls for several substances, including e-cigarettes (Stein et al., 2018b), cigarettes (Białaszek et al., 2017), alcohol (Moody et al., 2017), cocaine (Bickel et al., 2011a,b), opioids (Madden et al., 1997), and methamphetamine (Yoon et al., 2018). Moreover, DD is reliably and robustly associated with drug initiation, abuse, severity of use, and relapse (Amlung et al., 2017; Bickel et al., 2017). Collectively, these findings provide strong evidence that DD can serve as a useful behavioral marker of addiction (Bickel et al., 2014).
The rate at which rewards are discounted depends on the type of the commodity being evaluated, which is also known as a domain effect (Baker et al., 2003). The majority of DD studies have presented choices involving the same rewards with varying magnitudes and delays (MacKillop et al., 2011), providing a measure of single-commodity discounting (SCD). Money is the most commonly assessed commodity (Hamilton et al., 2015) as it allows the easy comparison across populations, although various studies have explored SCD for illicit and licit substances including cigarettes, alcohol, marijuana, cocaine, and heroin (Bickel et al., 2011a,b; Johnson et al., 2010; Madden et al., 1997; Moody et al., 2017; Yoon et al., 2009). Studies comparing SCD between two different commodities (e.g. money now and later vs. drugs now and later) have reliably observed that individuals with SUDs reliably discount their drug of abuse more steeply than comparable amounts of money (Baker et al., 2003; Bickel et al., 2011a,b; Coffey et al., 2003; Friedel et al., 2014; Johnson et al., 2010; Madden et al., 1997; Moody et al., 2017; Petry, 2001). However, it should be noted that the SCD approach separately compares the absolute discounting of a given reinforcer (e.g. money now and later vs. cigarettes now and later), and consequently may not fully reflect the real-world decisions made between alternatives (e.g., drug use now or health later) (Bickel et al., 2011a,b). Cross-commodity discounting (CCD) is another approach to explore DD wherein not only both the magnitude and delay of the reinforcer change, but different reinforcers are directly compared (e.g. choice between drug now and money later). Additionally, CCD may more closely model the drug-related decision-making processes that individuals make in the real world (Bickel et al., 2011a,b).
Assessing both SCD and CCD allows the simultaneous evaluation of the relative and absolute perceived value of distinct commodities, providing an opportunity to determine, in a wide array of circumstances, how preferences may change towards or away from a commodity among individuals with SUDs depending on how it is discounted and its utility. More specifically, the use of SCD for different commodities (money and drug) permits exploration of whether the discounting rate differs between commodities whereas the use of CCD for different commodities allows exploration of whether the utility of two commodities is functionally different, namely if they function as substitutes. To date, existing studies have studied both SCD and CCD among dependent individuals for alcohol (Moody et al., 2017), cigarettes (Mitchell, 2004; Yoon et al., 2009), methamphetamine (Yoon et al., 2018), and cocaine (Bickel et al., 2011a,b).
The available studies examining discounting among e-cigarette users have only assessed SCD of money (Stein et al., 2018a; Weidberg et al., 2017; Białaszek et al., 2017). SCD for money may provide important clinical information including predicting treatment outcomes (abstinence vs relapse) (González-Roz et al., 2019; Secades-Villa et al., 2016; Washio et al., 2011; Yoon et al., 2007) and discriminating across populations (moderate alcohol drinkers vs hazardous drinkers) (Moody et al., 2017). Nonetheless, there is evidence that drug discounting may be a better predictor of clinically relevant outcomes compared to monetary discounting (Field et al., 2007; Giordano et al., 2002). For e-cigarette users, the commodity being consumed (i.e. during vaping) is e-cigarette liquid (also known as e-liquid). Recent evidence suggests that e-cigarettes may have lower abuse potential compared to combustible cigarettes (Evans and Hoffman, 2014), which may be reflected by relatively decreased discounting for e-liquid compared to money across SCD and CCD tasks. Understanding the factors involved in decision making for e-cigarette may be useful for developing effective e-cigarette cessation interventions. For example, understanding e-cigarette users’ decision making could inform interventions that attempt to influence short-term decisions by providing alternative, immediate rewards for abstinence (e.g., contingency-management (CM) based treatments) (Stanger et al., 2013), as well as these interventions that try to shift temporal orientation to more salient future-oriented goals (e.g., episodic future thinking) (Snider et al., 2016).
To date, relatively little research has been conducted on SCD among e-cigarette users (Białaszek et al., 2017; Stein et al., 2018b; Weidberg et al., 2017), and more importantly, no prior study has explored CCD of e-cigarettes vs. money. The primary purpose of this secondary analysis of a clinical laboratory study was to explore SCD (money vs money and e-liquid vs e-liquid) and CCD (e-liquid now vs money later) among experienced e-cigarette users. A secondary goal was to explore the associations between single commodity (i.e., money vs money and e-liquid vs e-liquid) and cross-commodity (e-liquid vs money) discounting rates and e-cigarette use.
2. Methods
2.1. Participants
Participants were experienced e-cigarette users who were enrolled in a parent study aimed at exploring abuse liability of e-cigarettes (Pericot-Valverde et al., 2018). Participants were recruited from the greater Burlington, Vermont area. Prior to study enrollment, participants received a brief phone screen to assess initial eligibility. Potential candidates were invited to the laboratory for an in-person screening to verify their eligibility. Eligibility criteria for participation included being ≥18 years old, using ≥1 mL of e-cigarette liquid per day with a nicotine concentration of ≥3 mg/mL, and having used an e-cigarette for at least 3 months. Participants were excluded for using combustible cigarettes, having a current diagnosis for a psychiatric or medical condition based on the Mini-International Neuropsychiatric Interview (Sheehan et al., 1998), or for using a prescription medication other than birth control. Of the initial 40 participants screened, 2 were deemed ineligible during the research session for reporting use of combustible cigarettes. Thus, 38 participants were considered for inclusion in the current analyses. All participants provided written informed consent and procedures were approved by University of Vermont’s Institutional Review Board.
2.2. Instruments and variables
2.2.1. Socio-demographics
Basic socio-demographic characteristics including age, gender, race/ethnicity, and educational attainment were obtained using a brief questionnaire developed specifically for this study.
2.2.2. Cigarette smoking and e-cigarette assessments
After providing informed consent, participants completed questionnaires assessing their e-cigarette use history, the Penn State E-cigarette Dependence Scale (PSECDI) (Foulds et al., 2014), and the Brief Questionnaire for Vaping Urges (QVU-B) (Dowd et al., 2018). Participants were also asked to provide biological samples to ensure their eligibility. Biological measures collected included breath carbon monoxide (CO) and urine sample. Non-cigarette smoking status and current e-cigarette use was verified using a cut-off ≤6 parts per million (ppm) in CO and a value 100 ng/ml (i.e., ≥3) for cotinine in urine (NicAlert™ by Nymox Pharmaceutical Corporation) (Benowitz, 2017; Marrone et al., 2011).
2.2.3. Delay discounting tasks
Participants were initially instructed on how to complete the two SCD tasks and the CCD task. Participants were informed that the questions were hypothetical, but to respond as if the choices were real. Additionally, participants were instructed to assume that when choosing e-cigarette liquid that it was for their own use and that it was their preferred brand.
Prior to starting the discounting tasks, e-cigarette users were asked to report the cost of their preferred e-liquid and the amount of mL available in the container. With this information the research staff then calculated the cost of a single mL of e-liquid. For example, if a participant reported that their e-cigarette liquid cost $10 for 5 ml of e-liquid, then the cost for each mL was determined as $2. Delay discounting tasks included two SCD tasks and one CCD task. The SCD tasks assessed either money now vs money later in dollar units (M-M) or e-liquid now vs e-liquid later in milliliter units (mL-mL). The CCD task assessed e-liquid now vs money later (mL-M). In the M-M task, participants were presented with repeated choices between various amounts of money now vs. $1000 presented after a fixed delay. Similarly, in the mL-mL task, e-cigarette users were presented with repeated choices between various amounts of e-liquid now vs. a fixed amount of e-liquid worth $1000 presented after a delay. Finally, in the mL-M task, participants were presented with choices between various amounts of e-liquid available immediately and $1000 after a fixed delay. When presenting the immediate e-liquid option, the monetary value was also explicitly presented (e.g., $50 ml of e-liquid (worth $150)). The magnitude of the fixed, delayed reward was set at $1000 because (1) using lower magnitudes have been demonstrated to be less sensitive in detecting differences in responding (Mellis et al., 2017, 2018) and (2) $1000 is a common magnitude used in tobacco research (García-Rodríguez et al., 2013; Yoon et al., 2009; Weidberg et al., 2017), making it easier to compare the results of the current study with the broader tobacco literature. Delays (1day, 1 week, 1 month, 6 months, 1 year, 5 years, and 25 years) were presented in an ascending order. The equivalent amount of e-liquid was individually determined for each participant. For all three discounting tasks, the value of the immediate option was adjusted via a titrating procedure that determined an individual’s indifference point based on their responses. The titration procedure took the lower (initial $0) and upper ($1000) limit possible of values and divided this total range by 2, 3, or 4 to obtain an interval value. The value of the immediate choice was determined as one interval value below or above the lower and upper limits. In the case that the immediate value was outside the limits ($0- $1000), another value was randomly chosen based on participant’s response.
All three tasks were presented to participants via a laptop running the Windows Operating system. Each task took approximately 5 min to complete and the order of presentation was randomized for each participant.
2.3. Data analyses
DD data were initially assessed for systematicity (Johnson and Bickel, 2008). Data were defined as being non-systematic if either of the following criteria were met: (1) any indifference point (starting at the second delay) was greater than the prior indifference point by a magnitude greater than 20 % of the larger later reward; and (2) the last indifference point (i.e., 25 years) was not ≤ 10 % of the value of the first indifference point obtained at the 1 day delay (Johnson and Bickel, 2008). Data were used only for participants that demonstrated systematicity across all three discounting tasks. Data from eleven participants (11/38 or 28.9 %) were excluded because they produced non-systematic delay discounting data in any of the three delay discounting tasks (i.e., M-M, mL-mL, and mL-M). More specifically, four participants provided non-systematic data in one of the tasks, six participants in two tasks, and one participant in all three tasks.
Obtained indifference points from each DD assessment were fitted to the hyperbolic equation defined by Mazur (1987):
(1) |
Eq. (1) illustrates how the value (V) of a reinforcer of some amount (A) is discounted as the delay (D) to receipt increases. The free parameter k describes the rate of discounting, with higher k values indicating greater discounting.
We also assessed discounting using a relatively new variation of area under the curve (AUC) (Myerson et al., 2001). In this variation, the delay values are initially log-transformed (Borges et al., 2016), which mitigates the disparate influence of areas from larger delays (Yoon et al., 2017). AUClog correlates closely with log k, and both measures have been observed to detect clinically relevant differences under conditions that AUC did not (Borges et al., 2016; Yoon et al., 2017, 2018).
(2) |
Eq. (2) describes how the AUClog is obtained, where D1log and D2log are the delays associated with successive indifference points, and Vi and V2 are the indifference point values associated with these delays in a log scale. Note that both the logged delay values and indifferences points are normalized to a 0–1 scale prior to calculating AUClog. Obtained indifference points were fit to Eqs (1 and 2) using the curve-fitting program GraphPad Prism 6 (GraphPad Software, San Diego, CA).
Statistical analyses were performed on log-transformed k values and AUClog values (Borges et al., 2016; Yoon and Higgins, 2008; Yoon et al., 2007). Skewness, kurtosis, and D-Agostino and Pearson normality tests were conducted for the distribution of logk and AUClog values for each discounting task (i.e. M-M, mL-mL, mL-M) in order to test for normality of the data. A one-way repeated analysis of variance (ANOVA) was conducted to explore whether logk and AUClog values differed as a function of condition (M-M, mL-mL, mL-M). Post-hoc comparisons between conditions were performed by means of the Least Significant Difference (LSD) test. Effect sizes were calculated using Cohen’s d, with values of 0.2, 0.5, and 0.8 indicative of small, medium, and large effect size, respectively (Cohen, 1988). Pearson correlations were used to describe the associations of the logk and AUClog values between the three tasks. Correlations were also conducted to explore associations between logk and AUClog values and measures of e-cigarette use and dependence. According to the guidelines proposed by Cohen (1988), correlations were considered as small (r = .10–.29), medium (r = .30–.49) and large (r = .5 0 to 1 .0). To investigate the robustness of the findings obtained in the correlation analyses, we performed sensitivity analyses by means of comparing average scores on logk and AUClog using t-tests. Significance for all statistical analyses was defined as p ≤ .05 and the statistical package used was SPSS version 25 (IBM Corp, Armonk, NY).
3. Results
The final sample consisted of 27 e-cigarette users. Participants were on average 20.7 years old, male (70.4 %), White (96.3 %), and had some college education (77.8 %). Average duration of e-cigarette use was 12.3 months with an average of 3.4 mL of e-liquid consumed daily. Average PSECDI and QVU-B scores were 9.3 and 32.7, respectively (Table 1).
Table 1.
Characteristic | M ± SD/% |
---|---|
Age (years) | 20.7 ± 5.6 |
Gender (male) | 70.4 |
Race/ethnicity (White) | 96.3 |
Educational attainment | |
≤ High school graduate | 22.2 |
≥ Some college | 77.8 |
PSECDI | 9.3 ± 3.7 |
QVU-B | 32.7 ± 11.1 |
Attempts to quit vaping | 1.5 ± 1.6 |
Months of regular e-cigarette use | 12.3 ± 8.9 |
Nicotine e-liquid (mg/mL) | 20.9 ± 2.3 |
Daily use of e-liquid (mL) | 3.4 ± 4.1 |
Preferred flavor | |
Tobacco | 11.1 |
Fruity | 33.3 |
Menthol | 29.6 |
Sweet | 11.1 |
Other | 14.8 |
Note: PSECDI = Penn State Electronic Cigarette Dependence Index; PSCI scoring: 0-3 = not dependent; 4-8 = low dependence; 9-12 = medium dependence; 13+ = high dependence; QVU-B = Brief Questionnaire for Vaping Urges; mg = milligram; mL = milliliters.
Fig. 1 depicts histograms of logk and AUClog data for both SCD tasks (M-M and mL-mL) and CCD task (mL-M). Values were normally distributed and none failed the D’-Agostino and Pearson normality tests.
Both logk (F(2, 25) = 20.22, p < .001) and AUClog (F(2, 25) = 23.46, p < .001) were significantly different across the three discounting tasks (Fig. 2). Greater discounting was observed in the mL-mL (M logk =−2.05, SD logk = 0.94; M AUClog = 0.49, SD AUClog = 0.20) vs. M-M (M logk =−2.66, SD logk=0.84; M AUClog = 0.64, SD AUClog = 0.17) task and mL-M (M logk =−3.43, SD logk = 1.20; M AUClog = 0.75, SD AUClog = 0.19) task. Post-hoc comparisons showed that all delay discounting conditions were significantly different from one another logk (M-M vs. mL-mL (p < .001), M-mL vs. mL-mL (p < .001), M-M vs. mL-M (p = .001) and AUClog (M-M vs. mL-mL (p < .001), M-mL vs. mL-mL (p < .001), M-M vs. mL-M (p = .003)). Effects sizes were 0.79 and 0.86 for logk and AUClog, respectively.
Table 2 shows correlations for delay discounting rates across the M-M, mL-mL, and mL-M discounting tasks. All correlations were significant with the majority of associations being large in magnitude. As expected, logk values were positively correlated to each other while negatively correlated with AUClog values and vice versa. Note that greater discounting is indicated by relatively higher logk values and lower AUClog values. The strongest correlations were found between SCD indices, wherein correlations between M-M and mL-mL values were r = 0.708 (p < .001), and r = 0.737 (p < .001) for logk and AUClog, respectively.
Table 2.
M-M logk | M-M AUClog | mL-mL logk | mL-mL AUClog | mL-M logk | mL-M AUClog | |
---|---|---|---|---|---|---|
M-M logk | – | |||||
M-M AUClog | −.984*** | – | ||||
mL-mL logk | .708*** | −.737*** | – | |||
mL-mL AUClog | −.705*** | .743*** | −.978*** | – | ||
mL-M logk | .514** | −.481* | .439* | −.428* | – | |
mL-M AUClog | −.499** | .481* | −.436* | .427* | −.946*** | – |
Note:
indicates p < .001
indicates p < .01;
indicates p < .05;
mL: milliliters; M-M = money vs. money; mL-mL = milliliters vs. milliliters; mL-M = milliliters vs. money.
Finally, Table 3 shows correlations across discounting rates from the three discounting tasks (M-M, mL-mL, and mL-M) and measures of e-cigarette use, attempts to quit vaping, and nicotine dependence. Attempts to quit vaping was positively associated with AUClog for M-M and mL-mL tasks (M-M: r = −.389, p < .05; mL-mL: r = −.474, p < .05) and negatively related to logk (M-M: r = −.392, p < .05; mL-mL: r = −.460, p < .05) in both SCD tasks. These findings persisted after sensitivity analyses wherein it was found that logk and AUClog scores for both SCD tasks differed significantly among those who never tried to quit vaping (0 attempts) compared to those who ever tried to quit vaping (≥ 1 attempts). Specifically, ever quit attempters exhibited significantly higher delay discounting than never quit attempters, M-M logk(25)=−2.259, p≤ .05; M-M AUClog(25)=2.229, p< .05; mL-mL logk(25)=−3.333, p < .01; mL-mL AUClog(25) = 3.118, p < .01. The amount of daily use of e-liquid was negatively associated with M-M task in logk units (−.349, p < .05). No other significant correlations were found.
Table 3.
M-M logk | M-M AUClog | mL-mL logk | mL-mL AUClog | mL-M logk | mL-M AUClog | |
---|---|---|---|---|---|---|
PSECDI | .127 | −.111 | .126 | −.106 | −.080 | .052 |
QVU-B | .158 | −.154 | 0.28 | −.068 | .292 | −.387 |
Attempts to quit vaping | .392* | −389* | .460* | −.474* | .248 | −.280 |
Months of regular e-cigarette use | −.074 | .075 | −.174 | .234 | .124 | .024 |
Nicotine e-liquid (mg/mL) | .244 | −.209 | .099 | −.129 | .043 | −.191 |
Daily use of e-liquid (mL) | −.394* | .367 | −.306 | .352 | −.158 | .265 |
Note: PSECDI = Penn State Electronic Cigarette Dependence Index; QVU-B = Brief Questionnaire for Vaping Urges; mg = milligrams; mL = milliliters; M-M = money vs. money; mL-mL = milliliters vs. milliliters; mL-M = milliliters vs. money
indicates p < .001
indicates p < .01;
indicates p < .05.
4. Conclusions
This study aimed to explore delay discounting among e-cigarette users for two commodities (i.e., money and e-liquid) using two SCD tasks (M-M and mL-mL) and one CCD task (mL-M). A secondary objective was to explore relations in discounting across the different SCD and CCD tasks, as well as measures of e-cigarette use and dependence.
In the SCD tasks (i.e., M-M and mL-mL), experienced e-cigarette users discounted e-cigarette liquid (i.e., mL-mL) at a significantly steeper rate than money (i.e, M-M). This result is consistent with previous studies showing that individuals with SUDs (e.g., conventional cigarettes, opioids, cocaine, marijuana, alcohol) (Bickel et al. 2011a,b; Johnson et al., 2010; Madden et al., 1997; Moody et al., 2017; Petry, 2001; Yoon et al., 2009) exhibit greater discounting for their drug of choice compared to money. Importantly, the current results are the first to generalize these findings to e-cigarette users. It should be noted that prior studies have shown that consumable commodities (e.g., food, drugs), especially those that can be immediately used, are more rapidly discounted than these that serve as an exchangeable commodity (e.g., money) (Hamilton et al., 2015; Jarmolowicz et al., 2014), which may explain why e-cigarette users discount e-cigarette liquid at higher rates than money.
Additionally, this study demonstrates differences in discounting across the three discounting tasks. Specifically, results from the SCD tasks (i.e., mL-mL < M-M) demonstrate that discounting of e-liquid is greater than discounting of money. Taken together with the results showing that discounting was greater in the M-M task compared to the mL-M task, our findings may be indicating that the utility (i.e., reward preference) of receiving e-liquid is not functionally equivalent to the utility of receiving money (i.e., they do not function as perfect substitutes). This is a novel and interesting finding regarding the relative preference of e-liquid depending on the delayed commodity available (money vs e-liquid) which can have several explanations. This finding could mean that the availability of alternative consumable reinforcers (i.e. money) may effectively compete with the immediate availability of e-cigarette liquid, resulting in a reduced preference for e-liquid (Higgins, 1997). Alternatively, this finding could be interpreted as e-cigarette users in this study value e-liquid to a lesser extent when other alternatives may be available in the future, due to them not suffering a severe dependence for nicotine.
This study also reveals significant and positive correlations between SCD and CCD discounting rates, suggesting that e-cigarette users that discount at higher rates in the SCD conditions (M-M and mL-mL) also show elevated discounting rates in the CCD condition (mL-M). This finding is consistent with the Bickel et al. (2011a,b) study, that reported that individuals who steeply discount a given commodity would also exhibit relatively greater discounting for other commodities. The current results support the findings from prior studies that suggest steeper discounting may be a trait component wherein the individual has relatively unchanging patterns of behaviours regarding the preference for smaller-sooner rewards over larger-later rewards (Odum, 2011b). These results also support the hypothesis that the tendency to make impulsive decisions may be a trans-disease process underlying a wide array of problematic behaviours (e.g. additive behaviours, poor health practices) (Bickel et al., 2012; Hamilton et al., 2015). Overall, these findings support the idea that DD of certain commodities may be closely related and the results of one discounting task among e-cigarette users, either SCD or CCD, may be generalized to the results of other commodities. Accordingly, in specific contexts, such as in clinical settings, the use of only one task may be a cost-efficient approach to explore delay discounting among e-cigarette users.
We also found significant associations among discounting and clinically relevant measures of e-cigarette use. More specifically, we found that greater discounting in the SCD tasks was associated with more unsuccessful attempts to quit vaping. This finding is in agreement with prior studies that reported that DD predicts (Dallery and Raiff, 2007; Yoon et al., 2007) relapse among users of other tobacco products. Nicotine is a powerfully addictive substance (Benowitz, 2009) and an appealing commodity relative to alternative reinforcers independent of the source of delivery (Abrams et al., 2018; Stiles et al., 2018). Even though prior studies have suggested that e-cigarettes may have low addictive potential, our findings provide evidence that e-cigarettes have the potential for abuse liability (Stiles et al., 2017, 2018). This finding is relevant since it provides evidence that DD tasks can serve as prognostic indicator of relapse among e-cigarette users, which has already been established for other tobacco products (i.e., cigarettes) (Dallery and Raiff, 2007; Sheffer et al., 2014; Yoon et al., 2007).
Some clinical implications of this study merit comment. We found that the preference for the immediately available e-cigarette liquid was less pronounced when an alternative commodity was available after a delay, in this case money. This information could be relevant for interventions that link drug abstinence with long-term outcomes. For example, interventions that focus on specific aspects of recovery associated with personally salient long-term goals (e.g., Motivational Interviewing) may be useful to reduce e-cigarette use. We also found that discounting rates for SCD and CCD were highly correlated which indicates that e-cigarette users that steeply discount e-liquid also steeply discount other commodities such that they may have an overall tendency to make impulsive choices. Therefore, it is possible that an intervention that reduced delay discounting for a given commodity could also impact the discounting of other commodities, and consequently reduce the pattern of poor decision making. Another implication is that delay discounting rates can be used to determine the most appropriate treatment for e-cigarette users. For example, individuals who tend to discount more steeply may need interventions focused on promoting consideration of delayed outcomes (e.g., episodic thinking) (Stein et al., 2018a). Finally, until more refined measures to assess dependence among e-cigarette users are developed, delay discounting tasks could be used as a behavioral marker of dependence among e-cigarette users.
There are several limitations of the current study that should be noted. First, the delay discounting task is a hypothetical measure of preference. Although, it should be noted that prior studies have shown that hypothetical choices are associated with real world behaviours (Lawyer et al., 2011). Second, delay discounting task data was obtained from moderately dependent individuals, reducing the generalizability to more dependent e-cigarette users. Third, 28.9 % of the eligible participants were excluded because they provided non-systematic discounting data. It should be noted that this rate of exclusion is lower than in prior studies exploring SCD and CCD (Moody et al., 2017). Fourth, while the amount of e-liquid offered during the discounting task was equivalent in cost to the money option, we could not be certain whether the value for the same amount of e-liquid and money were subjectively equal since neither exchange rate or utility were assessed in this study, as in most prior literature exploring SCD and CCD (Moody et al., 2017; Yoon et al., 2009, 2018). Fifth, this study only included the CCD task when the substance (i.e., juice) was immediately available and money was offered after a delay (mL-M). However, given the overwhelming evidence indicating that individuals with a SUD are less likely to use drugs when the option of use is presented as a future outcome rather than an immediate option and the study’s time constraints, this condition was not included. Finally, delay discounting was evaluated while e-cigarette users were relatively satiated, discounting rates may have differed under abstinence. In this regard, there is evidence that short-term nicotine abstinence increases delay discounting rates among smokers (Field et al., 2006; Mitchell, 2004). Future research should explore whether nicotine deprivation influences discounting rates among e-cigarette users.
Despite these limitations, the current study provides the first step forward in understanding both SCD and CCD discounting among e-cigarette users. Future studies are still needed to explore whether these results may differ based on other commodities (e.g. conventional cigarettes) or e-cigarette users characteristics (e.g. low-socioeconomic individuals).
Acknowledgements
The authors thank Dr. Moonseong Heo for comments and suggestions.
Role of funding source
Research reported in this publication was supported by the Center of Biomedical Research Excellence award P20GM103644 from the National Institute of General Medical Sciences (NIGMS), and National Institute of Drug Abuse (NIDA), and NIDA/FDA grant U54DA031659. The content is solely the responsibility of the authors and does not necessarily represent the official views of NHLBI, NIDA, NIGMS, or the FDA
Footnotes
Declaration of Competing Interest
The authors have no conflict of interest.
References
- Abrams DB, Glasser AM, Pearson JL, Villanti AC, Collins LK, Niaura RS, 2018. Harm minimization and tobacco control: reframing societal views of nicotine use to rapidly save lives. Ann. Rev. Public. Health 39, 193–213. 10.1146/annurev-publhealth-040617-01384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Amlung M, Vedelago L, Acker J, Balodis I, MacKillop J, 2017. Steep delay discounting and addictive behavior: a meta-analysis of continuous associations. Addiction 112, 51–62. 10.1111/add.13535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baker F, Johnson MW, Bickel WK, 2003. Delay discounting in current and never-before cigarette smokers: similarities and differences across commodity, sign, and magnitude. J. Abnorm. Psychol 112, 382–392. 10.1037/0021-843X.112.3.382. [DOI] [PubMed] [Google Scholar]
- Benowitz NL, 2009. Pharmacology of nicotine: addiction, smoking-induced disease, and therapeutics. Annu. Rev. Pharmacol. Toxicol 49, 57–71 10.1146/annurev.pharmtox.48.113006.094742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Benowitz NL, 2017. Biochemical Verification of Tobacco Use and Cessation SRNT Seminar. University of California, San Francisco: Available via https://www.srnt.org/general/custom.asp?page=biochemical_verific. Accessed 2 October 2019. [Google Scholar]
- Białaszek W, Marcowski P, Cox DJ, 2017. Differences in delay, but not probability discounting, in current smokers, E-cigarette users, and never smokers. Psychol. Rec 67, 223–230. 10.1007/s40732-017-0244-l. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Jarmolowicz DP, Mueller ET, Koffarnus MN, Gatchalian KM, 2012. Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence. Pharmacol. Ther 134, 287–297. 10.1016/j.pharmthera.2012.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Koffarnus MN, Moody L, Wilson AG, 2014. The behavioral-and neuro-economic process of temporal discounting: a candidate behavioral marker of addiction. Neuropharmacology. 76, 518–527. 10.1016/j.neuropharm.2013.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Landes RD, Christensen DR, Jackson L, Jones BA, Kurth-Nelson Z, Redish AD, 2011a. Single-and cross-commodity discounting among cocaine addicts: the commodity and its temporal location determine discounting rate. Psychopharmacology 217, 177–187. 10.1007/s00213-011-2272-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Marsch LA, 2001. Toward a behavioral economic understanding of drug dependence: Delay discounting processes. Addiction 96, 73–86. 10.1046/j.1360-0443.2001.961736.x. [DOI] [PubMed] [Google Scholar]
- Bickel WK, Snider SE, Quisenberry AJ, Stein JS, 2017. Reinforcer pathology: the behavioral economics of abuse liability testing. Clin. Pharmacol. Ther 101,185–187. 10.1002/cpt.443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickel WK, Yi R, Landes RD, Hill PF, Baxter C, 2011b. Remember the future: working memory training decreases delay discounting among stimulant addicts. Biol. Psychiatry 69, 260–265. 10.1016/j.biopsych.2010.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Borges AM, Kuang J, Milhorn H, Yi R, 2016. An alternative approach to calculating Area-Under-the-Curve (AUC) in delay discounting research. J. Exp. Anal. Behav 106, 145–155. 10.1002/jeab.219. [DOI] [PubMed] [Google Scholar]
- Breland A, Soule E, Lopez A, Ramoa C, El-Hellani A, Eissenberg T, 2017. Electronic cigarettes: what are they and what do they do? Ann. N. Y. Acad. Sci 1394, 5–30. 10.1111/nyas.12977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Coffey SF, Gudleski GD, Saladin ME, Brady KT, 2003. Impulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals. Exp. Clin. Psychopharmacol 11, 18–25. 10.1037/1064-1297.11.1.18. [DOI] [PubMed] [Google Scholar]
- Cohen J, 1988. Statistical Power Analysis for the Behavioral Sciences. Erlbaum, New Jersey. [Google Scholar]
- Dallery J, Raifif BR, 2007. Delay discounting predicts cigarette smoking in a laboratory model of abstinence reinforcement. Psychopharmacology 190, 485–496. 10.1007/s00213-006-0627-5. [DOI] [PubMed] [Google Scholar]
- Dowd AN, Motschmam CA, Tiffany ST, 2018. Development and validation of the questionnaire of vaping craving. Nicotine Tob. Res 21, 63–70. 10.1093/ntr/nty046. [DOI] [PubMed] [Google Scholar]
- Evans SE, Hoffman AC, 2014. Electronic cigarettes: abuse liability, topography and subjective effects. Tob. Control 23, ii23–ii29 10.1136/tobaccocontrol-2013-051489. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Field M, Christiansen P, Cole J, Goudie A, 2007. Delay discounting and the alcohol Stroop in heavy drinking adolescents. Addiction 102, 579–586. 10.1111/j.1360-0443.2007.01743.x. [DOI] [PubMed] [Google Scholar]
- Field M, Santarcangelo M, Sumnall H, Goudie A, Cole J, 2006. Delay discounting and the behavioural economics of cigarette purchases in smokers: the effects of nicotine deprivation. Psychopharmacology 86, 255–263. 10.1007/S002-006-0385-4. [DOI] [PubMed] [Google Scholar]
- Foulds J, Veldheer S, Yingst J, Hrabovsky S, Wilson SJ, Nichols TT, Eissenberg T, 2014. Development of a questionnaire for assessing dependence on electronic cigarettes among a large sample of ex-smoking E-cigarette users. Nicotine Tob. Res 17, 185–192. 10.1093/ntr/ntu204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Friedel JE, DeHart WB, Madden GJ, Odum AL, 2014. Impulsivity and cigarette smoking: discounting of monetary and consumable outcomes in current and non-smokers. Psychopharmacology 231, 4517–4526. 10.1007/s00213-014-3597-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- García-Rodríguez O, Secades-Villa R, Weidberg S, Yoon JH, 2013. A systematic assessment of delay discounting in relation to cocaine and nicotine dependence. Behav. Process 99, 100–105. 10.1017/j.beproc.2013.07.007. [DOI] [PubMed] [Google Scholar]
- Giordano LA, Bickel WK, Loewenstein G, Jacobs EA, Marsch L, Badger GJ, 2002. Mild opioid deprivation increases the degree that opioid-dependent outpatients discount delayed heroin and money. Psychopharmacology 163, 174–182. 10.1007/s00213-002-1159-2. [DOI] [PubMed] [Google Scholar]
- González-Roz A, Secades-Villa R, Pericot-Valverde I, Weidberg S, Alonso-Pérez F, 2019. Effects of delay discounting and other predictors on smoking relapse. Span. J. Psychol 22, E9 10.1016/sjp.2019.11. [DOI] [PubMed] [Google Scholar]
- Hamilton KR, Mitchell MR, Wing VC, Balodis I, Bickel WK, Fillmore M, Lane SD, 2015. Choice impulsivity: definitoins, measurement issues, and clinical implications. Personal. Disord 6, 182–198. 10.1037/per0000099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Higgins ST, 1997. The influence of alternative reinforcers on cocaine use and abuse: a brief review. Pharmacol. Biochem. Behav 57, 419–427. 10.1016/S0091-3057(96)00446-7. [DOI] [PubMed] [Google Scholar]
- Huerta TR, Walker DM, Mullen D, Johnson TJ, Ford EW, 2017. Trends in e-cigarette awareness and perceived harmfulness in the US. Am. J. Prev. Med 52 (339), 346 10.1016/j.amepre.2016.10.017. [DOI] [PubMed] [Google Scholar]
- Jarmolowicz DP, Landes RD, Christensen DR, Jones BA, Jackson L, Yi R, Bickel WK, 2014. Discounting of money and sex: effects of commodity and temporal position in stimulant-dependent men and women. Addict. Behav 39, 1652–1657. 10.1016/j.addbeh.2014.04.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson MW, Bickel WK, 2008. An algorithm for identifying nonsystematic delay-discounting data. Exp. Clin. Psychopharmacol 16, 264–274. 10.1037/1064-12222222297.16.3.264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson MW, Bickel WK, Baker F, Moore BA, Badger GJ, Budney AJ, 2010. Delay discounting in current and former marijuana-dependent individuals. Exp. Clin. Psychopharmacol. 18, 99–107. 10.1037/a0018333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnston LD, O’Malley PM, Miech RA, Bachman JG, Schulenberg JE, 2015. Monitoring the Future: National Results on Adolescent Drug Use2014. Avaiable via http://www.monitoringthefuture.org/pubs/monographs/mtf-overview2014.pdf Accessed 2 October 2019. .
- Koffarnus MN, Johnson MW, Thompson-Lake DG, Wesley MJ, Lohrenz T, Montague PR, Bickel WK, 2016. Cocaine-dependent adults and recreational cocaine users are more likely than controls to choose immediate unsafe sex over delayed safer sex. Exp. Clin. Psychopharmacol 24, 297–304. 10.1037/pha0000080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lawyer SR, Schoepflin F, Green R, Jenks C, 2011. Discounting of hypothetical and potentially real outcomes in nicotine-dependent and nondependent samples. Exp. Clin. Psychopharmacol 19, 263–274. 10.1037/a0024141. [DOI] [PubMed] [Google Scholar]
- Leventhal AM, Strong DR, Kirkpatrick MG, Unger JB, Sussman S, Riggs NR, Audrain-McGovern J, 2015. Association of electronic cigarette use with initiation of combustible tobacco product smoking in early adolescence. JAMA 314, 700–707. 10.1001/jama.2015.8950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Loukas A, Marti C, Cooper M, Pasch KE, Perry CL, 2018. Exclusive e-cigarette use predicts cigarette initiation among college students. Addict. Behav 76 10.1016/j.addbeh.2017.08.023.434-347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, Munafo MR, 2011. Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology 216, 305–321. 10.1007/s00213-011-2229-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Madden GJ, Petry NM, Badger GJ, Bickel WK, 1997. Impulsive and self-control choices in opioid-dependent patients and non-drug-using control patients: drug and monetary rewards. Exp. Clin. Psychopharmacol 5, 256–262. 10.1037/1064-1297.5.5.256. [DOI] [PubMed] [Google Scholar]
- Marrone GF, Shakleya DM, Scheidweiler KB, Singleton EG, Huestis MA, Heishman SJ, 2011. Relative performance of common biochemical indicators in detecting cigarette smoking. Addiction 106, 1325–1334. 10.1111/j.1360-0443.2011.03441.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mazur JE, 1987. An adjusting procedure for studying delayed reinforcement In: In: Commons Mazur JE, Nevin A, Rachlin H (Eds.), Quantitative Analyses of Behavior Vol 5 The effect of delay and of intervening events on reinforcement value. Erlbaum, New Jersey, pp. 55–74. [Google Scholar]
- Mellis AM, Athamneh LN, Stein JS, Sze Y, Epstein LH, Bickel WK, 2018. Less is more: negative income shock increases immediate preference in cross commodity discounting and food demand. Appetite 129, 155–161. https://doi.org/10.10l6/j.appet.2018.06.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mellis AM, Woodford AE, Stein JS, Bickel WK, 2017. A second type of magnitude effect: reinforcer magnitude differentiates delay discounting between substance users and controls. J. Exp. Anal. Behav 107, 151–160. 10.1002/jeab.235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mitchell SH, 2004. Effects of short-term nicotine deprivation on decision-making: delay, uncertainty and effort discounting. Nicotine Tob. Res 6, 819–828. 10.1080/14622200412331296002. [DOI] [PubMed] [Google Scholar]
- Moody LN, Tegge AN, Bickel WK, 2017. Cross-commodity delay discounting of alcohol and money in alcohol users. Psychol. Rec 67, 285–292. 10.1007/S40732-017-0245-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Myerson J, Green L, Warusawitharana M, 2001. Area under the curve as a measure of discounting. J. Exp. Anal. Behav 76, 235–243. 10.1901/jeab.2001.76-235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Odum AL, 2011a. Delay discounting: I’m ak, you’re ak. J. Exp. Anal. Behav 96, 427–439. 10.1901/jeab.2011.96-423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Odum AL, 2011b. Delay discounting: trait variable? Behav. Process 87, 1–9. 10.1016/j.beproc.2011.02.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Odum AL, Madden GJ, Badger GJ, Bickel WK, 2000. Needle sharing in opioid-dependent outpatients: psychological processes underlying risk. Drug Alcohol Depend. 60, 259–266. 10.1016/S0376-8716(00)00111-3. [DOI] [PubMed] [Google Scholar]
- Pericot-Valverde I, Tromblee M, Diaz V, Nesheim-Case R, Gallagher T, Rogers E, Gaalema DE, Higgins ST, 2018. Modeling the effects of stress on vaping relapse behavior. Poster Presented at the Substance Use and Addiction Conference Avaialble via https://www.abainternational.org/events/program-details/event-detail.aspx?sid=57232andby=author#s5_8. Accessed 2 October 2019. [Google Scholar]
- Pericot-Valverde I, Gaalema DE, Priest JS, Higgins ST, 2017. E-cigarette awareness, perceived harmfulness, and ever use among US adults. Prev. Med 104, 92–99. 10.1016/j.ypmed.2017.07.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Petry NM, 2001. Delay discounting of money and alcohol in actively using alcoholics, currently abstinent alcoholics, and controls. Psychopharmacology 154, 243–250. 10.1007/s002130000638. [DOI] [PubMed] [Google Scholar]
- Reynolds B, 2006. A review of delay-discounting research with humans: relations to drug use and gambling. Behav. Pharmacol 17, 651–667. 10.1097/FBP.0b013e3280115f99. [DOI] [PubMed] [Google Scholar]
- Sawdey MD, Day HR, Coleman B, Gardner LD, Johnson SE, Limpert J, Pearson JL, 2018. Associations of risk factors of e-cigarette and cigarette use and susceptibility to use among baseline PATH study youth participants (2013–2014). Addict. Behav 91, 51–60. 10.1016/j.addbeh.2018.ll.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Secades-Villa R, Pericot-Valverde I, Weidberg S, 2016. Relative reinforcing efficacy of cigarettes as a predictor of smoking abstinence among treatment-seeking smokers. Psychopharmacology 233, 3103–3112. 10.1007/s00213-016-4350-6. [DOI] [PubMed] [Google Scholar]
- Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Dunbar GC, 1998. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychiatry 59, 22–33. [PubMed] [Google Scholar]
- Sheffer CE, Christensen DR, Landes R, Carter LP, Jackson L, Bickel WK, 2014. Delay discounting rates: a strong prognostic indicator of smoking relapse. Addict. Behav 39, 1682–1689. 10.1016/j.addbeh.2014.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Snider SE, La Conte SM, Bickel WK, 2016. Episodic future thinking: expansion of the temporal window in individuals with alcohol dependence. Alcoholism Clin Exp Res. 40, 1558–1566. 10.1111/acer.13112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spindle TR, Hiler MM, Cooke ME, Eissenberg T, Kendler KS, Dick DM, 2017. Electronic cigarette use and uptake of cigarette smoking: a longitudinal examination of US college students. Addict. Behav 67, 66–72. 10.1016/j.addbeh.2016.12.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stanger C, Budney AJ, Bickel WK, 2013. A developmental perspective on neuroeconomic mechanisms of contingency management. Psychol. Addict. Behav 27, 403–415. 10.1037/a0028748. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stein JS, Heckman BW, Pope DA, Perry ES, Fong GT, Cummings KM, Bickel WK, 2018a. Delay discounting and e-cigarette use: an investigation in current, former, and never cigarette smokers. Drug Alcohol Dependen. 191,165–173. 10.1016/j.drugalcdep.2018.06.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stein JS, Tegge AN, Turner JK, Bickel WK, 2018b. Episodic future thinking reduces delay discounting and cigarette demand: an investigation of the good-subject effect. J. Behav. Med 41, 269–276. 10.1007/sl0865-017-9908-l. [DOI] [PubMed] [Google Scholar]
- Steward T, Mestre-Bach G, Vintro-Alcaraz C, Agiiera Z, Jimenez-Murcia S, Granero R, Fernandez-Aranda F, 2017. Delay discounting of reward and impulsivity in eating disorders: from anorexia nervosa to binge eating disorder. Eur. Eat. Disord. Rev 25, 601–606. 10.1002/erv.2543. [DOI] [PubMed] [Google Scholar]
- Stiles MF, Campbell LR, Graff DW, Jones BA, Fant RV, Henningfield JE, 2017. Pharmacodynamic and pharmacokinetic assessment of electronic cigarettes, combustible cugarettes, and nicotine gum: implications for abuse liability. Psychopharmacology 234, 2643–2655. 10.1007/s00213-17-4665-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stiles MF, Campbell LR, Jin T, Graff DW, Fant RV, Henningfield JE, 2018. Assessment of the abuse liability of three menthol Vuse Solo electronic cigarettes relative to combustible cigarettes and nicotine gum. Psychopharmacology 235, 2077–2086. 10.1007/s00213-018-4904-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- US Department of Health and Human Service, 2016. E-Cigarette Use Among Youth and Young Adults: A Report of the Surgeon General. Available via https://e-cigarettes.surgeongeneral.gov/documents/2016_sgr_full_report_non-508.pdf. Accessed 2 October 2019. .
- Washio Y, Higgins ST, Heil SH, McKerchar TL, Badger GJ, Skelly JM, Dantona RL, 2011. Delay discounting is associated with treatment response among cocaine-dependent outpatients. Expl. Clin. Psychopharmacol 19, 243–248. 10.1037/a0023617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weidberg S, Gonzalez-Roz A, Secades-Villa R, 2017. Delay discounting in e-cigarette users, current and former smokers. Int. J. Clin. Health Psychol 17, 20–27. 10.1016/j.ijchp.2016.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yoon JH, De La Garza R, Newton TF, Suchting R, Weaver MT, Brown GS, Haliwa I, 2017. A comparison of Mazur’sk and area under the curve for describing steep discounters. Psychol. Rec 67, 355–363. 10.1007/s40732-017-0220-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yoon JH, Higgins ST, 2008. Turning k on its head: comments on use of an ED50 in delay discounting research. Drug Alcohol Dependen. 95, 169–172. 10.1016/j.drugalcdep.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yoon JH, Higgins ST, Bradstreet MP, Badger GJ, Thomas CS, 2009. Changes in the relative reinforcing effects of cigarette smoking as a function of initial abstinence. Psychopharmacology 205, 305–318. 10.1007/s00213-009-1541-4. [DOI] [PubMed] [Google Scholar]
- Yoon JH, Higgins ST, Heil SH, Sugarbaker RJ, Thomas CS, Badger GJ, 2007. Delay discounting predicts postpartum relapse to cigarette smoking among pregnant women. Expl. Clin. Psychopharmacol 15, 176–186. 10.1037/1064-1297.15.2.186. [DOI] [PubMed] [Google Scholar]
- Yoon JH, Weaver MT, De La Garza R, Suchting R, Nerumalla CS, Omar Y, Brown GS, Haliwa I, Newton TF, 2018. Comparison of three measurement models of discounting among individuals with methamphetamine use disorder. Am. J. Addict 27, 425–432. 10.1111/ajad.12761. [DOI] [PubMed] [Google Scholar]