Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Feb 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2020 Aug 6;30(1):39–50. doi: 10.1037/pha0000423

Effects of substance use on monetary delay discounting among people who use stimulants with and without HIV: an ecological momentary assessment study

Yunan Xu 1,*, Sheri L Towe 1, Shakiera T Causey 1, Paul A Dennis 1,2, Christina S Meade 1
PMCID: PMC8407024  NIHMSID: NIHMS1731619  PMID: 32757596

Abstract

Exploration of the real-time relationship between substance use and delay discounting may reveal potential mechanisms driving high-risk behaviors. We conducted an ecological momentary assessment (EMA) study to investigate the effects of substance use on delay discounting in a sample of people who use stimulants (HIV+: 30; HIV−: 34). Participants completed multiple EMAs throughtout the day for 28 days. The EMAs collected data on delay discounting and substance use (time since last substance use and level of intoxication). Delay discounting was assessed using a brief Monetary Choice Questionnaire (MCQ). Analyses were conducted using linear mixed effects modeling. Most participants (99.1%) used cocaine as their primary stimulant. Among participants without HIV, MCQ score remained relatively stable during the first 2 hours after stimulant use, followed by an increase during 2–6 hours (p<0.05), before decreasing again. For alcohol and marijuana, the MCQ score was stable during the first 4 hours after use, with a sharp increase at 4–6 hours (p<0.05), before decrease again. Among participants with HIV, there were no changes in MCQ score as a function of time since recent substance use. These findings provide evidence of a plausible connection between delay discounting and acute withdrawal that may have relevance for risky behaviors.

Keywords: substance use, delay discounting, HIV, ecological momentary assessment

Introduction

Stimulants remain a commonly used class of drug in the United States (Piper et al., 2018). In 2018, the prevalence of past month use was estimated to be 2.2 million for cocaine and 0.8 million for methamphetamine (Bose, Hedden, R.N., & E, 2018). Both cocaine and methamphetamine produce an intensely pleasurable “rush”, followed by euphoria along with increased energy, attention, wakefulness, and activity (National Institute on Drug Abuse (NIDA), 2016b, 2019). Acute withdrawal symptoms that can emerge within hours of last use include exhaustion, hypersomnia, dysthymia, irritability, and lack of cravings (Jenner & Saunders, 2005). Many people who use stimulants also use other substances, including alcohol, marijuana, and opioids (Herbeck et al., 2013; John & Wu, 2017). While both cocaine and methamphetamine are neurotoxic (Jablonski, Williams, & Vorhees, 2016; Mader, Ramos, Cruz, & Branch, 2019), chronic substance use in general disrupts neural circuitry implicated in reward-based decision making (Volkow et al., 2010; Volkow, Wang, Tomasi, & Baler, 2013). These changes are associated with a wide range of risky behaviors (Wardle, Gonzalez, Bechara, & Martin-Thormeyer, 2010).

Delay discounting has been applied to examine pathological decision making across a range of psychological disorders and unhealthy behaviors (Amlung, Petker, Jackson, Balodis, & MacKillop, 2016; Bickel, Johnson, Koffarnus, MacKillop, & Murphy, 2014; Jackson & MacKillop, 2016). Delay discounting indexes the subjective reduction in the value of a future reward as the delay to that reward increases (Mazur, 1987; McKerchar et al., 2009). Money is most often used in these delay discounting tasks because it is a secondary reinforcer that is universally understood and has broad applicability. Monetary delay discounting, although specific to financial impulsivity, has become a domain-general measurement in the delay discounting paradigm. It has been shown to be associated with a number of health-related behaviors, such as unprotected sex, gambling, smoking initiation, substance use, and relapse following substance use cessation (Albein-Urios, Martinez-Gonzalez, Lozano, & Verdejo-Garcia, 2014; Audrain-McGovern et al., 2009; Jones et al., 2018; Yoon et al., 2007).

Numerous studies have reported steeper delay discounting of monetary reward among individuals with stimulant, alcohol, and marijuana use disorders compared with non-drug using controls (Ashe, Newman, & Wilson, 2015; MacKillop et al., 2011). While the effects of chronic substance use on delay discounting have been well-established, the moment-to-moment variability of delay discounting within individuals following substance use is not well understood. Identifying the acute effects may be critical to understanding the processes involved in risk behavior and identifying targets for behavioral interventions. Findings of laboratory studies have suggested that monetary delay discounting does not change following acute administration of cocaine, alcohol, or cannabidiol (Adams, Attwood, & Munafo, 2017; Hindocha et al., 2018; M. W. Johnson, Herrmann, Sweeney, LeComte, & Johnson, 2016; P. S. Johnson, Sweeney, Herrmann, & Johnson, 2016). However, these studies were conducted in strictly controlled lab environments with precise dosage of the administered subtance and timing of delay discounting measurements. As such, findings of these studies may not reflect real-life situations, which can be influenced by social and environmental cues that are absent in the laboratory.

Ecological momentary assessment (EMA) methods provide an opportunity to evaluate the real-world effects of substance use on delay discounting in daily life. With numerous time points, it is possible to assess temporal changes of a phenomenon with respect to a specific event such as patterns of delay discounting associated with substance use. Mobile health (mHealth) technologies, including smartphone apps, have been developed and applied in medical care and public health studies to implement EMA methods. Research findings have supported the validity, reliability, feasibility, and acceptability of EMA using mHealth technology in a wide range of health behaviors and conditions, including chronic medical diseases (Kamarck, Li, Wright, Muldoon, & Manuck, 2018; Kratz, Murphy, & Braley, 2017), lifestyle (Biddle, Gorely, Marshall, & Cameron, 2009; Schumacher et al., 2018), risky sexual behaviors (Mackesy-Amiti & Boodram, 2018; Newcomb & Mustanski, 2014), and substance use (Diener & Emmons, 1984; Moskowitz & Young, 2006; Yang et al., 2015). However, to our knowledge, EMA methods have not been applied to study within-person change in delay discounting in relation to recency of substance use.

The association between HIV infection, substance use, and delay discounting is inconclusive. Some studies have linked HIV infection to impairments in decision-making capacities (Iudicello et al., 2013; E. M. Martin et al., 2013; Vassileva et al., 2013). Prior work within our research group has also shown that HIV infection can alter brain functioning in regions that support intertemporal decision making (Meade et al., 2016). This study found that participants with HIV demonstrated significantly larger increases than participants without HIV in activation in the prefrontal cortex when they made choices between smaller, sooner monetary rewards and larger, delayed rewards. These changes were significantly associated with lower nadir CD4+ T cell counts. In addition, studies have found an interactive effect of cocaine use and HIV infection on the neural circuitry of monetary decision making (Meade et al., 2018; Meade et al., 2017). However, a behavioral study did not find an independent effect of HIV on delay discounting in a sample of persons with a history of substance use (E. M. Martin, Gonzalez, Vassileva, & Bechara, 2019).

Using mHealth technology, we conducted this EMA study to collect real-time data on delay discounting and substance use in a sample of people who use stimulants. Participants completed a monetary delay discounting task at randomly prompted times throughout the day and reported on recent substance use on a smartphone. We aimed to evaluate the within-person changes in monetary delay discounting in relation to recency of substance use. In contrast to experimental studies that have not identified a relationship between acute intoxication and delay discounting, we expected that delay discounting would be highest immediately following substance use. In addition, given the mixed findings of the effects of HIV and substance use on delay discounting, we also tested whether HIV infection modulates the relationship between acute substance use and delay discounting.

Methods

Participants

Participants were enrolled in a cohort study of people who use stimulants with and without HIV. Inclusion criteria for the mHealth study were: 1) 18–59 years old; 2) stimulant use (cocaine or methamphetamine) in the past 30 days; 3) sexual activity in the past 30 days; 4) literacy; and 5) willingness to use a smartphone to complete the assessments for 28 days. Participants who met these eligibility criteria were invited to participate in the mHealth study between June 2017 and May 2019.

Procedures

After providing written informed consent, study staff provided training on how to use the study app, an overview of the assessments, and an explanation of the compensation schedule. Participants had the option of borrowing a study smartphone (iPhone SE or 6S) or using their own smartphone.

To account for the reality that participants would not be able to complete EMAs at all times of the day due to work schedules, sleep, and other restricting activities, participants received four prompts each day between 12:00PM and 2:00AM. The prompts were programmed to occur randomly within four fixed time blocks (12:00PM-3:30PM, 4:00PM-7:30PM, 8:00PM-11:30PM, and 12:00AM-2:00AM). Participants were notified of each assessment via a push notification on their smartphone. If the assessment was not initiated within 30 minutes, a second push notification was sent. After 60 minutes, the assessment period closed and responses could no longer be submitted. All assessments were programmed using the MetricWire mobile app (Trafford, 2015). The EMAs included a brief delay discounting questionnaire, followed by questions on substance use and related variables. The delay discounting items were administered first in order to avoid potential priming effects. At the end of 28 days, participants returned to the lab for a follow-up session during which they returned the study equipment or deleted the study app from their personal phone.

Participants were compensated $25 each for the enrollment visit and the follow-up visit. Participants who borrowed a study phone received a $50 bonus when the phone equipment was returned, while participants who used their own phone received a $50 bonus to compensate them for data usage. To incentivize survey compliance, participants were compensated on a weekly basis for the number of completed EMAs. Participants received: $16 for completing ≥23 surveys; $12 for 20–22 surveys; $10 for 17–19 surveys; $8 for 14–16 surveys; $6 for 7–13 surveys; and $3 for 1–6 surveys. All payments were made on a reloadable debit card. Study procedures were approved by the institutional review board at Duke University Health System (Protocol number: Pro00069412; Protocol title: Project Avenir - New Approaches to Decision Making and HIV Risk Research)

Measures

Delay discounting

This study used an adapted version of the Monetary Choice Questionnaire (MCQ-36) (Kirby, Petry, & Bickel, 1999; Towe, Hobkirk, Ye, & Meade, 2015). The MCQ measures an individual’s preferences for immediate versus delayed rewards by asking them to choose between rewards available immediately and larger rewards available after a delay. The relationship between the perceived value of the immediate reward to the delayed reward can be described using the following hyperbolic function: Vimmediate = Vdelayed / (1+kD), where V is the size of the reward in US dollars and D is the duration of the delay in days (Mazur, 1987). The k-value (1/days) is a free delay discounting parameter, with higher k-values (1/days) indicating a greater preference for smaller, immediate rewards over larger, delayed rewards.

The MCQ-36 has 36 choices associated with 12 k-values (1/days) ranging from 0.00016 to 4.0 across small, medium, and large delayed rewards (Towe et al., 2015). For the present study, in order to minimize the length of the EMA assessment, we restricted to items from the large reward category. To minimize the likelihood that participants would respond rotely due to seeing the same questions multiple times per day, we developed six sets of 12-item scales that had delayed options of $70–99 and delays of 2–90 days. The immediate options were computed by fitting the above hyperbolic function for each of the 12 base k-values (see Appendix Table 1a and 1b). For example, participants were asked, “Would you prefer $27 today or $89 in 36 days?” Questions were presented in random order at each administration. Participants’ k-value at each EMA was estimated based on the pattern of choices across the 12 questions using standard scoring procedures (Kirby et al., 1999), where the proportion of choices that were consistent with each k-value was calculated to determine the most likely k-value (1/days) (Towe et al., 2015). To create a final MCQ score, the estimated k-values (1/days) were natural log transformed to correct for skewness and obtain an approximately normal distribution for analysis.

For quality assurance, MCQ data were examined to check that participants appeared to understand the task and respond in a consistent manner. If the highest proportion of consistent choices was ≤0.70 or if ≥2 k-values were tied as the highest proportion, the response pattern was considered invalid and the entire event was excluded from analysis. Of 3716 MCQ responses, 3372 (90.7%) were valid. The mean proportion of valid responses was similar for participants with (88.7%, SD= 9.1) and without (90.0%, SD= 1.8) HIV [t(62)= 0.81, p= .42]. Validity of MCQ responses was not associated with the time since substance use and being intoxicated (Appendix Table 2).

Substance use

Participants were first asked, “Within the last 6 hours, which of these drugs have you used?” The list included cocaine, other stimulants, marijuana, alcohol, opioids, and other drugs. For other drugs, participants typed the name of the drug. Participants who reported any substance use in the past 6 hours were asked the following two questions for each specific substance: 1) “How long has it been since you used [substance]?” (<1 hour ago, 1–2 hours ago, 2–4 hours ago, and >4 hours ago); and 2) “How high on [substance] are you feeling right now?” (0–10 scale). Cocaine and other stimulants were combined for analysis by taking the most recent time of last use and the highest rating of intoxication.

Other variables

Descriptive characteristics were collected as part of the cohort visit immediately preceding enrollment into the mHealth study. Participants completed an audio computer-assisted self-interview (ACASI), which assessed age, gender, race, education, and employment. Frequencies of substance use in the past 30 days were obtained using Timeline Follow-back methodology (Sobell & Sobell, 1996). Self-reported HIV status was confirmed using a rapid oral fluid rapid test (OraQuick®) or by chart review for participants with an established HIV diagnosis. HIV clinical data (nadir CD4+ T cell count, most recent viral load and CD4+ T cell count, years since HIV diagnosis, and current antiretroviral therapy) were abstracted from medical records. Data on adherence to HIV medication were collected using 100-point Visual Analog Scale (VAS), with 0 indicating “no doses of medication taken” and 100 indicating “all doses of medication taken” in the last 4 weeks. The adherence to each medication was asked and the overall VAS score was computed by averaging the VAS score of each medication. VAS score 95% or higher was defined as “optimal adherence” (Mai et al., 2018).

Statistical analysis

Sample characteristics for participants with and without HIV were compared using t-tests for continuous variables with normal distribution, Wilcoxon rank test for continuous variables non-normally distributed, and chi-square tests for all other categorical variables. Multilevel models were used to test the effects of time since substance use on MCQ scores. This approach adjusts for the repeated measurements of multiple waves of longitudinal data and is robust to missing data. We constructed three-level linear mixed effects models because randomly prompted entries (first level) were nested within days (second level) which were nested within subjects (third level). Each model allowed for a random intercept, and “≤1 hour ago” was used as the reference group. The fixed effects of time since last use were used to examine the effects of substance use on delay discounting. The primary analysis examined the relationship between recency of any substance use and MCQ scores. Secondary analyses were conducted for any stimulants, cocaine specifically, marijuana, and alcohol to determine if the pattern of the relationship differed by drug class, with separate models for each substance. Preliminary analyses determined that there was no relationship of MCQ scores to age, race, education, and employment, so subsequent analyses were performed without additional adjustment of covariates. All models were first stratified by HIV status, followed by a full model that combined participants with and without HIV. The full model incorporated HIV status, substance use, and a product term of HIV by substance to test the significance of the interaction. In addition, we also conducted the analysis stratified by CD4 nadir (<200 vs. ≥200) to determine whether history of immunosuppression may impact the effects of acute substance use on delay discounting. Analyses were conducted using SAS 9.4 (Cary, NC). Statistical significance was set at p<0.05.

Results

Participant characteristics

A total of 64 participants (HIV+: 30; HIV−: 34) were enrolled in the mHealth study. Sample characteristics are described in Table 1. The majority of the sample was male (62.5%), African American (90.6%), and in their mid-40s (M= 46.5 years, SD= 11.1). In the past 30 days prior to their last cohort visit, 60 (93.8%) participants used only cocaine, 2 (3.1%) used other stimulants but not cocaine, and 2 (3.1%) used cocaine plus another stimulant. Participants reported stimulant use on an average of 13.0 days (SD= 10.4), alcohol on an average of 11.4 days (SD=10.8), marijuana on an average of 12.6 days (SD=12.8), and polysubstance use on an average of 12.9 days (SD= 10.4). There were no differences between participants with and without HIV on these characteristics. Among participants with HIV, the majority (93.3%) were currently on antiretroviral therapy and had CD4+ T cell counts ≥500 cells/μL (70%), but only 66.7% had suprresed viral loads at <50 copies/mL. About half (46.7%) had a nadir CD4+ T cell count <200 cells/μL. Among participants on antiretroviral therapy, 64.3% had optimal ART adherence. The median of years since HIV diagnosis was 15 (IQR: 9, 21).

Table 1.

Participant characteristics

Variable HIV+ (n=30) HIV− (n=34) Statistic p-value
Age, mean (SD) 47.0 (12.4) 44.2 (9.5) t(62)=−0.99 0.325
Gender identity, n (%) X2(1)=0.42 0.518
 Female 10 (33.3) 14 (41.2)
 Male 20 (66.7) 20 (58.8)
Race, n (%) X2(1)=1.04 0.378
 African American 26 (86.7) 32 (94.1)
 Other 4 (13.3) 2 (5.9)
Education, n (%) X2(1)=0.38 0.539
 Less than high school 9 (30.0) 6 (17.7)
 High school or above 21 (70.0) 28 (82.3)
Employment status, n (%) X2(1)=0.28 0.594
 Unemployed, disabled, or retired 17 (56.7) 17 (50.0)
 Employed full or part time 13 (43.3) 17 (50.0)
Days of use in last 30 days, median (IQR)
 Stimulant 8 (3, 23) 12 (6, 22) R=900 0.312
 Marijuana 6 (1, 26) 6 (1, 30) R=969 0.940
 Alcohol 8 (2, 26) 8 (1, 13) R=928 0.530
 Poly-substance 11 (4, 23) 8 (2, 13) R=922 0.475
Open in a new tab

IQR: Interquartile range; SD: Standard error.

EMA data characteristics

Participants completed one or more EMA surveys on an average of 25.8 days (SD=5.6) out of the 28-day study. A total of 3372 events with valid MCQ responses (HIV+: 1629 and HIV−: 1743) were collected. The proportion of responses during which substance use was endorsed was high, with 1664 (49.4%) for any substance, 1126 (33.4%) for stimulants (1111 for cocaine only, 10 for other stimulant only, and 5 for cocaine plus other stimulant), 687 (20.4%) for alcohol, 662 (19.6%) for marijuana, 44 (1.3%) for opioids, and 6 (0.18%) for other drugs. MCQ scores varied markedly both between and within participants across the 28-day study (Figure 1). While the MCQ score was slightly higher for participants with HIV (−2.4, SD= 2.6) than participants without HIV(−2.7, SD= 2.3), this difference was not significant [t(3334)= −0.62, p= .45].

Figure 1. Box plot of MCQ score across the study period among participants with (n=30) and without (n=34) HIV.

Figure 1

Open in a new tab

The solid black line within each box is the median. The circle within each box is the mean. The length of the box is the interquartile range, with the upper bound representing the 75th percentile and the lower bound representing the 25th percentile. The vertical lines issuing from the box extend to the maximum and minimum values.

As expected, time since last use was highly correlated with subjective level of intoxication for stimulants, alcohol, and marijuana. This association was similar for participants with and without HIV (Figure 2).

Figure 2. Association between subjective level of intoxication and time since last substance use among participants with (n=30) and without (n=34) HIV. (a) participants with HIV; (b) participants without HIV.

Figure 2

Open in a new tab

The Y-axis is the least square means of intoxication adjusted for multiple measurements of substance use within days and within subjects using mixed effects model.

Association between substance use and delay discounting

The primary results are summarized in Table 2 and illustrated in Figure 3. For participants without HIV, substance use was associated with increased MCQ score occurring several hours after last use. For stimulants and cocaine, relative to the first hour after use, there was no difference in MCQ score 1–2 hours after use. However, at 2–4 hours and 4–6 hours after use, MCQ score was significantly higher. This increase in MCQ score was no longer evident after 6 hours of use. The pattern was similar for marijuana and alcohol, except the increase in MCQ score only emerged during the 4–6 hours since last use. For participants with HIV, there was no relationship between recency of substance use and MCQ score. The association was similar for each class of substances among participants with nadir CD4+ T cell count <200 and ≥200 cells/μL (p>0.05).

Table 2.

Association between MCQ score and time since substance use among participants with and without HIV

b (SE)
Variable HIV+ (n=30) HIV− (n=34)
Any substance
 <1 hour ago Ref. Ref.
 1–2 hours ago −0.04 (0.07) 0.04 (0.09)
 2–4 hours ago 0.01 (0.09) 0.15 (0.04)*
 4–6 hours ago −0.08 (0.12) 0.32 (0.11)**
 None in the past 6 hours −0.05 (0.06) 0.07 (0.06)
Stimulants
 <1 hour ago Ref. Ref.
 1–2 hours ago 0.02 (0.10) 0.06 (0.10)
 2–4 hours ago 0.09 (0.12) 0.25 (0.08)**
 4–6 hours ago 0.02 (0.15) 0.28 (0.13)**
 None in the past 6 hours 0.03 (0.08) 0.12 (0.07)
Cocaine
 <1 hour ago Ref. Ref.
 1–2 hours ago 0.09 (0.12) 0.05 (0.10)
 2–4 hours ago 0.02 (0.10) 0.24 (0.08)**
 4–6 hours ago −0.06 (0.16) 0.28 (0.13)*
 None in the past 6 hours 0.03 (0.08) 0.12 (0.07)
Marijuana
 <1 hour ago Ref. Ref.
 1–2 hours ago −0.04 (0.11) 0.19 (0.12)
 2–4 hours ago −0.01 (0.14) 0.07 (0.13)
 4–6 hours ago −0.35 (0.20) 0.42 (0.17)*
 None in the past 6 hours −0.06 (0.10) 0.01 (0.09)
Alcohol
 <1 hour ago Ref. Ref.
 1–2 hours ago −0.02 (0.09) 0.05(0.13)
 2–4 hours ago −0.09 (0.12) −0.15 (0.15)
 4–6 hours ago 0.08 (0.17) 0.44(0.18)**
 None in the past 6 hours −0.05 (0.08) 0.13 (0.10)
Open in a new tab
*

p<0.05

**

p<0.01

b (SE) was the unstandardized estimates (standard error) estimated using linear mixed effects models.

Figure 3. Association between MCQ score and time since last substance use among participnats with (n=30) and without (n=34). a) Any substance; b) Any stimulants; c) Cocaine; d) Marijuana; and e) Alcohol.

Figure 3

Figure 3

Open in a new tab

Values are least square means adjusted for multiple measurements of MCQ score within days and within subjects using mixed effects models.

Consistent with the discrepant patterns between participants with and without HIV, there were significant HIV by time since last use interaction effects on MCQ score for any substance [F(4,1988)=2.7, p=0.03], alcohol [F(4,1988)=2.4, p=0.04], and marijuana [F(4,1988)=2.5, p=0.04], but not for stimulants [F(4,1988)= 1.3, p=0.26] and cocaine [F(4,1977)= 1.5, p=0.19].

Discussion

To our knowledge, this is the first study to delineate patterns of monetary delay discounting relative to recency of substance use in the real world using mHealth technology. While we had expected MCQ scores to be highest in the first hours after substance use, instead we found that MCQ scores for participants without HIV were highest several hours after last use. The pattern was similar when examining stimulants, marijuana, and alcohol separately, except that the increase was observed earlier for stimulants (by 2–4 hours after last use) than for alcohol and marijuana (by 4–6 hours after last use). Among participants with HIV, there was no change in delay discounting following substance use, with scores remaining stable across the 6-hour timeframe.

Consistent with lab-based studies, there was no within-person change in delay discounting among participants without HIV in the first 2 hours after stimulant use. However, beginning 2–4 hours after last use, there was a significant increase in delay discounting that persisted through 4–6 hours after last use. For alcohol and marijuana, the increased MCQ score was observed only during 4–6 hours after last use. We speculate that the effects of substance use on monetary delay discounting may relate to the negative physiological effects of substance withdrawal, rather than intoxication as expected. We found a continuous decrease in intoxication over time after substance use, including stimulant, marijuana, and alcohol. The peak of delay discounting did not occur right after substance use when the intoxication was presumably the highest, but after several hours when the intoxication would have subsided. This is consistent with drive-reduction theory, which suggests that the spike in cognitive effects is a response to users seeking gratification after experiencing withdrawal-related symptoms (de Wit, 2009). Prior studies have found cognitive/affective changes accompanied with substance withdrawal, like craving and dysphoria (Moulin et al., 2018; Robles-Martinez et al., 2019).

In contrast to findings from experimental studies that generally did not report changes in monetary delay discounting following acute drug administration, we observed an increase in monetary delay discounting occurring 2–4 hours after last substance use for participants without HIV. The null findings from these experimental studies may be due to an insufficient time window of measuring delay discounting. In prior studies, delay discounting was only measured once and generally collected less than 3 hours post-administration. For example, delay discounting was measured 20 minutes after cocaine administration (M. W. Johnson et al., 2016), 15 and 30 minutes after alcohol administration (Adams et al., 2017; P. S. Johnson et al., 2016), and 150 minutes after cannabidiol administration (Hindocha et al., 2018). The time points of measurement were determined based on the time to reach the peak plasma concentrations of each substance (Hindocha et al., 2018; M. W. Johnson et al., 2016; P. S. Johnson et al., 2016). Relative to cocaine, marijuana and alcohol have a longer half life and take longer time to reach their peak concentration (Isaksson, Walther, Hansson, Andersson, & Alling, 2011; Isenschmid, Fischman, Foltin, & Caplan, 1992; Mitchell, Teigen, & Ramchandani, 2014; National Institute on Drug Abuse (NIDA), 2016a, 2018; Smith-Kielland, Skuterud, & Morland, 1999). This may also explain why the increase in delay discounting after recent stimulant use was observed earlier than the increases after alcohol and marijuana use in our study.

For participants with HIV, delay discounting remained stable with no significant change in the 6 hours after substance use. The mechanism underlying these differential effects is unclear. Given that there was no difference between participants with and without HIV on subjective level of intoxication as a function of time since substance use, it appears that the changes in “high” with substance use does not explain the difference. Rather, other cognitive mechanisms such as information processing may be responsible for the dampening effect of HIV. One possible explanation may be the synergistic effects of HIV and chronic stimulant use on dopamine reuptake, which is a primary mechanism underlying cognitive and psychological changes commonly experienced immediately after drug use (Gaskill, Miller, Gamble-George, Yano, & Khoshbouei, 2017; National Institute on Drug Abuse (NIDA), 2016a). For participants with HIV, this potential dysfunction of the dopaminergic system due to chronic stimulant use may have impacted the effects of recent substance use on delay discounting. Further studies are necessary to advance our understanding of substance process among people with HIV.

While multiple studies have successfully used EMA to assess a wide range of constructs in persons who are actively using drugs, this is the first to assess monetary delay discounting in real-life settings. There are also some noteworthy limitations. First, our prompts occurred randomly within four fixed time blocks throughout the day, rather than in response to substance use events. Thus, overlap of timeframes of substance use was possible. Since we only assessed substance use in the past 6 hours, we were unable to determine the effects of substance use on delay discounting after 6 hours. Likewise, we do not have precise time of last substance use for every MCQ administration. As a result, we cannot estimate the association of MCQ with subsequent substance use. In addition, it may be that participants with the highest delay discounting were less compliant with survey prompts than those with the lowest delay discounting. Although the valid MCQ responses did not differ based on level of intoxication, it is possible that participants were less willing or less likely to respond when they were acutely intoxicated or “partying.” The entire events were excluded if the MCQ was invalid. The results may have been impacted if the effects of substance use on delay discounting were different between participants who provided valid and invalid MCQ responses. Moreover, our study shows that patterns between substance use and delay discounting were similar for stimulant, alcohol, marijuana, and any substance use. While the sample was defined by current stimulant use, poly-substance use was very common. Consequently, the identified effects may not have been specific to a single drug, but rather the combined effects of poly-drug use. Finally, because the vast majority of stimulant events involved cocaine use (99.1%), our results may not represent the effects of stimulants other than cocaine. While our study showed similar patterns of changes in delay discounting for stimulants and cocaine, the duration of the “high” and half-life vary for cocaine and other stimulants (National Institute on Drug Abuse (NIDA), 1998). Thus, future research should explore the effect on delay discounting of methamphetamines or other stimulants specifically.

In conclusion, our study employed EMA methods to assess the effects of substance use on monetary delay discounting among people with and without HIV. In contrast to previous lab-based studies with strictly controlled experimental conditions, this EMA study may have more ecological validity because the data were collected in a natural context. We found there was a significant increase in delay discounting 2–6 hours after stimulant use and 4–6 hours after marijuana and alcohol use in participants without HIV. Our findings provide evidence of a plausible connection between delay discounting and substance withdrawal that may have relevance for risky behaviors. Because our study did not test the direct effects of delay discounting on risky behaviors, further research is needed to determine whether elevations in delay discounting following substance use are linked to risky behaviors. Targeted interventions aimed at reducing risk behaviors among people who use stimulants may need to address the potential effects of substance withdrawal on delay discounting.

Supplementary Material

Supplemental Material

Public Significance Statement:

This ecological monmentary assessment study provides evidence of a plausible connection between delay discounting and substance withdrawal that may have relevance for risky behaviors. The finding suggests that targeted interventions aimed at reducing risk behaviors among people who use stimulants may need to address the potential effects of substance withdrawal on delay discounting.

Disclosures and Acknowledgments:

This study was funded by grant DP2-DA040226 from the United States National Institutes of Health. This funding source had no other role other than financial support. All authors contributed in a significant way to the manuscript and had read and approved the final manuscript. All authors declared no conflict of interest.

References

  1. Adams S, Attwood AS, & Munafo MR (2017). Drinking status but not acute alcohol consumption influences delay discounting. Hum Psychopharmacol, 32(5). doi: 10.1002/hup.2617 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Albein-Urios N, Martinez-Gonzalez JM, Lozano O, & Verdejo-Garcia A (2014). Monetary delay discounting in gambling and cocaine dependence with personality comorbidities. Addict Behav, 39(11), 1658–1662. doi: 10.1016/j.addbeh.2014.06.001 [DOI] [PubMed] [Google Scholar]
  3. Amlung M, Petker T, Jackson J, Balodis I, & MacKillop J (2016). Steep discounting of delayed monetary and food rewards in obesity: a meta-analysis. Psychol Med, 46(11), 2423–2434. doi: 10.1017/S0033291716000866 [DOI] [PubMed] [Google Scholar]
  4. Ances BM, Vaida F, Cherner M, Yeh MJ, Liang CL, Gardner C, … Group HIVNRC (2011). HIV and chronic methamphetamine dependence affect cerebral blood flow. J Neuroimmune Pharmacol, 6(3), 409–419. doi: 10.1007/s11481-011-9270-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Ashe ML, Newman MG, & Wilson SJ (2015). Delay discounting and the use of mindful attention versus distraction in the treatment of drug addiction: a conceptual review. J Exp Anal Behav, 103(1), 234–248. doi: 10.1002/jeab.122 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Audrain-McGovern J, Rodriguez D, Epstein LH, Cuevas J, Rodgers K, & Wileyto EP (2009). Does delay discounting play an etiological role in smoking or is it a consequence of smoking? Drug Alcohol Depend, 103(3), 99–106. doi: 10.1016/j.drugalcdep.2008.12.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bickel WK, Johnson MW, Koffarnus MN, MacKillop J, & Murphy JG (2014). The behavioral economics of substance use disorders: reinforcement pathologies and their repair. Annu Rev Clin Psychol, 10, 641–677. doi: 10.1146/annurev-clinpsy-032813-153724 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Biddle SJ, Gorely T, Marshall SJ, & Cameron N (2009). The prevalence of sedentary behavior and physical activity in leisure time: A study of Scottish adolescents using ecological momentary assessment. Prev Med, 48(2), 151–155. doi: 10.1016/j.ypmed.2008.10.025 [DOI] [PubMed] [Google Scholar]
  9. Bose J, Hedden SL, R.N. L, & E P-L (2018). Key Substance Use and Mental Health Indicators in the United States: Results from the 2017 National Survey on Drug Use and Health. Retrieved from https://www.samhsa.gov/data/.
  10. de Wit H (2009). Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addict Biol, 14(1), 22–31. doi: 10.1111/j.1369-1600.2008.00129.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Diener E, & Emmons RA (1984). The independence of positive and negative affect. J Pers Soc Psychol, 47(5), 1105–1117. [DOI] [PubMed] [Google Scholar]
  12. Gaskill PJ, Miller DR, Gamble-George J, Yano H, & Khoshbouei H (2017). HIV, Tat and dopamine transmission. Neurobiol Dis, 105, 51–73. doi: 10.1016/j.nbd.2017.04.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Golub SA, Thompson LI, & Kowalczyk WJ (2016). Affective differences in Iowa Gambling Task performance associated with sexual risk taking and substance use among HIV-positive and HIV-negative men who have sex with men. J Clin Exp Neuropsychol, 38(2), 141–157. doi: 10.1080/13803395.2015.1085495 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Herbeck DM, Brecht ML, Lovinger K, Raihan A, Christou D, & Sheaff P (2013). Poly-drug and marijuana use among adults who primarily used methamphetamine. J Psychoactive Drugs, 45(2), 132–140. doi: 10.1080/02791072.2013.785824 [DOI] [PubMed] [Google Scholar]
  15. Hindocha C, Freeman TP, Grabski M, Crudgington H, Davies AC, Stroud JB, … Curran HV (2018). The effects of cannabidiol on impulsivity and memory during abstinence in cigarette dependent smokers. Sci Rep, 8(1), 7568. doi: 10.1038/s41598-018-25846-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Isaksson A, Walther L, Hansson T, Andersson A, & Alling C (2011). Phosphatidylethanol in blood (B-PEth): a marker for alcohol use and abuse. Drug Test Anal, 3(4), 195–200. doi: 10.1002/dta.278 [DOI] [PubMed] [Google Scholar]
  17. Isenschmid DS, Fischman MW, Foltin RW, & Caplan YH (1992). Concentration of cocaine and metabolites in plasma of humans following intravenous administration and smoking of cocaine. J Anal Toxicol, 16(5), 311–314. doi: 10.1093/jat/16.5.311 [DOI] [PubMed] [Google Scholar]
  18. Iudicello JE, Woods SP, Cattie JE, Doyle K, Grant I, & The H. I. V. Neurobehavioral Research Program Group. (2013). Risky decision-making in HIV-associated neurocognitive disorders (HAND). The Clinical Neuropsychologist, 2(256–275), 256. doi: 10.1080/13854046.2012.740077 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jablonski SA, Williams MT, & Vorhees CV (2016). Mechanisms involved in the neurotoxic and cognitive effects of developmental methamphetamine exposure. Birth Defects Res C Embryo Today, 108(2), 131–141. doi: 10.1002/bdrc.21130 [DOI] [PubMed] [Google Scholar]
  20. Jackson JN, & MacKillop J (2016). Attention-Deficit/Hyperactivity Disorder and Monetary Delay Discounting: A Meta-Analysis of Case-Control Studies. Biol Psychiatry Cogn Neurosci Neuroimaging, 1(4), 316–325. doi: 10.1016/j.bpsc.2016.01.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Jenner J, & Saunders JB (2005). Psychostimulant withdrawal and detoxification. In Baker A, Lee NK, & Jenner L (Eds.), Models of intervention and care for psychostimulant users (2 ed., pp. 105–107). [Google Scholar]
  22. John WS, & Wu LT (2017). Trends and correlates of cocaine use and cocaine use disorder in the United States from 2011 to 2015. Drug Alcohol Depend, 180, 376–384. doi: 10.1016/j.drugalcdep.2017.08.031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Johnson MW, Herrmann ES, Sweeney MM, LeComte RS, & Johnson PS (2016). Cocaine administration dose-dependently increases sexual desire and decreases condom use likelihood: The role of delay and probability discounting in connecting cocaine with HIV. Pharmacology, 234, 599–612. doi: 10.1007/s00213-016-4493-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Johnson PS, Sweeney MM, Herrmann ES, & Johnson MW (2016). Alcohol Increases Delay and Probability Discounting of Condom-Protected Sex: A Novel Vector for Alcohol-Related HIV Transmission. Alcohol Clin Exp Res, 40(6), 1339–1350. doi: 10.1111/acer.13079 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Jones J, Guest JL, Sullivan PS, Sales JM, Jenness SM, & Kramer MR (2018). The association between monetary and sexual delay discounting and risky sexual behavior in an online sample of men who have sex with men. AIDS Care, 30(7), 844–852. doi: 10.1080/09540121.2018.1427851 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kamarck TW, Li X, Wright AGC, Muldoon MF, & Manuck SB (2018). Ambulatory Blood Pressure Reactivity as a Moderator in the Association Between Daily Life Psychosocial Stress and Carotid Artery Atherosclerosis. Psychosom Med, 80(8), 774–782. doi: 10.1097/PSY.0000000000000627 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kirby KN, Petry NM, & Bickel WK (1999). Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. Journal of Experimental Psychology: General, 128(1), 78–87. [DOI] [PubMed] [Google Scholar]
  28. Kratz AL, Murphy SL, & Braley TJ (2017). Ecological Momentary Assessment of Pain, Fatigue, Depressive, and Cognitive Symptoms Reveals Significant Daily Variability in Multiple Sclerosis. Arch Phys Med Rehabil, 98(11), 2142–2150. doi: 10.1016/j.apmr.2017.07.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. MacDuffie KE, Brown GG, McKenna BS, Liu TT, Meloy MJ, Tawa B, … Group T (2018). Effects of HIV Infection, methamphetamine dependence and age on cortical thickness, area and volume. Neuroimage Clin, 20, 1044–1052. doi: 10.1016/j.nicl.2018.09.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Mackesy-Amiti ME, & Boodram B (2018). Feasibility of ecological momentary assessment to study mood and risk behavior among young people who inject drugs. Drug Alcohol Depend, 187, 227–235. doi: 10.1016/j.drugalcdep.2018.03.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, & Munafo MR (2011). Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology (Berl), 216(3), 305–321. doi: 10.1007/s00213-011-2229-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Mader EC Jr., Ramos AB, Cruz RA, & Branch LA (2019). Full Recovery From Cocaine-Induced Toxic Leukoencephalopathy: Emphasizing the Role of Neuroinflammation and Brain Edema. J Investig Med High Impact Case Rep, 7, 2324709619868266. doi: 10.1177/2324709619868266 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Mai HT, Le GM, Tran BX, Do HN, Latkin CA, Nguyen LT, … Ho RC (2018). Adherence to antiretroviral therapy among HIV/ AIDS patients in the context of early treatment initiation in Vietnam. Patient Prefer Adherence, 12, 2131–2137. doi: 10.2147/PPA.S175474 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Martin E, Gonzalez R, Vassileva J, Maki PM, Bechara A, & Brand M (2016). Sex and HIV serostatus differences in decision making under risk among substance-dependent individuals. J Clin Exp Neuropsychol, 38(4), 404–415. doi: 10.1080/13803395.2015.1119806 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Martin EM, DeHaan S, Vassileva J, Gonzalez R, Weller J, & Bechara A (2013). Decision making among HIV+ drug using men who have sex with men: a preliminary report from the Chicago Multicenter AIDS Cohort Study. Journal of Clinical and Experimental Neuropsychology, 35(6), 573–583. doi: 10.1080/13803395.2013.799122 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Martin EM, Gonzalez R, Vassileva J, & Bechara A (2019). Double dissociation of HIV and substance use disorder effects on neurocognitive tasks dependent on striatal integrity. AIDS, 33(12), 1863–1870. doi: 10.1097/QAD.0000000000002291 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Mazur JE (1987). An adjusting procedure for studying delayed reinforcement.. In Commons ML, Mazur JE, Nevin J, & Rachlin H (Eds.), The Effect of Delay and of Intervening Events on Reinforcement Value (Vol. 5, pp. 55–73): Lawrence Erlbaum Associates Inc. [Google Scholar]
  38. McKerchar TL, Green L, Myerson J, Pickford TS, Hill JC, & Stout SC (2009). A comparison of four models of delay discounting in humans. Behav Processes, 81(2), 256–259. doi: 10.1016/j.beproc.2008.12.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Meade CS, Addicott M, Hobkirk AL, Towe SL, Chen NK, Sridharan S, & Huettel SA (2018). Cocaine and HIV are independently associated with neural activation in response to gain and loss valuation during economic risky choice. Addict Biol, 23(2), 796–809. doi: 10.1111/adb.12529 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Meade CS, Cordero DM, Hobkirk AL, Metra BM, Chen NK, & Huettel SA (2016). Compensatory activation in fronto-parietal cortices among HIV-infected persons during a monetary decision-making task. Hum Brain Mapp, 37(7), 2455–2467. doi: 10.1002/hbm.23185 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Meade CS, Hobkirk AL, Towe SL, Chen NK, Bell RP, & Huettel SA (2017). Cocaine dependence modulates the effect of HIV infection on brain activation during intertemporal decision making. Drug Alcohol Depend, 178, 443–451. doi: 10.1016/j.drugalcdep.2017.05.043 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mitchell MC Jr., Teigen EL, & Ramchandani VA (2014). Absorption and peak blood alcohol concentration after drinking beer, wine, or spirits. Alcohol Clin Exp Res, 38(5), 1200–1204. doi: 10.1111/acer.12355 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Moskowitz DS, & Young SN (2006). Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. J Psychiatry Neurosci, 31(1), 13–20. [PMC free article] [PubMed] [Google Scholar]
  44. Moulin V, Golay P, Palix J, Baumann PS, Gholamrezaee MM, Azzola A, … Conus P (2018). Impulsivity in early psychosis: A complex link with violent behaviour and a target for intervention. Eur Psychiatry, 49, 30–36. doi: 10.1016/j.eurpsy.2017.12.003 [DOI] [PubMed] [Google Scholar]
  45. National Institute on Drug Abuse (NIDA). (1998). Comparing Methamphetamine and Cocaine. Retrieved from https://archives.drugabuse.gov/news-events/nida-notes/1998/06/comparing-methamphetamine-cocaine
  46. National Institute on Drug Abuse (NIDA). (2016a). Cocaine. Retrieved from https://www.drugabuse.gov/publications/research-reports/cocaine.
  47. National Institute on Drug Abuse (NIDA). (2016b). Research Report Series: Cocaine. Retrieved from https://www.drugabuse.gov/node/pdf/1141/cocaine.
  48. National Institute on Drug Abuse (NIDA). (2018). Marijuna. Retrieved from https://www.drugabuse.gov/publications/drugfacts/marijuana.
  49. National Institute on Drug Abuse (NIDA). (2019). Research Report Series: Methamphetamine. Retrieved from https://www.drugabuse.gov/node/pdf/1793/methamphetamine.
  50. Newcomb ME, & Mustanski B (2014). Diaries for observation or intervention of health behaviors: factors that predict reactivity in a sexual diary study of men who have sex with men. Ann Behav Med, 47(3), 325–334. doi: 10.1007/s12160-013-9549-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Paydary K, Mahin Torabi S, SeyedAlinaghi S, Noori M, Noroozi A, Ameri S, & Ekhtiari H (2016). Impulsivity, Sensation Seeking, and Risk-Taking Behaviors among HIV-Positive and HIV-Negative Heroin Dependent Persons. AIDS Res Treat, 2016, 5323256. doi: 10.1155/2016/5323256 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Piper BJ, Ogden CL, Simoyan OM, Chung DY, Caggiano JF, Nichols SD, & McCall KL (2018). Trends in use of prescription stimulants in the United States and Territories, 2006 to 2016. PLoS One, 13(11), e0206100. doi: 10.1371/journal.pone.0206100 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Robles-Martinez M, Garcia-Carretero MA, Gibert Rahola J, Rodriguez-Cintas L, Palma-Alvarez RF, Abad AC, … Roncero C (2019). Relationship between craving and impulsivity in patients with alcohol dependence with or without dual disorders in an outpatient treatment center: a descriptive study. Actas Esp Psiquiatr, 47(3), 88–96. [PubMed] [Google Scholar]
  54. Schumacher LM, Martin GJ, Goldstein SP, Manasse SM, Crosby RD, Butryn ML, … Forman EM (2018). Ecological momentary assessment of self-attitudes in response to dietary lapses. Health Psychol, 37(2), 148–152. doi: 10.1037/hea0000565 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Smith-Kielland A, Skuterud B, & Morland J (1999). Urinary excretion of 11-nor-9-carboxy-delta9-tetrahydrocannabinol and cannabinoids in frequent and infrequent drug users. J Anal Toxicol, 23(5), 323–332. doi: 10.1093/jat/23.5.323 [DOI] [PubMed] [Google Scholar]
  56. Sobell LC, & Sobell MB (1996). Timeline Follow-back User’s Guide: A Calendar Method for Assessing Alcohol and Drug Use. Toronto: Addiction Research Foundation. [Google Scholar]
  57. Towe SL, Hobkirk AL, Ye DG, & Meade CS (2015). Adaptation of the Monetary Choice Questionnaire to accommodate extreme monetary discounting in cocaine users. Psychol Addict Behav, 29(4), 1048–1055. doi: 10.1037/adb0000101 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Trafford E (2015). MetricWire (Version 2.2.1) [Mobile application software]. Retrieved from http://research.metricwire.com
  59. Vassileva J, Ahn WY, Weber KM, Busemeyer JR, Stout JC, Gonzalez R, & Cohen MH (2013). Computational modeling reveals distinct effects of HIV and history of drug use on decision-making processes in women. PLoS One, 8(8), e68962. doi: 10.1371/journal.pone.0068962 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Volkow ND, Wang GJ, Fowler JS, Tomasi D, Telang F, & Baler R (2010). Addiction: decreased reward sensitivity and increased expectation sensitivity conspire to overwhelm the brain’s control circuit. Bioessays, 32(9), 748–755. doi: 10.1002/bies.201000042 [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Volkow ND, Wang GJ, Tomasi D, & Baler RD (2013). Unbalanced neuronal circuits in addiction. Curr Opin Neurobiol, 23(4), 639–648. doi: 10.1016/j.conb.2013.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Wardle MC, Gonzalez R, Bechara A, & Martin-Thormeyer EM (2010). Iowa Gambling Task performance and emotional distress interact to predict risky sexual behavior in individuals with dual substance and HIV diagnoses. J Clin Exp Neuropsychol, 32(10), 1110–1121. doi: 10.1080/13803391003757833 [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Yang C, Linas B, Kirk G, Bollinger R, Chang L, Chander G, … Latkin C (2015). Feasibility and Acceptability of Smartphone-Based Ecological Momentary Assessment of Alcohol Use Among African American Men Who Have Sex With Men in Baltimore. JMIR Mhealth Uhealth, 3(2), e67. doi: 10.2196/mhealth.4344 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Yoon JH, Higgins ST, Heil SH, Sugarbaker RJ, Thomas CS, & Badger GJ (2007). Delay discounting predicts postpartum relapse to cigarette smoking among pregnant women. Exp Clin Psychopharmacol, 15(2), 176–186. doi: 10.1037/1064-1297.15.2.186 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material
NIHMS1731619-supplement-Supplemental_Material.pdf (106.5KB, pdf)

RESOURCES