Abstract
Introduction
The major aim of this study was to test whether abstinence from e-cigarettes causes withdrawal symptoms in former smokers.
Methods
We conducted an unblinded, within-participants, pre–post clinical trial in which 109 former smokers who were current daily electronic cigarette (e-cigarette) users used their own e-cigarette for 7 days followed by 6 days of biologically confirmed abstinence engendered via an escalating contingency payment system. Participants monitored symptoms of nicotine withdrawal daily via an Interactive Voice Response system. They also attended three laboratory visits per week for carbon monoxide and cotinine testing to verify abstinence.
Results
Half of participants completely abstained for a week. All the Diagnostic and Statistical Manual, Fifth Edition (DSM-5) tobacco withdrawal symptoms, craving for e-cigarettes, craving for tobacco cigarettes, and the four possible new withdrawal symptoms (anhedonia, impulsivity, mood swings, and positive affect) increased during abstinence. Weight increased and heart rate decreased with abstinence. Symptoms showed the prototypical inverted U time pattern of a withdrawal state. The magnitude of withdrawal appeared to be somewhat less than that in a prior study of abstinent daily tobacco cigarette smokers. More severe withdrawal on the first 2 days of abstinence did not predict abstinence on the last day of the study.
Conclusions
Former smokers who are daily e-cigarette users transfer physical dependence on tobacco cigarettes to dependence on e-cigarettes. The severity of withdrawal from e-cigarettes appears to be only somewhat less than that from daily tobacco cigarette use. Replication tests that include placebo controls, testing for pharmacological specificity, and including never-smokers, non-daily e-cigarette users and dual users are indicated.
Implications
Our results indicate e-cigarettes can maintain physical dependence. This adverse effect should be included in any risk vs. benefit calculation. Also, potential and current e-cigarette users should be informed that abrupt cessation of e-cigarettes can cause withdrawal symptoms.
Trial registration
Introduction
There are several concerns about use of electronic cigarettes (e-cigarettes): for example, will they act as a starter product for young adults,1 and will they promote or undermine motivation to stop smoking.2 Another concern is the dependence liability of e-cigarettes.3 A major aspect of dependence liability is physical dependence; that is, tolerance and withdrawal. The major predictors of physical dependence are (1) rapid onset of drug effects, and (2) sufficient dose.4 In some e-cigarette users, e-cigarettes produce a rapid uptake of nicotine and produce nicotine levels similar to that for tobacco cigarette use5; thus, one would expect some e-cigarette users to be physically dependent on nicotine and have withdrawal symptoms when they stop e-cigarettes. On the other hand, e-cigarette users consistently report dependence on e-cigarettes that is less than that on tobacco cigarettes.6 More specifically, two retrospective surveys of current e-cigarette users found e-cigarette withdrawal severity in past attempts to quit was one-half or three-quarters less than tobacco withdrawal severity.6–9 Given the known problems of retrospective recall, we conducted the first experimental test of whether abstinence from e-cigarettes causes withdrawal and its magnitude.
There are several reasons to assess withdrawal. One major harm of nicotine withdrawal is that it can make cessation of a product more difficult.10 In addition, in a minority of users, nicotine withdrawal can produce functional consequences such as poor cognitive functioning and clinical depression.10 Also, the occurrence of withdrawal is associated with a greater severity of addiction.11 Given this, if e-cigarettes were found to maintain significant withdrawal upon abstinence, then this would clearly need to be conveyed in the product labeling. In addition, the negative effects of withdrawal and the resultant increased risk of addiction and difficulty in stopping e-cigarettes would need to be factored into any risk/benefit assessment of e-cigarettes.12
Methods
The inclusion criteria were (1) at least 18 years old; (2) history of smoking tobacco cigarettes in the past for at least 1 year; (3) have not used more than five tobacco cigarettes in the last month; (4) currently using a nicotine containing refillable tank e-cigarette daily for the last 2 months (no JUUL users were enrolled); (5) used cannabis less than or equal to five times in past month and agree to not use cannabis during the study; (6) no current anxiety, depression, attention deficit, or drug use disorder that would make interpreting withdrawal symptoms difficult or that would impair the ability to understand the study tasks; (7) have daily access to a phone; (8) have a carbon monoxide level of less than or equal to 8 ppm (Micro+Basic Smokerlyzer, Covita and the Vitalograph BreathCO); (9) a negative urinary cannabis level based on the One-Screen Marijuana Test strip; and (10) a urinary cotinine level of at least 100 ng/ml based on the NicAlert test strip. We used daily users of second-generation products with high nicotine levels because this is the sample most likely to incur withdrawal symptoms.12,13 We chose a carbon monoxide level of 8 ppm and a cotinine level of 100 ng/ml because it would likely include the upper 60% of daily e-cigarette users.5 To measure cotinine, we used a score on the NicAlert dipstick of at least 3. Initially, the study planned to recruit only former smokers; however, we received several applicants who were never-smokers (ie, <100 lifetime cigarettes). Given that little is known about never-smokers using e-cigarettes, during the study we began to also include never-smokers. Our results with never-smokers will be reported in a separate study.
Our sample size calculation was based on a study that used the same withdrawal scale, a similar pre–post design and used procedures almost identical to our study.14 In this prior study, the increase in the mean withdrawal score after stopping tobacco cigarettes at the peak time (2 days post-cessation) was an increase of +3.8 units (on a 27 point scale; ie, nine symptoms rated from 0 to 3), with a between-participants SD of 5.6, and a within-participants deviation of 4.2. We chose a sample size to be sufficient to detect an increase of 50% of that observed for tobacco cigarette cessation because the data cited earlier suggest withdrawal from stopping e-cigarettes is likely to be less than from tobacco cigarettes. If we assume an alpha of 0.05 and power of 0.80, then a two-tailed test of own e-cigarette use versus abstinence required a mean of 60 compliant participants (see later for definition of compliant) using a within-participant design.
The University of Vermont in Burlington, VT (20% of those enrolled), and Battelle in Baltimore, MD (80%), were the study sites. Participants were enrolled between July of 2016 and October of 2018. The major methods of recruitment were Facebook (51% of enrolled), Instagram (17%), and referral (15%). We excluded a large proportion of applicants; the major reasons for exclusion are outlined in the flow chart (Figure 1).
Participants attended a baseline session to examine final inclusion criteria, to obtain consent, to complete surveys, and to obtain baseline carbon monoxide and cotinine levels. Participants were asked to use their own e-cigarette as usual and call an Interactive Voice Response (IVR) system15 nightly to record e-cigarette, tobacco cigarette, and other tobacco product use and withdrawal symptoms. On Monday, Wednesday, and Friday of each week, participants attended the site to obtain biological verification of compliance, and weight and heart rate measurements. They provided a breath sample for carbon monoxide and a urine sample for cotinine and cannabis testing. Finally, they completed the E-Cigarette Purchase Task in which they indicated how much money they would spend for various amounts of e-cigarette use.16 The results from this outcome will be reported in a separate study.
Participants were reimbursed $110 for completing forms and attending sessions. We used monetary contingencies to increase compliance to attendance and abstinence. During the abstinence week, if a participant was compliant they would earn an extra $15 on a Monday session, $20 if still compliant on a Wednesday session, and $25 if still compliant at the Friday session. In the first week, continued use of their own nicotine containing e-cigarettes and continued abstinence from tobacco and other nicotine products should result in low levels of carbon monoxide and continued high levels of cotinine. On the basis of laboratory studies of e-cigarette only use,17 we required a breath carbon monoxide of less than or equal to 8 ppm. To assess cotinine levels, we tested urine samples using the NicAlert semiquantitative dipsticks. Prior research has found high concordance between NicAlert scores and gas chromatography and mass spectrometry cotinine levels.18
Abstinence from e-cigarettes in the second week should continue low levels of carbon monoxide and show a decline in cotinine levels. The half-life of cotinine is 20 hours. Thus, we required that the cotinine levels drop at least one NicAlert level from the lowest level during the vaping period to the last day of e-cigarette abstinence. Although this may seem lenient, we wanted to reduce the number of exclusions to decrease selection bias as much as possible.
Withdrawal symptoms were collected via nightly IVR calls. Participants were reimbursed $2/IVR call with a bonus of $10 for each week in which all seven daily calls were completed. The nightly IVR asked if the participant had used an e-cigarette and if so the number of milliliters of e-liquid they used that day. The IVR then asked the two urge questions of the Mood and Physical Symptoms Scale19 adopted separately for e-cigarette and for tobacco cigarettes; that is, “how strong have the urges to vape/smoke been” and “how much of the time have you felt the urge to vape/smoke.” Both were rated on a five point scale. It next asked participants to rate the seven Diagnostic and Statistical Manual, Fifth Edition (DSM-5)11 withdrawal criteria (anger, anxiety, difficulty concentrating, depression, hunger insomnia, and restlessness) plus four plausible withdrawal symptoms (anhedonia, impulsivity, mood swings, and positive affect), and three control symptoms known not to increase with withdrawal (diarrhea, headache, and tremor).20 All of these were rated from 0 = not at all, 1 = a little, 2 = some, and 3 = a lot. It also asked participants to rate whether the withdrawal symptoms “interfered with your functioning at work, school, or other settings” either “not at all, a little, some, or a lot.” Finally, the IVR asked about any use of tobacco cigarettes, smokeless, cigars, or nicotine replacement.
Nicotine withdrawal symptoms almost always increase and very often peak in the first 2 days of abstinence10; thus, our major analysis was comparing withdrawal ratings in the first 2 days of abstinence with those on the last 2 days of vaping among compliant participants. We have previously discussed the pros and cons of using only a subset of abstinent participants versus using all enrolled participants and conducting statistical corrections such as imputing missing data or using abstinence as a time varying covariate.20 We chose to use the subset of those compliant (n = 59; 54% of those enrolled) because it is the most common method and the resultant sample size has been adequate in prior studies. We defined compliance as (1) no self-reported vaping during the second week, (2) a drop in cotinine level at the last visit of abstinence of at least one level from the lowest level during the vaping period, (3) no cannabis, non-e-cigarette nicotine, or tobacco use during the 2 weeks of the study, (4) at least 3 days of IVR ratings during vaping and 2 days during abstinence plus and, (5) having data from all six study visits. We also conducted a sensitivity test in which we only required abstinence on the first 2 days of the abstinence week to obtain a less selective sample.
For the main analyses we used the mean withdrawal score (mean of the seven DSM criteria). Secondary analyses examined individual withdrawal symptoms, craving scores, and possible newly proposed withdrawal symptoms.10 We entered mean values of these outcomes across the last 2 days of vaping versus the first 2 days of abstinence into a within-participants t test. We used a two-tailed p ≤ 0.05 to determine statistical significance. We did not use corrections for the number of tests because many believe this is not appropriate during an initial study in an area.21
Results
Participants were mostly middle-aged white men (range = 18–65) (Table 1). Few said they were unable to quit e-cigarettes or had withdrawal symptoms when stopping e-cigarettes; however, their mean self-rating of addiction to e-cigarettes was high and similar to ratings of addiction to tobacco cigarettes. Almost all were former daily smokers; that is, few were former non-daily smokers. They smoked about 15 cigarettes/day in the past. We calculated mean nicotine intake during the vaping period by multiplying the milliliters of e-liquid used by the milligram per milliliter of nicotine and found the median nicotine intake for the 59 completers to be 26 mg ml−1 day−1 (25th and 75th percentile: 14,43). To assess generalizability, we compared our compliant and noncompliant participants to a similar group in the Population Assessment of Tobacco and Health (PATH) survey conducted at approximately the same time (Wave 1).22 In the PATH, we selected current daily e-cigarette users that used non-disposable and non-cartridge e-cigarettes, used nicotine e-liquids, previously used tobacco cigarettes regularly, had used less than five tobacco cigarettes in the last month, and who had not used cannabis or non-e-cigarette tobacco products in the last month. The major difference between our sample and the PATH sample was that our sample had a higher prevalence of men (Table 1).
Table 1 .
Completers | Noncompleters | PATH Wave 1 | |
---|---|---|---|
n = 59 | n = 50 | n = 132 | |
Demographics | |||
Age | 32 (10) | 32 (10) | 34 |
% Men | 81 | 78 | 56 |
% >High school | 75 | 76 | 62 |
% Unemployed | 14 | 24 | 24 |
% White | 77 | 69 | 81 |
E-cigarette use | |||
Duration (months) | 26 (17) | 19 (16)** | |
% Started e-cigs after quit smoking | 53 | 54 | |
% Stated unable to stop using e-cigs | 7 | 8 | |
Current addiction to e-cigs (0–10) | 6.6 (1.9) | 7.0 (1.8) | |
% Ever stopped e-cigs ≥3 days | 36 | 24 | |
Among those who stopped, | |||
% with ≥4 of 7 DSM-5 withdrawal symptoms | 5 | 8 | |
Tobacco cigarette use | |||
Former daily smokers | 92% | 96% | 95% |
Age started smoking tobacco regularly | 18 (3) | 18 (5) | |
Cigs/day when smoked | 15 (11) | 14 (7) | 19 |
FTCD | 4.0 (2.6) | 4.2 (2.2) | |
Among those who stopped | |||
% with ≥4 of 7 DSM-5 withdrawal symptoms | 29 | 36 | |
Addiction to tobacco cigs when last smoked (0–10) | 7.8 (2.1) | 8.4 (2.0) |
DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; FTCD = Fagerstrom Test for Cigarette Dependence; PATH = Population Assessment of Tobacco and Health.
aAll data are means (standard deviation in parentheses) unless indicated otherwise.
**p < .05.
In terms of internal validity, despite that the study only required 6 days of abstinence and provided monetary awards, only 59 (54%) were able to abstain and complete procedures. Compliant and noncompliant participants were similar on almost all demographics, and on measures of e-cigarette and tobacco use (Table 1). Withdrawal ratings during vaping were stable or decreasing over time (Figure 2). The three non-withdrawal symptom ratings decreased slightly during the vaping period.
The mean withdrawal score increased with abstinence and peaked on days 1 or 2 when using only the 59 compliant participants (Figure 2; Table 2). This also occurred in the sensitivity analysis which only required abstinence on the first 2 days (n = 79, p < .001). The increase in mean withdrawal among compliant participants from smoking to abstinence was 0.41 units (95% CI = 0.29 to 0.54). In comparison, in a study using a similar 0–3 scale (0 = not present, 1 = mild, 2 = moderate, 3 = severe) among daily tobacco cigarette smokers not trying to quit found an increase in mean withdrawal of 0.55.14 All seven DSM-5 withdrawal symptoms increased with abstinence as did craving for both e-cigarettes and tobacco cigarettes. The four possible new withdrawal symptoms increased as well. Weight increased from 194.4 pounds to 196.0 pounds and heart rate decreased from 77.2 bpm to 70.7 bpm during the abstinence week. Two of the three non-withdrawal control symptoms did not change with abstinence; however, tremor showed a statistically significant but small increase during abstinence (Table 2).
Table 2.
Vaping | Abstinent | Increase | t value | |
---|---|---|---|---|
Mean | Mean | Mean | ||
Withdrawal (0–3) | ||||
Mean | 0.16 | 0.57 | 0.41 | 6.5*** |
Angry, irritable | 0.21 | 0.88 | 0.67 | 6.1*** |
Anxious, nervous | 0.14 | 0.59 | 0.45 | 4.1*** |
Increased appetite | 0.13 | 0.62 | 0.49 | 5.1*** |
Difficulty concentrating | 0.10 | 0.52 | 0.41 | 4.6*** |
Depressed, sad | 0.08 | 0.28 | 0.21 | 3.6*** |
Insomnia | 0.26 | 0.38 | 0.12 | 2.1* |
Restlessness | 0.17 | 0.71 | 0.53 | 5.1*** |
Craving (0–5) | ||||
E-Cig craving | ||||
How much of time urge | 1.97 | 2.47 | 0.49 | 3.7*** |
How strong urge | 1.94 | 2.62 | 0.68 | 4.9*** |
Tobacco cigarette craving | ||||
How much of time urge | 0.11 | 0.40 | 0.28 | 2.7* |
How strong urge | 0.08 | 0.38 | 0.30 | 2.9** |
Potential withdrawal items (0–3) | ||||
Impatient, impulsive | 0.10 | 0.57 | 0.47 | 4.5*** |
Less enjoy pleasant events | 0.03 | 0.31 | 0.28 | 3.1** |
Less positive outlook | 0.04 | 0.27 | 0.22 | 2.7** |
Mood swings | 0.05 | 0.41 | 0.36 | 3.9*** |
Control items (0–3) | ||||
Diarrhea | 0.04 | 0.07 | 0.03 | 0.6 |
Headache | 0.19 | 0.33 | 0.14 | 1.9 |
Tremors | 0.00 | 0.15 | 0.15 | 3.4** |
*p < .05, **p < .01, ***p < .001.
To test for the typical inverted U time pattern of withdrawal, we tested whether the mean withdrawal score decreased after the first 2 days. The slope during the last 4 days was −0.07 units per day (p = .001). Nevertheless, we found 41% of participants continued to have withdrawal symptoms at the end of the study; that is, their mean withdrawal score at the end of the study remained higher than their mean pre-abstinence vaping scores (increase of 0.11 units, p < .05).
The mean withdrawal score increased from the last 2 days of vaping to the first 2 days of abstinence in 81% of participants. The strength of craving for e-cigarettes increased in 62% of participants, and craving for tobacco cigarettes increased in 29%. When asked about whether symptoms interfered with functioning, this increased from 12% of participants during the last 2 days of vaping to 38% during the first 2 days of abstinence. Half (48%) of participants had at least four symptoms that increased with abstinence and thus met the symptom number criterion for DSM-5 nicotine withdrawal. When symptoms were also required to interfere with functioning then 33% met full DSM-5 criteria. To test for the clinical significance of withdrawal, we examined whether the mean withdrawal score across the first 2 days of abstinence predicted abstinence on the sixth day of abstinence when the study ended. The mean withdrawal score for those who relapsed was not greater than those who remained abstinent (0.62 vs. 0.57). No participants had a serious adverse event. None of the participants reported a strong urge to smoke cigarettes during the vaping period. A few participants (6, 10%) reported a “strong” urge to smoke tobacco cigarettes during abstinence.
To examine moderators, we tested the correlation between baseline variables with the increase in mean withdrawal in the first 2 days of abstinence using the Spearman correlation statistic (see Supplementary Appendix). The three most robust predictors were ratings of past withdrawal from stopping tobacco cigarettes (n = 33, rsp = .57, increase of .19 vs. .76 in a median split of low vs. high scores), ratings of past withdrawal from stopping e-cigarettes (n = 20, rsp = .41, .20 vs. .46), and negative expectancy score23 (n = 59, rsp = .41, .26 vs. .57). Neither duration of use of e-cigarettes nor gender moderated results. We did not have enough variety to examine the type of e-cigarette as a moderator.
Discussion
Our major findings were that, among daily users of second-generation e-cigarettes who are former smokers, (1) abstinence causes withdrawal symptoms, and (2) this appears to be a true withdrawal effect. Our results paint a mixed picture about the clinical and policy significance of e-cigarette withdrawal. The magnitude of withdrawal in our study appears to be about 25% less than that for tobacco cigarette withdrawal, which replicates the results from cross-sectional surveys.7 One third of participants met DSM-5 criteria for withdrawal and one quarter reported additional difficulty functioning during abstinence. Initial severity of e-cigarette withdrawal did not prospectively predict relapse to e-cigarette use, although ours was a somewhat weak test of this.
One limitation of our study was the lack of blindness; thus, expectancy effects could account for our results. Although the absence of a substantial increase in our non-withdrawal symptoms suggests this might not be the case, this is a weak test of expectancy. Another limitation is the absence of a concurrent comparison group of tobacco cigarette abstinence. We attempted to mitigate this by using a historical control; that is, a study that used an almost identical sample, pre–post design, time-period, and withdrawal measure; however, cross-study comparisons can be misleading. We did not use a population-based sample and excluded many applicants; however, our sample appeared similar to daily e-cigarette users in the PATH. We did not study dual users, the most common users of e-cigarettes and we did not examine gradual reduction, which may be more common than abrupt cessation. Finally, our study did not examine how much of the withdrawal effects was due to the reduction in nicotine per se.
Our study had several strengths: (1) within-participants design, (2) measures at multiple timepoints during each condition to test for a withdrawal time pattern, (3) methods to obtain a high rate of abstinence, (4) use of a validated withdrawal scale, (5) inclusion of measures of the functional significance of withdrawal effects, and (6) inclusion of newly proposed withdrawal symptoms.
Given the aforementioned limitations, exact and conceptual replication tests are indicated. Perhaps the more important tests are testing for pharmacological specificity and testing among less intensive users of e-cigarettes, those using first generation e-cigarettes, never-smokers, and dual users. In addition, population-based epidemiological studies of the incidence of withdrawal and other features of dependence are needed.
In summary, our study shows that e-cigarettes maintain physical dependence in former smokers. This is important for basic science, clinical practice, and regulation. In terms of basic science, as the only psychoactive ingredient in e-cigarettes is nicotine, our results provide a replication that the use of nicotine per se can maintain dependence. The clinical significance of our findings is unclear. On the one hand, the difference in withdrawal severity between e-cigarette and tobacco cigarette withdrawal appears to be small. On the other hand, e-cigarette withdrawal severity did not appear to undermine the ability to remain abstinent. Finally, in our study only half of participants were able to abstain for 1 week, despite the monetary contingencies and knowledge that they could return to vaping at the end of the week. This suggests that dependence on e-cigarettes can be substantial and, thus, treatments to help daily e-cigarette users to stop need to be developed. In terms of policy, the negative effects of withdrawal and the resultant increased risk of addiction and difficulty in stopping e-cigarettes would need to be factored into any risk vs. benefit assessment of e-cigarettes.12 Also, we believe that the fact that abrupt cessation can cause withdrawal should be communicated to potential and current e-cigarette users.
Funding
This work was supported by a research grant CA-192940 from the National Cancer Institute.
Declaration of Interest
JRH has received consulting and speaking fees from several companies that develop or market pharmacological and behavioral treatments for smoking cessation or harm reduction and from several nonprofit organizations that promote tobacco control. He currently receives consulting fees from Swedish Match and has received fees in the past from Altria and Philip Morris to assist their efforts to develop less-risky tobacco products. CP’s spouse’s employer markets nicotine replacement products. JE, EP, PC, EO, and NM have nothing to disclose.
Supplementary Material
Acknowledgments
The authors wish to thank Joy Benner, Quintella Bester, Jessie McNabb, Jennifer Oletsky, Jeff Priest, Joel Vincent, and Lauren Viray for help in the conduct of the study and Konstantinos Farsalinos and Saul Shiffman in the planning of the study or in interpretation of the study results.
References
- 1. Etter JF. Gateway effects and electronic cigarettes. Addiction. 2018;113(10):1776–1783. [DOI] [PubMed] [Google Scholar]
- 2. Malas M, van der Tempel J, Schwartz R, et al. . Electronic cigarettes for smoking cessation: a systematic review. Nicotine Tob Res. 2016;18(10):1926–1936. [DOI] [PubMed] [Google Scholar]
- 3. Foulds J, Veldheer S, Yingst J, et al. . Development of a questionnaire for assessing dependence on electronic cigarettes among a large sample of ex-smoking and e-cigarette users. Nicotine Tob Res. 2015;17(2):186-192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Schuckit M. Drug and Alcohol Abuse: A Clinical Guide to Diagnosis and Treatment. 6th ed. New York, NY: Springer Science+Business Media, Inc; 2006. [Google Scholar]
- 5. Etter JF, Bullen C. Saliva cotinine levels in users of electronic cigarettes. Eur Respir J. 2011;38(5):1219–1220. [DOI] [PubMed] [Google Scholar]
- 6. Etter JF, Eissenberg T. Dependence levels in users of electronic cigarettes, nicotine gums and tobacco cigarettes. Drug Alcohol Depend. 2015;147:68–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Rostron BL, Schroeder MJ, Ambrose BK. Dependence symptoms and cessation intentions among US adult daily cigarette, cigar, and e-cigarette users, 2012–2013. BMC Public Health. 2016;16(814). doi:10.1186/s12889-016-3510-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Liu G, Wasserman E, Kong L, Foulds J. A comparison of nicotine dependence among exclusive E-cigarette and cigarette users in the PATH study. Prev Med. 2017;104:86–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Browne M, Todd DG. Then and now: consumption and dependence in e-cigarette users who formerly smoked cigarettes. Addict Behav. 2018;76:113–121. [DOI] [PubMed] [Google Scholar]
- 10. Hughes JR. Effects of abstinence from tobacco: valid symptoms and time course. Nicotine Tob Res. 2007;9(3):315–327. [DOI] [PubMed] [Google Scholar]
- 11. American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5™ (5th ed.). Arlington, VA: American Psychiatric Publishing, Inc. [Google Scholar]
- 12. Evans SE, Hoffman AC. Electronic cigarettes: abuse liability, topography and subjective effects. Tob Control. 2014;23(suppl 2):ii23–ii29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Boykan R, Goniewicz ML, Messina CR. Evidence of Nicotine Dependence in Adolescents Who Use Juul and Similar Pod Devices. Int J Environ Res Public Health. 2019;16(12):2135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Hughes JR, Hatsukami D. Signs and symptoms of tobacco withdrawal. Arch Gen Psychiatry. 1986;43(3):289–294. [DOI] [PubMed] [Google Scholar]
- 15. Corkrey R, Parkinson L. Interactive voice response: review of studies 1989-2000. Behav Res Methods Instrum Comput. 2002;34(3):342–353. [DOI] [PubMed] [Google Scholar]
- 16. Cassidy RN, Tidey JW, Colby SM, Long V, Higgins ST. Initial development of an e-cigarette purchase task: a mixed methods study. Tob Regul Sci. 2017;3(2):139–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. McRobbie H, Phillips A, Goniewicz ML, et al. . Effects of switching to electronic cigarettes with and without concurrent smoking on exposure to nicotine, carbon monoxide, and acrolein. Cancer Prev Res (Phila). 2015;8(9):873–878. [DOI] [PubMed] [Google Scholar]
- 18. Marrone G, Paulpillai M, Evans R, Singleton E, Heishman S. Breath carbon monoxide and semiquantitative saliva cotinine as biomarkers for smoking. Hum Psychopharm Clin. 2009;25(1):80–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. West R, Hajek P. Evaluation of the mood and physical symptoms scale (MPSS) to assess cigarette withdrawal. Psychopharmacology (Berl). 2004;177(1-2):195–199. [DOI] [PubMed] [Google Scholar]
- 20. Hughes JR. Measurement of the effects of abstinence from tobacco: a qualitative review. Psychol Addict Behav. 2007;21(2):127–137. [DOI] [PubMed] [Google Scholar]
- 21. Feise RJ. Do multiple outcome measures require p-value adjustment? BMC Med Res Methodol. 2002;2:8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Hyland A, Ambrose BK, Conway KP, et al. . Design and methods of the Population Assessment of Tobacco and Health (PATH) Study. Tob Control. 2017;26(4):371–378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Wetter DW, Smith SS, Kenford SL, et al. . Smoking outcome expectancies: factor structure, predictive validity, and discriminant validity. J Abnorm Psychol. 1994;103(4):801–811. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.