E-Cigarettes and Smoking Cessation: Insights and Cautions From a Secondary Analysis of Data From a Study of Online Treatment-Seeking Smokers

Abstract

Background:

Evidence from observational studies regarding the association between electronic cigarette (e-cigarette) use and cessation is mixed and difficult to interpret. Utilizing 2 analytic methods, this study illustrates challenges common in analyses of observational data, highlights measurement challenges, and reports associations between e-cigarette use and smoking cessation.

Methods:

Data were drawn from an ongoing web-based smoking cessation trial. The sample was comprised of 2,123 participants with complete 3-month follow-up data. Logistic regression models with and without entropy balancing to control for confounds were conducted to evaluate the association between e-cigarette use and 30-day cigarette smoking abstinence.

Results:

At follow-up, 31.7% of participants reported using e-cigarettes to quit in the past 3 months. E-cigarette users differed from nonusers on baseline characteristics including cigarettes per day, Fagerström score, quit attempt in the past year, and previous use of e-cigarettes to quit. At follow-up, e-cigarette users made more quit attempts and employed more cessation aids than smokers who did not use e-cigarettes to quit. E-cigarette use was negatively associated with abstinence after adjustment for baseline characteristics; however, the association was not significant after additional adjustment for use of other cessation aids at 3 months.

Conclusions:

The magnitude and significance of the estimated association between e-cigarette use and cessation in this study were dependent upon the analytical approach. Observational studies should employ multiple analytic approaches to address threats to validity. Future research should employ better measures of patterns of and reasons for e-cigarette use, frequency of e-cigarette use, and concurrent use of cessation aids.

Introduction

Ever use of electronic cigarettes (e-cigarettes) among U.S. adult smokers has increased rapidly in recent years, from 9.8% in 2010¹ to over 32% in 2012.² This increase may be due, in part, to the widespread perception that e-cigarette use is an effective means to reduce the harm associated with smoking or to quit smoking altogether.^2–6 Though there is great interest in whether e-cigarettes are effective smoking cessation aids, only two randomized trials to date have examined the efficacy of these devices for smoking cessation.^7,8 Both studies suggest that the use of e-cigarettes may promote smoking reduction or cessation, but their generalizability is limited and more research is needed.

In contrast, many observational studies have examined the association of e-cigarette use and smoking cessation.^3,9–15 Three cross-sectional studies of smokers found an inverse relationship between e-cigarette use and cessation among U.S. adults⁹ and U.S. and Korean youth smokers.^10,11 Conversely, a cross-sectional study of nearly 6,000 English smokers found that e-cigarette use specifically for smoking cessation was associated with higher odds of abstinence compared to use of nicotine replacement therapy (NRT) or unaided quitting.¹² Three web-based cross-sectional surveys of e-cigarette users found that 31%–74% reported at least some periods of smoking abstinence.^3,13,14 Another web-based survey of e-cigarette users found that 91.1% were former smokers,¹⁵ though it is unclear whether respondents quit using e-cigarettes.

Six longitudinal studies have examined the association between e-cigarette use and cessation.^6,8,16–19 Three prospective studies reported a positive relationship between e-cigarette use and cigarette reduction or cessation, even among smokers who had no intention to quit.^8,16,17 However, two of these studies had small sample sizes and all used e-cigarette devices that are now obsolete (e.g., smaller batteries, less vapor production, and poorer nicotine delivery than current models), so the generalizability of these studies is diminished. Another study of smokers in the United States, Canada, United Kingdom, and Australia¹⁸ reported that e-cigarette users were more likely to have reduced their cigarettes per day between waves than nonusers, but cessation rates did not differ between the two groups. In a study of U.S. smokers, Grana et al.¹⁹ reported that past 30-day e-cigarette use at baseline was not associated with reduced cigarette consumption or cessation 1 year later. In a different study of motivated quitters in six U.S. states,⁶ e-cigarette ever users were less likely than never users to have quit smoking cigarettes at 7-month follow-up.

All of these observational studies have limitations that preclude drawing inferences about the effectiveness of e-cigarettes as a smoking cessation or reduction aid. Selection bias cannot be ruled out when estimating the association between e-cigarette use and smoking cessation because of lack of randomization to the exposure—namely, e-cigarette use for cessation.²⁰ Any number of measured or unmeasured factors could confound an observed relationship between use of e-cigarettes and smoking cessation. Additionally, it is difficult to discern whether e-cigarettes are effective cessation tools in some existing studies^{6,9–11,18,19} due to imprecise measurement of the reasons for e-cigarette use. These studies examined quit rates among all smokers who had ever tried e-cigarettes for any reason, rendering it impossible to interpret their findings as indicative of e-cigarette use facilitating or hindering smoking cessation or reduction efforts. The mix of both negative and positive associations of e-cigarette use and smoking cessation from these studies illustrates the challenge of analyzing and interpreting results from observational data.

A variety of statistical methods, such as entropy balancing, have been developed to reduce confounding/selection bias when estimating treatment effects. Entropy balancing is a data preprocessing method to achieve covariate balance between the treated and control groups in observational studies.²¹ This approach relies on a maximum entropy reweighting scheme that generates observation-level weights such that the treated group and the reweighted control group are balanced with respect to a potentially large set of prespecified covariates. Typically, entropy balancing “balances” the two groups as to have equal means for the covariates of interest (the two groups may also be balanced with respect to higher moments if data allows); the weights that provide balance are subsequently used as survey weights to estimate the treatment effect using a regression model. As a consequence, the estimates of treatment effect based on the preprocessed data can exhibit lower model dependency compared to estimates from regression models applied directly to the original (i.e., unprocessed) data. It is important to note that entropy balancing assures perfect balance on all covariates included in the reweighting, and does not result in any direct loss of data as often occurs with alternative methods, such as propensity score matching methods. Because of the introduction of weights, entropy balancing will reduce statistical precision. This method may also fail to achieve balance on all variables if there is insufficient overlap between the treated and control groups on some of the variables.

This study examined the association of e-cigarette use and abstinence within the context of an ongoing smoking cessation trial using logistic regression models with and without entropy balancing to account for the lack of randomization to the exposure (e-cigarette use for cessation). Using logistic regression models applied to both the unbalanced and entropy-balanced data, we examined the effect of past 3-month use of e-cigarettes to quit (use vs. no use) on 30-day smoking abstinence measured at 3 months. Our findings illustrate important methodological considerations in the interpretation of observational studies of e-cigarette use.

Methods

Study Overview

Data were drawn from an ongoing web-based smoking cessation trial that compares the efficacy of an interactive, evidence-based smoking cessation website (WEB) alone and in conjunction with (a) a theory-driven, empirically-informed social network (SN) protocol designed to integrate participants into the online community, and (b) access to a free supply of NRT products designed to encourage NRT uptake.²² The study uses a 2 (SN integration, no SN) × 2 (access to free NRT, no access) randomized, controlled factorial design with repeated measures at baseline, 3, and 9 months and randomizes participants to: (a) WEB, (b) WEB + SN, (c) WEB + NRT, or (d) WEB + SN + NRT. The primary outcome of the parent study is 30-day point prevalence abstinence (ppa) at 9 months; the current analyses focus on 30-day ppa at 3 months. Study eligibility criteria are current smoking, age 18 years or older, and U.S. residence; exclusion criteria are contraindications to NRT. Randomization is stratified by gender and baseline motivation to quit.

Recruitment

The study protocol is described in detail elsewhere.²² Briefly, current smokers were recruited to the trial immediately following registration on BecomeAnEX, a free, publicly-available web-based cessation program. Eligibility screening, informed consent, and the baseline survey were conducted online via an automated web-based clinical trials management system. Participants were randomized to treatment upon completion of the baseline survey. Recruitment began in March 2012. Analyses focus on the 2,123 participants (of 3,408 randomized; 62.3% response rate) who had completed the 3-month follow-up assessment as of December 31, 2013.

Interventions

All participants had full access to the BecomeAnEX website which provided assistance setting a quit date, training to enhance self-efficacy for quitting and problem-solving skills, assistance in selecting and using pharmacotherapy, and social support through a large online community.^23,24 Participants randomized to the two SN treatment arms received proactive communications from established members of the BecomeAnEX community designed to integrate them into the community. Three former smokers who were well-known, active members of the community served in this role. Using a secure, dedicated interface within the study clinical trials management system, these individuals were notified when new participants were randomized to this treatment arm. Within 24hr, they posted a public message on the new member’s “wall” within BecomeAnEX to welcome them to the site, and encouraged them to fill out their profile or comment on some aspect of an existing profile. Participants randomized to the two NRT arms were mailed a free supply of the NRT product of their choice (patch, gum, or lozenge). NRT was provided as an over-the-counter product (that is, with no additional support or guidance provided) to parallel the experience participants would have if they purchased NRT on their own. A printed calendar with study contact information was included with the NRT shipment for participants to mark the days they used the product. Figure 1 depicts the study procedures from recruitment through follow-up assessment.

Figure 1.

Study procedures from enrollment through follow-up data collection.

Data Collection

Data for these analyses were obtained via web-based self-report assessments at baseline and 3 months.

Measures

The primary outcome for these analyses was self-reported 30-day point prevalence abstinence (ppa) at 3-months.

Our main exposure variable of interest was use of e-cigarettes as a cessation aid reported at the 3-month follow-up. Participants were asked which methods they had used to quit in the past 3 months and were presented a list of common quit methods.²⁵ Participants were considered e-cigarette users if they selected “e-cigarettes” in response to this question or if they entered terms like “vapors,” “vaping,” “vape,” or “ecigs” in the “other quit methods” open-ended response option.

We used a broad range of variables from the baseline assessment as control variables. Demographic characteristics included age, gender, race, ethnicity, marital status, employment, education, and frequency of Internet use.²⁶ Health history items included current health status,²⁷ history of an illness caused or made worse by smoking, and advice to quit smoking from a healthcare provider in the past year. Smoking variables included the number of cigarettes smoked per day, the Fagerström Test for Nicotine Dependence (FTND),²⁸ use of other tobacco products (including chewing tobacco, snuff, or snus), stage of change,²⁹ desire and confidence to quit smoking (1 = not at all, 5 = very much), and the number of quit attempts made in the past year. Past year quit methods were also assessed at baseline and were categorized as unassisted (i.e., participant endorsed no quit methods or selected “cold turkey/unassisted/willpower only”), NRT, prescription medication (i.e., Zyban/Wellbutrin/bupropion, Chantix/varenicline), behavioral interventions (pamphlet or book, individual or group counseling, telephone counseling, Internet quit smoking program other than BecomeAnEX), brand switching or cutting back to quit, and alternative quit methods (e.g., acupuncture, hypnosis, herbal therapies, lasers, homeopathic medicines). Use of e-cigarettes to quit in the past year was also assessed at baseline. Other smoking-related measures included the Smoking Situations Confidence Questionnaire (SSCQ)³⁰ and a series of questions about perceptions of nicotine replacement products adapted from existing instruments^31,32 (e.g., “NRT products double the chance of quitting compared to cold turkey,” “NRT products are too expensive,” and “The nicotine in nicotine stop smoking products is more dangerous than the nicotine in cigarettes”). Psychosocial measures included the “appraisal” and “belonging” subscales of the 12-item Interpersonal Support Evaluation List³³ and select items (Openness to Experience, Extraversion, Emotional Stability) from the Ten-Item Personality Inventory³⁴ that have been linked to social media use.³⁵

In two models, we also included past 3-month use of other quit aids and the number of quit attempts participants had made since enrolling in the trial measured at the 3-month follow-up assessment. These other cessation methods may have been used prior to, concurrent with, or after use of e-cigarettes as a quit aid. The items used to assess quit methods at the 3-month follow up were the same as those used at baseline.

Statistical Analyses

Chi-square and Mann-Whitney tests were used to compare groups of interest (i.e., e-cigarette users and nonusers) with respect to baseline demographic, smoking, and psychosocial characteristics of the full sample. To assess the association between use of e-cigarettes to quit at the 3-month follow-up and 30-day ppa using traditional regression methods, we constructed three models: Model 1 was a unadjusted (bivariate) logistic regression; Model 2 was a multivariable logistic regression that controlled for gender, age, race/ethnicity, education, parent study treatment allocation, Fagerström score, number of quit attempts in the past year, cigarettes per day, self-efficacy to quit, and stage of change; Model 3 was a multivariable logistic regression that controlled for all baseline variables included in Model 2, as well as use of other cessation aids in the past 3 months and the number of quit attempts in the past 3 months. Conventional measures of nicotine dependence, quit history, self-efficacy to quit, and stage of change were included in Models 2 and 3 for comparability with prior smoking cessation research.

Models 4 and 5 employed entropy balancing for all available baseline variables to adjust for differences in baseline characteristics between participants who did and did not report e-cigarette use to quit at the 3-month follow up. The goal of this procedure was to balance the covariate distributions of the two groups of smokers such that there were no measured differences between those who did and did not use e-cigarettes to quit. For Model 5, we also balanced with respect to use of other cessation methods in the past 3 months as recommended by Rosenbaum,³⁶ as the order of use of e-cigarettes and other quit aids was unclear. Entropy balancing yielded weights that were employed as survey weights using the “svy” command in Stata for Models 4 and 5.

In an attempt to further investigate the association between e-cigarette use and cessation and remove the confound of concurrent use of multiple cessation aids, we examined the association of e-cigarette use for cessation to 3-month ppa in the absence of other cessation aids by comparing a subset of smokers that reported exclusive use of e-cigarettes to quit to a subset of smokers that reported exclusive unassisted quit attempt(s) (i.e., cold-turkey) at the 3-month follow up. We followed a similar approach as the one described for the full sample. Due to a small sample size and limited overlap between the treated and control groups in some variables, we were unable to balance the sample on the following 7 covariates: use of other tobacco products, the social and craving subscales of the SSCQ, desire to quit, and 3 items on perceptions of NRT. Given the definition of the subset, further balancing was performed only for the number of quit attempts in the past 3 months. Entropy balancing was performed using ebalance,²¹ and all analyses were implemented in Stata/IC 13.1.

Results

Sample Characteristics and Association With E-Cigarette Use

The sample was primarily female (68.4%), non-Hispanic White (81.0%), and had completed at least some college (50.9%) or had a college degree (24.2%). Median age was 42 years (IQR [interquartile range] = 30–52). Participants smoked a median of 15 cigarettes per day (IQR = 10–20) and had a median FTND score of 5 (IQR = 4–7). The majority (69.1%) had tried to quit smoking in the past year: the most commonly endorsed quit methods (not mutually exclusive) were unassisted (60.9%), NRT (28.7%), e-cigarettes (26.1%), and switching or cutting back (23.9%) (Table 1).

Table 1.

Sample Characteristics of Smokers Who Enrolled in the Parent Study and Completed the 3-Month Follow-Up Survey Between March 2012 and December 2013 Overall and by Use of E-Cigarettes as a Cessation Aid at Follow-Up (N = 2,123)

	Overall (N = 2,123)	E-cigarettes for cessation: yes (n = 672)	E-cigarettes for cessation: no (n = 1,451)	p value*
Baseline variables
Sex, % (n)
Male	31.6 (670)	31.5 (212)	31.6 (458)	.994
Female	68.4 (1,453)	68.5 (460)	68.4 (993)
Race/ethnicity, % (n)
White, non-Hispanic	81.0 (1,719)	81.7 (549)	80.6 (1,170)	.090
Black, non-Hispanic	11.7 (248)	11.3 (76)	11.9 (172)
Hispanic	4.2 (90)	3.0 (20)	4.8 (70)
Other	3.1 (66)	4.0 (27)	2.7 (39)
Age, median (IQR)	42.0 (30–52)	41.0 (30–52)	42.0 (30–52)	.848
Education
College	24.2 (513)	21.9 (147)	25.2 (366)	.058
Some college	50.9 (1,081)	52.5 (353)	50.2 (728)
HS diploma/GED	20.8 (441)	22.6 (152)	19.9 (289)
<HS	4.1 (88)	3.0 (20)	4.7 (68)
Cigarettes per day, median (IQR)	15.0 (10–20)	17.0 (10–20)	15.0 (10–20)	<.001
Fagerström score, median (IQR)	5.0 (4–7)	6.0 (4–7)	5.0 (4–7)	.006
Quit attempt past year, % (n)
Yes	69.1 (1,468)	72.6 (488)	67.5 (980)	.018
No	30.9 (655)	27.4 (184)	32.5 (471)
Number of quit attempts in past year, median (IQR)	2.0 (0–3)	2.0 (0–3)	2.0 (0–3)	.060
Confident about quitting, % (n)
Not at all	5.0 (106)	5.1 (34)	5.0 (72)	.205
A little	14.7 (312)	16.2 (109)	14.0 (203)
Somewhat	43.4 (922)	42.9 (288)	43.7 (634)
A lot	21.3 (452)	22.6 (152)	20.7 (300)
Very much	15.6 (331)	13.2 (89)	16.7 (242)
Stage of change
Quitting in next 30 days	81.3 (1,726)	83.0 (558)	80.5 (1,168)	.378
Quitting in next 6 months	18.4 (390)	16.7 (112)	19.2 (278)
Not thinking of quitting	0.3 (7)	0.3 (2)	0.3 (5)
Past use of cessation aids at baseline, % (n)
E-cigarettes	26.1 (555)	45.1 (303)	17.4 (252)	<.001
Unassisted	60.9 (1,292)	62.2 (418)	60.2 (874)	.387
NRT	28.7 (610)	31.0 (208)	27.7 (402)	.124
Varenicline or bupropion	12.3 (262)	14.1 (95)	11.5 (167)	.087
Behavioral intervention	15.3 (324)	15.9 (107)	15.0 (217)	.564
Switched or cut back	23.9 (507)	24.9 (167)	23.4 (340)	.476
Alternative methods	7.5 (160)	8.3 (56)	7.2 (104)	.344
3-Month follow-up variables
Use of cessation aids at 3 month, % (n)
E-cigarettes	31.7 (672)	100.0 (672)	0.00 (0)	–
Unassisted	69.1 (1,467)	77.4 (520)	65.3 (947)	<.001
NRT	55.9 (1,186)	62.8 (422)	52.7 (764)	<.001
Varenicline or bupropion	4.9 (105)	6.3 (42)	4.3 (63)	.059
Behavioral intervention	21.7 (460)	24.3 (163)	20.5 (297)	.049
Switched or cut back	34.1 (725)	48.7 (327)	27.4 (398)	<.001
Alternative methods	9.6 (204)	13.1 (88)	8.0 (116)	<.001
Number of quit attempts past 3 months, median (IQR)	2.0 (2–4)	3.0 (2–5)	2.0 (1–4)	<.001
30-day point prevalence abstinence	20.0 (425)	15.8 (106)	22.0 (319)	.001

E-cigarette = electronic cigarette; GED = General EdDevelopment; HS = high school; IQR= interquartile range; NRT = nicotine replacement therapy.

*All p values were obtained using chi-square or Mann-Whitney tests, as appropriate.

At the 3-month follow-up, 31.7% of participants (n = 672) reported using e-cigarettes to quit in the past 3 months. E-cigarette users differed on a number of baseline variables compared to nonusers: they reported smoking a greater number of cigarettes per day (CPD; Median [M] = 17, IQR = 10–20 vs. M = 15, IQR = 10–20, p ≤ .001), had a higher Fagerström score (M = 6.0, IQR = 4–7 vs. M = 5.0, IQR = 4–7; p ≤ .05), were more likely to have made at least one quit attempt in the past year (72.6% vs. 67.5%; p ≤ .05), and were more likely to report using e-cigarettes to quit during the past year (45.1% vs. 17.4%; p ≤ .001). At follow-up, e-cigarette users reported making more quit attempts (M = 3, IQR = 2–5 vs. 2, IQR = 1–4; p ≤ .001) and using a greater number of quit methods (not mutually exclusive), including unassisted (77.4% vs. 65.3%; p ≤ .001), NRT (62.8% vs. 52.7%; p ≤ .001), behavioral interventions (24.3% vs. 20.5%; p ≤ .05) switching or cutting back (48.7% vs. 27.4%; p ≤ .001), and alternative methods (13.1% vs. 8.0%; p ≤ .001). Among participants who continued to smoke, both groups reported a reduction in CPD at follow up. However, e-ciga rette users reported a greater average reduction at follow-up than non-e-cigarette users (7.1 CPD, standard deviation [SD] = 8.3 vs. 5.1 CPD, SD = 7.9; p ≤ .05).

In bivariate analyses, those who reported e-cigarette use at 3 months were less likely to report 30-day ppa than those who reported no e-cigarette use at 3 months (15.8% vs. 22.0%; p ≤ .001).

Logistic Regression Models

Table 2 presents unadjusted and adjusted odds ratios for the association between e-cigarette use and 30-day ppa at 3 months. In the unadjusted logistic regression model (Model 1), the odds of abstinence were lower among smokers who used e-cigarettes to quit compared to those who did not use e-cigarettes to quit (odds ratio [OR] = 0.66, 95% confidence interval [CI] = 0.52, 0.85). In the multivariable logistic regression model that controlled only for baseline covariates (Model 2), the negative association between e-cigarette use and abstinence was attenuated but still significant (adjusted odds ratio [AOR] = 0.68, 95% CI = 0.53, 0.87). Additional adjustment for recent use of other cessation aids and number of quit attempts in the past 3 months (Model 3) further weakened the association between e-cigarette use and cessation (AOR = 0.77, 95% CI = 0.59, 1.00).

Table 2.

Unadjusted and Adjusted Odds Ratios for the Association (Main Effects) Between E-Cigarette Use and 30-Day Smoking Abstinence Among Smokers Who Enrolled in the Parent Study and Completed the 3-Month Follow-Up Survey Between March 2012 and December 2013 (N = 2,123)

	Model 1: logistic regression: OR (95% CI)	Model 2: logistic regression: AOR a (95% CI)	Model 3: logistic regression: AOR a,b (95% CI)	Model 4: logistic regression with entropy balancing: AOR c (95% CI)	Model 5: logistic regression with entropy balancing: AOR c,d (95% CI)
3-month follow-up
Cessation method—past 3 monthse
E-cigarettes	0.66 (0.52, 0.85)**	0.68 (0.53, 0.87)*	0.77 (0.59, 1.00)*	0.67 (0.51, 0.87)*	0.83 (0.63, 1.10)
NRT			1.60 (1.20, 2.14)**
Varenicline/bupropion			2.80 (1.78, 4.39)**
Behavioral intervention			1.32 (1.00, 1.73)*
Unassisted			1.04 (0.80, 1.34)
Alternative methods			1.37 (0.95, 1.98)
Switched or cut back			0.22 (0.16, 0.30)**
Number of quit attempts—past 3 months			1.01 (1.00, 1.02)

AOR = adjusted odds ratio; e-cigarette = electronic cigarette; NRT = nicotine replacement therapy; OR = odds ratio.

^{aAdjusted for the following characteristics, measured at baseline: gender, age, race/ethnicity, education, parent study treatment allocation, Fagerstrom score, number of quit attempts in the past year, cigarettes per day, self-efficacy to quit, and stage of change.bAdjusted for use of other quit methods in the past 3 months and number of quit attempts in the past 3 months.cBalanced on sociodemographic variables, parent study treatment allocation, health history, tobacco use history, previous use of quit methods at baseline, perceptions of NRT, and psychosocial measures.dBalanced on use of other quit methods in the past 3 months and number of quit attempts in the past 3 months.eReference condition is no use of that cessation method in the past 3 months.}

*p ≤ .05; **p ≤ .001.

Entropy Balancing

In the logistic regression model with balance for baseline covariates using entropy balancing weights (Model 4; Table 2), the odds of 30-day ppa were 33% lower among e-cigarette users than nonusers (AOR = 0.67, 95% CI = 0.51, 0.87). The logistic regression model with additional balance for concurrent use of other cessation aids and quit attempts in the past 3 months (Model 5) found a nonsignificant association between e-cigarette use and abstinence (AOR = 0.83, 95% CI = 0.63, 1.10). We report the results with and without balance on the use of other cessation aids in the past 3 months to highlight how results change according to different analytical approaches. We note that the association between e-cigarette use and 30-day ppa may be confounded by lack of adjustment for other cessation aids in Models 1, 2, and 4, while the association reported in Model 5 may be hampered by posttreatment bias.

To isolate the relationship between e-cigarette use and cessation, we created a smaller dataset with two subsets of smokers: (a) those who reported e-cigarettes as their only quit method at 3 months (N = 73, “e-cig only”), and (b) those who reported only unassisted quit attempts (N = 221, “unassisted”) at 3 months. At baseline, e-cig only participants smoked more cigarettes per day (M = 18.0, IQR = 11–20 vs. M = 12.0, IQR = 10–20; p ≤ .001) and were more likely to report past-year e-cigarette use to quit at baseline (43.8% vs. 15.4%; p ≤ .001) than unassisted participants. In the unadjusted logistic regression model, e-cig only participants had 52% lower odds of cessation than unassisted participants (OR = 0.48, 95% CI = 0.23, 0.96). After adjusting for baseline characteristics as in Model 2, the magnitude of the association remained nearly the same, though the result was no longer statistically significant (AOR = 0.52; 95% CI = 0.24, 1.12). The entropy-balanced versions of these models revealed nonsignificant results and adjusted odds ratios moving towards 1.0 (AOR = 0.67, 95% CI = 0.23, 1.90; AOR = 0.87, 95% CI = 0.31, 2.42). We note the smaller sample size of this subset and that although the directions of the estimated ORs are consistent, their range in the subset (0.48–0.87) is wider than in the main analysis (0.66–0.83).

Discussion

In this study, e-cigarette use was negatively associated with abstinence in the full sample using unadjusted logistic regression models both with and without entropy balancing. E-cigarette use was also negatively associated with abstinence after adjusting for baseline characteristics both with and without entropy balancing. However, after further adjustment for use of other cessation aids and quit attempts in the past 3 months—variables that may have preceded, co-occurred, or followed e-cigarette use—the association between e-cigarette use and cessation was attenuated and nonsignificant. These findings serve to illustrate the importance of multiple analytical approaches and the value of careful measurement of other cessation aids in secondary analyses of data from studies that were not originally meant to investigate the association between e-cigarette use and cessation. Observational studies should employ a range of analytic approaches to address potential challenges inherent in nonrandomized designs such as selection bias and lack of information on timing of use of other cessation aids.

The inverse association between e-cigarette use for cessation and abstinence in adjusted models could be explained by the lack of randomization to e-cigarette use: smokers in our sample who used e-cigarettes to quit smoked more cigarettes per day, had higher Fagerström scores, were more likely to have tried e-cigarettes to quit in the past year, and made more quit attempts and used a greater variety of quit methods at 3 months than those who did not use e-cigarettes to quit. In subset analyses, some of these between-group differences were even more pronounced when comparing smokers who exclusively used e-cigarettes to those who made an unassisted quit attempt, though differences became nonsignificant after adjustment. While some statistical methods such as entropy balancing are capable of reducing the impact of selection bias, they may not eliminate its effect. Although tightly controlled randomized trials have lower external validity, there is no substitute for randomization when investigating the initial efficacy of a new intervention such as e-cigarettes. At the same time, careful observational studies can make valuable contributions to the literature when they clarify how smokers use e-cigarettes on a population level in “real-world” situations.

Our findings that e-cigarette users made a greater number of quit attempts and used more quit methods than those who had not used e-cigarettes are consistent with other observational studies showing that e-cigarette use is associated with use of NRT or prescription medication for cessation, a higher number of quit attempts, and a higher number of quit attempts lasting at least 24hr.^2,6 We cannot make causal statements about the association between e-cigarette use, use of other quit methods, and quit attempts, but note that these associations have now been reported in at least three studies and are worthy of further investigation. E-cigarettes may positively influence public health if their widespread availability stimulates quit attempts.³⁷

It is also notable that smokers who reported e-cigarette use at the 3-month follow up also used nearly every other cessation method to a greater extent than smokers who did not try e-cigarettes to quit smoking, including switching or cutting back as a quit method (48.7% vs. 27.4%, p < .001). While e-cigarettes may positively affect public health if they encourage quit attempts, they may negatively affect public health if they distract smokers from more effective quit methods. Future research should assess how smokers use cessation aids in combination with e-cigarettes, how smokers use e-cigarettes to reduce cigarette consumption, and how these behaviors are associated with the number and success of subsequent quit attempts.

Measurement bias is a limitation of our study and of previously published observational studies of the association between e-cigarette use and smoking cessation.^{6,9–11,18,19} The limitations inherent in the field’s current approach to measuring e-cigarette use have been noted in other research.^10,19,38 Our study improves upon previous observational studies in that we evaluated the association between e-cigarette use specifically for smoking cessation, rather than associating any e-cigarette use with cessation. However, we do not have data about the extent or patterns of e-cigarette use and, thus, have likely conflated casual or one-time e-cigarette use with persistent and long-term e-cigarette use. Additionally, the type of e-cigarette device used, frequency and duration of use, nicotine fluid concentration, and efficiency of nicotine delivery are all important variables that were not available from the parent trial. Dose and adherence are integral to the effectiveness of any cessation aid.^39,40 Information on how e-cigarettes are used during a quit attempt would shed light on the potential of these devices as cessation aids and inform ways to maximize their effectiveness with appropriate instructions. At a minimum, future observational studies should include items on daily versus non-daily e-cigarette use in addition to the purpose of e-cigarette use.

We were not able to evaluate which cessation aids were associated with a successful quit attempt, nor were we able to discern which aids were used singly or in combination among participants who reported multiple cessation aids at the 3-month follow up. Lack of data on the order of cessation aids may make this analysis susceptible to posttreatment bias, in that the use of e-cigarettes for cessation could have later influenced a participant’s decision to use another cessation aid. As we cannot ascertain the order of the cessation aids, nor whether use of an additional aid was influenced by e-cigarette use or some other baseline characteristic (such as nicotine dependence), we chose to include models with (Models 3 and 5) and without (Models 2 and 4) use of other cessation aids, acknowledging the results from Models 3 and 5 may be sensitive to posttreatment bias.⁴¹ Future observational studies should include items about the most recent quit attempt to ascertain the effect of an aid or combination of aids on the subsequent attempt.

Entropy balancing has some inherent limitations that deserve discussion. As with all weighting methods, entropy balancing reduces precision in statistical analyses. In the subset, we were also unable to balance on all pretreatment variables that were used to balance the parent sample due to limited overlap between the treated and control groups on these characteristics. Additional limitations include the risk that, despite the numerous covariates included in the entropy balancing procedure, we may have failed to assess some characteristic that affects both e-cigarette use for cessation and 30-day ppa, resulting in unbalanced groups. It is also possible that study participants who quit successfully using methods not promoted by the parent study may have been more likely to fail to complete the follow-up, leading to worse apparent outcomes for smokers who used e-cigarettes or other non-promoted cessation aids (e.g., varenicline) to quit. Finally, results are limited to this sample of motivated quitters that had enrolled in a web-based cessation program and may not be generalizable to all smokers.

At baseline, smokers who used e-cigarettes to quit were fundamentally different from smokers who did not and had more complex quit attempt histories at follow-up. These differences pose complex challenges to analyses of observational data of the relationship between e-cigarette use and smoking cessation. Examining the real-world effectiveness of e-cigarettes for cessation using observational data is essential to assess the public health impact of e-cigarettes. Careful use of various analytic methods is critical to ensure that research provides data-driven conclusions to inform ongoing debates on the potential merits and harms of these products. Bias from unmeasured confounders cannot be fully controlled for in observational data, and this analysis illustrates that even sophisticated statistical approaches cannot fully address fundamental shortfalls in the study design regarding participant selection and lack of information (e.g., timing of use of other cessation aids). When evaluating a novel product like e-cigarettes, untestable assumptions should be avoided and caution must be exercised when drawing inferences from observational data, even when using multiple techniques.

Funding

This study was supported by funding from the National Cancer Institute of the National Institutes of Health (#5R01CA155489-03; P30CA051008) and the National Institute on Drug Abuse of the National Institutes of Health (#1K01DA037950-01). The study is registered at ClinicalTrials.gov (NCT01544153).

Declaration of Interests

JLP, SC, RSN, and ALG are employees of Legacy, a non-profit public health foundation that runs becomeanex.org, an online tobacco cessation intervention.