Smoking cessation in individuals who use vaping as compared with traditional nicotine replacement therapies: a systematic review and meta-analysis

Abstract

Objectives

Despite the aggressive marketing of electronic nicotine device systems (ENDS) as smoking cessation tools, the evidence of their effectiveness is mixed. We conducted a systematic review of randomised controlled trials to determine the effect of ENDS on cigarette smoking cessation, as compared with other types of nicotine replacement therapies (NRT).

Design

Systematic review and meta-analysis using the Grading of Recommendations Assessment, Development and Evaluation approach.

Data sources

MEDLINE, Embase, the CENTRAL Trials Registry of the Cochrane Collaboration using the Ovid interface, ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform trials registries were searched through 17 June 2020.

Eligibility criteria for studies

Randomised controlled trials in which any type of ENDS was compared with any type of NRT, in traditional cigarette users.

Data extraction and synthesis

The primary outcome was smoking cessation, defined as abstinence from traditional cigarette smoking for any time period, as reported in each included study, regardless of whether abstinence is self-reported or biochemically validated. Secondary outcomes included smoking reduction, harms, withdrawal and acceptance of therapy. A random-effect model was used, and data were pooled in meta-analyses where appropriate.

Results

Six studies were retained from 270. Most outcomes were judged to be at high risk of bias. The overall quality of evidence was graded as ‘low’ or ‘very low’. Pooled results showed no difference in smoking cessation (rate ratio (RR) 1.42, 95% CI 0.97 to 2.09), proportion of participants reducing smoking consumption (RR 1.25, 95% CI 0.79 to 1.98), mean reduction in cigarettes smoked per day (mean difference 1.11, 95% CI −0.41 to 2.63), or harms (RR 0.96, 95% CI 0.76 to 1.20), between groups.

Conclusion

We found no difference in smoking cessation, harms and smoking reduction between e-cigarette and NRT users. However, the quality of the evidence was low. Further research is needed before widespread recommendations are made with regard to the use of ENDS.

PROSPERO registration number

Systematic review registration number: protocol registered with the International Prospective Register of Systematic Reviews (PROSPERO) on February 27^th, 2020; CRD42020169416.

Strengths and limitations of this study.

• This study provides up to date meta-analyses of direct comparisons of vaping with nicotine replacement therapy for smoking cessation, studied through randomised controlled trials (RCTs).

• We examined harms associated with vaping, which are becoming increasingly concerning.

• This study makes extensive efforts to obtain unreported data from investigators.

• Careful consideration is given to the potential impact of risk of bias and methodological heterogeneity.

• As we included only RCTs, many studies that used weaker study designs were ineligible for this review.

Introduction

Background

Despite a significant lack of rigorous pharmacological testing, the use of electronic nicotine device systems (ENDS), otherwise known as vaping devices, has been aggressively marketed as an effective method to quit smoking. In Canada, 32% of current and former smokers report having used ENDS as a smoking cessation aid.¹ In addition to delivering nicotine to the user, ENDS are thought to replace some of the habitual behaviours and sensations associated with smoking, such as the action of bringing a cigarette to the mouth. By doing so, ENDS may provide coping mechanisms that other traditional nicotine replacement therapies (NRT) do not offer, and therefore, may help with the behavioural component of smoking reduction and cessation.² While vaping is believed to be less harmful than cigarette smoking, a large number of emerging reports on the health impacts of vaping are worrisome. In addition, the evidence on the effectiveness of ENDS as a smoking cessation aid is mixed.

In 2016, a meta-analysis of 20 studies found that people using ENDS had a 28% reduction in the odds of stopping cigarette smoking as compared with those not using ENDS.³ However, in a 2019 recent randomised controlled trial (RCT), individuals randomised to nicotine-containing e-cigarettes were more likely to abstain from smoking at 1 year compared with individuals randomised to nicotine patches (18% compared with 9.9%, rate ratio (RR) 1.83; 95% CI 1.30 to 2.58).⁴ A Cochrane review⁵ found that nicotine-containing e-cigarettes were more effective than non-nicotine containing e-cigarettes for smoking cessation, but was not able to compare ENDS products to traditional NRT.

Little information is known about the long-term health impacts of ENDS. Reports of acute toxicity have recently captured the public’s attention. In late 2019 and early 2020, ‘e-cigarette, or vaping, product use-associated lung injury’ (EVALI) caused 2807 illnesses and 68 deaths in the USA,⁶ and 19 cases in Canada.⁷ Other short-term adverse events reported with the use of ENDS include cardiovascular changes such as increased heart rate and blood pressure, cough, wheeze⁸ and mucus production.⁹ Burn injuries have also been reported, as well as fatalities from drinking or injecting the e-liquid.⁸

There are no long-term data available on the relationship between ENDS and oral, respiratory and cardiovascular health, as well as cancer. There is, however, available data linking the chemicals present in e-liquids with cellular DNA damage and carcinogenicity.^{9 10} There is some evidence that the use of ENDS is associated with asthma exacerbations.¹¹ No human long-term data exist on the use of ENDS in pregnancy and their impact on the developing fetus.

Given the large number of smokers using ENDS as a potential smoking cessation tool, there is a need to review and synthesise the evidence of trials examining a head to head comparison of ENDS versus traditional NRT for smoking cessation.

Objective

The objective of this review is to systematically review the evidence found in RCTs to determine the effect of ENDS on cigarette smoking cessation in smokers, as compared with other types of NRT.

Methods

Protocol and registration

The protocol for this systematic review was submitted to International Prospective Register of Systematic Reviews (PROSPERO) on February 27^th, 2020 (CRD42020169416) and uploaded as a preprint on Open Science Framework (OSF) Preprints on May 12^th 2020.¹²

Patient and public involvement

No patient involved.

Criteria for study inclusion

Study characteristics

RCTs in which ENDS were compared with non-electronic NRT in smokers were included. We restricted our inclusion to RCTs to minimise the risk of bias. No language limits were imposed. No date limits were imposed either, although we did not anticipate studies published prior to 2003, since this is when the first e-cigarette was invented.¹³ There was no geographical restriction of studies.

Study population

All traditional cigarette users were included, regardless of age, amount of traditional cigarette use and motivation to quit.

Intervention of interest

The intervention of interest comprised all types, models and brands of ENDS.

Comparators

All included studies compared ENDS with non-electronic NRT. NRT comprised, but were not limited to, nicotine patch, gum, lozenge, nasal spray, inhalator, mouth spray, mouth strips, microtabs and combination of products.

Outcome measures

The primary outcome measure is traditional cigarette smoking cessation defined as abstinence from traditional cigarette smoking for any time period, as reported in each included study, regardless of whether abstinence is self-reported or biochemically validated.

Secondary outcomes include reduction in the number of traditional cigarettes smoked in any given time period, adverse events, withdrawal symptoms and participants’ acceptance of therapy. We had planned on collecting quit attempts information but none of the studies reported on this outcome.

Settings

All healthcare and community settings were included.

Study identification

The following databases were searched through 17 June 2020: MEDLINE (1946 to June 2020), Embase (1947 to June 2020) and the CENTRAL Trials Registry of the Cochrane Collaboration (May 2020 Issue) using the Ovid interface. The MEDLINE search was limited using the Cochrane Highly Sensitive Search Strategy and the Embase search was limited using the recommended limit for controlled trials.¹⁴ Searches were developed by a librarian experienced in systematic reviews, using a method designed to optimise term selection.¹⁵ ClinicalTrials.gov and WHO International Clinical Trials Registry Platform trials registries were searched for registered intervention studies, regardless of their completion status. Electronic search strategies are presented in online supplemental material 1. The reference lists of included studies and any applicable review studies were searched.

Authors of protocols identified through registries were contacted electronically, to request data for the review. In addition, clinical experts in the field of vaping and smoking cessation were contacted to enquire about any unpublished research fulfilling our inclusion criteria.

Selection of studies

Records retrieved by the electronic search were downloaded and imported into a Reference Manager database for duplicate removal, and then uploaded to Covidence. Throughout the review, newly identified records were integrated into the set for screening.

Each title and abstract was independently screened by two review authors (from CMP, JZZ and ATK) against the eligibility criteria.¹⁴ Full text of all studies deemed potentially eligible was obtained and reviewed independently by two of the same review authors to determine eligibility. For screening, data extraction and risk of bias assessment, disagreements were resolved by discussion, and with a third reviewer when needed.

Data extraction and management

For studies that fulfilled the inclusion criteria, two reviewers (CMP and JZZ) extracted the data into an electronic data collection form, which was piloted by both reviewers (online supplemental material 2). The data collection was revised, based on feedback from the reviewers. Study authors were contacted electronically to obtain relevant but unavailable data.

Risk of bias assessment for included studies

Two reviewers (CMP and JZZ) independently conducted the risk of bias assessment for each study at the outcome level using the Revised Cochrane risk-of-bias tool for randomised trials.¹⁶

Measures of treatment effect

Dichotomous data were analysed by calculating the prevalence RR, using the longest follow-up time reported, as well as the 95% CI. The prevalence RR for smoking cessation was calculated as such:

$${\displaystyle \mathrm{R}\mathrm{R}=\frac{\frac{\mathrm{N}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{s}\mathrm{a}\mathrm{b}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{f}\mathrm{r}\mathrm{o}\mathrm{m}\mathrm{s}\mathrm{m}\mathrm{o}\mathrm{k}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{N}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}}{\frac{\mathrm{N}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{s}\mathrm{a}\mathrm{b}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{f}\mathrm{r}\mathrm{o}\mathrm{m}\mathrm{s}\mathrm{m}\mathrm{o}\mathrm{k}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}}{\mathrm{N}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}}}}$$

Continuous data for the secondary outcomes were analysed through mean differences (MD) between groups as the same scales were used. In the case of studies with multiple arms, we only extracted data for the groups relevant to this review.

Data synthesis

We provide a synthesis of the included studies (table 1). Where appropriate, data have been pooled for meta-analyses, and random effects were used for all analyses in RevMan.¹⁴ The inverse-variance random-effects and the MD approach (using SD and sample sizes) were used for dichotomous and continuous outcomes, respectively, to assign the weight given to each study. Participants with missing data were considered as still smoking.⁵ The proportion of adverse events reported was based on the number of people available for outcome assessment. For the reduction of the number of cigarettes smoked, missing values were assumed to be 0.

Table 1.

Characteristics of included studies

Characteristics of RCTs measuring smoking cessation at 6 months or later
Author and year of publication	Design	Country	No of participants	Main eligibility criteria	Intervention	Comparator	Main outcome of interest
Bullen, 201318	3 group, parallel, single centre	Australia	657 total, 584 included in this review (2 of 3 groups)	>18 years, smoked >10 cigarettes per day in the past year, motivated to quit	First-generation e-cigarette × 12 weeks	Nicotine patch x 12 weeks	Continuous abstinence 6 months after quit day
Hajek 20194	2 group, parallel, multicentre	UK	884	Adults with no strong preference towards e-cigarette or NRT	Any type of e-cigarette	Any nicotine-replacement therapy	Continuous abstinence 52 weeks after quit day
Lee, 201921	2 group, parallel, single centre	Republic of Korea	150	>18 years, smoked >10 cigarettes per day in the past year, motivated to quit	e-cigarette × 24 weeks	Nicotine gum × 24 weeks	Continuous abstinence 24 weeks after quit day
Lee, 201921	2 group, parallel, single centre	USA	30	Adults, smoked >2 cigarettes per day in the past year, smoked at least once in last 7 days	e-cigarette × 6 weeks	Nicotine patch × 5 weeks, then placebo patch × 1 week	7-day point prevalence abstinence at 6 months
Characteristics of RCT measuring smoking cessation earlier than 6 months
Hatsukami, 201920	4 group, parallel, multicentre	USA	264 total, 152 included in this review (2 of 4 groups)	>18 years, smoked >5 cigarettes per day	e-cigarettes	Nicotine gum or nicotine lozenge	7-day point prevalence abstinence at 8 months
Characteristics of RCT measuring other outcomes
Eisenhofer, 201519	2-group, parallel, single centre	USA	11	Veterans who met criteria for tobacco disorder	e-cigarettes × 3 weeks	Nicotine patch × 3 weeks	Reduction in no of cigarettes smoked per day at 3 weeks

NRT, nicotine replacement therapies; RCT, randomised controlled trial.

Assessment of heterogeneity

A p value of 0.10 for the χ² test (Cochrane Q) and an I² value of >50% were used as indicators of substantial heterogeneity. This, however, needs to be interpreted with caution given the small number of studies available for the meta-analysis. Clinical and methodological diversity was also explored.

We planned to assess reporting/publication bias using funnel plots of effect estimate against SE, and testing for funnel plot asymmetry, however, the number of included studies was too low (<10).

We also planned on conducting a number of sensitivity analyses to determine the robustness of the results of the meta-analyses; subgroup analyses to investigate potentially modifying factors such as age and smoking intensity; as well as meta-regression to study the impact of covariates such as motivation to quit smoking, provision of training and other factors,¹⁷ but minimum data thresholds were not met.

We present a ‘Summary of Findings’ table (table 2) for all outcomes. We used the five Grading of Recommendations Assessment, Development and Evaluation considerations (study limitations, consistency of effect, imprecision, indirectness and publication bias)¹⁴ to assess the quality of evidence for each outcome and to draw conclusions about the robustness of evidence within this review.

Table 2.

Summary of findings table

Nicotine-containing Electronic cigarettes (ENDS) vs Nicotine Replacement Therapies (NRT) for smoking cessation
Population: current smokers at enrolment into trials
Intervention: Nicotine-containing e-cigarettes
Comparison: Nicotine-replacement therapies
Outcomes ENDS as compared with NRT	Relative effect (95% CI)	No of participants (studies)	Quality of the evidence (GRADE)	Comments
Cessation	RR 1.42 (0.97 to 2.09)	1800 (5 studies)	⊕⊕OO*† low
Smoking reduction Proportion of people decreasing cigarette consumption by 50% Mean decrease in cigarettes per day	RR 1.25 (0.79 to 1.98) MD 1.11 (-0.41 to 2.63)	1460 (4 studies) 633 (3 studies)	⊕⊕OO*† low ⊕⊕OO*† low
Adverse events (AEs)	RR 0.96 (0.76 to 1.20)	758 (4 studies)	⊕OOO*†‡ Very low	No severe AEs related to investigated products were reported
Withdrawal symptoms	Summary data not available	4 studies	⊕OOO*†‡ Very low	Withdrawal measures included Minnesota Nicotine Withdrawal Scale, QSU scores, frequency of urge and strength of urge score and prespecified symptoms of depressed mood, irritability, restlessness and hunger
Acceptance of therapy	Summary data not available	4 studies	⊕OOO*†‡ Very low	Acceptance defined as wanting to recommend product to friends, helpfulness, taste, satisfaction, psychological reward, enjoyment of sensation, aversion, and ability to reduce craving depending on study

GRADE Working Group grades of evidence.

High quality: Further research is very unlikely to change our confidence in the estimate of effect.

Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.

Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.

Very low quality: We are very uncertain about the estimate.

*Downgraded one level because of risk of bias.

†Downgraded one level because of heterogeneity.

‡Downgraded one level because of imprecision of results.

GRADE, Grading of Recommendations Assessment, Development and Evaluation; MD, mean difference; QSU, Questionnaire on Smoking Urges; RR, rate ratio.

Results

Our initial bibliographic search yielded 270 records, and after screening and full-text review, we retained six RCTs. An updated search conducted in June 2020 yielded an additional 116 records (for a total of 386 records), none of which were included after screening (figure 1).

Figure 1.

Study flow diagram.

We identified six RCTs (Bullen et al,¹⁸ Eisenhofer et al,¹⁹ Hajek et al,⁴ Hatsukami et al,²⁰ Lee et al,²¹ Lee et al.²² Of these, five contributed data to our primary outcome of smoking cessation.4 18 20–22 Four studies4 18 21 22 examined cessation at 6 months or longer, while one²⁰ examined short-term cessation (<6 months). Table 1 includes the salient features of the included studies. A more detailed description of included studies can be found in online supplemental material 3.

Risk of bias in included studies

We assessed risk of bias for each included study. A detailed report of the risk of bias assessment can be found in online supplemental material 4.

Figure 2 illustrates the risk of bias for each outcome.

Figure 2.

Risk of bias for each outcome.

Effect of interventions

Smoking cessation

Five of the six studies reported on smoking cessation.4 18 20–22 When comparing e-cigarettes to NRT in the context of smoking cessation, there was no significant difference between groups in verified self-reported continuous abstinence at 6 months (21/289 vs 17/295, RR 1.26, 95% CI 0.68 to 2.34, p=0.46) in the Bullen et al¹⁸ study, and in continuous abstinence from 9 to 24 weeks (16/75 vs 21/75, RR 0.76, 95% CI 0.43 to 1.34, p=0.344) in the Lee et al²¹ study. In addition, the Lee et al²² study showed no difference between groups for the 7-day point prevalence abstinence at 6 months in the context of perioperative smoking cessation (5/20 vs 1/10, RR 2.50, 95% CI 0.34 to 18.63, p=0.63).

In the Hajek et al ⁴ study, self-reported, verified continuous abstinence at 1 year was found to be higher in the e-cigarette group (79/438 vs 44/446, RR 1.83, 95% CI 1.30 to 2.58, p<0.001), and smoking cessation assessed by 7-day point prevalence at 8 weeks in the Hatsukami et al²⁰ trial was also higher in the e-cigarette group (25/76 vs 13/76, RR 1.92, 95% CI 1.07 to 4.37, p=0.039).

We combined data from all five studies comparing smoking cessation between e-cigarettes and NRT and obtained a pooled RR 1.42, 95% CI 0.97 to 2.09 (figure 3).

Figure 3.

Pooled results per outcome. ENDS, electronic nicotine device systems; NRT, nicotine replacement therapies.

Smoking reduction

All six studies^{4 18–22} assessed smoking reduction. Bullen et al,¹⁸ Eisenhofer et al,¹⁹ Hajek et al⁴ and Lee et al²² reported the proportion of participants reducing smoking by at least 50%. While Lee et al²¹ also reported on this outcome, the size of the reduction was not specified. Bullen et al¹⁸ and Lee et al²¹ reported an absolute reduction, and Hatsukami et al²⁰ reported a relative reduction in cigarettes per day from baseline.

In the Bullen et al study,¹⁸ mean cigarette consumption at 6 months decreased by 9.7 (SE 0.4) in the e-cigarette group, and by 7.7 (SE 0.4) in the NRT group. MD between groups was 1.9 (SE 0.6) (p=0.002). After excluding people who successfully quit smoking, the RR of decreasing cigarette smoking by at least 50% when comparing the e-cigarette to the NRT groups was 1.61 (95% CI 1.31 to 1.99).

Eisenhofer et al¹⁹ compared week 3 to week 1, and showed that both e-cigarettes (t=5.3, p=0.013) and NRT (t=3.4, p=0.015) significantly reduced (∼50%) self-reports of cigarettes smoked in the previous 24 hours. This was confirmed by significant reductions of breath carbon monoxide (CO) levels in both groups No additional information could be obtained from the abstract and none of the authors could be reached.

In the Hajek et al⁴ study, 44 of 345 participants in the e-cigarette group, and 29 of 393 participants in the NRT group experienced a CO-validated reduction in smoking of >50% in participants without abstinence between weeks 26 and 52, yielding a relative risk of smoking reduction of 1.73 (1.11–2.70).

Hatsukami et al²⁰ defined smoking reduction by the estimated ratio of cigarettes smoked at 8 weeks as compared with baseline, with a result of 0.25 (0.17, 0.37) in the e-cigarette group, and 0.29 (0.21, 0.39) in the NRT group (p=0.185). Additional data obtained from the author showed that 19 participants in the e-cigarette group and 22 participants in the NRT group reduced smoking consumption by 50% (RR 0.86, 95% CI 0.51 to 1.46) at 8 weeks, and that mean cigarette consumption decreased by 9.22 (SD 7.95) in the e-cigarette group, and by 7.61 (SD 8.27) in the NRT group. The MD between groups was 1.61, (95% CI −0.97 to 4.19).

In the Lee et al²¹ study, mean cigarette consumption decreased at 24 weeks by 6.5±2.87 (SD) in the e-cigarette group, and by 6.60±3.75 (SD) in the NRT group (p=0.974). In addition, 31 out of 75 participants (41.3%) in the e-cigarette group and 19 out of 75 participants (25.3%) in the NRT group reduced their daily cigarette consumption (p=0.038), but no information on size of smoking reduction is provided. After excluding abstainers, an RR of 1.49, (95% CI 0.97 to 2.31) was obtained for decrease in daily cigarette consumption.

Lastly, in the Lee et al,²² one participant in the END group and four participants in the NRT group reduced their cigarette consumption by at least half, resulting in an RR 0.15, 95% CI 0.02 to 1.14).

We combined data from the Bullen et al,¹⁸ Hajek et al,⁴ Hatsukami et al²⁰ and Lee et al²² studies comparing smoking reduction of at least 50% between e-cigarettes and NRT, as they used similar measures. Pooled results comparing the difference in smoking reduction between the e-cigarette and the NRT groups produced an RR of 1.25, with the line of equivalence falling within the 95% CI (0.79 to 1.98) (figure 3).

We also combined data from the Bullen et al,¹⁸ Hatsukami et al²⁰ and Lee et al²¹ comparing mean reduction of cigarettes per day from baseline for ENDs and NRT. Meta-analysis yielded an MD of 1.11, with the line of equivalence falling within the 95% CI (−0.41 to 2.63) (figure 3).

Harms

Five studies reported on harms (Bullen et al,¹⁸ Hajek et al,⁴ Hatsukami et al,²⁰ Lee et al,²¹ Lee et al.²¹ None of the included studies reported serious adverse events (SAEs) related to e-cigarettes or NRT.

In the Bullen et al¹⁸ study, 107 participants in the e-cigarette group reported 137 adverse events, while 96 participants in the NRT group (patches) reported 119 events, and, using the number of participants available for analysis at 6 months, there was no difference in the incidence of adverse events between groups (RR 0.99, 95% CI 0.81 to 1.22). No difference between groups was also observed in the Hatsukami et al²⁰ study, where additional data provided by the author showed that 51 of 69 participants in the e-cigarette group and 53 of 72 participants in the NRT group reported adverse events (1.00, 95% CI 0.82 to 1.22), and in the Lee et al²² study, where no significant difference in the incidence of adverse events between groups was seen at 8 weeks (RR 1.24, 95% CI 0.54 to 2.84).

Hajek et al⁴ defined adverse events of interest as nausea, sleep disturbances, and throat and mouth irritation. There were 27 SAEs in the e-cigarette group and 22 in the NRT group, none felt to be related to the intervention or control products. Based on the number of participants available at the 12-month follow-up, e-cigarettes were found to be less likely associated with nausea (RR 0.78, 95% CI 0.66 to 0.92) and sleep disturbances (RR 0.88, 95% CI 0.83 to 0.95), but more likely associated with throat/mouth irritation (RR 1.24, 95% CI 1.13 to 1.37). These numbers, however, should be interpreted with caution as it was not possible to determine with certainty the denominator from the data.

In the Lee et al study,²¹ 5 participants in the e-cigarette group and 13 participants in the nicotine gum group reported adverse events. There were no SAEs. Based on the number of participants who completed the study, e-cigarettes were less likely to be associated with adverse events (RR 0.13, 95% CI 0.12 to 0.87).

We combined data from the Bullen et al,¹⁸ Hatsukami et al,²⁰ Lee et al,²¹ Lee et al²² studies comparing harms between e-cigarettes and NRT. Hajek et al⁴ were excluded as they did not clearly report the number of participants that experienced any adverse events and reported only on specific adverse events. Pooled results comparing ENDS to NRT yielded an RR of 0.96, (95% CI 0.76 to 1.20) (figure 3).

Withdrawal symptoms

Four studies reported on the results of withdrawal symptoms (Eisenhofer et al,¹⁹ Hajek et al,⁴ Hatsukami et al²⁰ and Lee et al²² and all used different scales. Eisenhofer et al¹⁹ assessed withdrawal with the Questionnaire on Smoking Urges, Hajek et al⁴ used a composite urge score (frequency and strength of urge to smoke), Hatsukami et al²⁰ measured the severity of withdrawal using the Minnesota Nicotine Withdrawal Scale, and Lee et al²¹ assessed withdrawal symptoms as part of their adverse event assessment. In light of the differences in outcome assessment measures, the data were not pooled.

In Eisenhofer et al,¹⁹ urges and cravings to smoke were significantly reduced in the e-cigarette group (t=3.8, p=0.03), but not in the NRT group (t=2.1, p=0.08).

In Hajek et al,⁴ urges for e-cigarette users decreased more than for NRT users at 1 week (MD: −0.4 (95% CI −0.6 to -0.2)) and at 4 weeks (MD: −0.3 (95% CI −0.5 to −0.1)). E-cigarette users also reported a smaller increase from baseline in irritability, restlessness, inability to concentrate, hunger and depression. The withdrawal symptoms disappeared mostly for both groups by week 4.

In Hatsukami et al,²⁰ participants in the e-cigarette group reported lower median (min/max) changes from baseline on the severity scale compared with participants in the NRT group at all measurement points, with week 1 (3.0 (−9.0/25.0) vs 3.5 (−20.0/32.0)), week 2 (1.0 (−13.0/25.0) vs 3.0 (−13.0/39.0)) and week 4 (1.0 (−17.0/30.0) vs 2.5 (−28.0/29.0). The planned pairwise comparisons were significant with p<0.017. As well, fewer participants (5.3%) withdrew from the complete substitution e-cigarettes group than from the NRT group (15.8%) for product-related reasons (disliking product or experiencing withdrawal symptoms; p value not reported).

Lee et al²² only reported on withdrawal symptoms for the NRT group, and did not report on withdrawal symptoms for the e-cigarette group.

Acceptance of therapy

Four studies reported on acceptance of therapy (Bullen et al,¹⁸ Hajek et al,⁴ Hatsukami et al²⁰ and Lee et al,²² and all used different scales. In light of the difference in outcome assessment measures, the data were not pooled.

In the Bullen¹ study,¹⁸ 230 out of 260 participants (88%) in the e-cigarettes group said they would recommend their allocated product to a friend at 1 month, as compared with 130 out of 232 participants (56%) in the NRT group (RR 1.58, 95% CI 1.40 to 1.78)). At 6 months, 205 out of 241 participants (85%) in the e-cigarettes group said they would recommend their allocated product as compared with 107 out of 215 participants (50%) in the NRT group (RR 1.71, 95% CI 1.48 to 1.97)).

In the Hajek et al study,⁴ acceptance of therapy was measured with a Likert scale (1–5, with a higher score associated with higher acceptance). At 4 weeks postquit date, helpfulness of e-cigarettes was rated 4.3 (SD 0.9) while that of NRT was 3.7 (SD 0.9) (MD 0.6 (0.4, 0.7)). Taste was scored at 3.5 (SD 1.3) for the e-cigarette group and 3.1 (SD 1.5) (MD 0.4 (0.2,0.6)), and satisfaction was rated at 2.7 (SD 1.1) and 2.3 (SD 1.2), respectively, for the e-cigarette and NRT groups (MD 0.5 (0.3, 0.6)).

In the Hatsukami et al study,²⁰ acceptance of therapy was defined as satisfaction with the product, psychological reward, enjoyment of sensation, aversion and ability to reduce craving. Results are reported for the NRT group as an estimated MD and 95% CI in product evaluation subscales using the e-cigarette group as a reference. The following results are reported; satisfaction: −0.6 (−1.0, –0.1), psychological reward: −0.4 (−0.8, 0.01), enjoyment of sensation: −0.6 (−1.1, –0.1), aversion: 0.1 (−0.2, 0.4), and ability to reduce craving: −0.3 (−0.8, 0.2).

Lastly, the Lee et al trial²² defined acceptance of therapy as satisfaction with the assigned product, measured with a Likert scale (1–7, with a higher score associated with higher satisfaction). Median scores and IQR are reported. Participants randomised to the e-cigarette group reported scores of 6 (4–7), 5.5 (2.5–7) and 6 (5–7), respectively, while participants randomised to the NRT group reported scores of 5 (3–7), 5 (3–6) and 7 (6–7), respectively, for the following questions. ‘The product is helpful for quitting smoking’, ‘I was satisfied with the product to help with quitting’, ‘I would recommend the product to someone interested in quitting smoking’.

Risk of bias across studies

The review process we used was thorough, and we took every precaution to minimise the risk of bias due to publication bias or selective reporting. We reached out to clinical experts to enquire about unpublished reports, examined protocol registries, and contacted the authors of identified protocols to request unpublished results. Given the low number of retained studies, we did not include a funnel plot.

Sensitivity, subgroup and meta-regression analyses

We performed a sensitivity analysis for the smoking cessation outcome by removing the Lee et al study.²² While the other four studies aimed to assess smoking cessation in general, Lee et al were targeting a perioperative population, who may have had different motivations to quit smoking. The pooled data, once Lee et al²² is removed, yield an RR of smoking abstinence of 1.39 (95% CI 0.92 to 2.11) when comparing ENDS to NRT (figure 4).

Figure 4.

Sensitivity and subgroup analyses. ENDS, electronic nicotine device systems; NRT, nicotine replacement therapies.

We had planned on undertaking multiple subgroup analyses. We were unable to perform the subgroup analyses based on age (all participants were adults), smoking intensity (no study enrolled smokers >25 cigarettes per day), or biochemically validated smoking cessation (all studies used biochemical validation). We also could not perform a subgroup analysis of studies with ties to industry as only Bullen et al¹⁸ was found to have ties to the vaping industry.

We did, however, perform the following subgroup analyses: limiting comparator to nicotine patches (Bullen et al¹⁸ and Lee et al,²¹ and including only studies assessing continuous/sustained smoking abstinence >6 months given that smoking cessation is defined as sustained abstinence for at least 6 months²³; (Bullen et al,¹⁸ Hajek et al,⁴ Lee et al²¹ (figure 4)).

Meta-regression analyses were not performed as our threshold of 10 eligible studies was not met.

Discussion

In our review, there was no significant difference in smoking cessation, smoking reduction or harms between e-cigarette and NRT users. However, we report on results from a limited number of RCTs, and the level of evidence is low. Our efficacy results are similar to those described in a 2016 Cochrane review,⁵ which also showed no difference between abstinence rates between the nicotine e-cigarette group and NRT group. Their review only included one study,¹⁸ also included in our review for this particular outcome. Similar to the evidence we are presenting, none of the studies examined in the Cochrane review reported SAEs considered to be related to e-cigarette use.

Although our meta-analysis of the five trials that examined smoking cessation showed no significant difference between e-cigarette and NRT, there was a trend towards favouring e-cigarettes. Interestingly, our sensitivity analysis limiting inclusion to studies reporting smoking cessation of 6 months or greater yielded a smaller point estimate than the one obtained from the main analysis, although still with no difference between groups. It could be hypothesised that additional benefits that may be attributed to e-cigarette early on in smoking cessation may be attenuated as time progresses. This again should be interpreted with caution given the small number of studies^{4 18 20} and the very significant heterogeneity.

In all comparisons, our results need to be interpreted carefully. There was significant clinical heterogeneity between studies in terms of the population enrolled, smoking intensity at baseline, type and nicotine concentration of e-cigarettes, type and dose of NRT, as well as methodological heterogeneity in terms of study conduct, and intervention and control protocols. For instance, one of the included studies¹⁸ used first-generation e-cigarettes, with nicotine delivery about 20% of that obtained from cigarette smoking. While e-cigarette users were couriered the supplies needed, NRT users had to redeem vouchers from community pharmacies to obtain their patches. The low nicotine content of the e-cigarettes, the extra step in obtaining NRT supplies, and the low intensity of additional co-interventions likely contributed to the low rate of smoking abstinence at 6 months in both groups, limiting the generalisability of the results. Another included study⁴ allowed for multiple types and concentrations of ENDS, as well as upwards of 10 NRT products and doses, complicating the interpretation of the results. Nicotine concentrations reported in the trials ranged from 0.01 to 48 mg/mL,4 18 20–22 making comparisons between studies difficult.

Given that the risk of bias was assessed as high in five of six included studies,^{4 18–21} our smoking cessation outcome results need to be interpreted with caution. In addition, it is interesting to note that all studies verified self-reported smoking cessation with an exhaled CO test, however, different cut-off values were used. Additionally, there are limitations to using CO as a way to verify smoking cessation. CO has a relatively short half-life and is eliminated from the body within 24 hours; it can, therefore, lead to false negative results. However, this issue is somewhat mitigated by the fact that smoking cessation study participants tend to be daily smokers.

All studies included in this review examined smoking reduction. There was no difference between groups in the mean reduction of cigarettes from baseline in the studies that measured that outcome, or in the proportion of participants successfully reducing their smoking consumption.

None of the included studies reported severe adverse events related to ENDS or NRT, and, for the four studies with data that could be pooled, there was no difference between groups in terms of harms related to either therapy. However, in addition to the clinical heterogeneity mentioned above, there was significant methodological heterogeneity in how adverse events were collected. We evaluated the quality of the evidence as very low, given the high risk of bias of included studies, the significant heterogeneity, and the inability to accurately determine the number of subjects involved in this outcome, thus leading to result imprecision.

Since the included trials were powered to detect a difference in the primary outcome, it is possible that rare or unexpected harms were not detected due to a lack of power for this specific outcome. Also, it is important to acknowledge that these studies are limited by their short time frame. Data on long-term side effects of ENDS are lacking. The recent EVALI epidemic is a reminder that further research is needed before widespread recommendations can be made with regard to the use of ENDS. In addition, there are now emerging concerns that respiratory disease caused by the novel coronavirus SARS-CoV-2, the virus responsible for the COVID-19 pandemic, could be exacerbated by exposure to ENDS.^24–26

Finally, although there seemed to be increased acceptance of therapy towards e-cigarettes in the four studies that considered it,4 18 20 22 high risk of bias, significant heterogeneity and the small number of studies using widely different scales leading to imprecise measures, mean that the results should be interpreted with extreme caution. In addition, given that the trials were unblinded, participants who were disappointed with their treatment allocation may have reported less acceptability than their counterparts.

Limitations at review level

We restricted our search to RCTs to try to minimise the risk of bias, however, this considerably limited the number of available studies for this review. It is surprising that, given the widespread availability of e-cigarettes and how aggressively they have been marketed as smoking cessation agents, there are so few head-to-head trials comparing ENDS and traditional NRT. While there may be some unpublished studies that our review did not capture, our literature search was thorough and included personal communications to multiple experts in the field.

Our review identified seven ongoing trials^27–33 that potentially met our inclusion criteria, totalling over 1500 targeted participants. None of the investigators had any data ready to be shared, however it is hoped that this ongoing research can shed light on the effectiveness of ENDS as smoking cessation tools, as compared with traditional NRTs. Long-term research is also needed to investigate the long-term effects of ENDS, as well as the optimal dosing and method of delivery.

Conclusion

We found no difference in smoking cessation, harms and smoking reduction between e-cigarette and NRT users. However, the quality of the evidence was low. Further research is needed before widespread recommendations can be made with regard to the use of ENDS. Research is also needed to investigate the long-term effects of ENDS, as well as optimal dosing.