Patterns of Use of e-Cigarettes and Their Respiratory Effects: A Critical Umbrella Review

Abstract

E-cigarettes (ECs) are a well-established consumer product. To study their respiratory health effects, there is the issue of heterogenous patterns of use: concurrently with cigarette smoking (dual use), exclusive use after smoking cessation (exclusive use), or use initiated without any prior or current use of cigarettes (naïve use). Our primary goal was to synthesize the evidence on the respiratory effects of ECs use in adults, categorized by their pattern of use. Additionally, we identified the highest quality systematic reviews and critically evaluated the current literature on this topic. The review was developed with published umbrella review guidelines. The database searches were Medline, Scopus, Cochrane, Epistemonikos, LILACS, and grey literature databases. The criterion for inclusion of systematic reviews was analyses of respiratory tests from randomized controlled trials or cohort studies. Quality assessments were performed with AMSTAR2 and a checklist of reporting biases. A narrative analysis was synthesized by test method: spirometry, impulse oscillometry, breath gasses, biomarkers, and clinical serious adverse events. Twelve systematic reviews were included. The findings on respiratory functioning were statistically non-significant across all patterns of use. Reporting bias was frequently observed. Based on the current research, there is no evidence of significant change in the short or medium term in respiratory function with any pattern of ECs use. We attribute the null findings to the weaknesses of acute studies, the participants’ smoking history masking testing, and the inclusion of participants with a low frequency of use.

Introduction

E-cigarettes (ECs), also known as vapes, were introduced to the international market in 2007. ¹ Since their introduction, they have gained popularity among people who smoke as a potential tool for harm reduction and smoking cessation,^2,3 appreciated for their cost-effectiveness ⁴ and ability to replicate the smoking experience without combustion or smoke production.^5,6 In 2023 worldwide sales had topped $23 billion (US dollars). ⁷ ECs, now a well-established consumer product, appear to offer a less harmful alternative to cigarettes. ECs emit significantly lower levels of toxins than cigarette smoke ⁸ and eliminate many altogether.^9-11 While the use of ECs, commonly referred as to vaping, can generate certain unique toxicants, emission studies of newer vaping products have shown reductions in chemical emissions, particularly carbonyl compounds. ¹² According to toxicological assessments, analyses show that the cancer potencies of ECs are less than 1% of those associated with tobacco cigarettes, ¹³ and the estimated lung cancer risk from vaping is approximately 5 orders of magnitude lower than that from smoking. ¹⁴

Based on the evidence of the substantially reduced exposure to toxicants in ECs aerosol emissions compared to cigarette smoke, ECs have been proposed as a potential tobacco harm reduction strategy.^3,15-17 To evaluate ECs for this purpose, robust evidence on the health effects of ECs use is essential. Indeed, although e-cigarettes contain fewer toxicants than conventional tobacco, they are not risk-free. This umbrella review focuses specifically on the impact of ECs on respiratory function.

Research on respiratory effects of ECs is challenging due to their varied pattern of use compared to the more uniform cigarette use, which averages 9-13 cigarettes daily worldwide. ¹⁸ ECs use may occur alongside smoking (dual use), exclusive use after smoking cessation (exclusive use), or as an initial nicotine experience (naïve use).

Each usage pattern impacts respiratory health differently and presents specific challenges. With dual use, sustained health risks depend on the level of dual usage, with health impacts varying based on the frequency of use of cigarettes because continued exposure to tobacco smoke, even from as few as 1 or 2 cigarettes per day, still poses significant risks for tobacco-related diseases.^19-21 With exclusive use previous smoking may already impaired respiratory health despite the toxin reduction obtained from switching to EC.^20,22

A limited number of surveys offer prevalence data on the patterns of use. The International Tobacco Control Survey (Canada, US, England, and Australia) in 2018 reported dual use at 57.2%, ²³ the 2019 Canadian Tobacco and Nicotine Survey prevalence was 38.3%, ²⁴ and in the US 2019 Current Population Survey, 39.1%. ²⁵ Surveys reporting EC use by persons who have never smoked are difficult to interpret because EC use was defined as any use in the past month. The prevalence rates of 36.7% in Canada ²⁴ and 23.1% in the US ²⁵ likely counted many respondents who use EC infrequently.

Research Question

The primary goal of our umbrella review is to synthesize all the available evidence on the respiratory effects of EC use in adults, categorized by their usage patterns: dual use of EC concurrently with cigarette smoking, exclusive use after quitting cigarettes, and naïve use by adults who have never smoked. We aim to determine whether each of these usage patterns leads to benefits, no changes, or negative effects on respiratory function. Additionally, we seek to identify the highest quality systematic reviews by critically evaluating their methodology and reporting.

Methods

This umbrella review was developed with the Methods for Overviews of Reviews (MOoR) framework^26,27 and the PRIOR statement (Preferred Reporting Items for Overviews of Reviews). ²⁸ The protocol was registered with PROSPERO CRD42024540034 and published in a peer reviewed journal. ²⁹

Population, Intervention, Comparator, and Outcomes Criteria

The scope of our umbrella review is defined by the following PICO (Population, Intervention, Comparator, Outcome) criteria:

Population: Adults aged 18 years and older, including those who currently smoke cigarettes, those who have quit smoking, and those who have never smoked more than 100 cigarettes in their lifetime.

Intervention: EC use.

Comparator: Changes within the same individuals over time; a control group or arm (including those receiving a placebo); or a longitudinal cohort.

Outcome: Any alteration in respiratory function, whether self-reported or clinically measured. Both respiratory symptoms and the results of respiratory function tests were included. The detailed list of outcomes included is provided in the protocol. ²⁹

Search strategy and study selection

A full description of the search and selection processes was provided in the published protocol. ²⁹ A brief description follows. The search databases were Medline (via PubMed), Scopus (Elsevier), Cochrane Database of Systematic Reviews, Epistemonikos, LILACS and grey literature databases were MedNar, National Technical Information Service (NTIS), and WorldWideScience.org.

The database searches were conducted between April 3 and 8, 2024. There were no restrictions on language. The timeframe for the publication dates is January 1, 2021 to the date the search was conducted as per protocol. ²⁹ Systematic reviews with an advance publication date before 2021 were excluded. This time frame was chosen because only the current systematic reviews will have included primary studies conducted with the newest devices. Newer EC devices have substantially changed in their designs, and models sold before 2017 are no longer on the market.^30,31 Nicotine delivery is higher in the newer models than the earlier ones. ³² Findings from the latest primary studies are critical for evidence on the EC models in actual use Therefore we excluded older systematic reviews so that our analyses would not be over weighted by studies with discontinued ECs.

The search terms “electronic nicotine” OR “e-cigarette” OR “vaping” were combined with the respiratory keywords “respiratory,” “lungs,” and “pulmonary.” The researcher determined the keywords, fields, and filters specific to each database and constructed the search accordingly (see Supplementary Materials S1).

The retrieved records were downloaded into EndNote X9 (Thomson Reuters, New York, NY, USA).

Two researchers independently screened the records for all PICO criteria in the title and abstract; publications lacking any PICO criterion were excluded. Inter-rater agreement was 100%.

The inclusion criteria were:

• • Searches conducted in at least 2 databases.

• • Analyses based on randomized clinical trials (RCTs) or prospective cohort studies.

• • Published in peer reviewed journals, government publications, or medical organization reports.

• • Reporting data by the pattern of use, dual use, exclusive use, or naïve use, on changes in respiratory disease symptoms, clinical test measurements, or self-reported health.

Systematic reviews were excluded if all their analyses were combined with data from cross-sectional, survey, or case studies, or if the review had no analyses (ie, solely a narrative description of individual primary studies).

Two researchers independently evaluated the systematic reviews for inclusion, achieving an inter-rater agreement of 95%, with 1 discrepancy resolved by the Project Leader.

After completing the database searches, we conducted a second round of secondary searches. One involved a citation chase (snowball search) in Google Scholar, and the other was a review of the references in the included systematic reviews. These 2 searches were performed independently by 2 reviewers. Publications from the secondary searches were included or excluded based on the full paper criteria. Finally, 2 experts in respiratory diseases reviewed and confirmed the list of included systematic reviews.

Quality assessment and data extraction

To ensure efficiency, data extraction and quality assessments of a systematic review (including its supplementary materials and protocol) were conducted concurrently. The specific items for data extraction including bibliographic information and study details, were chosen by the research team and pilot tested on 2 systematic reviews. All items for data extraction are available in the protocol. ²⁹ For systematic reviews that included multiple study designs, data were extracted solely from the analyses of RCTs and cohort studies. One researcher performed the data extraction, and it was 100% cross-checked by a second reviewer.

We used the AMSTAR2 checklist (A MeaSurement Tool to Assess Systematic Reviews) ³³ to assess the methodological quality of the systematic reviews and to identify the most rigorous reviews. Two reviewers independently completed the checklist, and there were no unresolved discrepancies. For the overall confidence rating, the Project Leader scored its confidence rating as high, moderate, low, or critically low (see Deviations from Protocol below).

A second quality assessment examined reporting issues in 3 areas: unreported deviations from protocol, textual reporting biases (checklist drawn from the Oxford Catalogue of Bias,^29,34 see Supplementary Material), and internal data reporting discrepancies. ³⁵

Analyses

As anticipated in the protocol, a statistical meta-analysis was deemed unfeasible due to the heterogeneity in the participants, EC devices, study duration, and test selection of the systematic reviews. To mitigate bias in our data synthesis, the systematic reviews rated as AMSTAR2 moderate or high confidence were selected for the data analyses. The results were synthesized with a narrative summary by measurement method: spirometry, impulse oscillometry, breath gases, and clinical serious adverse respiratory events.

To confirm the potential accuracy of our narrative synthesis, we assessed the overlap of primary studies across the systematic reviews with the corrected coverage area (CCA) calculation. ³⁶ The purpose of the CCA is to identify if the over-representation of specific primary studies might skew the results of our umbrella review.

One sensitivity analysis was conducted, comparing the findings of analyses with those from the systematic reviews rated at low confidence.

Deviations from protocol

In the protocol, we included the International Prospective Register of Systematic Reviews (PROSPERO) among the databases to screen. However, in practice, we only checked PROSPERO for any ongoing systematic reviews on the respiratory effects of EC. In addition, we added LILACS to the databases screened.

We made two modifications to the AMSTAR2 scoring. One modification was for Question 4, literature search strategy, that requires the systematic review authors justify publication language restrictions. Instead of having this one item be a criterion for a “no” on Question 4, we decided to evaluate as “partial yes” those reviews in which language restrictions were not justified but that fulfilled all other criteria. In our experience, almost all systematic reviews state (or assume) without any further comment that the search was limited to publications in English. We believe that the absence of a language justification is not sufficient to rate the entire search strategy as a critical flaw in the conduct of the review.

In our second modification of AMSTAR2, we changed the scoring of Question 7, a list of excluded studies, from a critical flaw to a non-critical weakness. We made this modification because such a list is not required by PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). ³⁷

Three planned analyses were not conducted. First and most important, we could not perform the Vote Counting of Direction of Effect because none of the results reported in the systematic reviews showed a significant change in comparisons or outcomes. With no differences between the findings of the reviews, an analysis of publication bias was a moot point. Lastly, a sensitivity analysis comparing the conclusions of reviews funded by commercial entities to reviews with other funders was not necessary because no industry or commercial reviews were retrieved.

Results

Search and selection results summary

The search and selection processes are summarized in the PRIOR flow diagram (Figure 1). The systematic reviews excluded during the full paper review are listed with the reasons for their exclusion, in Supplementary Materials S2. A total of 13 systematic reviews were identified.^2,22,38-48 One of the systematic reviews retrieved was our prior publication. ⁴⁸ To preclude possible bias, it was segregated and referenced in the Discussion.

Figure 1.

Flow chart of the study selection process.

The final selection of systematic reviews was 12. The general characteristics of these reviews, including the funders, number of primary studies, study designs, outcomes, and AMSTAR2 ratings, are summarized in Table 1 and Supplementary Materials S3.

Table 1.

Key Characteristics of the Systematic Reviews.

First Author Year	Funder	Analysis Method	Research Topic	AMSTAR2 rating
Alqahtani 2023	Not funded	Meta-analysis	Prevalence of clinical respiratory symptoms	Critically low
Anandan 2023	Not reported	Meta-analysis	Clinical respiratory symptoms EC vs NRT	Critically low
Honeycutt 2022	Canadian Institutes for Health Research	Narrative	EC and lung function	Moderate
Larue 2021	Canadian Institutes for Health Research, Fonds de Recherche du Québec, Canadian Cancer Society	Narrative	Acute physiological effects	Low
Levett 2023	Not funded	Meta-analysis	EC as cessation treatment	Moderate
Lindson 2024	U. of London; U. of Auckland, Oxford U; National Institute for Health Research, UK; Cancer Research UK	Meta-analysis	EC as cessation treatment	High
McNeill 2022	Public Health England, dept. of Health and Social Care	Narrative	EC perceptions and prevalence in england, and health risks	Moderate
Novelli 2022	Not reported	Narrative	Human and in vitro pulmonary effects	Critically low
Pisinger 2022	Not funded	Narrative	Health effects dual use vs exclusive use	Low
Quigley 2020	Health research Board, Ireland	Narrative	EC as cessation treatment	Low
Song 2022	Liaoning Social science planning fund	Meta-analysis	Lung function < 1 month EC use in healthy and asthmatic participants who smoke and participants not smoking	Critically low
Wasfi 2022	Not reported	Narrative	Chronic health effects	Moderate

The AMSTAR2 evaluation is reported in Supplementary Materials S4. The methodological quality ratings of the systematic reviews were 5 critically low, “should not be relied on”;^{38,39,41,44,47} 2 low, “may not provide an accurate and comprehensive summary”^43,45,46; 4 moderate “may provide an accurate summary”^22,40,42,43; and 1 high confidence, “an accurate and comprehensive summary of the results of the available studies”. ²

The 5 systematic reviews with moderate or high AMSTAR2 ratings were selected for the synthesis. They included a total of 41 primary studies^49-89 (see Table 2). They were published between 2013 and 2023, with 34 of 41 studies published 2018 or later, achieving our goal of including studies with the most recent EC models. The respiratory outcome measurements were spirometry, impulse oscillometry, breath gases, and clinical serious adverse respiratory events (see Table 3).

Table 2.

Citation Index.

Primary Study	Honeycutt	Levett	Lindson	McNeill	Wasfi
Antoniewicz 2019				X
Barna 2019				X
Begh 2021			X
Boulay 2017	X
Brozek 2019				X
Bullen 2013		X	X
Carpenter 2017			X
Chaumont 2019				X
Chaumont 2020				X
Coppeta 2018	X			X
Edmiston 2022			X
Eisenberg 2020		X	X
ESTxENDS trial (Auer 2024)			X
Ferrari 2015	X
George 2019			X
Hajek 2019		X	X
Hatsukami 2020			X
Hickling 2019				X
Holliday 2019			X
Kerr 2019				X
Kerr 2020			X
Kotoulas 2020				X
Lappas 2018				X
Lee 2018			X
Lucchiari 2020		X
Mcclelland 2021				X
Meo 2019				X
Myers Smith 2022			X
Palamidas 2017	X
Palamidas 2013	X
Polosa 2017	X			X	X
Polosa 2018				X	X
Polosa 2020					X
Pratt 2022			X
Puliyakote 2021	X			X
Pulvers 2020			X		X
Ruther 2021				X
Staudt 2018	X			X
Wagener 2023			X
Walele 2018			X	X
Walker 2020		X

Table 3.

Systematic Reviews in the Analyses.

First Author Year	Primary Studies	Last Search Date	Measurements
Honeycutt 2022	8	07/12/2021	Spirometry: FEF25-75, FEV1, FEV1/FVC%, FVC
			Impulse oscillometry: Airway resistance, respiratory resistance, respiratory flow resistance, specific airway conductance
			Breath gases: oxygen saturation
Levett 2023	5	01/9/2022	Respiratory serious adverse events
Lindson 2024	17	07/1/2023	Spirometry: FEF25-75, FEV1, FEV1/FVC%, FVC, PEF, PEF25-75
			Breath gases: oxygen saturation, FeNO
			Serious adverse events
McNeill 2022	18	07/1/2021	Spirometry: FEV1, FEV1/FVC%, FVC, PEF
			Impulse oscillometry: Impedance, resistance, reactance, reactance area
			Breath gases: FeNO
Wasfi 2022	4	01/29/2021	Spirometry: FEF25-75, FEV1, FEV1/FVC%, FVC

FEF – forced expiratory flow (interval in volume).

FEV – forced expiratory volume (in 1 second).

FVC – forced vital capacity.

PEF – peak expiratory flow (interval in volume).

FeNO - fractional exhaled nitric oxide.

Analyses

Synthesis of findings

The overlap of primary studies between the systematic reviews was classified as slight at 4.8% (Corrected Coverage Area calculation presented in Supplementary Materials S5). Therefore we could proceed with the synthesis without any study over-weighted in the synthesis.

Spirometry

Four systematic reviews examined spirometry tests: forced expiratory flow and peak expiratory flow. One review found no changes with naïve use, ⁴⁰ 3 reviews observed no significant changes with exclusive use,^2,22,43 and 1 review showed no differences with dual use. ⁴³

Impulse oscillometry (IO)

Two systematic reviews looked at IO testing. Honeycutt examined IO measurements in 2 small-n studies, both pre-2017, indicating a decrease in respiratory functioning with acute EC use in participants who smoked and in participants who did not smoke. ⁴⁰ The authors stated that they found no acute changes in lung function associated with vaping. McNeill discussed only 1 acute study, with no significant changes measured with dual use, and the authors stressed the need for more IO studies. ⁴³

Breath Gases

Three systematic reviews examined measurements of oxygen saturation or fractional exhaled nitric oxide (FeNO). Oxygen saturation tests showed no statistically different change with naïve use or dual use, ⁴⁰ and no clinically relevant changes with exclusive use. ² McNeill reported that almost all FeNO measurements showed no statistically significant differences. ⁴³ Lindson provided only 1 study on FeNO relevant to exclusive use, and it too had no significant change. ²

Serious adverse respiratory events

The systematic review by Levett calculated a meta-analysis from 3 studies of exclusive use and found that occurrences of “serious respiratory adverse events at maximum follow-up” were not significant. The authors labeled the analysis as “inconclusive” due to the lack of data. ⁴² The Cochrane review stated that with exclusive use, incidents of serious adverse events were low. ²

Analysis - Reporting Issues

Reporting issues were analyzed in the 12 systematic reviews. Table 4 displays a summary of the unreported deviations from protocol, data discrepancies, and narrative reporting biases. The incidences of data discrepancies and observations of reporting biases are reported in Supplementary Materials S6. At least 1 issue of reporting bias was observed in all 12 reviews.

Table 4.

Reporting Issues in all Systematic Reviews.

Review	Unreported Deviations from Protocol	Data Discrepancies	Reporting Biases
Alqahtani 2023	Databases searched quality assessment tools planned analysis primary outcome comparison groups	4	Missing supplementary materials
			One-sided references
			Outcome framing
			Overreliance on P
Anandan 2023	Population, comparators quality assessment tool data synthesis Search strategy	1	Omitted finding
			Overreliance on P
Honeycutt 2022	Review question inclusion and exclusion criteria primary outcomes analysis strategy subgroup analyses included study designs	None	3 omitted findings
			One-sided reference
			Overreliance on P
			Outcome framing
Laure 2021	study designs comparators quality assessment tool data synthesis	6	3 omitted findings
			One-sided references
			Outcome framing
			Overreliance on P
Levett 2023	study design secondary searches Outcomes	None	Omitted finding
			One-sided references
Lindson 2024	None	4	None
McNeill 2022	None	1	None
Novelli 2022	Protocol not published	1	2 omitted findings
			Over-reliance on P
			Outcome framing
			One-sided references
Pisinger 2022	Protocol not specified or published	3	Spin bias
			Omitted finding
Quigley 2020	Protocol not specified or published	3	None
Song 2022	Protocol not specified or published	11	3 omitted findings
			Outcome framing
Wasfi 2022	Definitions of EC use data extraction analysis strategy heterogeneity assessment	6	Spin bias
			Outcome framing
			One-sided references

Unreported deviations from protocol were a significant issue, observed in 6 reviews. Four reviews had no protocol specified or published and thereby were not transparent in the conduct of their reviews.

Data discrepancies were noted in ten systematic reviews, ranging from 1 to as many as 11 errors. The discrepancies are detailed in Supplementary materials.

Narrative reporting biases were prevalent. One-sided reference bias was identified in half of the reviews, and 6 of the reviews engaged in outcome framing. Overreliance on P-values rather than clinical relevance was also a common issue, noted in 5 systematic reviews. All the low and critically low rated reviews omitted a discussion of 1 or more of their findings, but only 1 of the higher rated reviews committed this reporting error. Observations of reporting bias are listed in Supplementary Materials.

Discussion

Summary of Findings

The findings of the 5 high and moderate quality systematic reviews displayed a remarkable consensus: that regardless of the pattern of use, EC use did not result in a statistically significant or clinically relevant change in respiratory function. These outcomes were based on 41 RCT or cohort primary studies, and covered a wide range of spirometry, impulse oscillometry, breath gases, and reports of serious adverse events in EC cessation trials. In considering the data in total, no clear differences in respiratory outcomes can be observed between the 3 patterns of EC use: dual use, exclusive use, or naïve use. The 5 systematic reviews in our analysis concluded that there is no definitive evidence of harm or benefit regardless of the pattern of EC use.

Discussion of Null Finding

We attribute the null findings to 2 major factors: the duration of the primary studies and heterogeneity in the patterns of EC use, particularly for dual use.

First and foremost, the majority of the primary studies were conducted with acute tests. Data from acute exposures to EC have limited value for evaluating longer-term respiratory functioning, as both Honeycutt and McNeill discussed.^40,43 Honeycutt characterized acute studies as “suggestive but not definitive”. ⁴⁰

For the longer-term follow-up studies, the majority of them were not of sufficient duration to observe changes or improvements with exclusive EC use, making it difficult to separate the effects of prior smoking from the effects of EC use. ²² This is a pragmatic concern for research as prior smoking history affects respiratory testing. The FEV₁ spirometry test requires at least 1 year of cigarette abstinence to show improvements.^90,91 To provide valid evidence, test data on the effects of EC use must account for recovery periods from smoking. ⁹² Study duration likely accounted for the non-significant findings in the systematic reviews in our analysis, with data from acute studies inconclusive and tests in long-term studies showing no changes due to the participants’ smoking histories.

The second potential contributor to our null finding is the heterogenous levels of EC exposure arising from the broad definition of EC use. The classification of current EC use is, almost without exception, defined as any past month use. This results in participants with substantially different frequencies of EC use being grouped together. For example, in the Canadian survey referenced earlier, ²⁴ 37.5% of those using EC were vaping less than once a week. Frequencies of EC use vary substantially in patterns of dual use. The most frequent pattern was daily smoking and non-daily vaping, ranging from 46.0% ⁹³ to 68.4%. ⁹⁴ In the US Population Assessment on Smoking and Health (PATH) 2019, 25.5% of participants classified as dual users were neither vaping nor smoking daily. ⁹³ The primary studies combining all these frequencies of EC use as dual use are “probably contributing to confusion about important issues”. ⁹⁴ It is likely that our analyses of dual use are confounded by participants with only a minimal use of EC.

The Methodological and Reporting Quality of the Literature

Turning to our critical evaluation of the systematic reviews, the state of the literature on our topic is poor. In the AMSTAR2 methodological evaluation, only 5 of 12 reviews were rated with high or moderate confidence in their findings. Reporting issues were all too common. Unreported deviations from protocol and the lack of a published protocol are at least an omission of concern and at worst a yellow flag for possible data manipulation. P-values took the place of a fulsome discussion of the clinical relevance of the findings. Last, but not least, data discrepancies in the reporting in all but 2 of the reviews reflected badly on the quality of the research.

Comparisons with Previous Literature and our Prior Systematic Review

Banks et al ⁹⁵ conducted an umbrella review on the health effects of EC based on 8 government publications published since 2018: 2 from the US,^96,97 3 from England,^98-100 and 1 each from Australia, ¹⁰¹ Ireland, ¹⁰² and the European Union. ¹⁰³ Thirty-three studies were surveillance studies or case reports on the June 2019 – February 2020 US EVALI (e-cigarette or vaping product use associated lung injury) event caused by black market tetrahydrocannabinol (THC) cartridges.^104,105 In contrast, only 11 studies were on respiratory functioning or diseases, 9 of which are included in our citation index.^{49,53,57,58,68,70,71,79,86} The government publications did not retrieve 19 relevant RCTs and cohort studies that were identified in the 5 systematic reviews in our analyses. Banks et al classified the evidence on respiratory resistance and impedance after acute exposure as limited, and the evidence for spirometry tests after acute exposure and breath gases were both labeled as insufficient. Overall, Banks et al reported that there was insufficient evidence to reach a conclusion on lung function outcomes in people who smoked and switched to exclusive EC use.

Travis 2022 umbrella review ¹⁰⁶ conducted a systematic search for systematic reviews on EC and health outcomes that included a narrative summary of acute respiratory effects, effects on COPD and asthma, in-vivo animal studies and invitro cell studies. They included 13 systematic reviews on respiratory studies, 5 of which were published before 2018. Five reviews on asthma or DNA markers were not relevant to our research topic. The review did not provide a citation index. They found that switching from smoking to vaping “showed a potential reduction in cardiovascular and pulmonary/respiratory harm.” They also suggested that dual use could be an “independent risk factor,” but this is not consistent with our findings. Interestingly, their Conclusions section solely critiqued the research literature but offered no final conclusions on EC and health. They addressed many of the same issues we have raised: the need for systematic reviews to adhere to “established reporting guidelines” such as PRISMA ³⁷ ; to develop and register a protocol; to analyze studies with newer EC; and to capture the effect modifications from participant smoking history.

Members of our review team were co-authors of a systematic review on the respiratory effects of EC substitution for smoking, exclusive use. ⁴⁸ Our results were in accord with the systematic reviews analyzed here. From a search date up to May 18, 2022, our analysis encompassed 66 respiratory test measurements (spirometry, impulse oscillometry, and lung capacity) from 16 studies (7 acute). We found that 65% of the tests had statistically non-significant results, and the tests showing statistically significant improvements or declines were not clinically relevant changes. Similar to the high frequency of instances of reporting bias that we observed in this umbrella review, in our systematic review we detected reporting errors and biases in 13 of 16 primary studies.

A recent systematic review ¹⁰⁷ by Glantz et al (2024) explored the respiratory health effects of exclusive e-cigarette use and dual use, highlighting differences in the risks associated with these behaviors. The review indicated improvements in asthma and COPD symptoms for current e-cigarette use, but an increased risk for dual use compared to cigarette smoking. However, this review was excluded from our analysis because their meta-analyses combined cross-sectional and longitudinal studies, with the majority of studies cross-sectional, which limits the ability to establish causal relationships. Additionally, most of the studies did not account for the duration and intensity of e-cigarette or cigarette use. These limitations underscore the need for further high-quality longitudinal studies to better understand the effects of exclusive and dual e-cigarette use on respiratory health, consistent with the findings of our own review.

Limitations

One of the major limitations discussed by the systematic review authors was the lack of relevant studies. All of the 5 review teams included in our data synthesis bemoaned the lack of data. McNeill stated, “we concluded that there was insufficient evidence from spirometry, FeNO, impulse oscillometer, and bronchoscopy and imaging studies as to whether vaping has any impact on lung function after acute, short to medium and long-term exposure. ⁴³ ” Levett et al. ⁴² described their results as inconclusive due to limited data. Lindson et al. ² noted a lack of robust evidence, with only 1 or 2 studies providing data to many of their analyses. While noting the need for more studies is routine, long-term follow-up respiratory studies on EC are truly lacking.

In the primary studies, some of the testing methods had limitations over and above the duration of follow-up. For example, the FEV₁ test shows improvements in individuals who quit smoking before the age of 30, but those who quit after the age of 40 may still experience a decline in FEV₁ giving results similar to individuals who continue smoking. ⁹⁰ Sometimes improvements in spirometry tests can be explained by the participants’ familiarity with the testing procedure and not from an increase in lung function. ¹⁰⁸ Impulse oscillometry is a newer diagnostic test, and more research is needed to prove its clinical value. ⁴³

The greatest limitation for our research question and umbrella review was that data on specific patterns of use was scarce. Some systematic reviews did not provide a clear differentiation in the data between exclusive and dual use of EC. Little data was available on naive use. In addition, the conclusions of the reviews were informed by cross-sectional studies as well as RCTs and cohort studies. Another limitation for our review was the low quality of the reporting of the systematic reviews that prevented us from getting a clear picture of their findings. By conforming to PRIOR guidelines and with the guidance of the respiratory specialist on our team, we have presented as strong a conclusion as the data has warranted, but perforce our conclusions are limited by the lack of studies and data specific to the patterns of use.

Finally, another major challenge identified in this review is the difficulty in determining a consistent and objective variable to measure the respiratory health effects of exclusive e-cigarette use vs dual use. Variables such as lung function (FEV1), oscillometry, FeNO, and respiratory symptoms have been used inconsistently across studies. While oscillometry provides a more sensitive measure of lung function, the number of studies employing this technique remain limited. Moreover, the scarcity of studies that focus specifically on exclusive e-cigarette users and dual users further complicates the ability to draw definitive conclusions. These limitations emphasize the need for high-quality, long-term studies that employ objective measures of respiratory health to better understand the impact of e-cigarette use in these populations. As such, the current review highlights not only the gaps in the literature but also the necessity for further research, particularly studies that reflect longer exposure periods to e-cigarettes and dual use.

Future Research

Based on our experience and the current review, we highly recommend that future systematic reviews and umbrella reviews conduct an examination of reporting bias of their studies. They should also keep a warry eye out for biases and data discrepancies in their own articles.

A more detailed measurement of the patterns of use is a must for future study designs to determine the effects of the frequency of EC use on health outcomes. Once a month use does not constitute a prolonged exposure to vaping.

As previously discussed, the main shortcomings of current studies on e-cigarettes include short follow-up periods, variations in study design, and a lack of standardized disease parameters. To fully understand the long-term effects of e-cigarette use on the respiratory system, long-term studies are essential. Instead of initiating entirely new long-term projects, researchers could attempt to reconnect with past participants for follow-up data on their e-cigarette use and health status. Additionally, longitudinal cohort studies using medical records could provide valuable insights. Given the variability in disease parameters discussed in the present review, future research would benefit from adopting standardized indicators to assess the progression of negative effects, thereby enhancing comparability and improving the reliability of findings.

Conclusion

Five systematic reviews rated at AMSTAR2 high and moderate confidence found no significant change in respiratory function with EC use, either with dual use with continued cigarette use, exclusive use after quitting smoking, or naïve use by consumers who have never smoked. The null findings can be attributed to the weaknesses of acute studies, by the smoking history of the participants masking potential changes in test results, and by the inclusion of participants with very little exposure to EC due to a low frequency of vaping. Our assessment of the methodological and reporting quality of 12 systematic reviews found that the majority were subpar, further reducing our confidence in the literature. For accuracy, future studies must conduct their analyses on the specific patterns and frequency of EC use. Choices of testing methods should be carefully evaluated for their appropriateness and accuracy. Based on the current research, there is insufficient evidence of any significant change in the short or medium term in respiratory function with any pattern of EC use.

ORCID iD

Giusy Rita Maria La Rosa https://orcid.org/0000-0001-5127-5299