The E-Cigarette Debate: What Counts as Evidence?

Abstract

Two major public health evaluations of e-cigarettes—one from the National Academies of Science, Engineering, and Medicine (NASEM), the other from Public Health England (PHE)—were issued back to back in the winter of 2018. While some have read these analyses as broadly consistent, providing support for the view that e-cigarettes could play a role in smoking harm reduction, in every major respect, they come to very different conclusions about what the evidence suggests in terms of public health policy. How is that possible?

The explanation rests in what the 2 reports see as the central challenge posed by e-cigarettes, which helped to determine what counted as evidence. For NASEM, the core question was how to protect nonsmokers from the potential risks of exposure to nicotine and other contaminants or from the risk of smoking combustible cigarettes through renormalization. A precautionary standard was imperative, making evidence that could speak most conclusively to the question of causality paramount. For PHE, the priority was how to reduce the burdens now borne by current smokers, burdens reflected in measurable patterns of morbidity and mortality. With a focus on immediate harms, PHE turned to evidence that was “relevant and meaningful.”

Thus, competing priorities determined what counted as evidence when it came to the impact of e-cigarettes on current smokers, nonsmoking bystanders, and children and adolescents. A new clinical trial demonstrating the efficacy of e-cigarettes as a cessation tool makes understanding how values and framing shape core questions and conclusive evidence imperative.

In September 2018, the Food and Drug Administration (FDA) launched a $60 million campaign targeted at adolescents who had used or might be tempted to use e-cigarettes. Employing graphic imagery, the campaign depicts hideous worm-like creatures crawling under the skin and into the lungs and brains of otherwise blemish-free adolescents. The ads sound an urgent warning: “There is an epidemic spreading” and “vaping can put dangerous chemicals into your lungs” (Figure 1).¹ Nicotine itself is identified as the ultimate threat. The agency’s “Don’t Get Hacked” campaign evokes Reefer Madness, suggesting that nicotine triggers a kind of wild-eyed mania or personality hacking, in which nicotine transforms adolescents into robots that lack the autonomy or charm of a chatbot (Figure A, available as a supplement to the online version of this article at http://www.ajph.org). The approach was a muscular counter to e-cigarette advertising that blurred the boundary between an addictive product and candy or cereal (Figure 2).²

FIGURE 1—

The Food and Drug Administration’s “An Epidemic Is Spreading” Antivaping Campaign

Source. Food and Drug Administration.¹

FIGURE 2—

Gummy Bear E-Cigarette Advertisement

Source. Stanford University.²

Making this forceful challenge to e-cigarettes remarkable is that, when he took office, FDA Commissioner Scott Gottlieb announced a new approach to tobacco control. A continuum of risk would define FDA policy: products involving lesser harms should edge out deadly combustible products. In making this sharp turn, the FDA was responding not only to a perceived epidemic of youth vaping but also to an evidence review from the National Academies of Science, Engineering, and Medicine (NASEM) that it had commissioned.³

The FDA approach could not stand in sharper contrast with the Public Health England (PHE) strategy. For PHE, which has published its own reviews, the evidence provided additional support for a national policy in which e-cigarettes had become an official part of a campaign to address morbidity and mortality from tobacco smoking. In October 2017, England’s expert national public health agency advised smokers, “stop smoking with an e-cigarette” (Figure 3). PHE has also produced guides on how to switch from smoking to vaping. Although we focused in this analysis on the divide that separates the United States and England, England is in fact a global outlier on the question of e-cigarettes. Australia’s national science research agency, for example, has taken a very different stance and maintains a ban on nicotine sales.⁴ Indeed, some have argued that, at its very origins, funds from the tobacco industry tainted the English conviction that people “smoke for the nicotine but die from the tar.”5,6(p1431)

FIGURE 3—

Public Health England’s 2017 Stoptober Campaign

Underpinning the 2 approaches are very different takes on the evidence. The 2 agencies issued their evaluations of the evidence nearly back to back in the winter of 2018. Some have read these analyses as broadly consistent, providing support for the view that e-cigarettes could play a role in smoking harm reduction. Yet, in every major respect, they come to very different conclusions about what the evidence suggests in terms of public health policy. The differences between the 2 reports turn on the profoundly important question of what should count as evidence for policymakers.

Fundamentally, the 2 reports differed on whose risk was to be given priority. For PHE, the central public health concern was how to protect the health of current smokers. For the United States, the pivotal issue was the protection of children and nonsmokers—innocent bystanders. The formulation of the questions and inclusion and exclusion criteria is always a value-based process. Understanding these different values is critical to mapping the politics of smoking harm reduction as debate intensifies about disruptive high-impact nicotine products like Juul,⁷ which has been at the center of a storm of concern over youths, and heated tobacco products like the Philip Morris IQOS, which some suggest could help to lower toxicity standards in a way that could affect combustible products. The debate is bound to become ever more acrimonious now that Altria, the manufacturer of Marlboro, has acquired a substantial financial interest in Juul.

SYSTEMATIC REVIEW APPROACHES

In February 2018, PHE issued “Evidence Review of E-Cigarettes and Heated Tobacco Products.”⁸ Its aim was to “to summarise evidence to underpin policy and regulation” of e-cigarettes.^8(p25) More importantly, it was the first in a series of annual updates required by the Tobacco Control Plan. It thus represented an ongoing commitment to monitor the emerging evidence of the risks and benefits of e-cigarettes in a nation that was at the forefront of promoting these devices as part of an explicit smoking harm-reduction campaign that prioritized reducing health risks over achieving total abstinence from nicotine.⁸

The report used “systematic review methods” focused on other systematic reviews, analyses of survey data, government reports, and “high profile studies” or studies that provided “new relevant information” or that generated media interest published between January 2015 and August 2017, the period since PHE’s previous systematic review.^{8(p28,29,150)} The report relied on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) standard to report results. It also investigated e-cigarette–related accidents and poisonings. Because it used its earlier published systematic reviews as a starting point, the authors made a strategic decision:

A full systematic review was not possible given the timeframe within which the report was commissioned and needed to be delivered, and the wide scope of the topics covered. However, a full systematic review was carried out for heated tobacco products.^8(p28)

Heated tobacco products were newly introduced into the market and have not been subjected to systematic review of potential health risks and benefits.⁸

While the report did weigh randomized controlled trials, it did not hold them to be the gold standard for its evidentiary review. The demands of randomized controlled trials, the PHE report noted, were “discordant with what happens in real life” and therefore were not generalizable at the policy level.^8(p126) PHE authors argued that new inclusion criteria, such as those proposed by Villanti et al.⁹ and a recent Cochrane review of e-cigarettes,¹⁰ were required. For example, Villanti et al. underscored that no systematic reviews to date have addressed all of the most pressing use issues that contribute to variations in findings. Those include but are not limited to whether studies adequately measured exposure to e-cigarettes (as opposed to use on 1 or 2 occasions), whether e-cigarettes were used with the intention of cessation, and whether e-cigarette exposure actually preceded smoking cessation.⁹

The NASEM report, “Public Health Consequences of E-Cigarettes,” released a month earlier, was quite different in tone and perspective. The Committee on the Health Effects of Electronic Nicotine Delivery Systems approach “incorporated major attributes of systematic reviews.”^11(p43) It included methods established by the Cochrane Collaborative, the Center for Reviews and Dissemination, the US National Toxicology Program’s Office of Health Assessment and Translation, and ROBIS (a new tool for assessing Risk of Bias in Systematic Reviews).¹¹ The official charge was to analyze the research literature, identify the need for research to fill evidentiary gaps, and make judgments about the short- and long-term health effects of e-cigarettes. With a focus on health effects and an emphasis on determining causality, the committee underscored that it “did not treat all bodies of evidence equally, and prioritized human studies.”^11(p43) Considering their strength in determining causality, randomized controlled trials and prospective longitudinal studies provided the most robust evidence. The report took note of population-based ecological data on the changing prevalence of smoking and e-cigarette use over time and noted where it contradicted experimental data or observational data, yet it applied a precautionary standard in which proof of no harm was required.¹¹

With these methodological differences as backdrop, we weighed how each report addressed the central challenges posed by e-cigarettes when it came to smokers, bystanders, and youths. Table 1 summarizes the findings that we discuss in detail in the following sections.

TABLE 1—

Opposing Perspectives on E-Cigarettes for Smokers, Bystanders, and Children

		PHE8	NASEM and Major US Agencies
Smokers	Risk of e-cigarettes compared with combustible products	Recognizes e-cigarettes are not “safe,” but safer	NASEM: Recognizes e-cigarettes are not “safe,” but commits only to endorsing as an alternative if smokers switch completely11
	Promotion of e-cigarette use	Endorses widespread availability of e-cigarettes as smoking harm reduction, ideally combined with behavioral intervention	NASEM: Finds insufficient evidence to promote broad-scale substitution of e-cigarettes for combustible products11
Bystanders	Risk of sidestream exposure to particulates and nicotine	Finds no evidence that second-hand vaping poses identifiable health risks to bystanders	NASEM: States e-cigarettes in indoor environments may involuntarily expose nonusers to nicotine and particulates, but at lower levels compared with combustibles11
		Concludes that harms of nicotine are “minor”	CDC: States “e-cigarette aerosol is not harmless. It can contain harmful and potentially harmful substances including nicotine”12
			Surgeon general: Calls to “prevent involuntary exposure to nicotine and other aerosolized emissions from e-cigarettes”13(p188)
Children	E-cigarettes as a gateway to combustible cigarettes	Despite some experimentation with these devices among never smokers, e-cigarettes are attracting very few young people who have never smoked into regular use	NASEM: Cites substantial evidence that e-cigarette use increases risk of ever using combustible tobacco cigarettes among youths and young adults11
		PHE report author (Bauld) describes the impact on youths as “negligible”14	FDA (Zeller): States “For kids who initiate on e-cigarettes, there is a great chance of intensive use of cigarettes”15

Note. CDC = US Centers for Disease Control and Prevention; FDA = US Food and Drug Administration; NASEM = National Academies of Science, Engineering and Medicine; PHE = Public Health England.

SMOKERS

The point at which the PHE and NASEM reports came closest to agreement centered on the risks of e-cigarettes compared with combustible products. Both reports stressed that e-cigarettes are “safer” but “not safe.” But quantifying the relative risks had been a point of contention for years: are e-cigarettes marginally safer, thus still too risky to substitute for combustible products, or are they substantially safer?

To be considered “conclusive,” the NASEM committee required evidence from “many supportive findings from good quality-controlled studies (including randomized and non-randomized controlled trials) with no credible opposing findings.”^11(p5) It is therefore noteworthy that NASEM concluded, “There is conclusive evidence that completely substituting e-cigarettes for combustible tobacco cigarettes reduces users’ exposure to numerous toxicants and carcinogens present in combustible tobacco cigarettes.”^11(p11)

Unresolved was how aggressively e-cigarettes should be promoted to smokers as safer. Here a sharp divide informed a reading of what the evidence demanded. In a public comment, the NASEM chair stated, “While one might conclude from our report that a smoker who switches to e-cigarettes has reduced his or her risk, there is some uncertainty and the evidence suggests that they must switch completely.”¹⁶ In other words, he and other committee members who discussed the report in public appeared troubled by lingering uncertainty. Despite the report findings, they were reluctant to endorse policy initiatives that might favor the broad-scale substitution of e-cigarettes for combustible products. They indicated little concern about the extent to which public opinion had come to see e-cigarettes as equally or more risky than combustibles.¹⁷

PHE’s position differed starkly: it not only endorsed but also promoted e-cigarettes. While acknowledging that there were some risks and uncertainties and that e-cigarettes could not be called “safe,” PHE has continued to maintain that vaping is “at least 95% less harmful than smoking”^8(p20) (Figure B, available as a supplement to the online version of this article at http://www.ajph.org). This risk assessment, which to some was an unwarranted overstatement based on limited evidence and conflicts of interest,¹⁸ was, to the report’s authors, essential: they aimed to “communicate the large difference in relative risk unambiguously so that more smokers are encouraged to make the switch from smoking to vaping.”^8(p20) A central concern of PHE was the extent to which surveys indicated that misperceptions about the relative safety of e-cigarettes discouraged tobacco smokers from switching. Underscoring the importance of the widespread availability of e-cigarettes, the press release for the PHE report stated, “To become truly smoke free, Trusts should ensure e-cigarettes, alongside nicotine replacement therapies are available for sale in hospital shops.”¹⁴ Martin Dockrell, Head of PHE Tobacco Control, envisioned policy that extended beyond the report recommendations: “We would certainly encourage [hospitals] to make at least some single occupancy rooms where people can vape.”¹⁹ He also called for the creation of shared lounges for vapers in hospitals.

BYSTANDERS

But what of the risks the “safer” but not “safe” e-cigarettes pose to nonsmokers? The imperative to protect “innocent bystanders” became central to tobacco control policy in the United States beginning in the 1970s, even before the evidence was definitive.²⁰ Public health defended bans on smoking in public settings based on an obligation to create healthy environments for nonsmokers. This policy commitment continues to define US tobacco control. Three years before the NASEM report, virtually the entire US public health community urged the Department of Housing and Urban Development (HUD) to ban both combustible and e-cigarettes in all public housing based on the risk to nonsmokers. In a letter to HUD Secretary Julián Castro, nearly 40 US public health organizations argued, “E-cigarette aerosol contains nicotine, which is absorbed by users and bystanders” and “is not as safe as clean air.”²¹ The New York State Public Health Association strongly supported the ban, pointing to “research indicating that harmful chemicals and carcinogens are in the aerosol coming from these devices.”²² Likewise, the 2016 US surgeon general’s report, “E-Cigarette Use Among Youth and Young Adults,” underscored the importance of protecting the public “from both secondhand smoke and secondhand aerosol” to “prevent involuntary exposure to nicotine and other aerosolized emissions from e-cigarettes.”^13(p188)

Against this backdrop, NASEM concluded that because e-cigarettes contained and emitted potentially toxic substances, “using e-cigarettes in indoor environment may involuntarily expose non-users to nicotine and particulates, but at lower levels compared with exposure to secondhand smoke from combustible tobacco cigarettes.”^11(p622) While the NASEM report discussed the general pharmacology of nicotine, it was “not intended to be a systematic review of the topic.”^11(p96) On the heels of the NASEM report, the US Centers for Disease Control and Prevention stated in information for consumers, “e-cigarette aerosol is not harmless. It can contain harmful and potentially harmful substances, including nicotine, heavy metals like lead, volatile organic compounds, and cancer-causing agents.”¹² Most notably, the potential health threat of nicotine itself, which in the past had not been the subject of systematic inquiry, has now emerged as a focus of biomedical research and policy concern.²³

In England, by contrast, health officials pointed out that the health risks of secondhand vape lacked an evidentiary foundation. PHE underscored, “There have been no identifiable health risks of vaping to bystanders.”^8(p162) Although the report underscored that “adolescent nicotine use (separate from smoking) needs more research,”^8(p12) the risk of nicotine exposure was of secondary importance given a longstanding understanding that “people smoke for the nicotine but die from the tars.”^6(p1431) Similar to NASEM, then, PHE also chose not to conduct a systematic review regarding nicotine. Rather, it looked to a Royal College of Physicians report, in addition to any new evidence that might suggest risk, to support the conclusion that the harms of nicotine were “very minor.”^8(p61) Ann McNeill, one of the authors on all of the PHE reports, remained unswayed about the dangers of nicotine even as Juul, which stirred apprehension in the United States based on a nicotine profile that mirrors tobacco cigarettes, was introduced into the UK market. In July 2018, Scientific American quoted McNeill, who said, “We need to de-demonize nicotine.”²⁴

CHILDREN

The 2 reports’ analyses of the threat to children represented yet another yawning gulf about what the evidence revealed. NASEM concluded, “There is substantial evidence that e-cigarette use increases risk of ever using combustible tobacco cigarettes among youth and young adults.”^11(p10) The committee identified 9 studies that met the evidentiary bar.¹¹ For the analysis of the relationship between e-cigarette use and smoking over the past 30 days, only 2 studies qualified.¹¹ While the report did note contradictory data, it determined that observational or ecological evidence could not provide a conclusive refutation of the risk to children. Only randomized controlled trials could meet that bar.¹¹ Conclusive proof, for NASEM, was the standard when it came to vulnerable populations like children.

Most important was how the head of FDA’s Tobacco Division, Mitch Zeller, read the evidence in light of the NASEM Report: “For kids who initiate on e-cigarettes, there is a great chance of intensive use of cigarettes. As the regulator, we’ve got to factor that in.”¹⁵ How that evidence should be factored in was clear to Shannon Lea Watkins, a member of Stanton Glantz’s research team, which had long warned of threats posed by e-cigarettes. “It comes down to this tradeoff between definitely hurting kids and maybe helping some adults,” she said. “To me the tradeoff sounds quite clear.”²⁵ An analysis conducted by Kozlowski and Warner threw into high relief how differently the tradeoff could be perceived.²⁶ Citing evidence from large, cross-national studies, they argued that adoption of e-cigarettes as a smoking harm-reduction tactic “might come at the cost of additional new smokers among the younger generation. While unpleasant to contemplate, this cost must be compared to the far more immediate benefit in terms of health consequences that would be realized by adults quitting smoking.”^26(p213)

For the PHE, the evidence could not be read as providing proof that e-cigarettes were serving as a gateway to tobacco for young people. “Despite some experimentation with these devices among never smokers, e-cigarettes are attracting very few young people who have never smoked into regular use.”^8(p75) Linda Bauld, one of the report’s authors, was unambiguous in calling the impact on youth “negligible.”¹⁴ Indeed, studies suggest that England’s focus on smokers will have an impact on youth uptake: adult smoking represents a risk factor for youth uptake.^27,28

THE POLITICS OF HARM REDUCTION

The public health culture that informs the approach to e-cigarettes and smoking harm reduction make England stand apart. PHE is engaged in continual assessment of a policy path it started down years ago. Its mission has been to act in every way possible to reduce the threat to the health and life of smokers. Their current commitment is not to determine whether to recommend e-cigarettes. They already do. Rather, the issue was whether to retreat from an established policy position because of new evidence. This focus on identifying harms made it imperative for PHE to systematically review heated tobacco products, which were new to the market in 2018. Overall, the focus remains trained on the grave health risks to smokers themselves.

The policy context in which the FDA commissioned the NASEM analysis of the impact of e-cigarettes on health outcomes and the limits of the available evidence favored precaution:

With only few exceptions the epidemiological literature is quite limited and even where it is strongest (assessing short-term cardiovascular and respiratory effects) it does not address the etiology of chronic diseases. In other cases such as cancer and reproductive health, there is simply no available epidemiological research to consider.^11(p46)

Lack of evidence of harm was insufficient for NASEM: they insisted on proof of safety, particularly when the youth uptake was in question.

On the question of efficacy, the research community has yet to reach consensus about what represents an adequate study design to support smoking harm reduction. Nonetheless, new evidence that seems to meet a high evidentiary bar is already widening the divide between England and the United States, underscoring the power of the underlying political framings. In January 2019, the New England Journal of Medicine published the results of a major randomized clinical trial on the efficacy of e-cigarettes.²⁹ It concluded that e-cigarettes were more effective for smoking cessation than nicotine replacement therapy among those who made the decision to attend a stop smoking service and when combined with a behavioral intervention lasting at least 4 weeks. After 1 year, 18.0% of the e-cigarette group was tobacco abstinent compared with only 9.9% of the nicotine replacement group. For e-cigarette users who were not tobacco abstinent, the problem of dual use remained. The study supports the PHE approach of combining easy access to e-cigarettes with behavioral intervention.³⁰

Of the just-released randomized controlled trial, the deputy director of the Tobacco and Alcohol Research Group at London’s University College told CNN, “This study should reassure policymakers and health professionals—mainly beyond the UK—who have until now been hesitant to recommend e-cigarettes for smoking cessation on the basis that there was a lack of high-quality trial evidence.”31 University of California at San Francisco researcher Neil Benowitz described the study to the New York Times as “seminal.”³² Others, however, focused not on the finding that e-cigarettes were twice as effective at helping cessation but that 80% of those in the e-cigarette group continued to vape after 1 year compared with only 9% in the nicotine-replacement group who were still using their assigned product. This raised concerns not only about the long-term health effects but also the risk to adolescents. Data from the Population Assessment of Tobacco and Health study have raised similar concerns for some,³³ while others see in them no evidence of a threat.³⁴ Increasingly, however, concerns are shifting away from the gateway threat and toward the threat that nicotine itself poses. The American Lung Association stated, “Switching to e-cigarettes does not mean quitting. . . . Quitting means truly ending the addiction to nicotine.”³⁵

The fear of “addicting a generation of youth” to nicotine continues to define US skepticism.³⁶ The most recent National Youth Risk Behavior Survey showed that vaping in the past 30 days increased from 0.6% to 4.9% among middle-school students and 1.5% to 20.8% among high-school students between 2011 and 2018. During that period, regulators were most alarmed by a spike between 2017 and 2018: past-30-day e-cigarette use increased 48% among middle schoolers and 78% among high schoolers. FDA Commissioner Gottlieb, describing the situation as an epidemic, said, “These data shock my conscience.”³⁶ In those same school-age groups, however, combustible cigarette smoking declined 4.3% and 15.8% between 2011 and 2017.³⁷ However, according to most recent Centers for Disease Control and Prevention data, “no significant change in frequent use was observed for other tobacco products.”³⁸ A recent 2019 PHE evidence update cites new Action on Smoking and Health data that show similar trends: between 2014 and 2018, the percentage of youths reporting that they had ever tried an e-cigarette increased from 6.5% to 11.7%; during that same period, the percentage of youths who were current vapers increased from 1.6% to 3.4%.³² Youth smoking use and experimentation decreased between 2017 and 2018. As in the United States, media attention focused on sizeable increases in the rate of e-cigarette experimentation.

It is not yet possible to definitively conclude that e-cigarettes divert youths from or drive them to combustible products, but the data “paint a consistent picture of accelerated reductions in youth and young adult smoking prevalence as vaping became more widespread.”^39(p6) Iowa Attorney General Tom Miller, who with 45 of his peers in other states led the lawsuit against the tobacco industry that produced the Master Settlement Agreement, cited a study in the November 2018 issue of Tobacco Control demonstrating that the majority of adolescents aged 15 to 21 years vaped fewer than 10 days a month.⁴⁰ Miller argued that it is imperative to take the threat to kids seriously. Nonetheless, he concluded, “If there is an epidemic, it is an epidemic of casual use. There is no real fear of addicting a generation—2 percent does not make a generation.”⁴¹

Even in the absence of worrisome trends in e-cigarette experimentation among youths, the FDA crackdown on underage sales would be ethically justified. But youths have a right to know the great difference between the risks associated with vaping nicotine and the risks associated with using combustible tobacco even if that information results in an increase in e-cigarette experimentation and use. Although uncertainty remains about how much safer e-cigarettes are when compared with combustible products, by either the NASEM or the PHE evidentiary standard, it is no longer possible to argue that e-cigarettes are as or more harmful than smoking. In its zeal to protect youths, public health leadership must refrain from misrepresenting the conclusive evidence that they are safer.

The position taken by the New England Journal of Medicine indicates just how difficult that will be: in the issue of the journal reporting the results of the clinical trial demonstrating that e-cigarettes are twice as effective at promoting cessation as combinations of nicotine replacement therapy, the editors painted an alarming epidemiological picture.⁴² The editors repeated NASEM conclusions without ambiguity, “There is substantial evidence that e-cigarette use by youth increases the risk of smoking combustible tobacco cigarettes.”^42(p679) In the United States, youths have been the focal point of that picture. With FDA Commissioner Gottlieb’s resignation, it is uncertain what path that agency will now chart, particularly as conservative antiregulatory advocacy groups—some funded by Altria—pressure the current presidential administration to remove regulatory obstacles to e-cigarettes and possibly combustible products. Likewise, whether England can maintain a broad field of vision that includes smokers, who are immediately in harm’s way, remains to be seen. Correcting worsening public health misperceptions that might prevent smokers from switching while regulating e-cigarettes and combustible products out of the hands of kids represents a kind of ethical bottom line. So, too, does monitoring the impact of e-cigarettes on smoking rates as Big Tobacco absorbs more of the vaping industry. But keeping our eyes on the full picture is also imperative: as noted previously, there is a dose–response relationship between adult smoking and youth uptake. A 2017 cohort study concluded, “Adult smoking is a preventable and modifiable risk factor for children smoking.”^43(p13) While trade-offs remain inevitable, smoking harm reduction that benefits current smokers does not simply throw youths under the bus. It is against such standards of candor that we must judge efforts to address the threats posed by smoking and vaping.