Need for Examination of Broader Range of Risks When Predicting the Effects of New Tobacco Products

Levy et al.¹ developed a cohort-based model that follows youth and young adults forward in time to assess health impacts of “vaporized nicotine products” (VNP, e-cigarettes and heat-not-burn) over time. Their conclusion is that “under the most plausible scenarios, [e-cigarette] use generally has a positive public health impact. However, very high [e-cigarette] use rates could result in net harms.”

The primary reason that they reached the conclusion of net benefits is that their primary analysis is based on the assumption that e-cigarettes are only 5% as dangerous as conventional cigarettes.² This opinion did not consider the evidence that e-cigarettes have substantial negative effects on the cardiovascular system.³ These cardiovascular effects are important because the dose-response is highly nonlinear, with large effects at low doses. Heart and vascular disease accounts for about half of smoking-induced deaths, more than cancer.⁴

Given this evidence, it is surprising that Levy et al. only considered risks of VNPs up to 25% of cigarettes, a risk that they considered “unlikely.” While their model shows net health benefits at the 25% risk, the results in their Table 1 show these benefits to be small, suggesting that the cross-over point into net harm in their model is around 30%.

Last year we⁵ published a model that also sought to address the population impact of e-cigarettes. The approach we took was to model the steady state situation after the new market is fully developed and stable under a variety of scenarios on future use patterns and a range of e-cigarette risks, ranging from 0% (harmless) to 50% as dangerous as cigarettes (if e-cigarettes had risks for cardiovascular and non-cancer lung disease as large as smoking). The broad conclusion that we drew was that, under the most likely future scenarios, the overall population effect of e-cigarette use would be positive if e-cigarettes are not very dangerous and negative if they are more than about 20%–30% as dangerous as cigarettes.

Thus, if our reading of Levy et al.’s Table 1 is correct, the results of these two different modeling exercises may have led to similar conclusions, which would be an important fact (One would expect the Levy et al. model to produce lower risks, ie, a higher cross-over point, because it does not include the effects of e-cigarettes on established smokers, which is likely to depress quitting.⁶). To be sure, it is important that Levy et al. report the results of their model for risks up to 50%.

Another similarity in the results of our two analyses is what their respective sensitivity analyses show are the important parameters in the model. Levy et al. found that net population health effects from e-cigarettes are especially sensitive to e-cigarette risks and e-cigarette use rates among those likely to smoke cigarettes. We found that the dominant determinants are the ongoing health risks following smoking cessation, followed by the e-cigarette risk and the increase in interest in quitting among smokers.

In addition to the fact that Levy et al. modeled a cohort of youth and young adults going forward in time whereas we modeled steady state for all users (youth and adults), another important difference between the two modeling approaches is that, we structured our model to make (almost) all the transition probabilities directly observable, so that the parameters in the model were based directly on data, Levy et al. did not. As a result, all the probabilities in their model (Figure 1 of their paper and the tables in the online appendix) are their estimates based on considering the evidence in the literature.

The reason for this problem is that the first node in the Levy et al. model (in Figure 1 in their paper) is whether a never smoker would transition to “Would have become a smoker in the absence of [e-cigarettes]” or “Would not have become a smoker in the absence of [e-cigarettes].” As they note in their paper, “The initial branch in Figure 1 leads to hypothetical states which cannot be observed and are inferred from past smoking patterns.” While theoretically reasonable in a model, this means that the key initial node in the model is unobservable, which means that the transition probabilities in all the subsequent nodes, which are conditional on the pathway defined by the first node, are unobservable. That is why Levy et al. could not cite specific data for the precise numbers used in the model, which heavily affect the results. This problem is why there is not a single direct citation to evidence associated with any of the probabilities in their model. The lack of a direct linkage between the numbers in the model and actual observed data can mask biases and errors in the model.

This situation contrasts with the approach that we took, which was to base the transition probabilities in the model as much as possible on directly observable data (See Table 1 in our paper⁵ and note that there are citations to the data that define almost all the parameters in the model.). Indeed, it was this desire that led us to do a steady state model with scenarios rather than the time-based approach that Levy et al. used; we did not think that there were yet data to support all the additional assumptions that are implicit in including time as a variable.

This is more than a small technical detail. Best practices for developing these kinds of models specify basing the transition probabilities based on directly observable data.⁷

Despite these problems, the tentative similarities in the results of the two models are encouraging in terms of the cross-over risk and sensitivity analysis. We request that Levy et al. publish the results for higher risks for VNP products (e-cigarettes) in their response to this letter. It would also be better if future versions of their model was based on observable probabilities and they posted the full model as a supplemental file with the paper so that others can freely explore it.

Funding

SAG’s work reported in this publication was supported by grant 1P50CA180890 from the National Cancer Institute and FDA Center for Tobacco Products (CTP) and does not necessarily represent the official views of the NIH or the Food and Drug Administration.

Declaration of Interests

None declared.