Abstract
Electronic cigarettes (ECIGs) are a relatively new class of tobacco products and a subject of much debate for scientists and policymakers worldwide. Objective data that address the ECIG risk/benefit ratio for individual and public health are needed, and addressing this need requires a multidisciplinary approach that spans several areas of psychology as well as chemistry, toxicant inhalation, and physiology. This multidisciplinary approach would benefit from methods that are reliable, valid, and swift. For this reason, we formed a multidisciplinary team to develop methods that could answer questions about ECIGs and other potential modified risk tobacco products. Our team includes scientists with expertise in psychology (clinical, community, and experimental) and other disciplines including aerosol research, analytical chemistry, biostatistics, engineering, internal medicine, and public health. The psychologists on our team keep other members focused on factors that influence individual behavior, and other team members keep the psychologists aware of other issues, such as product design. Critically, all team members are willing to extend their interests beyond the boundaries of their discipline to collaborate effectively with the shared goal of producing the rigorous science needed to inform empirically-based tobacco policy. In addition, our trainees gain valuable knowledge from these collaborations and learn that other disciplines are accessible, exciting, and enhance their own research. Multidisciplinary work presents challenges: learning other scientists’ languages and staying focused on our core mission. Overall, our multidisciplinary team has led to several major findings that inform the scientific, regulatory, and public health communities about ECIGs and their effects.
Keywords: electronic cigarettes, multidisciplinary, team
Introduction
Electronic cigarettes (ECIGs) are a relatively new class of tobacco products that have become a subject of much debate for scientists and policymakers worldwide (McNeill, Brose, Calder, Bauld, & Robson, 2018; National Academies of Sciences, Engineering, and Medicine, [NASEM] 2018; US Department of Health and Human Services [USDHHS], 2016). Generally, ECIGs use a battery-powered heating element to aerosolize for user inhalation a liquid that usually contains propylene glycol, vegetable glycerin, flavorants, and the dependence-producing, stimulant drug nicotine (Breland et al., 2017). Because ECIGs do not involve burning tobacco, ECIG proponents tout the public health benefits of ECIG-assisted smoking cessation and reduced exposure to toxicants commonly found in cigarette smoke (e.g., Hajek, 2014). For similar reasons, ECIG opponents highlight ECIG-associated smoking initiation (especially among youth), as well as users’ exposure to toxicants that are sometimes the same and sometimes different from the toxicants found in cigarette smoke (e.g., Pisinger, 2014). Given the proven history of dependence, disease, disability, and death associated with combustible cigarette smoking (US Department of Health and Human Services, 2014), the possibility of ECIGs facilitating elimination of combustible tobacco use represents a tremendous public health opportunity. Likewise, the possibility that ECIGs will contribute to a new generation of cigarette smokers represents a tremendous public health threat. Clearly, there is a need for objective data that address the ECIG risk/benefit ratio for individual and public health.
Given that ECIG use is increasingly prevalent in the U.S. and elsewhere, especially among youth and young adults (NASEM, 2018; USDHHS, 2016), addressing this need for data requires a multidisciplinary approach that spans chemistry, toxicant inhalation, physiology, and human behavior to evaluate ECIG effects on individual and public health. This multidisciplinary approach would benefit from methods that are reliable, valid, and swift. For this reason, we formed a multidisciplinary team to develop methods that could answer questions about ECIGs and other potential modified risk tobacco products (MRTPs). The multidisciplinary approach that we have used over the past several years is intended to answer an array of questions such as: “What is the product’s toxicant emission profile and what factors influence that profile?”; “How much nicotine does the product deliver to the user and with what effects?”; “Does long-term product use by smokers influence their exposure to harmful toxicants and reduce concurrent cigarette smoking?”; and “What positive and adverse effects are reported by experienced product users?”. The purpose of this paper is to describe this multidisciplinary team, detail the model we use to evaluate ECIGs and other potential MRTPs, and highlight key findings and lessons learned from our experience.
Team Composition and Goals
Our team members conceptualize tobacco use as a behavior that often involves user self-administration of the dependence-producing drug nicotine as well as other toxicants that are a threat to individual and public health. We recognize that tobacco products, including ECIGs and other potential MRTPs, are highly engineered products with effects that likely depend upon an interaction between product characteristics and user behavior. Further, we recognize that individual tobacco users may transition from one product to another and back again, requiring the ability to monitor and analyze the individual- and population-level effects of these transitions. Given this conceptualization, our team includes scientists with a variety of backgrounds in psychology, including behavioral pharmacology and clinical, experimental, and community psychology. Our team also includes scientists with expertise in aerosol research, analytical chemistry, biostatistics, engineering, internal medicine, and public health. A key feature of all team members is their willingness to extend their interests beyond the boundaries of their discipline to collaborate effectively with the shared goal of producing efficiently the rigorous science needed to inform empirically-based tobacco policy. Below we describe each team member (order of presentation is alphabetical by last name).
Robert L. Balster is a psychologist with expertise in behavioral pharmacology and abuse liability assessment. His expertise in abuse liability assessment and tobacco control policy informs multiple team projects.
Alison Breland is an experimental psychologist with a specialization in biopsychology with experience in the evaluation of the effects of novel tobacco products, including nicotine and toxicant delivery, and user behavior. Her expertise informs the work of our engineering/analytical chemistry group.
Caroline Cobb is an experimental psychologist with a specialization in biopsychology and experience with clinical laboratory evaluations of tobacco products and assessing population patterns of tobacco use. Her expertise in assessment at both the individual and population level is an essential element of cross-project collaboration.
Thomas Eissenberg is an experimental psychologist and is the team’s director. His primary area of research is the behavioral pharmacology of drugs of abuse. His expertise in product design, pre-clinical methods, tobacco user behavior, user toxicant exposure, and population-level assessment of tobacco use allow him to foster cross-project collaboration.
Pebbles Fagan is a public health scientist and her research focuses on the social, behavioral, and bio-behavioral factors associated with tobacco/nicotine use with an emphasis on tobacco-related health disparities. She leads a research group that uses quantitative and qualitative methods to evaluate tobacco products and user behavior as well as changes in product design/marketing. Her expertise ensures that our work generalizes to the population.
Jonathan Foulds is a clinical psychologist and an expert in nicotine psychopharmacology and the treatment of nicotine addiction. His expertise as a clinical psychologist ensures we maintain a focus on individual-level health outcomes.
J. Randy Koch is a community psychologist with experience in community-based research, the translation of research into practice and policy, and organizational leadership. In his role directing our pilot research program, he helps to ensure that we consistently are engaged in new projects that address emerging needs in tobacco regulatory science.
Thokozeni Lipato is a physician specializing in internal medicine. He is a medical monitor for our clinical research and his focus on participant health helps ensure the continued safety our research participants.
Najat A. Saliba is an analytical chemist with extensive experience in the analysis of airborne particles, including tobacco. She oversees chemical assays in our work focused on characterizing tobacco product aerosol emissions, and is responsible for using our study results on product design to inform the development of new methods for assessing nicotine yield and previously unexplored toxicants (e.g., furans).
Alan Shihadeh is a mechanical engineer with expertise in the chemistry, physics, and exposure science of particle pollutants, with an emphasis on tobacco smoke. His experience with product-specific factors that influence tobacco product emissions, as well as his appreciation of the role user behavior plays in influencing these emissions, allow him to collaborate directly and effectively with all other team members.
Shumei Sun is an expert in modeling complex cross-sectional and longitudinal data. She leads the biostatistics support group. This group assists in clinical trial design, sample size and power calculations, analysis of repeated measures data, and longitudinal modeling.
Six psychologists, with expertise in behavioral pharmacology, nicotine pharmacology, abuse liability, biopsychology, tobacco product evaluation, research design, tobacco user behavior, and the translation of research into policy, form a critical core component of our 11-member multidisciplinary team. The psychologists’ expertise is enriched by the other team members who bring expertise in public health, medicine, chemistry, engineering, and statistics.
Since core funding began in 2013, this multi-disciplinary team has collaborated to complete four interlinked projects to study ECIGs as an exemplar of MRTPs: 1) studying ECIGs and other tobacco products in the engineering laboratory to determine emissions profiles and the factors that influence them, 2) studying ECIGs and other tobacco products in the human laboratory to determine their nicotine delivery profile, user behavior patterns, and other effects under controlled conditions, 3) conducting an RCT to determine the effects of ECIG use on cigarette smoker toxicant exposure and concurrent other tobacco product use, and 4) conducting mixed qualitative/quantitative research to understand effects reported by ECIG users, ECIG use motivation, and product marketing/packaging. This work is conducted using a multidisciplinary evaluative model, described below. In addition to this work, team members lead several other federally-funded research grants on topics related to tobacco regulatory science. Also, our team developed a training program in tobacco regulatory science as well as a pilot grants program to facilitate the initiation of complementary areas of research. One of the keys to building and maintaining our multidisciplinary research program has been the interaction with other federally-funded regulatory science teams which, considered together, represent an even larger multidisciplinary effort for informing tobacco product regulation. There are biannual meetings of these teams as well as shared knowledge dissemination and these meetings have led to the formation of working groups that have helped shape our research and training programs as well as our participation in collaborative research projects across teams.
Evaluative Model for Potential Modified Risk Tobacco Products
FDA’s mandate to regulate tobacco products, including novel tobacco products like ECIGs and other potential MRTPs, arises from the Family Smoking Prevention and Tobacco Control Act (FSPTCA; FDA Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, 2016, 21 C.F.R. § 1100, 1140, and 1143). According to the FSPTCA, an MRTP is “. . . a tobacco product that is sold or distributed for use to reduce harm or the risk of tobacco-related disease associated with commercially marketed tobacco products” (p. 28987). The law makes clear that any product intended as an MRTP must be evaluated to determine if it is likely to reduce harm. However, there are few demonstrated methods for evaluating the effects of MRTPs in controlled settings and for predicting MRTP effects in the real world. In response to this dearth of methods, the Institute of Medicine (IOM) published Scientific Standards for Studies on Modified Risk Tobacco Products (IOM, 2012). That monograph makes clear that a variety of methods are required for comprehensive MRTP evaluation, including analytical methods that identify MRTP constituents before an MRTP is marketed; human laboratory methods that reveal the toxicant exposure, effects, and abuse liability associated with MRTP use in controlled settings before an MRTP is marketed; RCT methods used to study MRTP effects under real-world use conditions before an MRTP is marketed; and qualitative/related methods that reveal user attitudes, behaviors, and effects after an MRTP is marketed.
Our team was influenced by the IOM (2012) monograph regarding the methods that are critical for MRTP evaluation and subsequent science-based regulation by FDA. We noted that these and other methods will be most useful for regulatory purposes when they are well-integrated and used iteratively. Contributing expertise from their respective fields, and drawing from previous work (Hatsukami et al., 2005), team members developed an integrated, iterative model for evaluating any novel tobacco product including potential MRTPs and ECIGs (Figure 1). The model highlights analytic laboratory, human laboratory, RCT, and quantitative/qualitative methods, recognizing that other methods are also necessary (e.g., in vitro and in vivo non-human animal models that reveal toxicity; Bahl et al., 2012; Husari et al., 2016). In an ideal world, the analytic lab, human lab, and RCT methods are applied to study an MRTP initially, before it is marketed, with each evaluation phase providing relevant information to the next, and all results informing the first round of regulation (e.g., initial product labeling and user access). Then, after the product is marketed, quantitative and qualitative methods are applied to understand real-world use patterns/effects and thus inform subsequent rounds of the regulatory process (e.g., “Is current labeling adequate?” “Should access be more or less restricted?”). Also, these methods may reveal non-marketed, “unorthodox” uses of MRTPs, suggesting the need for additional analytic and human lab work that can inform subsequent regulation (e.g., design restrictions that limit potentially harmful unorthodox use). The model therefore is integrated in that each method informs others, and iterative, in that evaluation is on-going based upon what is learned in various phases. The sections below highlight briefly how each set of methods in the model can inform tobacco product regulation.
Figure 1:

An integrated, iterative, model for evaluating modified risk tobacco products (MRTPs; adapted from Hatsukami et al., 2005). This schematic illustrates how analytic laboratory, human laboratory, RCT, and qualitative/quantitative methods can be integrated and used iteratively to inform FDA about MRTP toxicant yield and effects under controlled, real-world, and unorthodox use conditions. For example, if analytic lab results suggest an MRTP with particular characteristics has reduced toxicant yield, human lab methods can explore how MRTPs with those characteristics influence toxicant exposure and abuse liability. If human lab results also suggest a potential for decreased risk/abuse, RCT methods can then be used to reveal toxicant exposure and adverse event profile under real-world conditions. Should the product come to market, quantitative/qualitative methods reveal MRTP prevalence, likelihood of multi-product use, and any non-marketed, unorthodox MRTP use.
How Can Analytic Lab Methods Inform Regulation?
The first step in our integrated, iterative model involves applying analytic methods to understanding an MRTP. Important questions to resolve include: “Does the product yield nicotine and other toxicants?”, “How is nicotine yield influenced by product design features?”, and “How does nicotine and toxicant yield differ under unintended, unorthodox use methods?” Questions like these were first posed of cigarettes decades ago, and a variety of analytical lab methods were designed to answer them. For example, by the 1930s, smoking machines were being used to generate and analyze smoke for the content (or “yield”) of nicotine, carbon monoxide (CO) and other toxicants (Bradford, Harlan, & Hanmer, 1936), and these methods were later standardized. Over time, cigarette manufacturers incorporated design features to reduce measured toxicant yields, such as ventilated filters that dilute the smoke (Kozlowski & O’Connor, 2002). Unfortunately, while reported yields dropped by more than 60% from the 1950s to the 1990s, these reductions did not reduce human toxicant exposure because cigarette manufacturers had thwarted the measurement standards and also because cigarette smokers puffed more intensively when using these products (National Cancer Institute [NCI], 2001). While the relation of toxicant exposure to human behavior long had been understood by the tobacco industry (Wayne & Carpenter, 2009), the regulatory failure to require that analytical methods account for human behavior led to widespread misunderstanding of the health risks of an entire “low-yield” class of tobacco products. Use of an integrated and iterative model such as ours, that involves collaborating investigators that understand analytic methods and user behavior, provides an opportunity for tobacco regulators to avoid repeating this unfortunate outcome with MRTP evaluation and regulation.
How Can Human Lab Methods Inform Regulation?
The second step in our model involves applying human laboratory methods to understanding MRTP effects. Important questions to resolve include: “How do people use MRTPs?”, “How effectively does an MRTP deliver nicotine and what are its other effects on users?”, and “How are MRTP effects influenced when MRTPs are used in an unorthodox manner?”. Questions like these were first posed of cigarettes several decades ago, and a variety of human lab methods were designed to answer them. For example, a detailed analysis of user puffing behavior (i.e., “puff topography”, the measurement of puff number, volume, duration, and flow rate) was critical to the realization that changes in cigarette filter design intended to reduce smoke toxicant yield failed to produce changes in smoker toxicant exposure: smokers take more, bigger, and/or longer puffs when they switch from full-flavor to low-yield brands (NCI, 2001). In the human laboratory, puff topography analysis is often coupled with physiological recording, blood sampling, and/or subjective response assessment to understand cigarette effects better (e.g., Malson, Lee, Murty, Moolchan, & Pickworth, 2003; Zacny & Stitzer, 1988). Behavioral tasks are also used to evaluate a product’s abuse liability: the likelihood that a tobacco product will maintain persistent use and dependence (e.g., Carter et al., 2009). Thus, human laboratory methods are ideal for evaluating a wide variety of outcomes relevant to understanding the effects of a potential MRTP.
How Can Randomized Controlled Trial Methods (RCT) Methods Inform MRTP Regulation?
The third step in our model involves applying RCT methods to understanding MRTPs. Important questions include: “What are the patterns of real-world use of the MRTP?”, “How does real-world MRTP use influence biomarkers of tobacco toxicant exposure and disease risk?”, “What is the withdrawal symptom and adverse event profile associated with long-term MRTP use?”, and “What is the influence of MRTP use on use of conventional tobacco products?” RCT methods answer these questions because they allow participants to use products at home, work, or play and allow investigators to contrast novel and conventional product effects on a variety of outcomes. RCT methods have been used in the past to understand the effects of tobacco products (e.g., Fagerström, Hughes, & Callas, 2002; Hatsukami et al., 2004). RCT outcomes include exposure to tobacco toxicants such as nicotine, CO, and carcinogenic nitrosamines (e.g., Fagerström et al., 2002; Hatsukami et al., 2004). The adverse event profile of the novel product is also important (e.g., Chen, 2013) as is its ability to suppress the effects of tobacco abstinence (e.g., Breland, Kleykamp, & Eissenberg, 2006). Another outcome in these studies is the effect of the novel product on concurrent use of and dependence on traditionally marketed tobacco cigarettes (Foulds et al., 2015; Hughes & Keely, 2004). RCT methods are informed by and extend human laboratory results by revealing the effects of novel tobacco product use under longer-term, real-world conditions.
How Can Qualitative and Quantitative Methods Inform MRTP Regulation?
The fourth step of the integrated, iterative model involves applying quantitative and qualitative methods to understanding MRTPs. Important questions include “What are the attitudes, beliefs, and perceived effects associated with MRTP use?” and “What are some of the unorthodox ways of using an MRTP?” One method that can be used to address some of these questions is “concept mapping” (Rosas & Kane, 2012). Concept mapping is a mixed-method participatory research approach that integrates qualitative group-level processes (brainstorming, sorting, and group interpretation) and quantitative multivariate statistical analyses (multidimensional scaling and hierarchical cluster analysis) to help understand complex relationships. It has been applied to tobacco issues (e.g., Trochim, Stillman, Clark, & Schmitt, 2003) and can provide a framework for organizing and structuring vast amounts of information that can be used to guide future empirical inquiry (e.g., hypothesis testing and validation studies). Another data source that is becoming increasingly important in understanding novel tobacco product use involves information posted by users online. For example, analysis of YouTube videos and other alternative data sources has informed understanding of perceptions of smokeless tobacco (Bromberg, Augustson, & Backinger, 2012), waterpipes (Carroll, Shensa, & Primack, 2012), and ECIG use behavior (Hua, Yip, & Talbot, 2011). Overall, a mix of qualitative and quantitative methods plays a key role in an integrated, iterative model of MRTP evaluation that informs science-based regulation.
The model described here is innovative because it is an integrated, iterative, and multidisciplinary approach in which collaborating investigators apply proven methods to learn about novel tobacco products swiftly. This type of evaluation was first proposed in 2005 (Hatsukami et al., 2005) and the authors noted that “An important point to remember is that the different stages involved in testing these products are bidirectional rather than unidirectional; that is, results from human evaluation can inform preclinical evaluation including product design. Similarly, post-marketing surveillance results can inform clinical or preclinical evaluation of products.” (p. 830). The model displayed in Figure 1 is tuned exquisitely to this “bidirectional” testing and articulates clearly some of the specific areas where bidirectionality and multidisciplinarity are essential. For example, integrating human topography data with analytic lab analysis (Shihadeh & Eissenberg, 2011) allows tobacco product emissions to be produced by a machine in the same way that humans produce them. This work requires product-specific technology that mimics human behavior exactly. Conceiving of and conducting this work highlights the value of involving engineers and psychologists as collaborative colleagues. Likewise, the understanding that real-world experience sometimes brings unorthodox product use invokes this bidirectional application of methods while highlighting the multidisciplinary efforts of scientists trained in field-based and human and analytical lab techniques. A final point to note about the model is that it is “plug-and-play”. That is, with variations in engineering and laboratory techniques, any tobacco product can be evaluated using the methods described.
Key Findings
Since funding began in September 2013, our multidisciplinary team has published over 100 peer-reviewed papers addressing the toxicant emissions and effects of a variety of tobacco products including waterpipe, “heat-not-burn”, and, primarily, ECIGs. Reviewing all of this work is beyond the scope of this paper. Instead, below we describe key findings that highlight the value of involving a collaborative multidisciplinary team in tobacco product evaluation. These key findings are that ECIGs are a heterogeneous product class and understanding this heterogeneity is critical for effective regulation; ECIG nicotine emissions can exceed those of a combustible tobacco cigarette and can be modeled mathematically; ECIG use is associated with subsequent cigarette smoking; and that we have developed and refined methods that apply to a variety of tobacco products.
ECIGs are a Heterogeneous Product Class.
There is great variability among ECIGs in terms of construction, electric power, and liquid ingredients, and the product class is evolving constantly in a largely unregulated environment (e.g., Malek et al., in press). All of these features influence the amount of nicotine and other toxicants that ECIGs emit. For example, we demonstrated that doubling ECIG device power increases nicotine emissions ~5-fold (Talih et al., 2015) and, accounting for the surface area of the heating element, increasing power also increases volatile aldehyde emissions (Talih, Salman, et al., 2017). A greater concentration of nicotine in the liquid also increases nicotine emissions (Talih et al., 2015) as well as nicotine delivery to users’ blood (Hiler et al., 2017). However, we demonstrated that labeled nicotine content is often not a good indicator of actual nicotine content and does not reveal if the liquid contains the more bioavailable “free base” form of nicotine (El Hellani et al., 2015). We also developed a method for measuring “free base” nicotine in ECIG liquids and aerosols (El Hellani et al., 2015). In addition, our mixed-method group noted that sweeteners are often added to ECIG liquids (Fagan et al., 2017). In response to this information, our analytic group found that, when heated, some sweeteners can expose users to toxic furans: the per-puff yield of some furans is comparable to values reported for combustible cigarettes (Soussy et al., 2016).
Overall, ECIG heterogeneity is an indicator of the need for regulation of this product class because, by virtue of their differing construction, power, and liquids, some members of the class produce far greater toxicant emissions than others, even when holding nicotine output constant. However, because these features interact, regulating only one of them and not the others may not achieve policy goals (Shihadeh & Eissenberg, 2015). For example, because nicotine emissions are a function of device power, liquid nicotine concentration, and user behavior, regulating nicotine concentration only, as in the European Union (http://ec.europa.eu/health/tobacco/products_en), is unlikely to limit nicotine emissions and subsequent nicotine delivery as intended. That is, higher wattage ECIGs (e.g., ~70 Watt) paired with low nicotine concentration liquids (~4 mg/ml) are capable of meeting or exceeding the nicotine delivery of a combustible cigarette (Wagener et al., 2016). For this reason, policymakers may want to consider regulating the rate of ECIG nicotine emission, rather than the device and liquid factors that influence it (i.e., nicotine “flux”; Shihadeh & Eissenberg, 2015; Talih, Balhas, et al., 2017). Importantly, the research questions that led to these findings would not have been asked, and their relationship to regulatory outcomes would not have been realized, without the input from team members from a variety of disciples that extend far beyond aerosol research and analytical chemistry to include psychology, public health, and policy.
ECIG Nicotine Emissions Can be Modeled Mathematically.
Policymakers may seek to regulate the rate of ECIG nicotine emissions (nicotine flux), a task that is challenged by ECIG heterogeneity. To reduce the challenge, we developed a physics-based mathematical model of ECIG nicotine emissions (Talih, Balhas, et al., 2017). The model accounts for the time it takes for a heating element to heat up after electricity begins flowing, and how much the element cools down between puffs. It also accounts for the various ways that heat can be transported from the element: by the air passing over it, by the latent heat of the liquid as it evaporates, by conduction through the metal solder to the body of the device, and by radiation to the surroundings. Inputs to the model include user behavior (puff velocity and duration, inter-puff interval), the length, diameter, electrical resistance and thermal capacitance of the heater coil, the composition and thermodynamic properties of the liquid (including nicotine concentration), and the ambient temperature. To validate the model, we compared model predictions of nicotine for 100 conditions in which ECIG power, user behavior, device type, and liquid composition were varied. The model accounted for 72% of the variability in nicotine flux in the conditions tested (Talih, Balhas, et al., 2017). The model can be used to predict the nicotine emissions of any ECIG based on design specifications, and thus allows regulators to determine in advance if a planned device will meet regulatory standards. The multidisciplinary nature of the team underpins every aspect of this mathematical model. The need for it was made clear by those of us familiar with regulatory action, the ability to develop it was made possible with those of us with expertise in statistics, modeling, and aerosol chemistry, and the necessity of including user behavior was made apparent by those of us familiar with the behavioral pharmacology of drugs of abuse. Working together, this multidisciplinary team was able to develop a tool that regulators can use to understand the nicotine emission profile of ECIGs now and in the future.
ECIG Use among Never Smokers is Associated with Subsequent Tobacco Cigarette Smoking.
A potential MRTP presents a public health concern if it encourages people who do not use tobacco products to begin using them; this concern is magnified if the product encourages cigarette smoking that kills thousands of Americans each year. Having seen reports that some never-smokers transition from ECIGs to tobacco cigarettes, we were curious to see if this effect occurs in college students. Accordingly, we analyzed data from over 3000 undergraduates who were surveyed twice (Time 1 and Time 2), with surveys separated by a year (Spindle, Hiler, Cooke, et al., 2017). Among participants reporting never smoking at Time 1, those who had ever tried ECIGs or currently were using ECIGs were more likely to have ever tried tobacco cigarettes by Time 2 relative to individuals who had not used ECIGs. Ever use of ECIGs also increased participants’ likelihood of being current cigarette smokers at Time 2. Working across institutions with collaborators outside of our team, we participated in a meta-analysis of this study and eight others, involving over 17,000 adolescents and young adults, and demonstrated that ECIG use was associated with over three times greater risk for subsequent cigarette smoking initiation and past 30-day cigarette smoking (Soneji et al., 2017). Given that never-smoking participants who had tried ECIGs were more likely to initiate cigarette use later, policymakers may want to consider limiting young adults’ access to these products. This key finding, and our contribution to it, is perhaps the best exemplar of the value of working in a multidisciplinary team. That is, it demonstrates how the multidisciplinary experience allows researchers to ask questions that stretch their expertise and then work with others from other disciplines to answer those questions using methods outside of their “comfort zone”.
Methods for Evaluating a Variety of Tobacco Products.
Novel tobacco products are proliferating in the US marketplace and include ECIGs and products marketed to heat but not burn tobacco (i.e., “heat-not-burn” products; Jenssen, Walley, & McGrath-Morrow, 2018). Effective regulation of these products requires objective data that reveals their effects in users. Building on our earlier work (Breland, et al., 2006; Shihadeh & Azar, 2006; Vansickel, Cobb, Weaver, & Eissenberg, 2010), we have continued to develop, refine, and standardize methods that meet this need. For example, our engineering group developed technology that is sensitive enough to measure low flow rate puffs observed in ECIG users and our human laboratory group demonstrated that the use of this technology does not influence study outcomes (Spindle, Breland, Karaoghlanian, Shihadeh, & Eissenberg, 2015). Also, we have refined methods for evaluating the nicotine delivery and subjective effect profile of novel products under controlled and free (ad lib) puffing conditions (Spindle, Hiler, Breland, et al., 2017) and used these methods to demonstrate the influence of liquid nicotine concentration and user behavior on ECIG nicotine delivery and subjective effects (Hiler et al., 2017). We have demonstrated that these methods also can be used to evaluate heat-not-burn products (Lopez, Hiler, Maloney, Eissenberg, & Breland, 2016), little cigars (Blank, Cobb, Eissenberg, & Nasim, 2016), and waterpipe (e.g., Ramôa, Shihadeh, Salman, & Eissenberg, 2016). These methods have been implemented by other researchers in this country and elsewhere (e.g., Dawkins & Corcoran, 2014; Farsalinos et al., 2015; Wagener et al., 2016). Policymakers require objective data in order to craft science-based tobacco product regulation. Our team has pioneered technology and methods to meet this regulatory need. This method of tobacco product evaluation is inherently multidisciplinary, and ongoing work continues to include all team members.
Lessons Learned From our Multidisciplinary Experience
There are several benefits and also challenges to working in a multidisciplinary team. Benefits include opportunities for better science (Bennett & Gadlin, 2012), more comprehensive trainee experiences, and a greater willingness to continue to collaborate across disciplines. With regard to better science, a multidisciplinary approach improves our study designs because each team member has a chance to ask questions and apply methods from their domains of expertise. For example, in reviewing previous RCTs examining the effects of ECIGs on cigarette smoking (e.g., Bullen et al., 2013), team members noted that the low power devices used in those trials were unlikely to deliver cigarette-like doses of nicotine to inexperienced users. Accordingly, the engineering group tested the nicotine emissions of a variety of ECIG products and nominated one for consideration for our RCT. Preliminary human laboratory testing revealed that the product could produce cigarette-like plasma nicotine concentrations when paired with a 36 mg/ml liquid (Ramôa et al., 2016) and this product with that liquid was included in the RCT (Lopez et al., 2016). In another example, the observation made by the human laboratory group that nicotine delivery from ECIGs was highly variable and difficult to predict inspired the engineering group to suggest and then develop the mathematical model of ECIG nicotine emissions (Talih, Balhas, et al., 2017).
A multidisciplinary approach has enhanced our ability to provide a comprehensive training experience for pre- and post-doctoral trainees. From the beginning, trainees have been involved in all of our work, helping to design studies, write Institutional Review Board (IRB) submissions, conduct research, analyze results, and, of course, authoring theses, dissertations, and manuscripts for publication. This latter effort allows us to quantify success: of our total publications, over half involve our trainees, and of those involving trainees, over half have trainees as the first author. Further, comprehensive training goes beyond publications and extends to experiential learning. Trainees in our team generally are assigned to one core faculty member/discipline, and they also are encouraged to gain experience with other projects and with other faculty members working in other disciplines (a “multiple mentor” model; Nash, 2008). Moreover, we prioritize trainees for off-site learning experiences, and trainees have accompanied their mentors to FDA-sponsored workshops, international conferences, visits to other international sites of collaboration (e.g., Jordan University of Science and Technology, in Irbid, Jordan), and a visit to World Health Organization headquarters in Geneva, Switzerland (June, 2015), where we sponsor 3-month trainee internships.
Trainees gain valuable knowledge from these opportunities and also learn that other disciplines are accessible, exciting, and enhance their own research. Such opportunities have led our trainees to engage in multidisciplinary work of their own. For example, two trainees noted that an ongoing longitudinal survey (Dick et al., 2014) might provide insight into ECIG effects on subsequent cigarette smoking. Those trainees approached the longitudinal study team, were granted access to the data, and published their results (Spindle, Hiler, Cooke, et al., 2017).
Indeed, that trainee experience is an example of one of the most meaningful outcomes of our multidisciplinary experience: a willingness to reach out to and collaborate with other scientists outside of our own disciplines. This willingness is apparent in recent publications, such as those having to do with brain imaging (Baldassarri et al., 2018; Hobkirk et al., 2018) or assessing the effects of a menthol cigarette ban in Ontario, Canada (Chaiton, Schwartz, Cohen, Soule, & Eissenberg, 2018), and is even more evident in recently submitted applications for federal funding. For example, team members who are psychologists have collaborated with a biologist and a biomedical engineer on an application to the National Institute of Dental and Craniofacial Research to study the effects of ECIG aerosols on craniofacial birth defects using a frog embryo model and a mouse model, and with a forensic toxicologist on an application to the National Institute of Justice to study the impact of inhaling ethanol in ECIGs on concurrent oral ethanol consumption. Another recent application to the National Cancer Institute involves team members working with experts in obesity and dietary assessment to adapt a computer application used to measure energy and nutrient intake (Martin et al., 2012) as a tool for estimating the nicotine intake of ECIG users. These ongoing collaborations and others with which our team members are involved are facilitated by our experience with the excitement, synergy, and productivity that comes with working in a multidisciplinary team.
Further, an important outcome of our team’s formation and receipt of core funding has been the expansion of our research through a pilot grant program as well as faculty recruitment. For example, we were able to add pharmacology expertise by funding a pilot project that assessed the role of menthol flavor on the abuse-related effects of nicotine. In addition, we recruited an expert on behavioral economics to help in the assessment of the abuse potential of ECIG products. Also, several team members have obtained independent funding for other tobacco product research that complements our core mission.
The opportunities of multidisciplinary work also come with challenges. One challenge is the steep learning curve necessary to collaborate productively with experts in other fields, thus requiring new training models for investigators at all levels (Hall, Feng, Moser, Stokols, & Taylor, 2008). This challenge can be especially acute with seasoned investigators who are themselves experts, albeit in a different discipline. For those who are beginning to learn a new discipline, there is a disconcerting sense of déjà vu – as though they have morphed from expert to neophyte. Recognizing the need to ask questions, the need to listen carefully to answers, and the need to engage wholly with a different field can be hard for an accomplished scientist but is necessary for multidisciplinary science to thrive. For experts who are helping others learn a new discipline, the challenge is different: explaining familiar concepts to a new and questioning audience unfamiliar with the basics of the field. Recognizing the potential vulnerability of the audience (confident expert morphed into disconcerted neophyte) and the need to communicate familiar information clearly and respectfully also can be hard for an accomplished scientist, but also is necessary for multidisciplinary science to thrive.
Drawing from the work of others (e.g., Bennett & Gadlin 2012; Nash, 2008), our team has come to refer to this effort as “learning to speak each other’s language” and we have oftentimes stumbled on our way to success. One anecdote is perhaps revealing: in discussing mathematical modeling of ECIG nicotine emissions, the engineering group continually referred to “mass transfer”. The phrase’s meaning was readily obvious to others, but was meaningless to the psychologist in the room. Eventually, the psychologist reluctantly posed the obvious question: “What do you mean, ‘mass transfer’?”. The engineering group stepped back from the detailed modeling work and explained they were referring to the amount (“mass”) of nicotine moving (“transfer”) through the device. The two-part message from this anecdote hits home: asking questions about what seems obvious to others is necessary and explaining “obvious” concepts clearly and respectfully facilitates valuable collaborations. This lesson continually informs our efforts as, for example, the human laboratory group installs, calibrates, troubleshoots and repairs complex measurement equipment developed by an engineering team that is thousands of miles away (Hiler et al., 2017) or, in another example, the mixed-method group learns the complex procedures of time series analyses (Maloney et al., in press). Learning to speak each other’s scientific language requires liberal helpings of patience, humility, good humor, and a willingness to ask “stupid” questions and answer them respectfully.
While writing manuscripts and grant applications with colleagues from diverse disciplines can be very productive, it also poses challenges. Terminology differs considerably, so jargon such as “puff topography” (puff number, volume, duration and flow rate) and “Stokes numbers” (describing the behavior of flowing particles), and “cognitive interview” (a qualitative data collection technique) require context and definition. The expectations of journal reviewers and editors differ across fields, so where small sample sizes and correlational analyses are appropriate for one author, others have different expectations that require thoughtful resolution. In some fields, analysis of statistical power is not included routinely in grant applications, where in others it is required. We have overcome these and other disciplinary differences by learning to listen and respond appreciatively and not defensively to questions and comments raised during the writing process; we find that this approach strengthens the final product.
Another challenge with which we struggle is to stay focused on our core mission – informing tobacco product regulation – without being distracted by other issues. We have learned that working outside own disciplines raises important scientific questions that our team would like to answer, but some of these questions drift away from tobacco product regulation. For example, many of us have a strong interest in evaluating ECIG-delivered nicotine as a smoking cessation medication. However, medication development is not in the purview of FDA tobacco product regulation and is therefore outside of the scope of our FDA/NIH core funding. Our multidisciplinary team remains focused on our core mission by continually asking: “How would answering this scientific question inform tobacco product regulation?”
Future Plans
Over the next few years, inspired by our work evaluating the Ontario ban on menthol cigarettes (Chaiton et al., 2018), our team plans to pivot from evaluating products to predicting the effects of potential regulatory actions. According to U.S. law, FDA tobacco regulation must protect public health, though regulations may also have unintended consequences that cause harm. For example, requiring “tar” and nicotine yields on cigarette packs was intended to provide health-promoting information to smokers, but drove some of them to switch to “low yield” cigarettes instead of quitting smoking, thus increasing their risk of tobacco-caused disease (NCI, 2001). Similar negative outcomes may occur with tobacco product regulation today. If FDA could predict a potential regulation’s effects, then refinements before the regulation is issued might maximize its health-promoting effects and minimize unintended consequences.
Methods exist for assessing a regulation’s effects once it is in place (e.g., Borland et al., 2009), but few data-driven models predict impact beforehand. Any such model must draw from several domains to assess how a potential regulation might change product toxicity, user behavior, and addiction/abuse liability. Also, the model must demonstrate the extent to which predictions about potential regulatory effects describe actual population-level outcomes. We hope to address this issue by testing hypotheses and generating predictions regarding the impact of several potential regulations (e.g., limiting the nicotine concentration of ECIG liquid) and then examining the validity of these predictions at the population level using a prospective cohort survey. Our vision is to provide tools that can guide regulation development so that, by the time a regulation goes into effect, validated methods have tested it, refined it, and generated data showing that its health-promoting effects are maximized and unintended consequences minimized. Over the next several years, our multidisciplinary team will continue to work together using our integrative, iterative approach to inform regulatory efforts. And, we will continue to learn to speak each other’s languages.
Overall Summary/Conclusions
Overall, our team of psychologists and others in public health, medicine, chemistry, engineering, and statistics, working together across disciplines, has informed the scientific, regulatory, and public health communities about ECIGs and their effects. Science that informs regulation requires an understanding of the factors that influence individual behavior, as individual behavior change can improve overall public health. In our team, expertise about individual-level behavior is provided primarily by psychologists. The psychologists on our team help to keep other members focused on how nicotine is a dependence-producing drug, that nicotine users self-administer this drug regularly, and that dependent users maintain their regular, frequent nicotine use in order to receive the drug’s reinforcing effects (positive and negative). Our behavioral focus challenged the engineering team to develop technology that allows us to measure nicotine self-administration behavior (i.e., puff topography) in ECIG users, and that challenge ultimately led to their inspiration to use puff topography records from actual users to understand ECIG toxicant emissions. Our team’s future work -- designed to give FDA a set of laboratory tools that can predict population-level policy effects -- is a direct result of our understanding that individual-level behavior change can be achieved through product regulation, and that pre-testing the effects of potential regulatory action at the individual level is a critical component of ensuring that policy has its intended consequences that protect public health while avoiding unintended consequences that do not. At the same time, we emphasize that other team members keep the psychologists focused on other critical issues, such as how product design changes can influence nicotine flux and how context (e.g., product labeling and marketing) can influence user perceptions and thus behavioral outcomes. In sum, on our team, psychologists play a key role by emphasizing an understanding of individual-level behavior and that emphasis is, in turn, influenced powerfully by the diversity of expertise provided by other team members.
Acknowledgements
Research reported in this publication was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number P50DA036105 and U54DA036105 and the Center for Tobacco Products of the U.S. Food and Drug Administration. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Food and Drug Administration. Dr. Eissenberg and Dr. Shihadeh are paid consultants in litigation against the tobacco industry and are named on a patent application for a device that measures the puffing behavior of electronic cigarette users. Dr. Foulds has done paid consulting for pharmaceutical companies involved in smoking cessation and has received grant funding from Pfizer Inc.
Footnotes
Disclosures
All other authors have no conflicts to report.
References
- Bahl V, Lin S, Xu N, Davis B, Wang YH, & Talbot P (2012). Comparison of electronic cigarette refill fluid cytotoxicity using embryonic and adult models. Reproductive Toxicology 34(4). 529–537. DOI: 10.1016/j.reprotox.2012.08.001 [DOI] [PubMed] [Google Scholar]
- Baldassarri SR, Hillmer AT, Anderson JM, Jatlow P, Nabulsi N, Labaree D, … Esterlis I (2018). Use of electronic cigarettes leads to significant beta2-nicotinic acetylcholine receptor occupancy: Evidence from a PET imaging study. Nicotine & Tobacco Research, 20(4), 425–433. DOI: 10.1093/ntr/ntx091 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bennett LM, & Gadlin H (2012). Collaboration and team science: From theory to practice. Journal of Investigative Medicine, 60(5), 768–775. DOI: doi: 10.231/JIM.0b013e318250871d. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blank MD, Cobb CO, Eissenberg T & Nasim A (2016). Acute effects of “hyping” a black&mild cigarillo. Nicotine & Tobacco Research, 18(4), 460–469. DOI: 10.1093/ntr/ntv063 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Borland R, Wilson N, Fong GT, Hammond D, Cummings KM, Yong HH, … & McNeill A (2009). Impact of graphic and text warnings on cigarette packs: Findings from four countries over five years. Tobacco Control, 18(5), 358–364. DOI: 10.1136/tc.2008.028043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bradford JA, Harlan WR, & Hanmer HR (1936). Nature of cigaret smoke: Technic of experimental smoking. Industrial & Engineering Chemistry, 28(7), 836–839. [Google Scholar]
- Breland AB, Kleykamp BA, & Eissenberg T (2006). Clinical laboratory evaluation of potential reduced exposure products for smokers. Nicotine & Tobacco Research, 8(6), 727–738. DOI: 10.1080/14622200600789585 [DOI] [PubMed] [Google Scholar]
- Breland A, Soule E, Lopez A, Ramôa C, El‐Hellani A, & Eissenberg T (2017). Electronic cigarettes: what are they and what do they do? Annals of the New York Academy of Sciences, 1394(1), 5–30. DOI: 10.1111/nyas.12977 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bromberg JE, Augustson EM, & Backinger CL (2012). Portrayal of smokeless tobacco in YouTube videos. Nicotine & Tobacco Research, 14(4), 455–462. DOI: 10.1093/ntr/ntr235 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bullen C, Howe C, Laugesen M, McRobbie H, Parag V, Williman J, & Walker N (2013). Electronic cigarettes for smoking cessation: a randomised controlled trial. The Lancet, 382(9905), 1629–1637. DOI: 10.1016/S0140-6736(13)61842-5 [DOI] [PubMed] [Google Scholar]
- Carter LP, Stitzer ML, Henningfield JE, O’Connor RJ, Cummings KM, & Hatsukami DK (2009). Abuse liability assessment of tobacco products including potential reduced exposure products. Cancer, Epidemiology, Biomarkers and Prevention, 18(12), 3241–3262. DOI: 10.1158/1055-9965.EPI-09-0948 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carroll MV, Shensa A, & Primack BA (2013). A comparison of cigarette-and hookah-related videos on YouTube. Tobacco Control, 22(5). 319–323. DOI: 10.1136/tobaccocontrol-2011-050253 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chaiton M, Schwartz R, Cohen JE, Soule E, & Eissenberg T (2018). Association of Ontario’s ban on menthol cigarettes with smoking behavior 1 month after implementation. JAMA Internal Medicine 178(5), 710–713. DOI: 10.1001/jamainternmed.2017.8650 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen IL (2013). FDA summary of adverse events on electronic cigarettes. Nicotine & Tobacco Research, 15(2), 615–616. DOI: 10.1093/ntr/nts145 [DOI] [PubMed] [Google Scholar]
- Dawkins L, & Corcoran O (2014). Acute electronic cigarette use: nicotine delivery and subjective effects in regular users. Psychopharmacology, 231(2), 401–407. DOI: 10.1007/s00213-013-3249-8 [DOI] [PubMed] [Google Scholar]
- Dick D, Nasim A, Edwards AC, Salvatore J, Cho SB, Adkins A, … & Kendler. KS (2014). Spit for Science: launching a longitudinal study of genetic and environmental influences on substance use and emotional health at a large US university. Frontiers in Genetics, 5, 47 DOI: 10.3389/fgene.2014.00047 [DOI] [PMC free article] [PubMed] [Google Scholar]
- El-Hellani A, El-Hage R, Baalbaki R, Salman R, Talih S, Shihadeh A & Saliba NA (2015). Free-base and protonated nicotine in electronic cigarette liquids and aerosols. Chemical Research in Toxicology, 28(8), 1532–1537. DOI: 10.1021/acs.chemrestox.5b00107 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fagan P, Pokhrel P, Herzog TA, Moolchan ET, Cassel KD, Franke AA, …& Addictive Carcinogens Workgroup (2017). Sugar and aldehyde content in flavored electronic cigarette liquids. Nicotine & Tobacco Research. Advance online publication DOI: 10.1093/ntr/ntx234 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fagerström KO, Hughes JR, & Callas PW (2002). Long-term effects of the Eclipse cigarette substitute and the nicotine inhaler in smokers not interested in quitting. Nicotine & Tobacco Research, 4(Suppl_2), S141–S145. DOI: 10.1080/1462220021000032771 [DOI] [PubMed] [Google Scholar]
- Family Smoking Prevention and Tobacco Control Act, 2009, (FDA), 111–31, (USA). [Google Scholar]
- Farsalinos KE, Spyrou A, Stefopoulos C, Tsimopoulou K, Kourkoveli P, Tsiapras D, … & Voudris V (2015). Nicotine absorption from electronic cigarette use: comparison between experienced consumers (vapers) and naïve users (smokers). Scientific Reports, 5, 11269 DOI: 10.1038/srep11269 [DOI] [PMC free article] [PubMed] [Google Scholar]
- FDA Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, 21 C.F.R. § 1100, 1140, and 1143, (2016). [Google Scholar]
- Foulds J, Veldheer S, Yingst J, Hrabovsky S, Wilson SJ, Nichols TT & Eissenberg T (2015). Development of a questionnaire for assessing dependence on electronic cigarettes among a large sample of ex-smoking e-cigarette users. Nicotine & Tobacco Research, 17(2), 186–192. DOI: 10.1093/ntr/ntu204 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hajek P (2014). Electronic cigarettes have a potential for huge public health benefit. BMC Medicine, 12(1), 225 10.1186/s12916-014-0225-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hall KL, Feng AX, Moser RP, Stokols D, & Taylor BK (2008). Moving the science of team science forward: collaboration and creativity. American Journal of Preventive Medicine, 35(2), S243–S249. DOI: 10.1016/j.amepre.2008.05.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hatsukami DK, Giovino GA, Eissenberg T, Clark PI, Lawrence D & Leischow S (2005). Methods to assess potential reduced exposure products. Nicotine & Tobacco Research, 7(6), 827–44. DOI: 10.1080/14622200500266015 [DOI] [PubMed] [Google Scholar]
- Hatsukami DK, Lemmonds C, Zhang Y, Murphy SE, Le C, Carmella SG, & Hecht SS (2004). Evaluation of carcinogen exposure in people who used “reduced exposure” tobacco products. Journal of the National Cancer Institute, 96(11), 844–852. DOI: 10.1093/jnci/djh163 [DOI] [PubMed] [Google Scholar]
- Hobkirk AL, Nichols TT, Foulds J, Yingst JM, Veldheer S, Hrabovsky S … & Wilson SJ (2018). Changes in resting state functional brain connectivity and withdrawal symptoms are associated with acute electronic cigarette use. Brain Research Bulletin 138, 56–63. DOI: 10.1016/j.brainresbull.2017.05.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hiler M, Breland A, Spindle T, Maloney S, Lipato T, Karaoghlanian N & Eissenberg T (2017). Electronic cigarette user plasma nicotine concentration, puff topography, heart rate, and subjective effects: Influence of liquid nicotine concentration and user experience. Experimental and Clinical Psychopharmacology, 25(5), 380–392. DOI: 10.1037/pha0000140 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hua M, Yip H, Talbot P. (2013) Mining data on usage of electronic nicotine delivery systems (ENDS) from YouTube videos. Tobacco Control 22(2), 103–106. DOI: 10.1136/tobaccocontrol-2011-050226 [DOI] [PubMed] [Google Scholar]
- Hughes JR, & Keely JP (2004). The effect of a novel smoking system—Accord—on ongoing smoking and toxin exposure. Nicotine & Tobacco Research, 6(6), 1021–1027. DOI: 10.1080/14622200412331296011 [DOI] [PubMed] [Google Scholar]
- Husari A, Shihadeh A, Talih S, Hashem Y, El Sabban M & Zaatari G (2016). Acute exposure to electronic and combustible cigarette aerosols: Effects in an animal model and in human alveolar cells. Nicotine & Tobacco Research, 18(5), 613–619. DOI: 10.1093/ntr/ntv169 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Institute of Medicine (US). Committee on Scientific Standards for Studies on Modified Risk Tobacco Products. (2012). Scientific Standards for Studies on Modified Risk Tobacco Products National Academies Press. [Google Scholar]
- Jenssen BP, Walley SC, & McGrath-Morrow SA (2018). Heat-not-burn tobacco products: Tobacco industry claims no substitute for science. Pediatrics, 141(1), e20172383 DOI: 10.1542/peds.2017-2383 [DOI] [PubMed] [Google Scholar]
- Kozlowski LT, & O’Connor RJ (2002). Cigarette filter ventilation is a defective design because of misleading taste, bigger puffs, and blocked vents. Tobacco Control, 11(suppl 1), i40–i50. DOI: 10.1136/tc.11.suppl_1.i40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lopez AA, Cobb CO, Yingst JM, Veldheer S, Hrabovsky S, Yen MS,. …& Eissenberg T (2016). A transdisciplinary model to inform randomized clinical trial methods for electronic cigarette evaluation. BMC Public Health, 16(1), 217 DOI: 10.1186/s12889-016-2792-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lopez AA, Hiler M, Maloney S, Eissenberg T & Breland AB (2016). Expanding clinical laboratory tobacco product evaluation methods to loose-leaf tobacco vaporizers. Drug & Alcohol Dependence, 169, 33–40. DOI: 10.1016/j.drugalcdep.2016.10.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Malek N, Nakkash R, Talih S, Lotfi T, Salman R, Karaoghlanian N, El-Hage R, … Shihadeh A (in press). A transdisciplinary approach to understanding characteristics of electronic cigarettes. Tobacco Regulatory Science
- Maloney SF, Soule EK, Palafox S, McFadden K, Guy MG, Eissenberg T, & Fagan P A longitudinal analysis of electronic cigarette forum participation. (in press). Addictive Behaviors [DOI] [PMC free article] [PubMed]
- Malson JL, Lee EM, Murty R, Moolchan ET, & Pickworth WB (2003). Clove cigarette smoking: biochemical, physiological, and subjective effects. Pharmacology Biochemistry and Behavior, 74(3), 739–745. DOI: 10.1016/S0091-3057(02)01076-6 [DOI] [PubMed] [Google Scholar]
- Martin CK, Correa JB, Han H, Allen HR, Rood JC, Champagne CM, … & Bray GA (2012). Validity of the remote food photography method (RFPM) for estimating energy and nutrient intake in near real‐time. Obesity, 20(4), 891–899. DOI: 10.1038/oby.2011.344 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McNeill A, Brose LS, Calder R, Bauld L, & Robson D (2018). Evidence review of e-cigarettes and heated tobacco products 2018. A report commissioned by Public Health England London: Public Health England, 6. [Google Scholar]
- Nash JM (2008). Transdisciplinary training: key components and prerequisites for success. American Journal of Preventive Medicine, 35(2), S133–S140. DOI: 10.1016/j.amepre.2008.05.004 [DOI] [PubMed] [Google Scholar]
- National Academies of Sciences, Engineering, and Medicine. (2018). Public health consequences of e-cigarettes Washington, DC: The National Academies Press. doi: 10.17226/24952. [DOI] [PubMed] [Google Scholar]
- National Cancer Institute. Risks associated with smoking cigarettes with low machine-measured yields of tar and nicotine. Smoking and Tobacco Control Monograph No 13 Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute, NIH Pub. No. 02–5074, October 2001. [Google Scholar]
- Pisinger C (2014). Why public health people are more worried than excited over e-cigarettes. BMC medicine, 12(1), 226.DOI: 0.1186/s12916-014-0226-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rosas SR, & Kane M (2012). Quality and rigor of the concept mapping methodology: A pooled study analysis. Evaluation and Program Planning, 35(2), 236–245. DOI: 10.1016/j.evalprogplan.2011.10.003 [DOI] [PubMed] [Google Scholar]
- Ramôa CP, Hiler MM, Spindle TR, Lopez AA, Lipato T, Karaoughlanian N, … Eissenberg T (2016). Electronic cigarette nicotine delivery can exceed that of combustible cigarettes: A preliminary report. Tobacco Control, 25(e1), e6–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ramôa CP, Shihadeh A, Salman R & Eissenberg T (2016). Group waterpipe tobacco smoking increases smoke toxicant concentration. Nicotine & Tobacco Research, 18(5), 770–776. DOI: 10.1093/ntr/ntv271 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shihadeh A, Antonios C, & Azar S (2005). A portable, low-resistance puff topography instrument for pulsating, high-flow smoking devices. Behavior Research Methods, 37(1), 186–191. [DOI] [PubMed] [Google Scholar]
- Shihadeh AL, & Eissenberg TE (2011). Significance of smoking machine toxicant yields to blood-level exposure in waterpipe tobacco smokers. Cancer Epidemiology and Prevention Biomarkers, 20(11), 2457–60. DOI: 10.1158/1055-9965.EPI-11-0586 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shihadeh A & Eissenberg T (2015). Electronic cigarette effectiveness and abuse liability: predicting and regulating nicotine flux. Nicotine & Tobacco Research, 17(2), 158–162. DOI: 10.1093/ntr/ntu175 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Soneji S, Barrington-Trimis JL, Wills TA, Leventhal AM, Unger JB, Gibson LA, …& Sargent JD (2017). Association between initial use of e-cigarettes and subsequent cigarette smoking among adolescents and young adults: A systematic review and meta-analysis. JAMA Pediatrics, 171(8), 788–797. DOI: 10.1001/jamapediatrics.2017.1488 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Soussy S, El-Hellani A, Baalbaki R, Salman R, Shihadeh A & Saliba NA (2016). Detection of 5-hydroxymethylfurfural and furfural in the aerosol of electronic cigarettes. Tobacco Control, 25(Issue Suppl 2), ii88–ii93. DOI: 10.1136/tobaccocontrol-2016-053220 [DOI] [PubMed] [Google Scholar]
- Spindle TR, Breland AB, Karaoghlanian NV, Shihadeh AL & Eissenberg T (2015). Preliminary results of an examination of electronic cigarette user puff topography: The effect of a mouthpiece-based topography measurement device on plasma nicotine and subjective effects. Nicotine & Tobacco Research, 17(2), 142–149. DOI: 10.1093/ntr/ntu186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spindle TR, Hiler MM, Breland AB, Karaoghlanian NV, Shihadeh AL & Eissenberg T (2017). The influence of a mouthpiece-based topography measurement device on electronic cigarette user’s plasma nicotine concentration, heart rate, and subjective effects under directed and ad libitum use conditions. Nicotine & Tobacco Research, 19(4), 469–476. DOI: 10.1093/ntr/ntw174 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spindle TR, Hiler MM, Cooke ME, Eissenberg T, Kendler KS & Dick DM (2017). Electronic cigarette use and uptake of cigarette smoking: a longitudinal examination of US college students. Addictive Behaviors, 67, 66–72. DOI: 10.1016/j.addbeh.2016.12.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Talih S, Balhas Z, Eissenberg T, Salman R, Karaoghlanian N, El Hellani A, … Shihadeh A (2015). Effects of user puff topography, device voltage, and liquid nicotine concentration on electronic cigarette nicotine yield: Measurements and model predictions. Nicotine & Tobacco Research, 17(2), 150–157. DOI: 10.1093/ntr/ntu174 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Talih S, Balhas Z, Salman R, El-Hage R, Karaoghlanian N, El-Hellani A, … Shihadeh A (2017). Transport phenomena governing nicotine emissions from electronic cigarettes: Model formulation and experimental investigation. Aerosol Science and Technology, 51(1), 1–11. DOI: 10.1080/02786826.2016.1257853 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Talih S, Salman R, Karaoghlanian N, El-Hellani A, Saliba N, Eissenberg T & Shihadeh A (2017). “Juice Monsters”: Sub-ohm vaping and toxic volatile aldehyde emissions. Chemical Research in Toxicology, 30(10), 1791–1793. DOI: 10.1021/acs.chemrestox.7b00212 [DOI] [PubMed] [Google Scholar]
- Trochim WMK, Stillman FA, Clark PI, & Schmitt CL (2003). Development of a model of the tobacco industry’s interference with tobacco control programmes. Tobacco Control, 12(2), 140–147. DOI: 10.1136/tc.12.2.140 [DOI] [PMC free article] [PubMed] [Google Scholar]
- US Department of Health and Human Services. (2014). The health consequences of smoking—50 years of progress: a report of the Surgeon General Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, 17. [Google Scholar]
- US Department of Health and Human Services. (2016). E-cigarette Use Among Youth and Young Adults: A Report of the Surgeon General Atlanta, GA: USDHHS, CDC, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health. [Google Scholar]
- Vansickel AR, Cobb CO, Weaver MF, & Eissenberg TE (2010). A clinical laboratory model for evaluating the acute effects of electronic “cigarettes”: nicotine delivery profile and cardiovascular and subjective effects. Cancer Epidemiology and Prevention Biomarkers, 19(8), 1945–1953. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wagener TL, Floyd EL, Stepanov I, Driskill LM, Frank SG, Meier E, … & Queimado L (2017). Have combustible cigarettes met their match? The nicotine delivery profiles and harmful constituent exposures of second-generation and third-generation electronic cigarette users. Tobacco Control, 26 e23–e28. DOI: 10.1136/tobaccocontrol-2016-05304 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wayne GF, & Carpenter CM (2009). Tobacco industry manipulation of nicotine dosing. In Nicotine Psychopharmacology (pp. 457–485). Springer, Berlin, Heidelberg: DOI: 10.1007/978-3-540-69248-5_16 [DOI] [PubMed] [Google Scholar]
- Zacny JP & Stitzer ML (1988). Cigarette brand-switching: effects on smoke exposure and smoking behavior. Journal of Pharmacology and Experimental Therapeutics, 246(2), 619–627. [PubMed] [Google Scholar]
