Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 1.
Published in final edited form as: Psychopharmacology (Berl). 2019 Nov 26;237(2):491–502. doi: 10.1007/s00213-019-05386-x

Effect of E-Cigarette Flavors on Nicotine Delivery and Puffing Topography: Results from a Randomized Clinical Trial of Daily Smokers

Natalie Voos 1, Danielle Smith 1,2, Lisa Kaiser 1, Martin C Mahoney 1, Clara M Bradizza 3, Lynn T Kozlowski 2, Neal L Benowitz 4, Richard J O’Connor 1, Maciej L Goniewicz 1
PMCID: PMC7691130  NIHMSID: NIHMS1647553  PMID: 31773209

Abstract

Rationale:

There is limited understanding regarding how various e-cigarette flavorings may influence the behavior of non-regular e-cigarette users who are regular cigarette smokers.

Objectives:

To assess differences in nicotine delivery, puffing topography, subjective effects, and user satisfaction from different flavored e-liquids.

Methods:

Eighteen daily smokers (average age 44.1±7.0; 9 males; average CPD 13.0±5.8) smoked their tobacco cigarettes during an initial visit and returned five times to try an e-cigarette (eGo type) refilled with a nicotine solution (24 mg/ml) of five different flavors: cherry, tobacco, espresso, menthol, and vanilla (randomized order). Assessments at each visit included puffing topography, blood samples for nicotine analysis, and subjective reports of nicotine effects and flavor satisfaction.

Results:

Vaping different flavors resulted in different levels of plasma nicotine. The flavor producing the highest plasma nicotine concentration (Cmax) was cherry (median 21.2 ng/ml), which was not significantly different than nicotine delivery from a combustible cigarette (29.2 ng/ml, p>.05). Vanilla e-liquid produced the lowest Cmax (9.7 ng/ml), and participants tended to puff less frequently on vanilla compared to tobacco flavor (p=.013). Flavors did not differ significantly in the speed of nicotine delivery (Tmax). During controlled use, puff duration for all flavors was significantly longer than a combustible cigarette (p<0.05). After controlling for nicotine delivery, significant differences in flavor enjoyment were detected. Menthol flavored e-liquid was rated as more enjoyable than vanilla and tobacco flavored e-liquids (p<0.05).

Conclusions:

Flavors tested in this study yielded different patterns of nicotine delivery and led to differences in reduction in smoking urges.

INTRODUCTION

Flavored e-liquids are commonly used in electronic cigarettes (e-cigarettes), with 64.6% of adult vapers in the United States (U.S.) reporting use of an e-liquid with a flavor other than tobacco within the past 30 days (Schneller et al., 2018). As of 2014, there were over 7,500 unique e-liquid flavors available for sale (Zhu et al., 2014), with increases in market share of menthol and “other assorted” e-liquid flavors (compared with tobacco flavored products) occurring between 2010 and 2016 (Cantrell et al., 2018). A recent analysis using nationally-representative U.S. data showed that the most common e-liquid flavor categories among adult e-cigarette users were menthol/mint (37.4%), fruit (31.2%), and candy/other sweets (16.2%) (Schneller et al., 2018). E-liquid flavorings can include potentially harmful toxicants (Kosmider et al., 2016; Farsalinos et al., 2014; Behar et al., 2014), produce toxic aldehydes upon thermal decomposition (Khlystov and Samburova, 2016), and increase the inhalation toxicity of e-cigarettes in in vitro models (Leigh et al., 2016). Nonetheless, a significant decrease in exposure to multiple known toxicants and carcinogens is associated with completely substituting e-cigarettes for combusted cigarettes (Goniewicz et al., 2017; Goniewicz et al., 2018; Shahab et al., 2017).

The marketing of flavored e-liquids is controversial. Flavored e-liquids may appeal to adolescents and young adults, which may increase uptake of e-cigarette use (Kong et al., 2015). However, many adult smokers who use an e-cigarette do so to either cut down or completely stop their cigarette smoking, and in a recent study smokers preferred fruit flavored e-liquids (compared to tobacco flavor) for smoking cessation (Harrel et al, 2017; Shiffman et al., 2015, Kalkhoran et al, 2017; Richardson et al., 2014, Hajek et al., 2019). It is important to understand how combustible cigarette users who are relatively naïve to e-cigarettes (not regular users) feel about different flavorings and perceive their effects, as this may help with transitions toward more established e-cigarette use. While a recent study showed that nicotine intake, subjective effects, and subjective liking varied across different e-cigarette devices, it is not presently well understood whether and how e-liquid flavors affect the experiences, perceptions, and subjective effects among regular cigarette smokers (Voos et al., 2019).

A recent study by St. Helen and colleagues examined the impact of e-liquid flavor on nicotine pharmacokinetics (St. Helen et al., 2017) and vaping behaviors (St. Helen et al., 2018) in experienced e-cigarette users. Those findings suggested that e-liquid flavors may influence nicotine delivery, likely due to differences in pH that may have a physiological effect on nicotine absorption. In addition, the second paper analyzed vaping behaviors (i.e. puff duration, count, clustering, and inter-puff interval) during ad lib e-cigarette use, and concluded that vaping behavior changes across different flavors and can influence nicotine intake. St. Helen and colleagues determined that further research is needed to understand the connection between puffing behavior changes, pH, subjective liking, and nicotine effects.

Our randomized pilot trial of daily smokers examined how regular cigarette smokers responded pharmacologically and behaviorally to different e-liquid flavors. We aimed to assess: 1) plasma nicotine levels after vaping different e-liquid flavors; 2) puffing topography when vaping different e-liquid flavors; and 3) subjective effects and satisfaction of different e-liquid flavors.

MATERIALS AND METHODS

Subjects

This randomized within-subject pilot trial recruited 18 daily, regular adult cigarette smokers between May 2016 and February 2018 using advertisements in local newspapers and websites (e.g. Craigslist, Buffalo Healthy Living magazine). Initial eligibility was assessed via phone screening, and final eligibility was confirmed at the screening visit. Eligible subjects were between the ages of 18 and 55, smoked ≥ 5 CPD, had nicotine dependence assessed as ≥ 2 on the Fagerstrom Test of Nicotine Dependence (FTND) (Heatherton et al., 1991), were healthy, reported no past year history of alcohol dependence, had an expired carbon monoxide (CO) breath test ≥ 6ppm, and had negative urine tests for illicit drug use (including marijuana) and pregnancy. There was no exclusion criteria for past e-cigarette use, however all participants were current daily cigarette smokers. Baseline flavor preferences were not assessed.

Flavors

Five e-liquid flavors were tested in this study: 1) cherry; 2) classic tobacco; 3) espresso; 4) menthol; and 5) vanilla. All flavored e-liquids were purchased from House of Vapor (Buffalo, NY). Each flavor was purchased from one batch, and all e-liquids were stored in a dark fridge at 4°C to ensure stability. All e-liquids had a listed nicotine concentration of 24 mg/ml, which were measured in laboratory before administering to study participants (range: 28.3 to 29.9 mg/ml). The SuperCig Automatic eGo 510 Battery 910mAh was used at each visit, and the CE4 clearomizer was replaced for each flavor (device delivered 4.1 volts and atomizer had 3.0 ohms of resistance).

Study Design

A complete schema for the study design can be seen in supplementary Figure 1. Participants attended seven sessions (visits), each one week apart. At visit 1, participants provided informed consent and eligibility was determined. Participants were randomized to the order of the five flavors by the study statistician using the Williams Design, a special case orthogonal latin squares design (Wang et al., 2009), generated using SAS v9.3 (Cary, NC). Neither participant nor study staff were blinded to flavor condition.

At visit 2, participants were asked to smoke their preferred brand of combustible cigarette ad lib (one cigarette was smoked, time period determined by participant). Video was collected for puffing behavior analysis. Throughout this visit, participants completed questionnaires pertaining to the subjective effects of nicotine delivered by the combustible cigarette, and blood samples were collected to assess the individual’s nicotine pharmacokinetic profiles from one combustible cigarette. Pharmacokinetics for the ad lib combustible smoking session were compared to the controlled session for each flavor. Participants did not undergo a controlled puffing session for the combustible cigarette. After visit 2, participants were given their first randomized flavor and instructed to practice with the flavor and device over the course of the week between visit 2 and 3 (this was done for each subsequent flavor as well) Participants were given the device with the tank pre-filled by the researchers.

During visits 3 to 7, participants underwent a controlled puffing session using the assigned flavor for 10 mins, puffing every 30 sec (total of 20 puffs). Throughout this visit, participants completed the subjective effect questionnaires and blood samples were taken. Video was collected during this session for puffing behavior analysis. A flavor satisfaction questionnaire was administered during this visit, following the puffing session.

During visits 2 through 7, participants were asked to abstain from smoking for at least 8 hrs prior to each visit, confirmed by an expired CO < 8 ppm to control for withdrawal and craving (any baseline nicotine levels were corrected for in pharmacokinetic analyses). Participants were not instructed to modify combustible cigarette use during the weeks between study visits. Participants were compensated $40 for their first visit, with a $10 increase in payment for each subsequent completed visit. Participants completing all study sessions received a $50 bonus payment. The study protocol was approved by the Institutional Review Board at Roswell Park Comprehensive Cancer Center.

Pharmacokinetic Analysis

Venous blood samples (4 ml) were collected to measure nicotine using a butterfly needle before and at 2, 4, 5, 6, 8, 10, 13, 15, 20, 30, 45, 60, 90, and 120 mins after the onset of use of each product. Plasma nicotine concentration was determined at the Clinical Pharmacology Laboratory at the University of California, San Francisco, using GC–MS/MS (Jacob et al., 1991) which was modified for tandem mass spectrometry for improved sensitivity (St. Helen et al., 2018). The limit of quantitation (LOQ) was 1.0 ng/mL. Any time points at which plasma nicotine concentration was below the LOQ were replaced with LOQ/2. To analyze changes in plasma nicotine attributable only to study product use, all nicotine concentrations were corrected for baseline using subtraction of a projected level based on log linear decline (Ct=C0e−Kt, K calculated using t1/2=.693/K, t1/2= 120 min)(Benowitz et al., 2009). Pharmacokinetic parameters were estimated using PCModfit for Excel v. 6.0 (Allen, 1990). The following parameters were estimated using a non-compartmental model and trapezoidal rule: maximum concentration of nicotine in plasma (Cmax), time to maximum concentration of nicotine in plasma (Tmax), and the area under the plasma concentration - time curve to 10 mins and 120 mins (AUC 0→10 min, AUC0→120 min). AUC0→10 min was measured to assess the rate of nicotine delivery over the 10 minute controlled puffing session. Tmax was determined from the start of the puffing session.

Puff Topography

Video was taken of the entire visit with the video positioned so the participant was against a dark background and aerosol could be seen when participants exhaled from the device. Video collected during the controlled puffing session was analyzed for puff duration, defined as the time the e-cigarette was placed in the mouth and the mouth was closed to the time when the e-cigarette was removed. Windows Movie Maker was used to analyze video at 29 frames per second. Video was used because topography devices have been shown to impact smoking experience (Ross and Juliano, 2016), and measurements of smoking topography exhibit little to no difference when using topography devices versus video analysis (Blank et al., 2009). One researcher analyzed the video data, and a second researcher provided quality control for this analysis by examining randomly chosen video clips and comparing results to the first analysis.

Subjective Measures

During visit 2 to 7, nicotine withdrawal symptoms were assessed using an adapted Minnesota Nicotine Withdrawal Scale (MNWS) (Hughes and Hatsukami, 1986) before product use (to control for baseline symptoms), and at 3, 15, and 110 mins after product use. The MNWS assesses five DSM-IV criteria for tobacco withdrawal (e.g. “Depressed,” “Irritable,” “Restless,” “Hungry,” and “Poor Concentration”) on a five-point Likert scale from “not at all” to “extremely.” The total score was assessed for this scale (Etter and Hughes, 2006). Withdrawal symptoms were analyzed by subtracting the MNWS total score prior to device use from all other time points. The score prior to use was considered the baseline, and all subsequent scores demonstrate a reduction in withdrawal symptoms from baseline.

Craving to smoke a combustible cigarette was measured using the Questionnaire of Smoking Urges-Brief (QSU-B) at 3, 15, and 110 mins after the controlled puffing session. This 10-item measure asked participants to score each statement from 0 (strongly agree) to 100 (strongly disagree) (e.g. “I have a desire for a cigarette right now”). This measure is used as a reliable global measure of craving (Cox et al., 2001).

Subjective responses to the effects of nicotine were measured using the Drug Effect Questionnaire (DEQ-5) at 1, 3, 5, 7, 10, 15, 20, 35, 50, 80, and 110 mins after the controlled puffing session. This five-item measure asked participants to indicate their responses to questions by placing a mark between the response anchors (“not at all” to “extremely”). Each DEQ-5 item is analyzed as a distinct construct, and has been found to be reflective of pharmacologically-induced effects (Morean et al., 2013).

A 12-item adapted product evaluation scale (Hatsukami et al., 2012) was used to assess satisfaction with and helpfulness of each flavor following use. Validity of this scale was supported by a significant relationship between subjective responses and product choice (Hatsukami et al., 2012). Participants were asked to rate each flavor on a scale from 0–100 on how satisfying, enjoyable, and dangerous to health the product was compared to a combustible cigarette. On this scale, a score of 50 indicated that the flavor was the same as the participant’s combustible cigarette. Additionally, participants were asked to rate each product on a Likert scale on measures such as how good the flavor tasted and how difficult the flavor was to puff on.

All measurement scales can be seen in Supplemental Tables 1–4.

Statistical Analysis

Sample size was estimated to detect an AUC difference at a two-sided alpha of .05, provided the true difference between the dosing regimens is 700 ng/mL × min, and the assumption that the within-patient standard deviation of response is 500 ng/mL × min (Benowitz et al., 2006). All analyses were performed using Stata 14.2 (Stata Corporation, College Station, TX). Descriptive statistics were used to characterize the study sample. Pharmacokinetic variables (Cmax, AUC0→120, AUC AUC0→10) and puff topography variables (puff duration, puff volume, puff flow, and inter-puff interval) were approximately log-normally distributed, and were log-transformed (using the natural log) for analysis. Due to correlated nature of the data and the need to account for missing pharmacokinetic data on select cases, (Holden et al., 2008) linear mixed models were used for analysis. The base model consisted of e-liquid flavor as the primary independent variable, and a lag variable to account for potential carryover effects from previous sessions (Jones and Kenward, 2014). The lag variable was used to avoid issues of power that are raised when many co-variates are used in a low sample size study. This variable accounted for the effect of both explanatory variable of interest (flavor used) and the past explanatory variable (previous flavor used) on the outcome being looked at. Models specified a random intercept for each participant, and were fit using restricted maximum likelihood estimation. Additional covariates (puff duration for pharmacokinetic outcomes, and Cmax for all other outcomes) were explored based upon the outcome under examination. Due to the small sample size and exploratory nature of this study, (Chen et al., 2017) all p-values presented reflect unadjusted comparisons, which were considered statistically significant at p<0.05. To examine potential effects of multiple testing on our results, Hochberg adjusted alpha values are provided alongside p-values in supplementary material. Mann-Whitney U tests were conducted to explore any potential differences between flavors with a pH<9 and those with a pH >9 on measures of pharmacokinetics and puff topography (pH of 9 was chosen as the cutoff due to the difference in amount of nicotine in the free-base form above and below that point (El-Hellani et al., 2015)).

We also conducted exploratory analyses to determine if satisfaction survey measures differed between participants who use mentholated versus non-mentholated combustible cigarettes using the Mann-Whitney U test. Satisfaction measures were analyzed by comparing all scores from menthol users against non-mentholated users. Visits where participants used the menthol flavored e-liquid were analyzed for differences between mentholated and non-mentholated smokers for measures of pharmacokinetic, puff topography, and subjective effects.

RESULTS

Participant Demographics

A flow diagram documenting how the study sample size was attained and reasons for attrition is shown in Figure 1. Overall, 27 subjects consented to take part in the study, of whom 18 completed all visits. The final completing sample consisted of 9 male and 9 female smokers with an average age of 44.1±7.0 years (mean±SD), FTND =5.2, and CPD =13.0. At baseline, the average CO level was 14.9±7.8 ppm. Of the 18 participants who completed the study, 11 used a mentholated combustible cigarette and 15 (83.3%) had prior experience with an e-cigarette. For those who had previously used an e-cigarette, 86.7% had last used an e-cigarette “a few months ago” to “more than a year ago.” Additionally, 92.4% of participants who had used an e-cigarette in the past used it for a duration of “a few days” to “a few months.” Demographics for those who consented, but did not complete the study and those who completed the study did not statistically differ (supplemental Table 5).

Figure 1.

Figure 1.

Open in a new tab

Participant flow chart.

Nicotine Delivery

A graph of plasma nicotine profiles for each flavor can be seen in Figure 2. The cherry flavored e-liquid delivered the highest Cmax (median=21.2 ng/ml), while vanilla delivered the lowest Cmax (median=9.73 ng/ml). AUC0 →10 was highest for cherry (median 85.2 ng/ml/min) and lowest for vanilla (median 50.5 ng/ml/min), and AUC0 →120 was highest for menthol (median 314.4 ng/ml/min) and lowest for tobacco flavor (median 172.0 ng/ml/min). The median Tmax for each flavor was 10 minutes. A summary of pharmacokinetic measures can be seen in Table 1a, and a summary for the linear mixed models can be seen in Table 1b. Findings from the base model (Model 1) indicated that Cmax for cherry was not significantly different than that of the participant’s regular combustible cigarette (p=.0887) but was significantly greater than Cmax for espresso. Puff duration was added as a covariate because it was the only puff topography measure that was determined by the participant during the controlled puffing session. When puff duration was added as a covariate to the model (Model 2) for Cmax it was both significant (p=.0160) and strengthened the relationship between all flavors and the participant’s regular combustible cigarette. When puff duration was controlled for, Cmax for cherry also became significantly different than the combustible cigarette (p=.0377). There were no differences between e-cigarette flavors for AUC0 →10 and AUC0 →120. All flavors had significantly lower averages for AUC0 →120 compared with a combustible cigarette (p<.05). In models for AUC0 →10 and AUC0 →120, puff duration was not a significant covariate (p=.0920, p=.0760). Specific values for each comparison can be viewed in supplementary Tables 6–11. pH was not significant in models for all pharmacokinetic measures when used as the only independent variable. When non-parametric t-tests were conducted, pharmacokinetic measures were not significantly different between flavors with a pH<9 and flavors with a pH>9 (p>.05).

Figure 2.

Figure 2.

Open in a new tab

Changes in plasma nicotine concentration after use of different flavored e-cigarette (nominally 24 mg/ml nicotine concentration) compared to preferred brand tobacco cigarette.

Table 1.

(A) Pharmacokinetic data from 18 smokers who used 5 different flavored e-liquids. Median (IQR). (B) Summary for linear mixed modeling of pharmacokinetic data and puff topography data.

(A)
Flavor Nicotine Content (mg/ml) pH Cmax (ng/ml) Tmax (min) AUC0→120 (ng/ml/min) AUC0→10 (ng/ml/min)
Vanilla Labeled: 24.0
Measured: 29.6		 8.53 9.73 (10.4) 10.0 (3.0) 174.3 (278.5) 50.5 (37.5)
Cherry Labeled: 24.0
Measured: 28.3		 8.28 21.2 (30.8) 10.0 (5.0) 293.0(318.0) 85.2 (106.4)
Menthol Labeled: 24.0
Measured: 29.4		 9.62 15.2 (21.2) 10.0 (3.0) 314.4 (415.0) 69.1 (104.0)
Espresso Labeled: 24.0
Measured: 28.8		 9.38 13.1 (12.5) 10.0 (3.0) 285.7 (307.8) 59.7 (64.4)
Classic Tobacco Labeled: 24.0
Measured: 29.9		 9.80 12.5 (12.5) 10.0 (3.0) 172.0 (299.9) 63.9 (62.2)
Tobacco Cigarette (reference) -- -- 29.2 (15.7) 5.0 (3.0) 702.6 (612.3) 132.2 (88.6)
(B)
Predictor Outcome Outcome
Cmax(ln) AUC0 →10 (ln) AUC0 →120 (ln) Puff Duration (ln,Video)
Model 1 (n=101) p>χ2=.0000 Model 2 (n=99) p>χ2=.0000 Model 1 (n=101) p>χ2=.0182 Model 2 (n=99) p>χ2=.0109 Model 1 (n=101) p>χ2=.0003 Model 2 (n=99) p>χ2=.0002 Model 1 (n=105) p>χ2= .0001 Model 2 (n=99) p>χ2= .0001
Intercept Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE)
3.45(.25)*** 3.08(.28)*** 4.73(.30)*** 4.41(.34)*** 6.48(.29)*** 6.14(.33)*** .578(.10)*** .266(.17)
Flavor (Tobacco Cigarette comparator)
 Vanilla −1.03(.29)*** −1.18(.31)*** −.641(.37) −.735(.39) −1.12(.36)** −1.24(.38)** .343(.11)** .415(.12)**
 Cherry −.558(.33) −.741(.36)* −.238(.41) −.366(.45) −1.44(.41)*** −1.59(.44)*** .409(.12)** .506(.13)***
 Menthol −.944(.30)** −1.05(.35)** −.673(.38) −.684(.44) −1.04(.37)** −1.11(.43)* .474(.13)*** .510(.14)***
 Espresso −1.15(.32)*** −1.32(.34)*** −.895(.40)* −1.00(.44)* −1.33(.39)** −1.46(.43)** .426(.12)** .527(.14)***
 Tobacco −1.08(.29)*** −1.23(.31)*** −.634(.37) −.739(.39) −1.32(.36)*** −1.46(.38)*** .364(.11)** .446(.13)***
Puff Duration(ln) -- .619(.26)* -- .540(.32) -- .556(.31) Cmax (ln) -- .095(.04)*
Variance components
 Intercept 557(.22) .387(.18) .662(.28) .508(.25) .623(.27) .480(.22) .105(.04) .072(.03)
 Residual .543(.09) .546(.09) .862(.14) .884(.15) .837(.13) .844(.14) .083(.01) .086(.01)
Open in a new tab

Outcome variables are log transformed using the natural log. Model 1 controls for flavor and potential carryover effects. Model 2 controls for flavor, potential carryover effects, and puff duration or Cmax (log transformed using the natural log). Bold indicates significant value.

*

p<.05,

**

p<.01,

***

p<.001.

Information criteria (AIC, BIC) for each model can be found in the supplemental material Tables 6 to 13.

Puffing Topography

During the controlled puffing sessions, puff duration (analyzed by video) was shortest for tobacco flavored e-liquid (2.81±.26 sec) and longest for menthol (3.31±.30 sec). All flavors had significantly longer puff durations than the combustible cigarette (p<.05). Findings related to puff topography can be viewed in Table 1B. The base model (model 1) showed that puff duration for all flavors was significantly longer than a combustible cigarette (p<.05) and puff duration was not significantly different between flavors (p>.05). When controlling for nicotine intake (Cmax) in model 2, the main effect of nicotine intake on puff duration emerged as significant (p=.0240), and strengthened the association between all flavors and puff duration.

Satisfaction and Subjective Effects

Menthol was rated as the most satisfying (47.2±6.6) and enjoyable (50.3±6.5) e-liquid flavor, and was significantly rated as being more satisfying than tobacco flavored e-liquid (p=.0471). When controlling for nicotine intake in model 2 (Cmax), menthol was significantly more satisfying than tobacco flavor (p=.0128) and cherry (p=.0092). Menthol was also more enjoyable than both tobacco flavor (p=.0040) and vanilla (p=.0106) when controlling for nicotine intake (Cmax). All flavors were viewed as being less dangerous than a combustible cigarette, and there were no significant differences between flavors (p>.05). In both models 1 and 2, there were no significant differences between flavors on how good they tasted (p>.05), however Cmax was a significant predictor of flavor taste (p=.006).There were significant differences on the measure of how difficult each flavor was to puff on. In the base model (model 1), Tobacco flavor (3.44±.29) was significantly harder to puff on than espresso (2.59±.3, p=.0085), menthol (2.39±.31, p=.0091), and cherry (2.17±.29, p=.0002). Vanilla was significantly harder to puff on compared to cherry (p=.0134). Cmax was not a significant covariate (p>.05). A summary of the results of linear mixed models can be seen in Table 2, and graphs for these measures can be seen in Figure 3.

Table 2.

Summary for linear mixed modeling of subjective ratings of different flavors.

Predictor Outcome
Satisfaction Enjoyment Perceived Health Risk How good flavor tasted How hard flavor was to puff from
Model 1 (n=90) p>χ2= .4044 Model 2 (n=84) p>χ2= .0106 Model 1 (n=89) p>χ2= .3327 Model 2 (n=83) p>χ2= .0032 Model 1 (n=90) p>χ2= .5800 Model 2 (n=84) p>χ2= .6369 Model 1 (n=90) p>χ2= .0483 Model 2 (n=84) p>χ2= .0007 Model 1 (n=89) p>χ2= .0207 Model 2 (n=83) p>χ2= .1235
Intercept Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE)
33.4(7.6)*** 11.9(10.5) 32.4(7.7)*** 10.2(10.3) 25.5(6.5)*** 34.3(9.2)*** 1.56(.31)*** .713(.42) 3.58(.38)*** 3.35(.55)***
Flavor (Tobacco comparator)
 Vanilla 6.82(6.3) 5.94(5.9) 3.18(6.9) 1.93(6.4) −1.98(6.0) −2.34(6.3) .040(.31) −.036(.30) −.470(.37) −.473(.38)
 Cherry .383(6.7) −2.85(6.8) 4.73(7.2) 3.07(7.2) 2.56(6.4) 6.51(7.2) .425(.33) .499(.34) −1.47(.40)*** −1.43(.43)**
 Menthol 12.8(6.5)* 15.1(6.1)* 16.6(6.9)* 18.7(6.5)** 5.29(6.1) 3.15(6.5) .057(.31) .036(.31) −.998(.38)** −.992(.39)*
 Espresso 5.37(6.7) 5.58(6.3) 4.21(7.1) 4.90(6.7) −3.91(6.3) −3.45(6.7) .275(.32) .421(.32) −1.05(.40)** −.994(.41)*
Cmax (ln) -- 8.74(3.19)** -- 9.01(3.2)** -- −3.52(2.8) -- .347(.13)** -- .087(.17)
Random Effects
 Intercept 440(176) 463(183) 371(156) 323(140) 208(93.9) 166(85.3) .368(.19) .305(.16) .462(.25) .568(.30)
  Residual 350(62.4) 290(54.6) 402(72.2) 332(63.3) 316(56.3) 334(62.7) .824(.15) .748(.14) 1.22(.22) 1.19(.23)
Open in a new tab

Model 1 controls for flavor and potential carryover effects. Model 2 controls for flavor, potential carryover effects, and Cmax (log transformed using the natural log. Outcome variables are not log adjusted. Bold indicates significant value.

*

p<.05,

**

p<.01,

***

p<.001.

Information criteria (AIC, BIC) for each model can be found in the supplemental material Tables 14 to 23.

Figure 3.

Figure 3.

Open in a new tab

Subjective rating of various flavors: (A) Satisfaction with flavors. (B) Enjoyment from using different flavors. (C) Perceived harm of flavors (Left bar indicates average for participants who do not use a mentholated combustible cigarette, right bar indicates average for participants who do use a mentholated combustible cigarette. (D) Taste of different flavor. (E) Difficulty puffing on different flavors.

*Horizontal bars indicate significant differences between flavors (p<.05, taken from pairwise comparisons when controlling for Cmax)

We also conducted exploratory analyses to determine differences between participants who use mentholated versus non-mentholated cigarettes. Sample sizes precluded our ability to use adjusted modelling approaches, but analyses showed that there were no significant differences on satisfaction, enjoyment, taste, and how difficult it was to puff; however, a statistical difference was observed for the measure of perceived health risk. Participants who smoked mentholated cigarettes viewed flavors as less dangerous to health compared with those who smoked non-mentholated cigarettes (p<.001). When added as a covariate with flavor in the model of perceived risk, there were still no significant differences between flavors (p>.05). Additionally there were no differences between menthol and non-menthol smokers on pharmacokinetic, puff topography, and subjective effect measures when using the menthol flavored e-liquid (p>.05).

Table 3 shows a summary of the linear mixed models of subjective effects of nicotine. The first question of the DEQ, which assessed subjective effects of nicotine, was analyzed. At 3 minutes post-use, the flavor that provided the greatest subjective effect of nicotine was menthol (mean score of 61.1±6.0). In the base model (model 1), only vanilla and tobacco flavor provided significantly lower subjective effects from nicotine than the combustible cigarette (p=.0121, p=.0035). At 3 minutes post-use, there was little evidence of significant differences in withdrawal symptom reduction among the flavors, or between the flavors and the combustible cigarette (p>.05). At 3 minutes post-use, participants felt the fewest smoking urges when using cherry (mean score of 384.5±69.5) compared with other flavors, and the level of smoking urges after using cherry was not significantly lower compared with a combustible cigarette (mean score of 319.1±80.8, p=.0547). Cmax was not significant in model 2 for all three subjective measures (p>.05). Supplementary Figure 2 illustrates change in DEQ, QSU-B, and MNWS scores over time following use of each flavor. Specific values for each comparison can be viewed in supplementary tables 14–29.

Table 3.

Summary for linear mixed modeling of subjective effect of different flavors.

Predictor Outcome
DEQ Nicotine (3 minutes) MNWS (3 minutes) QSU-B (3 minutes)
Model 1 (n=107) p>χ2= .0673 Model 2 (n=100) p>χ2= .0197 Model 1 (n=93) p>χ2= .6517 Model 2 (n=87) p>χ2= .4907 Model 1 (n=107) p>χ2= .0160 Model 2 (n=101) p>χ2= .0628
Intercept Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE) Estimate(SE)
68.9(6.4)*** 72.3(11.0)*** −1.60(.61)** −2.76(1.0)** 317.7(73.6)*** 346.8(110.7)**
Flavor (Tobacco Cigarette comparator)
 Vanilla −17.3(6.9)* −20.3(7.6)** .138(.59) .720(.68) 173.6(59.4)** 159.4(66.1)*
 Cherry −7.54(7.5) −3.28(8.1) −.027(.66) .247(76) 124.3(64.7) 142.9(70.5)*
 Menthol −10.7(7.1) −13.4(7.8) .388(.62) .910(.71) 198.5(61.0)** 183.6(67.2)*
 Espresso −9.89(7.4) −8.07(8.4) −.124(.65) .336(.77) 234.5(63.9)*** 225.7(72.5)*
 Tobacco −20.1(6.9)** −21.7(7.7)** .122(.60) .626(.70) 216.2(59.5)*** 209.0(66.4)*
Cmax (log) -- −.889(2.57) -- .274(.22) -- −8.49(23.6)
Random Effects
 Intercept 410(158) 339(137) 4.01(1.5) 3.52(1.4) 73122(26408) 74745(27818)
 Residual 308(49.0) 312(52.4) 1.91(.33) 2.04(.38) 22871(3639) 24086(4031)
Open in a new tab

Drug Effects Questionnaire (DEQ), Minnesota Nicotine Withdrawal Scale (MNWS), Questionnaire of Smoking Urges- Brief (QSU-B). Only the first question of the DEQ was analyzed (“Do you feel a nicotine effect right now?). Model 1 controls for flavor and potential carryover effects. Model 2 controls for flavor, potential carryover effects, and Cmax (log transformed using the natural log. Outcome variables are not log adjusted. Bold indicates significant value.

*

p<.05,

**

p<.01,

***

p<.001.

Information criteria (AIC, BIC) for each model can be found in the supplemental material Tables 24 to 29.

DISCUSSION

This is a unique study that examined changes in nicotine delivery, puff topography, and satisfaction associated with different flavors among regular smokers who were non-regular e-cigarette users. To our knowledge, this is the first study to do so among daily cigarette smokers, and we were able to explore the complex interaction between flavors and study outcomes.

Our study suggests that there is enhanced nicotine delivery from cherry and menthol flavors. Our study demonstrated that different flavors can result in varying nicotine delivery when used by daily cigarette smokers. We found that of the flavors tested, cherry yielded the highest nicotine delivery Cmax (median 21.2 ng/ml), which was not significantly different than a combustible cigarette (median 29.2 ng/ml). However, puff duration was significant when added to this model, and when controlling for puff duration during the controlled session, cherry was significantly lower than the combustible cigarette. This suggests that puff duration is variable and could influence nicotine delivery from different flavors. Nicotine Cmax was significantly higher for some flavors versus others, but AUC0 →120 was not, which suggests different sites of absorption. However, the lack of statistical differences for the AUC0 →120 measure could be related to a lack of statistical power to detect such an effect.

When examining puff topography during the controlled puffing session, puff duration was significantly longer with use of an e-cigarette compared to a combustible cigarette for all flavors, which is consistent with previous research (Hua et al., 2013; Hammond et al., 2005).

When analyzing the satisfaction survey, we attempted to determine the individual effects of flavor and nicotine on measures of satisfaction. Models for satisfaction and enjoyment were not significant with flavor alone in the model, but when Cmax was added these models were significant. When controlling for nicotine delivery, differences emerged between flavors on enjoyment indicating that degree of enjoyment from flavors is not just a product of the nicotine delivery. This is consistent with Goldenson et al. (2016) who concluded that flavor characteristics were significant in flavor appeal when controlling for nicotine. Although flavors differed on overall satisfaction and enjoyment, there were no differences between flavors on how good they tasted. However, Cmax was a significant predictor in the model for flavor taste, which suggests that taste is influenced by nicotine delivery. Cmax was not significant when modeling the difficulty of puffing on each flavor, meaning that this satisfaction measure could be explained by flavor alone. The cherry flavored e-liquid did not differ significantly from the participant’s combustible cigarette when looking at reduction in smoking urges. Interestingly, when modeling subjective nicotine effects (including smoking urges) we did not see a significant effect of Cmax, therefore other aspects of flavor contribute to these outcomes.

In general, participants rated all flavors as less harmful to their health than a combustible cigarette, and participants did not view flavors as differing in perceived risk. Interestingly, there was a significant difference in perceived harm of flavors between participants who smoke mentholated versus non-mentholated combustible cigarettes. Participants who smoke mentholated cigarettes viewed flavors as significantly less harmful to their health compared with participants who used non-mentholated cigarettes. Those who regularly use a mentholated cigarette may be more accustomed to inhaling a flavored substance, and therefore might view flavored e-liquids as less dangerous.

St. Helen et al. published two similar studies with experienced e-cigarette users. Our results concur with their conclusion that flavors affect nicotine delivery. However, we did not find differences between flavors when comparing AUC profiles. This could be due to the recruitment of inexperienced e-cigarette users in this study. It has been shown that experienced users of e-cigarettes attain higher nicotine doses (Farsalinos et al., 2015). Our results also agree with prior research indicating that puffing topography can influence nicotine intake (Talih et al., 2017); however, we did not find significant differences between flavors. Again, this could be due to the limited experience of our participants with e-cigarettes. The differences between our study and the St. Helen studies has interesting practical implications. Flavored e-liquids may draw smokers looking to quit, but these data suggest that there may be a learning curve and flavor alone may not deliver nicotine to the user, rather repeated exposure may be necessary to increase nicotine delivery. This could also be device contingent, but with the explosion in popularity of JUUL (and similar “pod mod” devices) a similar study utilizing the JUUL could provide more information on nicotine delivery (and the role that flavor plays in this) in inexperienced e-cigarette users.

St. Helen et al. (2017) suggested that pH may underlie the differences in nicotine delivery between flavors. Unfortunately, we were unable to explore pH in linear models due to collinearity in models between flavor and pH. However, exploratory analysis using non-parametric t-tests comparing flavors with a pH<9 and >9 showed no difference on all measures of pharmacokinetics and puff topography.

Our study has a number of strengths, including a comprehensive study design that allowed for the examination of associations between flavor, nicotine delivery, puffing topography, and satisfaction. This study also randomized the order in which combustible cigarette smokers tried 5 different e-cigarette flavors; participants were able to sample the flavors for 1 week prior to the study session, and participants used the flavors in a controlled session which allowed us to collect standardized pharmacokinetic data.

Our study also has several limitations, including a modest sample size, which may influence generalizability and impacted tour ability to detect small differences between flavors. Many comparisons were assessed, yet only a small percentage achieved significance. This limitation may be attributable to issues of adequate statistical power; therefore, further studies with larger sample sizes are needed. We were unable to examine the impact of pH, therefore future studies should also examine the role of pH in such contexts. Video analysis was only conducted by one researcher, so inter-rater reliability statistics cannot be analyzed. However, the present study provides a starting point for further exploration into the relationship between flavors, nicotine delivery, puff topography, and satisfaction.

Study results suggest associations between flavor and nicotine delivery, flavor and puff topography, and puff topography and nicotine delivery. Flavors in e-cigarettes appear to independently affect satisfaction and subjective effects, but flavors may also affect these indirectly by increasing nicotine delivery. While this study has a lot of data, it can be concluded that the flavors tested in this study do deliver differential amounts of nicotine and differences in enjoyment from flavors is not solely a product of its nicotine delivery ability. The complex nature of this relationship is also an important conclusion; this provides a background for further studies to more deeply explore flavor, nicotine delivery, and subjective effects. These findings can inform tobacco & nicotine regulatory science, and our understanding of how subjective and objective flavor effects may influence adoption and use of these products.

Supplementary Material

Supplementary Materials

Acknowledgements:

This research was supported in part by NIH NIDA grant R01DA037446 and NCI/FDA U54CA228110, and analytical chemistry resources grants DA012393 and S1ORR026437. Information and the views and opinions expressed in this presentation are those of the author only and do not necessarily represent the views, official policy, or position of the U.S. Department of Health and Human Services or any of its affiliated institutions or agencies

Footnotes

Conflicts of Interest: MLG has received research grant support from Pfizer and served as a member of the advisory board of Johnson & Johnson. MCM is on the speakers bureau, has served as a consultant to, and has received research support from Pfizer. NLB is a consultant to Pfizer and Achieve Life Sciences and has been a paid expert witness in litigation against tobacco companies. Other authors have no conflicts to declare.

Clinical Trial Registration: clinicaltrials.gov #NCT02575885

REFERENCES

  1. Allen GD (1990). “MODFIT: a pharmacokinetics computer program.” Biopharmaceutics & drug disposition 11(6): 477–498. [DOI] [PubMed] [Google Scholar]
  2. Behar R, Davis B, Wang Y, Bahl V, Lin S and Talbot P (2014). “Identification of toxicants in cinnamon-flavored electronic cigarette refill fluids.” Toxicology in vitro 28(2): 198–208. [DOI] [PubMed] [Google Scholar]
  3. Benowitz NL, Jacob P and Herrera B (2006). “Nicotine intake and dose response when smoking reduced–nicotine content cigarettes.” Clinical Pharmacology & Therapeutics 80(6): 703–714. [DOI] [PubMed] [Google Scholar]
  4. Benowitz NL, Hukkanen J and Jacob P (2009). Nicotine chemistry, metabolism, kinetics and biomarkers Nicotine psychopharmacology, Springer: 29–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Blank MD, Disharoon S and Eissenberg T (2009). “Comparison of methods for measurement of smoking behavior: mouthpiece-based computerized devices versus direct observation.” Nicotine & Tobacco Research 11(7): 896–903. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cantrell J, Huang J, Greenberg M, Willett J, Hair E and Vallone D (2018). “History and Current Trends in the Electronic Cigarette Retail Marketplace in the United States: 2010–2016.” Nicotine & Tobacco Research. [DOI] [PubMed] [Google Scholar]
  7. Chen S-Y, Feng Z and Yi X (2017). “A general introduction to adjustment for multiple comparisons.” Journal of thoracic disease 9(6): 1725. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cox LS, Tiffany ST and Christen AG (2001). “Evaluation of the brief questionnaire of smoking urges (QSU-brief) in laboratory and clinical settings.” Nicotine & Tobacco Research 3(1): 7–16. [DOI] [PubMed] [Google Scholar]
  9. El-Hellani A, El-Hage R, Baalbaki R, Salman R, Talih S, Shihadeh A and Saliba NA (2015). “Free-base and protonated nicotine in electronic cigarette liquids and aerosols.” Chemical research in toxicology 28(8): 1532–1537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Etter JF and Hughes JR (2006). “A comparison of the psychometric properties of three cigarette withdrawal scales.” Addiction 101(3): 362–372. [DOI] [PubMed] [Google Scholar]
  11. Farsalinos KE, Kistler KA, Gillman G and Voudris V (2014). “Evaluation of electronic cigarette liquids and aerosol for the presence of selected inhalation toxins.” Nicotine & Tobacco Research 17(2): 168–174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Farsalinos KE, Spyrou A, Stefopoulos C, Tsimopoulou K, Kourkoveli P, Tsiapras D, Kyrzopoulos S, Poulas K and Voudris V (2015). “Nicotine absorption from electronic cigarette use: comparison between experienced consumers (vapers) and naïve users (smokers).” Scientific reports 5: 11269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Goldenson NI, Kirkpatrick MG, Barrington-Trimis JL, Pang RD, McBeth JF, Pentz MA, Samet JM and Leventhal AM (2016). “Effects of sweet flavorings and nicotine on the appeal and sensory properties of e-cigarettes among young adult vapers: Application of a novel methodology.” Drug and alcohol dependence 168: 176–180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Goniewicz ML, Gawron M, Smith DM, Peng M, Jacob P and Benowitz NL (2017). “Exposure to nicotine and selected toxicants in cigarette smokers who switched to electronic cigarettes: a longitudinal within-subjects observational study.” Nicotine & Tobacco Research 19(2): 160–167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Goniewicz ML, Smith DM, Edwards KC, Blount BC, Caldwell KL, Feng J, Wang L, Christensen C, Ambrose B and Borek N (2018). “Comparison of nicotine and toxicant exposure in users of electronic cigarettes and combustible cigarettes.” JAMA network open 1(8): e185937–e185937. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hajek P, Phillips-Waller A, Przulj D, Pesola F, Myers Smith K, Bisal N, Li J, Parrott S, Sasieni P and Dawkins L (2019). “A randomized trial of e-cigarettes versus nicotine-replacement therapy.” New England Journal of Medicine 380(7): 629–637. [DOI] [PubMed] [Google Scholar]
  17. Hammond D, Fong GT, Cummings KM and Hyland A (2005). “Smoking topography, brand switching, and nicotine delivery: results from an in vivo study.” Cancer Epidemiology and Prevention Biomarkers 14(6): 1370–1375. [DOI] [PubMed] [Google Scholar]
  18. Harrell M, Weaver S, Loukas A, Creamer M, Marti C, Jackson C, Heath J, Nayak P, Perry C and Pechacek T (2017). “Flavored e-cigarette use: Characterizing youth, young adult, and adult users.” Preventive medicine reports 5: 33–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hatsukami DK, Zhang Y, O’connor RJ and Severson HH (2012). “Subjective responses to oral tobacco products: scale validation.” nicotine & tobacco research 15(7): 1259–1264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Heatherton TF, Kozlowski LT, Frecker RC and FAGERSTROM KO (1991). “The Fagerström test for nicotine dependence: a revision of the Fagerstrom Tolerance Questionnaire.” British journal of addiction 86(9): 1119–1127. [DOI] [PubMed] [Google Scholar]
  21. Helen GS, Dempsey DA, Havel CM, Jacob III P and Benowitz NL (2017). “Impact of e-liquid flavors on nicotine intake and pharmacology of e-cigarettes.” Drug and alcohol dependence 178: 391–398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Helen GS, Shahid M, Chu S and Benowitz NL (2018). “Impact of e-liquid flavors on e-cigarette vaping behavior.” Drug and alcohol dependence 189: 42–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Holden JE, Kelley K and Agarwal R (2008). “Analyzing change: a primer on multilevel models with applications to nephrology.” American journal of nephrology 28(5): 792–801. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hua M, Yip H and Talbot P (2013). “Mining data on usage of electronic nicotine delivery systems (ENDS) from YouTube videos.” Tobacco control 22(2): 103–106. [DOI] [PubMed] [Google Scholar]
  25. Hughes JR and Hatsukami D (1986). “Signs and symptoms of tobacco withdrawal.” Archives of general psychiatry 43(3): 289–294. [DOI] [PubMed] [Google Scholar]
  26. Jacob P III, Yu L, Wilson M and Benowitz NL (1991). “Selected ion monitoring method for determination of nicotine, cotinine and deuterium‐labeled analogs: absence of an isotope effect in the clearance of (S)‐ nicotine‐3′, 3′‐d2 in humans.” Biological Mass Spectrometry 20(5): 247–252. [DOI] [PubMed] [Google Scholar]
  27. Jones B and Kenward MG (2014). Design and Analysis of Cross-Over Trials, CRC Press Taylor & Francis Group. [Google Scholar]
  28. Kalkhoran S, Alvarado N, Vijayaraghavan M, Lum PJ, Yuan P and Satterfield JM (2017). “Patterns of and reasons for electronic cigarette use in primary care patients.” Journal of general internal medicine 32(10): 1122–1129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Khlystov A and Samburova V (2016). “Flavoring compounds dominate toxic aldehyde production during e-cigarette vaping.” Environmental science & technology 50(23): 13080–13085. [DOI] [PubMed] [Google Scholar]
  30. Kong G, Morean ME, Cavallo DA, et al. (2015) Reasons for Electronic Cigarette Experimentation and Discontinuation Among Adolescents and Young Adults. Nicotine Tob Res 17:847–854. doi: 10.1093/ntr/ntu257 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Leigh NJ, Lawton RI, Hershberger PA and Goniewicz ML (2016). “Flavourings significantly affect inhalation toxicity of aerosol generated from electronic nicotine delivery systems (ENDS).” Tobacco control 25(Suppl 2): ii81–ii87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Morean ME, de Wit H, King AC, Sofuoglu M, Rueger SY and O’Malley SS (2013). “The drug effects questionnaire: psychometric support across three drug types.” Psychopharmacology 227(1): 177–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Richardson A, Pearson J, Xiao H, Stalgaitis C and Vallone D (2014). “Prevalence, harm perceptions, and reasons for using noncombustible tobacco products among current and former smokers.” American Journal of Public Health 104(8): 1437–1444. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Ross KC and Juliano LM (2015). “Smoking through a topography device diminishes some of the acute rewarding effects of smoking.” Nicotine & Tobacco Research 18(5): 564–571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Shahab L, Goniewicz ML, Blount BC, Brown J, McNeill A, Alwis KU, Feng J, Wang L and West R (2017). “Nicotine, carcinogen, and toxin exposure in long-term e-cigarette and nicotine replacement therapy users: a cross-sectional study.” Annals of internal medicine 166(6): 390–400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Schneller LM, Bansal-Travers M, Goniewicz ML, McIntosh S, Ossip D and O’Connor RJ (2018). “Use of flavored electronic cigarette refill liquids among adults and youth in the US-Results from Wave 2 of the Population Assessment of Tobacco and Health Study (2014–2015).” PLoS One 13(8): e0202744. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Shiffman S, Sembower MA, Pillitteri JL, Gerlach KK and Gitchell JG (2015). “The impact of flavor descriptors on nonsmoking teens’ and adult smokers’ interest in electronic cigarettes.” Nicotine and Tobacco Research 17(10): 1255–1262. [DOI] [PubMed] [Google Scholar]
  38. Soule EK, Lopez AA, Guy MC and Cobb CO (2016). “Reasons for using flavored liquids among electronic cigarette users: A concept mapping study.” Drug and alcohol dependence 166: 168–176. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Talih S, Balhas Z, Salman R, El-Hage R, Karaoghlanian N, El-Hellani A, Baassiri M, Jaroudi E(2017). “Transport phenomena governing nicotine emissions from electronic cigarettes: model formulation and experimental investigation.” Aerosol science and technology 51(1): 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Voos N, Kaiser L, Mahoney MC, Bradizza CM, Kozlowski LT, Benowitz NL, O’Connor RJ and Goniewicz ML (2019). “Randomized within-subject trial to evaluate smokers’ initial perceptions, subjective effects and nicotine delivery across six vaporized nicotine products.” Addiction. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Wang B-S, Wang X-J and Gong L-K (2009). “The construction of a Williams design and randomization in cross-over clinical trials using SAS.” J Stat Softw 29(1): 1–10. [Google Scholar]
  42. Zhu S-H, Sun JY, Bonnevie E, Cummins SE, Gamst A, Yin L and Lee M (2014). “Four hundred and sixty brands of e-cigarettes and counting: implications for product regulation.” Tobacco control 23(suppl 3): iii3–iii9 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Materials
NIHMS1647553-supplement-Supplementary_Materials.docx (386KB, docx)

RESOURCES