Abstract
Aim
To describe systemic nicotine exposure and subjective effects of electronic cigarettes (e-cigarettes) in people who use both e-cigarettes and cigarettes (dual users), including within-subject comparisons of e-cigarette and cigarette use.
Design
Two-arm, counterbalanced crossover study. Participants used their usual brand of e-cigarette or cigarette during a standardized session in a 2-week study.
Setting
Research ward, San Francisco, California, USA.
Participants
Thirty-six healthy (8 women, 28 men) participants.
Measurements
Plasma nicotine was analyzed by gas chromatography–tandem mass spectrometry; nicotine withdrawal, urge to smoke and vape, affective states, craving, satisfaction, and psychological reward were measured by standardized questionnaires.
Findings
Compared with cigarettes, average maximum plasma nicotine concentration (Cmax) was lower with e-cigarettes [6.1±5.5 ng/ml (mean, SD) vs 20.2±11.1 ng/mL, p<0.001] and time of maximal concentration (Tmax) was longer [6.5±5.4 min vs 2.7±2.4 min, (p<0.001)]. Use of both products resulted in a reduction in the severity of withdrawal symptoms, negative affect, and urge to use either product. E-cigarettes were less rewarding and satisfying and reduced craving to a lesser degree than cigarettes. We were not able to detect any differences in withdrawal symptoms, affective states, and urge to smoke cigarettes between e-cigarette and cigarette use.
Conclusion
Systemic nicotine exposure was on average lower with single use of e-cigarettes compared with cigarettes and e-cigarettes was judged to be less satisfying and rewarding and reduced craving less than cigarettes.
Keywords: dual users, nicotine pharmacokinetics, subjective effects, e-cigarettes
INTRODUCTION
The efficiency of nicotine delivery and its potential for facilitating self-administration of electronic cigarettes (e-cigarettes) can be assessed through nicotine pharmacokinetic studies [1]. Although a majority of adults who use e-cigarettes are also cigarette smokers, i.e. they are dual users, most published pharmacokinetic studies of e-cigarettes have been conducted with e-cigarette-naïve smokers and/or experienced e-cigarette users who use e-cigarettes exclusively or primarily [1, 2]. Assessments of nicotine intake and subjective effects from e-cigarette use in dual users may inform questions of why dual users do not switch completely to e-cigarettes but instead use e-cigarettes as a supplemental source of nicotine to compensate for smoking reduction [3, 4] or to navigate smoking restrictions [5].
To the best of our knowledge, a study by Hajek and colleagues is the only published description of nicotine pharmacokinetics of commercially available e-cigarettes in dual users [6]. The study found that dual users were exposed to lower levels of nicotine from e-cigarette use compared to smoking cigarettes, and that more advanced e-cigarettes exposed users to higher levels of nicotine compared to cig-a-likes based on higher maximum plasma nicotine concentration (Cmax) and area under the plasma nicotine concentration-time curve (AUC) over a limited assessment time of 30 minutes. Subjective effects of nicotine or tobacco product administration were not presented, which limits our understanding of how these factors may interplay with nicotine pharmacokinetics to explain e-cigarette use behavior among dual users. In the present study, we describe a comprehensive assessment of pharmacokinetic/pharmacodynamic (PK/PD) effects of e-cigarette use among dual users, including: (a) a within-subject comparison of nicotine exposure and subjective effects of participants’ usual brands of e-cigarettes and cigarettes, and, (b) between-subject comparisons of nicotine exposure and subjective effects of different types of e-cigarettes.
METHODS
We conducted a 2-arm counterbalanced, crossover study over two consecutive weeks in healthy dual users of e-cigarettes and cigarettes. During each arm, participants used their usual brand of e-cigarettes or cigarettes over four days of outpatient ad libitum access followed by three days of product use in a hospital research ward. The hospital phase of each arm included a single-dose pharmacokinetic study on the first day of admission, the focus of this paper, followed by two days of ad libitum access to the assigned product to examine circadian nicotine intake, physiologic and subjective effects [7], and systemic toxicant exposure.
Participants
Thirty-six participants (8 women) recruited via Craigslist.com, Facebook, flyers, and college campus newspapers completed the study. Criteria for inclusion were: age 21 or over; healthy via medical history and a limited physical examination; smoke at least 5 cigarettes per day (CPD) over the past 30 days and use the same e-cigarette device at least once daily on 15 of the past 30 days; no intention to quit smoking or vaping over the next 3 months; and at the physical exam, have saliva cotinine and expired carbon monoxide (CO) of ≥50 ng/mL and ≥5 ppm, respectively, negative pregnancy test (if a woman), and negative urine illicit drug test, except for cannabis. The study was approved by the Institutional Review Board at the University of California San Francisco, and registered in ClinicalTrials.gov (NCT02470754). Written, informed consent was obtained from each participant and all participants were financially compensated.
Products
Participants used their usual brands of e-cigarettes and cigarettes, provided by the study. Details of the products are provided in Supplementary Table 1.
Experimental procedure
Participants arrived at the Clinical Research Center of the Zuckerberg San Francisco General Hospital between 7:00 to 8:00 AM on Day 5 of each study arm after overnight abstinence starting at 10 PM. We measured expired CO to verify abstinence from smoking (≤ 5 ppm).
At 9:00 AM, participants used the assigned e-cigarette or cigarette in a standardized protocol, taking one puff every 30 seconds [8]; puff duration was not controlled by the study. Cig-a-like and pod users took a total of 15 puffs while fixed-power and variable-power tank users took a total of 10 puffs. Based on our previous studies [8, 9], we estimated that 15 puffs from cig-a-likes/pods and about 10 puffs from tank devices would deliver similar nicotine levels as that of typical cigarettes (~1 mg) [10]. Cigarettes were smoked to completion, allowing for a more naturalistic control for comparisons of nicotine intake and subjective effects from e-cigarette use with cigarettes. We weighed the e-cigarettes before and after vaping to determine the amount of e-liquid consumed and nicotine inhaled. To quantify systemic nicotine dose from e-cigarettes, we collected the nicotine exhaled after each puff using gas traps [8, 11]. We collected blood samples before and 2, 5, 15, 30, 45, 60, 90, 120, 180, and 240 minutes after the last puff of each product through an intravenous (IV) line in the forearm.
Measures
At the screening visit, we administered questionnaires to assess demographics, smoking and e-cigarette history and tobacco dependence, including the Fagerström Test of Cigarette Dependence (FTCD) [12]. The E-cigarette Use Index, devised for this study, measures the frequency of e-cigarette use per day over the past 30 days as follows:
where, “e-cigarette times per day” is the number of times e-cigarettes are used on days that they are used, assuming each “time” consisted of around 15 puffs or lasted around 10 minutes [13] and “e-cigarette using days per month” is the number of days on which e-cigarettes were used.
The e-cigarette to cigarette use ratio = , assesses the relative frequency of use of products.
During the inpatient study, we measured nicotine withdrawal with the Minnesota Nicotine Withdrawal Scale (MNWS) [14]; Questionnaire for Smoking Urges (QSU-Brief) and the QSU-Brief modified for e-cigarettes to measure urge to smoke or vape (we administered both forms of the QSU during each arm); the Positive and Negative Affect Schedule (PANAS) to measure affective state [15]; and the modified Cigarette Evaluation Scale (mCES) to measure satisfaction, reward, aversive effects, enjoyment of sensation at the back of the throat and chest, and craving reduction after use of either product [16]. QSU Factor 1 assesses the positive reinforcement aspects of smoking or vaping and QSU Factor 2 assesses the negative reinforcing aspects of smoking or vaping [17]. We administered the MNWS, QSU, and PANAS questionnaires before and 5 minutes, 2 hours, and 4 hours after the last puff of each product while the mCES was administered at 5 minutes after the last puff.
Analytical chemistry
We measured nicotine in the 0.02 N HCl gas trap solution and in e-liquids by LC-MS/MS [8, 18]. The limit of quantitation (LOQ) was 0.5 ng/mL. We determined cotinine concentration in saliva by GC and nicotine concentration in plasma by a GC-MS/MS method [19] modified for tandem mass spectrometry for improved sensitivity. The LOQ for saliva cotinine and plasma nicotine were 10 ng/mL and 0.2 ng/mL, respectively.
Pharmacokinetic analysis
We estimated pharmacokinetic (PK) parameters from plasma nicotine concentrations using Phoenix WinNonlin 6.3 (Pharsight Corporation, Mountain View, CA). We computed Cmax, time to maximum concentration (Tmax), and AUC from 0 to 5 min (AUC0→5), 0 to 15 min (AUC0→15), 0 to 30 min (AUC0→30), 0 to 240 min (AUC0→240), and 0 to infinity (AUC0→∞) using a noncompartmental model and trapezoidal rule. To account for plasma nicotine concentrations prior to product administration, we corrected Cmax and AUCs for each day’s baseline values by subtracting the extrapolated blood nicotine level (based on log-linear decline) from the measured levels at each time point as described previously [8]. The PK-estimated dose for the session was computed as the product of average population clearance of nicotine (~1200 mL/min) and AUC(0→∞) [10]. As a measure of nicotine titration, which is the extent to which users match systemic nicotine exposure from cigarette smoking with e-cigarettes, we computed the quotient of AUC(0→∞) with e-cigarette to AUC(0→∞) with cigarette, where values closer to 1 indicate more complete titration.
Statistical analysis
The primary outcome on which we powered the study was the difference in daily nicotine exposure between e-cigarette vs cigarette use. We used within-subject variability data on plasma cotinine (mean, 200 ng/mL, CV 25%) in cigarette smokers whose cotinine had been measured on multiple occasions for estimation [20]. We estimate that with 36 participants in a 2 × 2 crossover design using ratios and a coefficient of variation of 25% in effect measure, we would have 86% power to determine significance of a ratio of 1.20 in the two conditions.
We calculated systemic retention of nicotine from e-cigarettes as described previously [8]. One pod user retained ~0% of the amount of nicotine inhaled from the pod and another pod user had minimal increase in blood nicotine levels after cigarette use. Given the small sample size of 3, we omitted pod users from the primary analyses.
For the primary analyses, we tested within-subject differences in blood nicotine levels, pharmacokinetic parameters, and subjective effects between e-cigarettes and cigarettes using mixed model analysis of variance (ANOVA). We included the order of product assignment in all models as a between-subject factor, and for models with outcomes measured multiple times during an arm, such as blood nicotine levels, an additional time variable was included. To examine whether changes in MNWS, QSU, and PANAS were different with e-cigarettes compared to cigarettes, we included a product assignment × time interaction term. Since subjective effects are influenced by both nicotine and non-nicotine factors [21], we repeated the primary analysis with plasma nicotine Cmax included as a covariate in separate models. Further, we repeated the primary analyses and included sex and FTCD as covariates since these factors have been shown to influence responses to cigarettes [22, 23]. Additional analyses included comparisons of nicotine pharmacokinetics and subjective effects across the three main types of e-cigarettes by Kruskal-Wallis nonparametric analysis.
We carried out all analyses using SAS v. 9.4 (SAS Institute, Inc., Cary, NC, USA). Statistical tests were considered significant at α<0.05 and multiple comparisons were corrected using Tukey’s method where applicable. Since the analysis was not pre-registered on a publicly available platform, the results should be considered exploratory.
RESULTS
The characteristics of participants enrolled in the study are presented in Table 1. On average, participants smoked 12.9 CPD, used e-cigarettes on 22.6 days of the past 30 days, and on days when e-cigarettes were used, they used e-cigarettes 8.1 times. Half of all participants (50%) used tobacco flavored e-cigarette liquid.
TABLE 1.
Demographic information and electronic cigarette and combustible cigarette history
| Variable | N (%) / Mean (SD) | Median (IQR) |
|---|---|---|
| Age (mean, SD) | 35.4 (11.7) | 32.5 (25 – 41.5) |
| Gender | ||
| Female (n, %) | 8 (22.2) | |
| Male (n, %) | 28 (77.8) | |
| Race | ||
| Asian (n, %) | 2 (5.6) | |
| Black (n, %) | 3 (8.3) | |
| Latino (n, %) | 4 (11.1) | |
| White (n, %) | 22 (61.1) | |
| Mixed (n, %) | 5 (13.9) | |
| Cigarettes per day (mean, SD) | 12.9 (6.4) | 10.0 (8 – 16.3) |
| Days of e-cigarette use in past 30 (mean, SD) | 22.6 (7.3) | 21.0 (18 – 30) |
| E-cigarette times per day (mean, SD) | 8.1 (7.2) | 5.0 (3 – 10) |
| E-cigarette use index | 6.5 (6.5) | 4.5 (1.9 – 10) |
| E-cigarette to cigarette use ratio | 0.57 (0.56) | 0.50 (0.16 – 0.67) |
| FTCD (mean, SD) | 4.4 (2.0) | 4.0 (3.0 – 5.25) |
| Time to first cigarette (TFC) | ||
| ≤ 5 min (n, %) | 6 (16.7) | |
| 5–30 (n, %) | 21 (58.3) | |
| 30–60 min (n, %) | 8 (22.2) | |
| > 60 min (n, %) | 1 (2.8) | |
| Time to first vape | ||
| ≤ 5 min (n, %) | 1 (2.8) | |
| 5–30 (n, %) | 12 (33.3) | |
| 30–60 min (n, %) | 4 (11.1) | |
| > 60 min (n, %) | 19 (52.8) | |
| Salivary cotinine, ng/mL (mean, SD) | 189 (92.8) | 190 (119 – 248) |
| E-cigarette type | ||
| Cig-a-like (n, %) | 12 (33.3) | |
| Fixed-power e-cigarette (n, %) | 15 (41.7) | |
| Variable-power e-cigarette (n, %) | 6 (16.7) | |
| Pod (n, %) | 3 (8.3) | |
| Flavor category | ||
| Dessert/candy | 8 (22.2) | |
| Fruit | 5 (13.9) | |
| Menthol | 5 (13.9) | |
| Tobacco | 18 (50.0) | |
Notes: SD is standard deviation; IQR is interquartile range; FTCD is Fagerström Test of Cigarette Dependence; E-cigarette times per day is the number of times e-cigarettes are used on days that they are used, assuming each “time” consisted of around 15 puffs or lasted around 10 minutes [ref [13]]; E-cigarette use index = [E-cigarette times per day × E-cigarette using days per month] ÷ 30 days; E-cigarette to cigarette use ratio = E-cigarette Use Index ÷ Cigarettes Per Day.
Nicotine pharmacokinetics: E-cigarettes vs combustible cigarettes
Although all subsequent analyses excluded pod users for reasons described before, the findings of the analyses with pod users included and those with pod users excluded were consistent. Compared to cigarette smoking, e-cigarette use resulted in lower average blood nicotine levels at all post-administration time points among all participants (minus pod users) (p<0.001) (Figure 1A). We found similar results when we grouped participants according to the types of e-cigarettes used (Figure 2 A–C). (The plasma nicotine concentration-time curves of the pod users are presented in Figure 2 D–F.)
FIGURE 1.
Average plasma nicotine concentration profile for electronic cigarettes and combustible cigarettes of all participants (A); within subject maximum plasma nicotine concentration (Cmax) when using electronic and combustible cigarettes (B); and, average plasma nicotine concentration profile for the main types of electronic cigarettes included in the study (C). Plot A, blood nicotine levels were significantly different between e-cigarettes and cigarettes at all time points. Plot B, solid black lines = cig-a-likes; solid grey line = fixed-power tanks and broken black line = variable-power tanks. Plot C, blood nicotine levels were significantly different between variable-power tanks and fixed-power tanks.
FIGURE 2.
Average plasma nicotine concentration profile for electronic cigarettes and combustible cigarettes for users of cig-a-likes (A), fixed-power tanks (B), and variable-power tanks (C). The plasma nicotine concentration profiles of the three pod users are given in plots D-F. Plots A-C: Blood nicotine levels were significantly different between e-cigarettes and cigarettes.
Considering all participants, plasma nicotine Cmax, AUCs, and the PK-predicted nicotine dose were higher with cigarettes while Tmax and the nicotine elimination half-life were longer with e-cigarettes (Table 2). Tmax was measured as the time from the last puff to Cmax for both products.
TABLE 2.
Comparison of average nicotine pharmacokinetic profile of electronic cigarettes compared to combustible cigarettes for all participants (A) and by the main types of e-cigarettes included in the study (B-D).
| Parameter | E-cigarette | Combustible cigarette | F† | p value |
|---|---|---|---|---|
| A. All participants (n = 33) | ||||
| Cmax (ng/ml) | 6.1 (5.5) | 20.2 (11.1) | 40.95 | < 0.001 |
| Tmax (min) | 6.5 (5.4) | 2.7 (2.4) | 14.64 | < 0.001 |
| Half-life (min) | 137.6 (39.3) | 121.2 (33.9) | 5.89 | 0.021 |
| AUC0→5 (ng/ml•min) | 22.2 (21.5) | 72.3 (38.4) | 43.16 | < 0.001 |
| AUC0→15 (ng/ml•min) | 72.0 (64.1) | 207.8 (103.5) | 41.48 | < 0.001 |
| AUC0→240 (ng/ml•min) | 550 (438) | 1368 (665) | 41.81 | < 0.001 |
| AUC0→∞ (ng/ml•min) | 779 (624) | 1849 (981) | 40.43 | < 0.001 |
| PK-estimated nicotine dose (mg) | 0.9 (0.7) | 2.2 (1.2) | 40.43 | < 0.001 |
| B. Users of cig-a-likes (n = 12) | ||||
| Cmax (ng/ml) | 5.6 (6.4) | 19.5 (11.5) | 13.08 | 0.004 |
| Tmax (min) | 9.3 (6.1) | 2.3 (0.9) | 16.84 | 0.002 |
| Half-life (min) | 125.1 (31.8) | 115.9 (31.2) | 0.83 | 0.381 |
| AUC0→5 (ng/ml•min) | 20.4 (25.6) | 73.9 (42.7) | 13.57 | 0.004 |
| AUC0→15 (ng/ml•min) | 67.2 (73.9) | 222.4 (123.9) | 13.57 | 0.004 |
| AUC0→240 (ng/ml•min) | 520 (390) | 1473 (861) | 14.63 | 0.003 |
| AUC0→∞ (ng/ml•min) | 718 (547) | 1915 (1116) | 13.71 | 0.004 |
| PK-estimated nicotine dose (mg) | 0.9 (0.7) | 2.3 (1.3) | 13.71 | 0.004 |
| C. Users of fixed-power e-cigarettes (n = 15) | ||||
| Cmax (ng/ml) | 4.9 (4.2) | 19.4 (11.4) | 24.83 | < 0.001 |
| Tmax (min) | 5.6 (5.1) | 3.3 (3.4) | 2.28 | 0.153 |
| Half-life (min) | 138.0 (43.0) | 116.5 (26.8) | 5.94 | 0.029 |
| AUC0→5 (ng/ml•min) | 17.5 (15.5) | 69.1 (40.6) | 25.12 | < 0.001 |
| AUC0→15 (ng/ml•min) | 55.2 (43.4) | 196.7 (103.7) | 26.98 | < 0.001 |
| AUC0→240 (ng/ml•min) | 388 (271) | 1173 (443) | 38.74 | < 0.001 |
| AUC0→∞ (ng/ml•min) | 542 (379) | 1521 (578) | 40.23 | < 0.001 |
| PK-estimated nicotine dose (mg) | 0.7 (0.5) | 1.8 (0.7) | 40.23 | < 0.001 |
| D. Users of variable-power e-cigarettes (n = 6) | ||||
| Cmax (ng/ml) | 10.0 (5.8) | 23.7 (12.4) | 5.71 | 0.060 |
| Tmax (min) | 3.5 (1.6) | 2.0 (0.0) | 5.00 | 0.076 |
| Half-life (min) | 161.5 (37.5) | 143.7 (49.5) | 0.56 | 0.500 |
| AUC0→5 (ng/ml•min) | 37.6 (21.7) | 77.1 (27.4) | 6.91 | 0.047 |
| AUC0→15 (ng/ml•min) | 123.4 (70.4) | 205.6 (62.2) | 4.15 | 0.097 |
| AUC0→240 (ng/ml•min) | 1015 (591) | 1646 (647) | 3.18 | 0.135 |
| AUC0→∞ (ng/ml•min) | 1495 (802) | 2535 (1258) | 3.33 | 0.127 |
| PK-estimated nicotine dose (mg) | 1.8 (1.0) | 3.0 (1.5) | 3.33 | 0.127 |
Notes: Degrees of freedom for analyses with: a) all participant = 1, 32; b) cig-a-like users = 1, 11; fixed-power tanks = 1, 14; and, variable-power tanks = 1, 5. Cmax is maximum plasma nicotine concentration; Tmax is time to maximum plasma nicotine concentration; AUC0→5 is area under the plasma nicotine concentration-time curve (AUC) from 0 to 5 minutes; AUC0→15 is AUC from 0 to 15 minutes; AUC0→240 is AUC from 0 to 240 minutes; AUC0→∞ is AUC from 0 to infinity; PK (pharmacokinetic)-estimated dose is the product of average population clearance of nicotine (~1200 mL/min) and AUC(0→∞) [ref [10]].
On average, plasma nicotine Cmax was 5.4 times higher with cigarettes compared to e-cigarettes (SD 4.4; median 4.3; range 0.6–20.1 times) (Figure 1B). Only three participants had higher Cmax with e-cigarettes compared to cigarettes: (1) variable-power tank user, 15.4 ng/mL vs 13.6 ng/mL; (2) variable-power tank user, 17.5 ng/mL vs 13.7 ng/mL; and, (3) cig-a-like user, 25.5 ng/mL vs 15.0 ng/mL. As a measure of the extent of nicotine titration, the average quotient of AUC(0→∞) with e-cigarette and AUC(0→∞) with cigarette of all participants was 0.49 (SD 0.53, range 0.10–2.92).
Nicotine pharmacokinetics: E-cigarette types
Comparisons of e-cigarette and cigarette use and dependence across types of e-cigarettes are shown in Table 3 (Section A) while comparisons of the pharmacokinetic profiles of the types of e-cigarettes are presented in Section B and Figure 1C. The average plasma nicotine concentrations across types of e-cigarettes were different (p=0.043), driven by differences between variable-power tank users and fixed-power tank users (Figure 1C). Titration among users of cig-a-likes [0.43 (0.36), mean (SD)], fixed-power tanks [0.39 (0.31)], and variable-power tanks [0.85 (1.03)] were similar (p=0.36).
TABLE 3.
Electronic and combustible cigarette history and nicotine pharmacokinetics by types of e-cigarettes used
| Variable | Cig-a-like | Fixed-power | Variable-power | Χ2 | p-value |
|---|---|---|---|---|---|
| n (%) | 12 (33.3) | 15 (41.7) | 6 (16.7) | 3.81 | 0.148 |
| A. Screening (mean, SD) | |||||
| Screening saliva cotinine (ng/mL) | 204 (103) | 193 (105) | 172 (58) | 0.19 | 0.909 |
| Cigarettes per day | 13.5 (6.4) | 12.3 (4.5) | 15.6 (10.9) | 0.16 | 0.925 |
| FTCD | 4.0 (1.9) | 4.6 (1.6) | 5.2 (2.9) | 1.22 | 0.544 |
| Days of e-cigarette use in past 30 | 18.2 (8.6) | 24.7 (5.9) | 24.5 (6.1) | 5.20 | 0.074 |
| E-cigarette Times Per Day | 6.7 (8.2) | 7.0 (5.0) | 13.3 (9.6) | 4.57 | 0.102 |
| E-cigarette Use Index | 5.1 (8.4) | 6.0 (4.8) | 10.0 (6.1) | 5.77 | 0.056 |
| E-cigarette to combustible cigarette use ratio | 0.4 (0.6) | 0.5 (0.5) | 0.8 (0.5) | 5.74 | 0.057 |
| B. Standardized session (mean, SD) | |||||
| Baseline plasma nicotine, e-cigarette arm (ng/mL) | 2.4 (4.1) | 1.6 (1.9) | 2.6 (2.4) | 2.25 | 0.325 |
| Baseline plasma nicotine, cigarette arm (ng/mL) | 2.3 (2.0) | 1.6 (1.5) | 3.7 (2.1) | 5.27 | 0.072 |
| E-liquid nicotine concentration (ug/mg) | 20.2 (13.4) | 12.2 (7.4) | 9.4 (3.9) | 8.43 | 0.015 |
| E-liquid consumed (mg) | 56.3 (24.5) | 88.8 (57.3) | 165.8 (85.6) | 7.17 | 0.028 |
| Amount of nicotine inhaled (mg) | 1.2 (1.1) | 1.0 (0.7) | 1.6 (0.9) | 2.49 | 0.288 |
| Amount of nicotine retained (mg) | 1.1 (0.9) | 0.9 (0.7) | 1.5 (0.9) | 2.51 | 0.285 |
| Systemic nicotine retention (%) | 97.9 (4.9) | 93.9 (15.8) | 94.2 (13.7) | 0.10 | 0.952 |
| PK-estimated nicotine dose (mg) | 0.9 (0.7) | 0.7 (0.5) | 1.8 (1.0) | 6.01 | 0.050 |
| Cmax (ng/ml) | 5.6 (6.4) | 4.9 (4.2) | 10.0 (5.8) | 4.81 | 0.090 |
| Tmax (min) | 9.2 (6.1) | 5.6 (5.1) | 3.5 (1.6) | 3.96 | 0.138 |
| Half-life (min) | 125.1 (31.8) | 138.0 (43.0) | 161.5 (37.5) | 5.33 | 0.069 |
| AUC0→5 (ng/ml•min) | 20.4 (25.6) | 17.5 (15.5) | 37.6 (21.7) | 4.61 | 0.100 |
| AUC0→15 (ng/ml•min) | 67.2 (73.9) | 55.2 (43.4) | 123.4 (70.4) | 5.11 | 0.078 |
| AUC0→240 (ng/ml•min) | 520 (390) | 388 (271) | 1015 (591) | 5.89 | 0.053 |
| AUC0→∞ (ng/ml•min) | 718 (547) | 542 (379) | 1495 (802) | 6.01 | 0.050 |
| AUC0→5 per retained dose (ng/ml•min per mg) | 19.4 (10.5) | 31.2 (35.5) | 28.6 (17.8) | 2.36 | 0.307 |
| AUC0→15 per retained dose (ng/ml•min per mg) | 65.3 (29.0) | 111.2 (161.2) | 95.4 (64.6) | 0.23 | 0.890 |
| AUC0→240 per retained dose (ng/ml•min per mg) | 528 (206) | 808 (1175) | 797 (578) | 2.14 | 0.343 |
| AUC0→∞ per retained dose (ng/ml•min per mg) | 738 (383) | 1119 (1563) | 1173 (747) | 1.99 | 0.370 |
| Nicotine titration | 0.43 (0.36) | 0.39 (0.31) | 0.85 (1.03) | 2.03 | 0.363 |
Notes: The degree of freedom for all tests is 2. FTCD is Fagerström Test of Cigarette Dependence; E-cigarette times per day is the number of times e-cigarettes are used on days that they are used, assuming each “time” consisted of around 15 puffs or lasted around 10 minutes [ref [13]]; E-cigarette use index = [E-cigarette times per day × E-cigarette using days per month] ÷ 30 days; E-cigarette to cigarette use ratio = E-cigarette Use Index ÷ Cigarettes Per Day; Cmax is maximum plasma nicotine concentration; Tmax is time to maximum plasma nicotine concentration; AUC0→5 is area under the plasma nicotine concentration-time curve (AUC) from 0 to 5 minutes; AUC0→15 is AUC from 0 to 15 minutes; AUC0→240 is AUC from 0 to 240 minutes; AUC0→∞ is AUC from 0 to infinity; PK (pharmacokinetic)-estimated dose is the product of average population clearance of nicotine (~1200 mL/min) and AUC(0→∞) [ref [10]].
Subjective effects: E-cigarettes vs combustible cigarettes
MNWS total score, PANAS negative, and QSU scores changed significantly after e-cigarette or cigarette administration (Figure 2) but we did not confirm a significant product assignment × time interaction term for these measures. Urge to vape [QSU Factor 1 (p<0.001) and Factor 2 (p=0.002)] were higher during the e-cigarette arm compared to the cigarette arm but there was no evidence of significant differences when plasma nicotine Cmax was included in the models. Inclusion of sex and tobacco dependence measures as covariates in the models did not change the outcome.
The average (and SD) of the five mCES subscales immediately after e-cigarette and cigarette use (in this order) were as follows: (1) enjoyment of sensation, 4.1 (1.5) vs 4.6 (1.6), p=0.05; (2) craving reduction, 4.2 (1.7) vs 5.6 (1.7), p<0.001; (3) satisfaction, 14.3 (4.3) vs 16.6 (3.3), p=0.001; (4) psychological reward, 19.7 (7.6) vs 23.2 (6.7), p=0.006; and (5) aversion, 5.1 (3.3) vs 5.5 (2.9), p=0.44. On inclusion of plasma nicotine Cmax in the models, we did not find evidence of significant differences between e-cigarettes and cigarettes.
Subjective effects: E-cigarettes types
During the e-cigarette arm, urge to smoke (QSU Factor 1, p=0.035 and Factor 2, p=0.009) and urge to vape (QSU Factor 2, p=0.004) were different across the types of e-cigarettes used in the study, driven by a larger decreases in these measures among variable-power tank users compared to cig-a-like and fixed-power tank users immediately after e-cigarette use. PANAS positive scores were different during the e-cigarette (p=0.003) and cigarette arms (p<0.001) across the types of e-cigarette used but this difference was driven by higher average PANAS positive scores at baseline for fixed-power tank users compared to cig-a-like and variable-power tank users.
Correlates of nicotine exposure and titration
The E-cigarette Use Index and number of times e-cigarettes are used during the day were moderately correlated with plasma nicotine Cmax during the e-cigarette arm [rho=0.33 (p=0.05) and rho=0.36 (p=0.03), respectively] but not during the cigarette arm [rho=−0.03 (p=0.86) and rho=−0.02 (p=0.89), respectively]. We could not confirm significant correlations between the degree of nicotine titration and the E-cigarette Use Index (rho=0.30, p=0.07) and number of times e-cigarettes are used during the day (rho=0.31, p=0.07). Correlations across e-cigarette types between degree of titration and E-cigarette Use Index were as follows: cig-a-like, rho=0.17, p=0.60; fixed-power, rho=0.24, p=0.38; and variable-power, rho=−0.03, p=0.96. Similarly, correlations between degree of titration and number of times e-cigarettes are used during the day were as follows: cig-a-like, rho=0.07, p=0.83; fixed-power, rho=0.25, p=0.37; and variable-power, rho=0.03, p=0.96.
DISCUSSION
While one study has described nicotine pharmacokinetics from e-cigarette use in dual users [6], our study is the first combined assessment of nicotine pharmacokinetics and subjective effects of commercially available e-cigarettes in dual users. Using their usual products in a standardized session, we show that e-cigarette use resulted in lower systemic nicotine exposure than cigarette smoking. Tmax and the elimination half-life of nicotine were significantly longer with e-cigarette use, likely due to greater oral deposition and/or swallowing of the e-cigarette aerosol compared to cigarette smoke. Relative to baseline, use of both products resulted in a reduction of the severity of withdrawal symptoms, negative affect, and urge to use either product. However, despite lower nicotine intake with e-cigarettes, changes in withdrawal symptoms, affective states, and urge to smoke cigarettes were not conclusively shown to be different or not between e-cigarette and cigarette use, suggesting that nicotine is not the sole driver of these effects. Consistent with previous studies [24, 25], we found that e-cigarettes were less rewarding and satisfying and reduced craving to a lesser degree than cigarettes. Interpreting these self-reported subjective effects data in the context of detailed nicotine intake and pharmacokinetic data from dual users is novel to our study and offers explanations for incomplete switching to e-cigarettes among smokers.
A strength of our study is that we enrolled users of commonly used designs of e-cigarettes, spanning a range of e-liquid nicotine content and electrical power. Across these devices, systemic exposure to nicotine was lower from e-cigarette use compared to cigarette smoking in most participants even though recent studies have shown that experienced e-cigarette users can achieve blood nicotine levels typically seen in smokers [8, 26–28]. Only four of 36 participants (11.1%) had plasma nicotine Cmax of at least 11 ng/mL, a typical nicotine boost from cigarettes [29], with e-cigarette use (15.4, 17.5, 18.9, and 25.5 ng/mL) compared to 26 participants (72.2%) with cigarette smoking (ranging from 11.0 to 46.1 ng/mL) (Figure 1B). Average systemic nicotine retention from e-cigarettes was high (>90%), as we have shown previously [8, 9] and the average retained dose of nicotine exceeded 1 mg. Thus, the relatively low blood nicotine levels following e-cigarette use are not explained by poor nicotine delivery from the e-cigarettes or low systemic retention.
Tobacco product user characteristics that might influence nicotine dose include sex, level of tobacco dependence, and puffing behavior. We found no evidence of significant effects of sex and tobacco dependence on nicotine intake. However, given the small number of women enrolled (8 of 36), our study was not adequately powered to determine a sex effect. Aerosol swallowing would result in lower blood nicotine levels due to slower rates of nicotine absorption in the stomach and first pass metabolism. Although we controlled the total number of puffs taken, we asked participants to puff on each product as they usually do, thus allowing participants the opportunity to alter their vaping behavior to titrate their desired nicotine dose [30]. Overall, nicotine titration with e-cigarettes relative to cigarettes was poor. Dual users may be using e-cigarettes in a manner that does not maximize their systemic exposure to nicotine. Users of variable-power tanks may be an exception, as they displayed a greater extent of nicotine titration. Users of variable-power tanks were exposed to significantly higher levels of nicotine than fixed-power tank users, and likely higher than cig-a-like users, but the small sample sizes across e-cigarette-types did not allow sufficient statistical power to detect significant differences.
Continued use of e-cigarettes as a supplementary rather than primary source of nicotine suggests that the abuse liability of e-cigarettes is lower than that of cigarettes [2, 4, 31]. Satisfaction, reward, and craving and urge reduction contribute to the reinforcing effect and abuse liability of tobacco products. Our data support the idea that the underlying reasons for differences in subjective effects of e-cigarettes and cigarettes include nicotine-related and non-nicotine-related factors. Thus, vaping behaviors that influence systemic exposure to nicotine, as well as non-nicotine factors, such as those associated with olfactory, taste and tactile sensations, may be important determinants of the extent of switching from cigarettes to e-cigarettes. Other determinants of the extent of switching from cigarettes to e-cigarettes (smoking cessation) include factors at the individual to societal levels, such as genetic and metabolic differences, socioeconomic status, presence of various stressors, and smokefree policies, including home smoking bans [32–34].
There is considerable public health concern due to reports of the surging popularity of JUUL among teens and questions about its addictiveness. We present the first independent pharmacokinetic profiles of JUUL, but generalizability is limited in that only three participants were JUUL users, and one user retained very little nicotine from JUUL. The reasons for poor systemic nicotine retention in that participant are not known but most likely was because the participant did not inhale the aerosol into their lungs. Given the high nicotine content of JUUL and the expected high nicotine levels per puff, JUUL users may be especially averse to an intense puffing regime, as done in this study [35].
The fixed puffing protocol may limit the generalizability and/or the ecological validity of our findings. For example, e-cigarette users tend to vape intermittently in small clusters of puffs during the day and so a session of a fixed number of puffs may not reflect actual use patterns. However, immediately after periods of nicotine abstention, we have shown that vapers often self-administer nicotine using clusters of greater than 10 puffs which are then followed by smaller clusters of puffs [36, 37], suggesting that the fixed puffing protocol of 10 to 15 puffs has real-world relevance. In addition, consistent with our findings from the fixed puffing protocol, we report in a separate manuscript that participants showed lower nicotine intake and reported lower psychological reward but similar withdrawal suppression with e-cigarettes compared to cigarettes during a day of ad libitum access [7]. These findings further support the idea that differences between e-cigarettes and cigarettes in nicotine intake and subjective effects from a fixed puffing protocol may be indicative of differences in nicotine intake and subjective effects from a longer period of ad libitum access. Further, participants could have smoked each cigarette more intensely than in their naturalistic setting in anticipation of not being able to smoke over the next four hours. Cigarettes allow for greater flexibility in puff duration, and therefore, nicotine intake, than e-cigarettes; long puffs on e-cigarettes can result in “dry puffs” and accompanying unpleasant sensations in the throat due to aldehyde generation, and are avoided by experienced e-cigarette users [38]. Furthermore, although cognitive expectancies influence craving of tobacco products, including e-cigarettes [39], it was not practical to blind participants to the study conditions. In assessing correlates of nicotine intake and extent of titration, the number of times e-cigarettes are used in the naturalistic setting are assumed to be either a session of 15 puffs or vaping for at least 10 minutes [13]. However, there is wide variability in e-cigarette use patterns and this assumption may not be generalizable to many e-cigarette users, including users of high nicotine content e-liquids who may take a few puffs at a time [35]. Finally, our study was primarily powered to detect differences in nicotine pharmacokinetic and subjective effects between e-cigarette use and smoking, and was not powered to detect effects of some of the null findings found, including differences across e-cigarette-types.
CONCLUSIONS
Nicotine intake and systemic exposure are on average lower with single use of e-cigarettes compared to cigarettes. Nicotine exposure was highest with variable voltage tank compared to other devices. Use of e-cigarettes resulted in a reduction of the severity of nicotine withdrawal symptoms, changes in affective states, and reduced urge to smoke comparable to cigarette smoking but e-cigarettes are judged to be less satisfying and rewarding and reduced craving less than cigarettes. These findings help explain why many dual users use e-cigarettes as a supplemental source of nicotine rather than switch completely to e-cigarettes.
Supplementary Material
FIGURE 3.
Changes in nicotine withdrawal (Minnesota Nicotine Withdrawal Scale, MNWS total score) (A), negative affect (Positive and Negative Affect Schedule, PANAS Negative subscale score) (B), urge to vape (Questionnaire for Smoking Urges-Brief, QSU Factor 1 and Factor 2 for vaping) (C, D), and urge to smoke cigarettes (QSU Factor 1 and Factor 2 for smoking) (E, F) before and after electronic and combustible cigarette use. The product assignment × time interaction term was not statistically significant for any of the measures. Open markers indicate significant difference from baseline and * indicates significant difference between e-cigarettes and combustible cigarettes.
ACKNOWLEDGEMENTS
We thank Marian Shahid, Annalise Davis, and Jennifer Ko for clinical research coordination; Kristina Bello, Trisha Mao, Polly Cheung, and Lisa Yu for performing analytical chemistry; and Zuckerberg San Francisco General Hospital Clinical Research Center nurses and staff for research participant care and study procedures.
Funding: This study was supported by grants R01DA039264 and P30DA012393 from the National Institute on Drug Abuse, 1 U54HL147127 from the National Cancer Institute, and was carried out in part at the Clinical Research Center at Zuckerberg San Francisco General Hospital (NIH/NCRR UCSF-CTSI UL1 RR024131). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health (NIH) or the Food and Drug Administration (FDA).
Footnotes
Declarations of Interest: Dr. Benowitz is a consultant to Pfizer and Achieve Life Sciences, companies that market or are developing smoking cessation medications, and has served as a paid expert witness in litigation against tobacco companies. The other authors declare no conflict of interest.
REFERENCES
- 1.Fearon IM, Eldridge AC, Gale N et al. Nicotine pharmacokinetics of electronic cigarettes: A review of the literature, Regul Toxicol Pharmacol 2018; 100: 25–34. [DOI] [PubMed] [Google Scholar]
- 2.National Academies of Sciences Engineering and Medicine Public health consequences of e-cigarettes (Washington, DC); 2018.
- 3.Piper ME, Baker TB, Benowitz NL, Kobinsky K & Jorenby DE Dual Users Compared to Smokers: Demographics, Dependence, and Biomarkers, Nicotine Tob Res 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nardone N, Ko J, Helen G St. & Benowitz NL Nicotine intake, dependence, and characteristics of electronic cigarette and dual users, Tob Regul Sci 2019; 5: 27–35. [Google Scholar]
- 5.Robertson L, Hoek J, Blank M-L et al. Dual use of electronic nicotine delivery systems (ENDS) and smoked tobacco: a qualitative analysis, Tob Control 2019; 28: 13–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Hajek P, Przulj D, Phillips A, Anderson R & Mcrobbie H Nicotine delivery to users from cigarettes and from different types of e-cigarettes, Psychopharmacology (Berl) 2017; 234: 773–779. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Harvanko AM, Helen G St, Nardone N, Addo N & Benowitz N Circadian subjective and pharmacological effects of ad libitum electronic and combustible cigarette use among dual users, Addiction 2019; Under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Helen G St, Havel C, Dempsey DA, Jacob P 3rd & Benowitz NL Nicotine delivery, retention and pharmacokinetics from various electronic cigarettes, Addiction 2016; 111: 535–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Helen G St, Dempsey DA, Havel CM, Jacob P 3rd & Benowitz NL Impact of e-liquid flavors on nicotine intake and pharmacology of e-cigarettes, Drug Alcohol Depend 2017; 178: 391–398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hukkanen J, Jacob Iii P & Benowitz NL Metabolism and disposition kinetics of nicotine, Pharmacol Rev 2005; 57: 79–115. [DOI] [PubMed] [Google Scholar]
- 11.Havel CM, Benowitz NL, Jacob P 3rd & St.Helen, G. An Electronic Cigarette Vaping Machine for the Characterization of Aerosol Delivery and Composition, Nicotine Tob Res 2017; 19: 1224–1231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Heatherton TF, Kozlowski LT, Frecker RC & Fagerstrom KO The Fagerstrom Test for Nicotine Dependence: a revision of the Fagerstrom Tolerance Questionnaire, Br J Addict 1991; 86: 1119–27. [DOI] [PubMed] [Google Scholar]
- 13.Foulds J, Veldheer S, Yingst J et al. Development of a Questionnaire for Assessing Dependence on Electronic Cigarettes Among a Large Sample of Ex-Smoking E-cigarette Users, Nicotine Tob Res 2015; 17: 186–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hughes JR & Hatsukami D Signs and symptoms of tobacco withdrawal, Arch Gen Psychiatry 1986; 43: 289–94. [DOI] [PubMed] [Google Scholar]
- 15.Becoña E, Vázquez FL, Fuentes M. a. J. & Del Carmen Lorenzo, M. A. Anxiety, affect, depression and cigarette consumption, Pers Indiv Differ 1998; 26: 113–119. [Google Scholar]
- 16.Rose JE, Westman EC, Behm FM, Johnson MP & Goldberg JS Blockade of smoking satisfaction using the peripheral nicotinic antagonist trimethaphan, Pharmacol Biochem Behav 1999; 62: 165–72. [DOI] [PubMed] [Google Scholar]
- 17.Cox LS, Tiffany ST & Christen AG Evaluation of the brief questionnaire of smoking urges (QSU-brief) in laboratory and clinical settings, Nicotine Tob Res 2001; 3: 7–16. [DOI] [PubMed] [Google Scholar]
- 18.Trehy ML, Ye W, Hadwiger ME et al. Analysis of electronic cigarette cartridges, refill solutions, and smoke for nicotine and nicotine related impurities, J Liq Chromatogr Relat Technol 2011; 34: 1442–1458. [Google Scholar]
- 19.Jacob P 3rd, Yu L, Wilson M & Benowitz NL Selected ion monitoring method for determination of nicotine, cotinine and deuterium-labeled analogs: absence of an isotope effect in the clearance of (S)-nicotine-3’,3’-d2 in humans, Biol Mass Spectrom 1991; 20: 247–52. [DOI] [PubMed] [Google Scholar]
- 20.Hendricks PS, Ditre JW, Drobes DJ & Brandon TH The early time course of smoking withdrawal effects, Psychopharmacology (Berl) 2006; 187: 385–396. [DOI] [PubMed] [Google Scholar]
- 21.Donny EC, Houtsmuller E & Stitzer ML Smoking in the absence of nicotine: behavioral, subjective and physiological effects over 11 days, Addiction 2007; 102: 324–334. [DOI] [PubMed] [Google Scholar]
- 22.Perkins KA Sex differences in nicotine reinforcement and reward: influences on the persistence of tobacco smoking The motivational impact of nicotine and its role in tobacco use, pp. 143–169 (Springer); 2008. [DOI] [PubMed] [Google Scholar]
- 23.Chiamulera C Cue reactivity in nicotine and tobacco dependence: a “multiple-action” model of nicotine as a primary reinforcement and as an enhancer of the effects of smoking-associated stimuli, Brain Res Rev 2005; 48: 74–97. [DOI] [PubMed] [Google Scholar]
- 24.Strasser AA, Souprountchouk V, Kaufmann A et al. Nicotine Replacement, Topography, and Smoking Phenotypes of E-cigarettes, Tob Regul Sci 2016; 2: 352–362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jorenby DE, Smith SS, Fiore MC & Baker TB Nicotine levels, withdrawal symptoms, and smoking reduction success in real world use: A comparison of cigarette smokers and dual users of both cigarettes and E-cigarettes, Drug Alcohol Depend 2017; 170: 93–101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Spindle TR, Breland AB, Karaoghlanian NV, Shihadeh AL & Eissenberg T Preliminary results of an examination of electronic cigarette user puff topography: The effect of a mouthpiece-based topography measurement device on plasma nicotine and subjective effects, Nicotine Tob Res 2015; 17: 142–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wagener TL, Floyd EL, Stepanov I et al. Have combustible cigarettes met their match? The nicotine delivery profiles and harmful constituent exposures of second-generation and third-generation electronic cigarette users, Tob Control 2016: tobaccocontrol-2016–053041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Ramôa CP, Hiler MM, Spindle TR et al. Electronic cigarette nicotine delivery can exceed that of combustible cigarettes: a preliminary report, Tob Control 2015: tobaccocontrol-2015–052447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Patterson F, Benowitz N, Shields P et al. Individual differences in nicotine intake per cigarette, Cancer Epidemiol Biomarkers Prev 2003; 12: 468–471. [PubMed] [Google Scholar]
- 30.Dawkins LE, Kimber CF, Doig M, Feyerabend C & Corcoran O Self-titration by experienced e-cigarette users: blood nicotine delivery and subjective effects, Psychopharmacology (Berl) 2016; 233: 2933–2941. [DOI] [PubMed] [Google Scholar]
- 31.Lee AS, Hart JL, Walker KL, Keith RJ & Ridner SL Dual Users and Electronic Cigarette Only Users: Consumption and Characteristics, Int J Healthc Med Sci 2018; 4: 111–116. [PMC free article] [PubMed] [Google Scholar]
- 32.National US Cancer Institute A Socioecological approach to addressing tobacco related health disparities. National Cancer Institute Tobacco Control Monograph 22. NIH Publication No. 17-CA-8035A, in: U.S. Department of Health and Human Services & National Institutes of Health, N. C. I; (Eds.) (Bethesda, MD. ); 2017. [Google Scholar]
- 33.Leventhal AM, Bello MS, Galstyan E, Higgins ST & Barrington-Trimis JL Association of Cumulative Socioeconomic and Health-Related Disadvantage With Disparities in Smoking Prevalence in the United States, 2008 to 2017, JAMA internal medicine 2019; 179: 777–785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Nollen NL, Mayo MS, Cox LS et al. Factors that Explain Differences in Abstinence between Black and White Smokers: A Prospective Intervention Study, JNCI: Journal of the National Cancer Institute 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Nardone N, Helen G St, Addo N, Meighan S & Benowitz NL JUUL electronic cigarettes: Nicotine exposure and the user experience Drug Alcohol Depend 2019; Accepted. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.St Helen G, Shahid M, Chu S & Benowitz NL Impact of e-liquid flavors on e-cigarette vaping behavior, Drug Alcohol Depend 2018; 189: 42–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Helen G St, Ross K, Dempsey D et al. Nicotine delivery and vaping behavior during ad libitum e-cigarette access, Tob Regul Sci 2016; 2: 363–376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Farsalinos KE, Voudris V & Poulas K E-cigarettes generate high levels of aldehydes only in ‘dry puff’conditions, Addiction 2015; 110: 1352–6. [DOI] [PubMed] [Google Scholar]
- 39.Palmer AM & Brandon TH How do electronic cigarettes affect cravings to smoke or vape? Parsing the influences of nicotine and expectancies using the balanced-placebo design, J Consult Clin Psychol 2018; 86: 486–491. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.