Abstract
Objectives:
Measurement of electronic cigarette (ECIG) puff topography provides an understanding of how product characteristics and user behavior affect nicotine delivery. However, mouthpiece-based topography devices may affect natural puffing behavior. This study was designed to compare ECIG topography measured by mouthpiece-based eTop computerized device and mouthpiece-free video recordings.
Methods:
ECiG-naïve cigarette smokers (N = 18) and ECIG-experienced users (N = 25) puffed on a standardized ECIG via eTop or conventionally; both sessions were videotaped. Following overnight abstinence, participants experienced one directed (10 puffs, 30 sec IPI) and 2 ad libitum puffing bouts. Heart rate and subjective response were measured throughout sessions.
Results:
No statistically significant differences between methods were observed for topography, heart rate, or abstinence-related subjective effects, and both methods were accurate and reliable. Use of a mouthpiece was perceived to alter aspects of ECIG puffing (eg, “reduce enjoyment).
Conclusions:
The mouthpiece-based eTop measures ECIG topography precisely as when no mouthpiece is used, and interferes minimally with subjective ECIG experience. Reliable and valid ECIG topography measurement methods are an important regulatory tool, as they can be used to understand the interplay between product design and user behavior to predict toxicant exposure.
Keywords: electronic cigarette, cigarette smoking, puff topography, methods
Electronic cigarettes (ECIGs) were designed to deliver nicotine without the other harmful chemicals associated with combustible cigarette smoking. Yet, the ability of an ECIG to deliver nicotine varies greatly, and is a function of product characteristics and user behavior. For instance, greater nicotine delivery to users’ blood is associated with devices with higher electrical power,1 liquids with higher nicotine concentrations, and users who take longer puffs.2 Complicating matters is that there exist thousands of device/liquid product combinations from which consumers can choose.3 To gain better understanding of how device/liquid combinations influence ECIG nicotine delivery, an evaluation of user behavior is crucial.
Analysis of user puff topography – puff number, volume, duration, inter-puff interval, and flow rate – reveals that it influences the nicotine delivery profile of cigarettes,4 cigars,5 and waterpipes.6 For example, smokers may take more, larger, longer, and/or more frequent puffs when they smoke cigarettes that yield less nicotine than their usual brand,7 presumably in an effort to obtain more nicotine. The ability to measure puff topography is facilitated by the use of computerized devices tailored to the characteristics of a given tobacco product.7,8 Our understanding of ECIG-derived nicotine exposure would thus benefit from computerized devices tailored to ECIG device characteristics.
Much of the extant work that addresses ECIG puff topography has relied on the use of computerized devices designed for cigarette smoking.9-11 Unfortunately, these devices impose constraints on ECIG topography measurement such as by limiting puff number to 43 per bout and puff durations to 5 seconds.9 At least under laboratory conditions, however, some ECIG users have been observed to take ~50% more puffs in a single use period,12,13 as well as puffs ~7 seconds in duration.14 One study demonstrated that puff durations were shorter, and puff number was higher, when ECIG puffing was measured via a cigarette topography device compared to natural puffing.15 Furthermore, such devices are less sensitive to lower flow rates,9 and thus, may fail to capture part or all of an ECIG puff. To improve upon these constraints, several computerized devices have been developed specifically for measurement of ECIG puff topography.9,12,14
One such device is the eTop (American University of Beirut), which incorporates a flowmeter design similar to that used for cigarette topography measurement but can detect flow rates well below those observed in experienced ECIG users (ie, 3 ml/s).12 This device has been used to demonstrate longer puff durations and larger puff volumes during ECIG use for ECIG-experienced users, relative to ECIG-naïve cigarette smokers.2 Still, its ability to measure topography requires the use of a mouthpiece that may or may not interfere with natural ECIG puffing. One study12 shows that, relative to no mouthpiece use, mouthpiece-based measurement does not affect subjective response, nicotine concentration, or puff number among ECIG-experienced users; however, other topography parameters were not assessed. The purpose of the present study was to compare ECIG puff topography as measured by mouthpiece-based eTop and direct observation via video recordings in ECIG-naïve cigarette smokers and experienced ECIG users.
METHODS
Sample Size
For within-subjects effects, 40 participants is sufficient to detect differences in topography outcomes between methods. This number of participants is required assuming a large correlation among repeated measures (r = 0.80)7 and relatively low effect sizes (f = 0.13), using a power of 0.80 and Type I error rate of 0.05. Note, however, that much larger effect sizes (fs > 0.4) have been reported in our previous work.7,16 Also in this previous work, moderate-large effect sizes (0.45 < f > 0.79) have been observed for differences between ECIG-experienced versus -naïve groups. Thus, 20 completers per group would allow for detecting effect sizes on the lower end of this range.
Inclusion Criteria
ECIG-naïve cigarette smokers reported smoking ≥ 10 cigarettes/day for at least one year, < 5 lifetime ECIG uses, and no ECIG use in the past month, and also provided an expired-air carbon monoxide (CO) > 10 ppm (CoVita; Haddonfield, NJ). ECIG-experienced users reported current ECIG use for ≥ 3 months, use ≥ 1 ml of ECIG liquid/day with a nicotine concentration ≥ 3 mg/ml, and smoking ≤ 5 cigarettes/day.
Exclusion Criteria
Individuals were excluded if they reported < 18 or > 55 years of age, a history of medical or psychiatric conditions, regular use of most prescription medications, regular use of other nicotine/tobacco products, or use of nicotine/tobacco pharmacotherapy. Women who reported current breastfeeding or pregnancy, or who tested positive for pregnancy via urinalysis, were excluded.
Procedures
Participants experienced 3, 3-hour sessions ordered by Latin square: mouthpiece-based eTop, mouthpiece-free eTop 2.0 (data not reported here), and direct observation. Sessions were separated by ≥ 48 hours and preceded by ~12 hours of nicotine/tobacco abstinence (expired air CO sample ≤ 10 ppm required pre-session). Expired air CO cannot verify abstinence from ECIGs, so participants also experienced a one-hour abstinence period at the beginning of the session.16 Continuous recording of heart rate and blood pressure then commenced, followed 30 minutes later by completion of baseline questionnaires that assessed subjective effects. Next, participants were asked to follow a standardized ECIG use bout in which they took 10 puffs from an ECIG with 30-second IPIs.2,12 Depending on condition, participants puffed on their ECIG using a computerized topography device or no device. All sessions were videotaped. Immediately following the standardized bout, participants again completed questionnaires to assess abstinence symptoms and nicotine effects. This same procedure (questionnaires, ECIG puffing bout, questionnaires) was repeated for 2 additional 5-minute bouts during which participants could puff ad libitum (30 minutes separated these 2 ad libitum bouts). Thus, each session included one standardized and 2 ad libitum bouts. At the end of the session, participants completed a final questionnaire that assessed their acceptability of the measurement method used.7 Participants were compensated $50 for session 1, $75 for session 2, and $75 for session 3.
Materials
ECIG devices and liquid.
All participants used the e-GO 3.3 V, 1000mAh battery (Joyetech; Irvine, CA) and a 510-style cartomizer with 1.5 ohms resistance and a dual-heating coil (SmokTech; Smoke Technology Co LTD; Shenzhen China). Cartomizers were filled with 1 ml of unflavored liquid with 18 mg/ml of nicotine and a 70%/30% propylene glycol/vegetable glycerin ratio. This device/liquid combination can deliver a physiologically active nicotine dose after 10 puffs to experienced ECIG users and ECIG-naïve cigarette smokers.2
Topography measurement equipment.
The eTop (American University of Beirut) continuously monitors, digitizes, and records airflow through a mouthpiece that is connected to an ECIG. A puff is detected by a pressure transducer whenever changes in pressure occur across an orifice that is incorporated into the mouthpiece. Orifice dimensions and transducer sensitivity provide valid measurements at puff flow rates as low as 3 ml/sec. The signal from the transducer is digitized using a National Instruments data acquisition device (NI DAQ USB-6008) and communicated to the computer via USB. For video recording of ECIG puffs, a Canon Vixia HF R42 (Canon USA, Inc; Melville, NY) camcorder was used. Video-based data were imported to Adobe Premier Pro 2015 (Adobe Systems, Inc; San Jose, CA), and analyzed using frame-by-frame time analysis.7
Outcome Measures
Puff topography.
Puff topography variables measured via eTop included puff number, duration (seconds), volume (ml), IPI (second), and flow rate (ml/second). Puff onset occurs when flow rate > 2 ml/second and offset occurs when the flow rates becomes < 2 ml/second. Video recordings were used to measure puff number, duration, and IPI, but could not capture puff volume or flow rate. Puff duration measurement via video recordings has been validated.7
Video data were scored using a “lip-based” definition.7,13 The first frame in which the lips were enclosed around the mouthpiece was counted as the puff onset and the last frame in which the lips were enclosed around the mouthpiece was counted as the puff offset. Puff number was the total number of puffs >300 ms that met this lip-based definition. Puff duration was defined as the time from the onset to the offset of a single puff, and IPI was defined as the time from the offset of one puff to the onset of the next puff. These video-based data were scored by 2 independent raters and then compared for reliability.7
Acceptability questionnaire.
An acceptability questionnaire was administered to assess participants’ subjective experience of how the measurement method used influenced their ECIG use (eg, “altered ECIG puffing behavior,” “made ECIG puffing less likely,” “reduced ECIG puffing enjoyment”). All items were presented as a visual analog scale (VAS) with scores that range from 0 to 100. This set of questions has been used previously to compare of methods to measure cigarette7 and ECIG12 topography.
Secondary Outcome Measures
Minnesota Nicotine Withdrawal Scale (MNWS).
The MNWS17 consists of 11 visual analog scale (VAS) items on a 0-100 scale to measure nicotine or tobacco withdrawal symptoms (eg, “craving,” “irritability/frustration/anger”).
Tiffany-Drobes Questionnaire on Smoking Urges (QSU): Brief Form.
The QSU-Brief18 consists of 10 Likert-scale items measuring multidimensional features of nicotine/tobacco cravings. Statements were centered above 7 boxes, with the leftmost labeled “strongly disagree” (score = 0) and rightmost labeled “strongly agree” (score = 6). For ECIG-experienced participants, the terms “cigarette” and “smoke” were replaced with “ECIG” and “vape”, respectively. Items were collapsed into intention to smoke (Factor 1) and anticipation of relief from withdrawal (Factor 2).
Direct Effects of Nicotine Scale (DENS).
The DENS19 consists of 10 VAS items to assess the severity of nicotine-associated side effects (eg, “dizzy,” “lightheaded,” “nervous).
Direct Effects of Product Use (DEPS).
The DEPS consists of 9 VAS items to assess effects commonly reported with cigarette smoking, but items are modified to ask about vaping (eg, “Was the ECIG satisfying?” and “Did the ECIG help you concentrate?”).2
Physiological measures.
Heart rate and blood pressure were measured continuously throughout each session (Model 506 NP3, Criticare Systems, Inc, Waukesha, WI). Heart rate was recorded every 20 seconds, and blood pressure every 5 minutes. Data for both measures were averaged into 5-minute bins to create a single value pre- and post-bout.
Data Analysis
Data preparation.
Prior to data analyses, puff topography data collected via the computerized device were cleaned by an automated procedure. First, all consecutive puffs separated by < 300 ms were combined into a single puff. Next, any remaining single puffs < 300 ms were considered artifacts and thus deleted.12 For topography data collected via video recordings, this cleanup procedure was applied manually to the values averaged between the 2 independent raters. Puff topography values within each bout were averaged across puffs to create a single value for puff duration, IPI, puff volume, and/or flow rate.7,12 Thus, 3 values (one for directed and one for each ad lib) for these topography variables were produced for each measurement method (except for puff volume and flow rate for video-based measurement).
Data analysis.
Interrater reliability between 2 independent raters for video-scored topography data was assessed using 2-way random Intraclass Correlations (ICC) for absolute agreement. The ICCs were consistently high (rs > 0.78 for puff number, > 0.91 for puff duration, and > 0.78 for IPI; all p’s < .001), and thus, the average rater scores were used in all analyses described below.
To assess the reliability of each measurement method, puff topography for the 2 ad lib bouts within each condition were correlated (Pearson’s r). Topography data from the mouthpiece-based device condition (eTop) also were correlated (Pearson’s r) with that from (1) the video-based condition (to assess measurement validity) and (2) video recordings of the mouthpiece-based device in use (to assess measurement accuracy).
Puff topography data were analyzed using a 2 x 2 x 3 mixed analysis of variance (ANOVA): group (ECIG-naïve cigarette smokers, ECIG-experienced users) by measurement method (eTop, video) by bout (1-3). Subjective and physiological data were analyzed in this same manner except that the factor of time (pre- and post-bout for each bout) also was included. Acceptability questionnaire data were analyzed using a 2 x 2 ANOVA (group and device). Huynh-Feldt corrections were used to adjust for potential violations of sphericity, and differences between means were examined using Tukey’s honestly significant different (HSD, p < .05) to control the familywise Type I error rate.
RESULTS
Participants
Eighteen ECIG-naïve cigarette smokers and 25 ECIG-experienced users completed the study. Groups did not differ in education level (M = 12.9, SE = 0.5 vs M = 14.0, SE = 2.1, respectively), t(41) = −1.78, p > .05), race (83.3% vs 80.0% white, respectively; χ2(1) = 0.08, p > .05), ethnicity (both 100% non-Hispanic), or sex (77.8% vs 96.0% male, respectively; χ2(1) = 3.38, p > .05). ECIG-naïve cigarette smokers (M = 30.4, SE = 2.1) were significantly older than ECIG-experienced users (M = 24.4, SE = 1.6), t(41) = 2.43, p < .05. There also were differences between ECIG-naïve and ECIG-experienced groups regarding employment: full time (22.2% and 36.0%, respectively), part time (16.7% and 20.0%, respectively), student (5.6% and 28.0%, respectively) and unemployed (55.6% and 16.0%, respectively), χ2(3) = 8.58, p < .05.
ECIG-naïve cigarette smokers used an average of 15.2 (SD = 4.7) cigarettes/day for 8.6 years (SD = 7.1), and had an average screening CO of 22.3 ppm (SD = 16.4) and an average score of 4.8 (SD = 1.6) on the Fagerström Test of Cigarette Dependence (FTCD).20 For ECIG-experienced users, N = 3 reported current cigarette smoking: 3.0 (SD = 0.0) cigarettes/day for 5.8 years (SD = 8.0; range = 0.5-15 years). This latter group also reported using 6.9 ml (SD = 4.9) of ECIG liquid/day for 1.8 years (SD = 1.2) with an average nicotine concentration of 7.4 mg/ml (SD = 6.3), and had an average score of 7.7 (SD = 4.8) on the Penn State Electronic Cigarette Dependence Index.21 Preferred device types were “tanks” (32.0%) and “mods” (68.0%), while preferred liquid flavors were fruit (44.0%), sweet (40.0%), and other/multiple (16.0%).
Measurement Method Reliability, Accuracy, and Validity
Correlations between the 2 ad lib bouts within the eTop condition were moderate to high for puff number [r(41) = 0.55], puff duration [r(42) = 0.91], IPI [r(42) = 0.56], puff volume [r(40) = 0.90], and flow rate [r(40) = 0.89, ps < .01]. For the video condition, correlations between these bouts were consistently high: puff number [r(41) = 0.83], IPI [r(42) = 0.94], and puff duration [r(42) = 0.88, ps < .01]. Table 1 displays correlations between eTop and video conditions, as well as between the eTop condition and video recordings of the eTop device in use. All correlations were moderate to high and statistically significant (all ps < .01).
Table 1.
Correlations for eTop versus Video or versus Video of eTop in Use
| eTop | |||
|---|---|---|---|
| Directed | Ad Lib 1 | Ad Lib 2 | |
| versus video | |||
| Puff Number | 0.47 | 0.66 | |
| Puff Duration | 0.68 | 0.66 | 0.53 |
| IPI | 0.83 | 0.63 | |
| versus video of eTop | |||
| Puff Number | 0.74 | 0.95 | |
| Puff Duration | 0.98 | 0.96 | 0.97 |
| IPI | 0.99 | 0.96 | |
Note.
All coefficients significant at p < .01
Puff Topography
Table 2 displays results for the mixed ANOVAs for topography outcomes, and Table 3 displays topography means for Group by Device by Bout. No interactions involving the Group or Bout factors were significant (all Fs < 2.92, ps > .05, ηp2 < 0.08), and thus, they are omitted from Table 2. The only significant effect observed was a main effect of bout for puff number [F(2, 80) = 8.99, p < .001, ηp2 = 0.18]. Participants took fewer puffs at the directed bout (M = 10.2, SE = 0.08) than at the second ad lib bout (M = 12.7, SE = 0.8; Tukey’s HSD, p < .05), but not the first ad lib bout (M = 11.5, SE = 0.6).
Table 2.
Statistical Analysis Results for Topography Outcomes
| Group | Device | Bout | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Outcome Measure | F | p | ηp2 | F | p | ηp2 | F | p | ηp2 |
| Puff Topography | |||||||||
| Numbera | 1.21 | .28 | 0.03 | 0.99 | .33 | 0.02 | 8.99 | .001 | 0.18 |
| Durationb | 0.02 | .89 | 0.00 | 1.20 | .28 | 0.03 | 0.31 | .66 | 0.01 |
| Inter-puff Intervalc | 0.46 | .50 | 0.01 | 0.48 | .49 | 0.01 | 3.22 | .06 | 0.08 |
| Volumeb | 0.21 | .65 | 0.00 | n/a | n/a | n/a | 1.74 | .19 | 0.04 |
| Flow Rateb | 0.24 | .63 | 0.00 | n/a | n/a | n/a | 2.30 | .12 | 0.06 |
Note.
dfgroup and dfdevice = (1,40); dfbout = (2,80)
dfgroup and dfdevice = (1,39); dfbout = (2,78)
dfgroup and dfdevice = (1,38); dfbout = (2,76)
Table 3.
Means (SEM) for Puff Topography Measures for Group by Device by Bout
| eTop | Video | |||||
|---|---|---|---|---|---|---|
| Directed | Ad Lib 1 | Ad Lib 2 | Directed | Ad Lib 1 | Ad Lib 2 | |
| Puff Number | ||||||
| ECIG-Naïve | 10.2 (1.1) | 10.5 (1.1) | 12.8 (1.3) | 10.0 (0.1) | 10.8 (1.1) | 11.5 (1.4) |
| ECIG-Experienced | 10.4 (0.1) | 11.4 (0.9) | 12.8 (1.0) | 10.1 (0.1) | 13.3 (0.9) | 13.7 (1.1) |
| Puff Duration | ||||||
| ECIG-Naïve | 3.8 (0.4) | 3.8 (0.3) | 3.9 (0.4) | 3.2 (0.3) | 3.3 (0.4) | 3.7 (0.4) |
| ECIG-Experienced | 3.9 (0.3) | 3.7 (0.3) | 3.5 (0.3) | 3.9 (0.3) | 3.6 (0.3) | 3.6 (0.3) |
| IPI | ||||||
| ECIG-Naïve | 30.1 (0.5) | 31.5 (4.7) | 22.9 (2.4) | 30.7 (0.4) | 28.6 (4.5) | 29.0 (4.4) |
| ECIG-Experienced | 29.4 (0.4) | 27.2 (4.1) | 22.8 (2.11) | 30.3 (0.4) | 26.0 (3.9) | 24.1 (3.8) |
| Puff Volume | ||||||
| ECIG-Naïve | 70.5 (9.7) | 72.2 (11.1) | 79.6 (14.4) | n/a | n/a | n/a |
| ECIG-Experienced | 62.5 (8.2) | 68.3 (9.3) | 71.7 (12.1) | n/a | n/a | n/a |
| Flow Rate | ||||||
| ECIG-Naïve | 19.0 (1.9) | 19.0 (2.3) | 19.4 (2.5) | n/a | n/a | n/a |
| ECIG-Experienced | 16.2 (1.6) | 17.6 (1.9) | 19.5 (2.1) | n/a | n/a | n/a |
Device Acceptability
Table 4 shows results for the acceptability measure, except for Group by Device interactions as none were observed to be significant [Fs(2,82) < 3.35, ps > .05, ηp2 < 0.08]. A significant main effect of group was found for the items of Alter Behavior [F(1, 41) = 6.97, p < .01, ηp2 = 0.15] and Less Likely [F(1, 41) = 7.22, p < .0p1, ηp2 = 0.15]. For both items, significantly higher ratings were reported by ECIG-naïve smokers than ECIG-experienced users (collapsed across device): 36.1 (SE = 4.7) versus 20.0 (SE = 4.0) respectively for Alter Behavior, and 29.4 (SE = 4.8) versus 12.4 (SE = 4.1) respectively for Less Likely.
Table 4.
Statistical Analysis Results for the Acceptability Questionnaire Items
| Groupa | Deviceb | |||||||
|---|---|---|---|---|---|---|---|---|
| Item | eTop | Video | F | p | ηp2 | F | p | ηp2 |
| Alter Behavior | 6.97 | .01 | 0.15 | 6.79 | .01 | 0.14 | ||
| ECIG-Naïve | 43.9 (6.4) | 28.4 (5.0) | ||||||
| ECIG-Experienced | 24.0 (5.4) | 16.0 (4.2) | ||||||
| Less Likely | 7.22 | .01 | 0.15 | 1.37 | .25 | 0.03 | ||
| ECIG-Naïve | 28.8 (5.3) | 29.9 (5.2) | ||||||
| ECIG-Experienced | 15.4 (4.5) | 9.4 (4.4) | ||||||
| Reduce Enjoyment | 0.32 | .57 | 0.01 | 8.56 | .010 | 0.17 | ||
| ECIG-Naïve | 27.6 (6.6) | 24.7 (5.3) | ||||||
| ECIG-Experienced | 30.7 (5.6) | 13.9 (4.5) | ||||||
| Affect Taste | 0.62 | .43 | 0.02 | 3.29 | .08 | 0.07 | ||
| ECIG-Naïve | 25.4 (6.8) | 17.1 (5.9) | ||||||
| ECIG-Experienced | 30.4 (5.7) | 23.4 (5.0) | ||||||
| More Difficult | 0.58 | .45 | 0.01 | 4.12 | .049 | 0.09 | ||
| ECIG-Naïve | 20.8 (6.2) | 20.0 (3.9) | ||||||
| ECIG-Experienced | 21.3 (5.3) | 10.5 (3.3) | ||||||
| Increase Awareness | 2.24 | .14 | 0.05 | 0.53 | .47 | 0.01 | ||
| ECIG-Naïve | 45.7 (7.3) | 47.9 (6.6) | ||||||
| ECIG-Experienced | 40.0 (6.2) | 31.7 (5.6) | ||||||
| Like to Know More | 0.45 | .51 | 0.01 | 0.09 | 0.77 | 0.002 | ||
| ECIG-Naïve | 60.8 (7.4) | 64.9 (7.4) | ||||||
| ECIG-Experienced | 69.8 (6.3) | 68.32 (6.3) | ||||||
Note.
df = (1,41)
df = (2,82)
A main effect of device also was significant for Alter Behavior, F(2, 82) = 6.79, p < .05, ηp2 = 0.14, with ratings (collapsed across group) higher for eTop (M = 33.9, SE = 4.2) relative to video (M = 22.2, SE = 3.3). A similar pattern was observed for the items of Reduce Enjoyment and More Difficult, with higher ratings for eTop than for video.
Heart Rate
The mixed ANOVA results for heart rate are separated into Tables 5 (Group and Device main effects) and 6 (Bout and Time main and interaction effects). Interactions involving the Group and Device factors are omitted from these tables as none were observed to be significant (Fs < 2.06, ps > .05, ηp2 < 0.05). For the effect of Bout by Time [F(2, 82) = 23.61, p < .001, ηp2 = 0.38], mean heart rate increased from before to after each bout: 71.3 (SE = 1.6) to 78.7 (SE = 1.6) respectively for directed, 72.2 (SE = 1.4) to 76.2 (SE = 1.4) respectively for ad lib bout 2, and 72.1 (SE = 1.5) to 75.5 (SE = 1.4) respectively for ad lib bout 3 (Tukey’s HSD; ps < .01 for pre-post comparisons at each bout).
Table 5.
Effects of Group and Device for Physiological and Subjective Outcomes
| Group |
Device |
|||||
|---|---|---|---|---|---|---|
| Outcome Measure | F | p | ηp2 | F | p | ηp2 |
| Heart Ratea | 0.00 | .98 | 0.00 | 0.29 | .60 | 0.01 |
| MNWSb | ||||||
| Urges | 5.97 | .02 | 0.13 | 1.41 | .24 | 0.03 |
| Irritable | 3.94 | .05 | 0.09 | 1.03 | .32 | 0.03 |
| Anxious | 4.20 | .05 | 0.09 | 0.01 | .94 | 0.00 |
| Diff Concentrate | 0.51 | .48 | 0.01 | 0.01 | .92 | 0.00 |
| Restless | 0.86 | .36 | 0.02 | 0.11 | .74 | 0.00 |
| Hunger | 8.18 | .01 | 0.17 | 0.07 | .80 | 0.00 |
| Impatient | 2.70 | .11 | 0.06 | 0.60 | .44 | 0.01 |
| Craving | 5.11 | .03 | 0.11 | 0.30 | .59 | 0.01 |
| Drowsy | 2.75 | .11 | 0.06 | 0.45 | .51 | 0.01 |
| Depressed | 1.03 | .32 | 0.03 | 0.24 | .63 | 0.01 |
| Desire Sweets | 1.38 | .25 | 0.03 | 1.35 | .25 | 0.03 |
| Tiffany Drobesb | ||||||
| Factor 1 | 10.12 | < .001 | 0.20 | 0.68 | .41 | 0.02 |
| Factor 2 | 5.23 | .03 | 0.11 | 0.51 | .48 | 0.01 |
| DEPSb | ||||||
| Satisfying | 0.54 | .47 | 0.01 | 0.45 | .51 | 0.01 |
| Pleasant | 0.56 | .46 | 0.01 | 0.00 | .95 | 0.00 |
| Taste Good | 0.02 | .88 | 0.00 | 0.02 | .88 | 0.00 |
| Dizzy | 0.00 | .96 | 0.00 | 0.13 | .72 | 0.00 |
| Calm | 0.06 | .80 | 0.00 | 1.46 | .24 | 0.03 |
| Concentrate | 0.12 | .74 | 0.00 | 2.02 | .16 | 0.05 |
| Awake | 0.05 | .83 | 0.00 | 0.75 | .39 | 0.02 |
| Reduce Hunger | 0.04 | .85 | 0.00 | 0.39 | .54 | 0.01 |
| Sick | 0.00 | .96 | 0.00 | 0.69 | .41 | 0.02 |
Note.
MNWS = Minnesota Nicotine Withdrawal Scale DEPS = Direct Effects of Product Scale
dfgroup = (1,39); all other df = (2,78)
dfgroup = (1,41); all other df = (2,82)
Nicotine/Tobacco Abstinence Effects
As with heart rate, the mixed ANOVA results for the abstinence-related measures are separated into Tables 5 and 6. Only 5 of a possible 130 interactions involving the Group and Device factors were observed to be significant (other Fs < 3.85, ps >.05, ηp2 < 0.09), and thus results for these interactions are not included in the tables. For the MNWS measure, several main effects of group were revealed (Fs > 4.20, ps < .05, ηp2’s > 0.09) such that ECIG-naïve cigarette smokers reported higher ratings of withdrawal than ECIG-experienced users. For instance, ratings for these groups were 42.7 (SE = 5.1) versus 26.5 (SE = 4.3) respectively for “urges to vape,” and 20.3 (SE = 3.4) and 1.3 (SE = 2.9) respectively for “anxious.” All MNWS items except “Hunger” revealed a significant Bout by Time interaction. For these items, scores generally decreased from pre- to post- bouts, with the most pronounced decreases observed at the first bout. Ratings of “craving” (item with the largest F value for Bout by Time) decreased from pre- to post-directed bout 1 (M = 55.6, SE = 5.1 vs M = 25.5, SE = 3.3, respectively), from pre- to post-ad lib bout 2 (M = 43.2, SE = 4.4 vs M = 22.6, SE = 3.5, respectively), and from pre- to post-ad lib bout 3 (M = 37.9, SE = 4.2 vs M = 19.7, SE = 2.9; Tukey’s HSD, ps < .01 for pre-post comparisons at each bout). A similar pattern was observed for all other outcomes significant for a Bout by Time interaction (Fs < 18.47, ps < .05).
Table 6.
Effects of Bout, Time, and Bout by Time for Physiological and Subjective Outcomes
| Bout | Time | Bout x Time | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Outcome Measure |
F | p | ηp2 | F | p | ηp2 | F | p | ηp2 |
| Heart Ratea | 4.05 | .03 | 0.09 | 81.08 | < .001 | 0.68 | 23.61 | < .001 | 0.38 |
| MNWSb | |||||||||
| Urges | 32.98 | < .001 | 0.45 | 51.34 | < .001 | 0.56 | 4.83 | .02 | 0.11 |
| Irritable | 15.65 | < .001 | 0.28 | 24.83 | < .001 | 0.38 | 8.01 | .002 | 0.16 |
| Anxious | 20.33 | < .001 | 0.33 | 27.89 | < .001 | 0.41 | 13.68 | < .001 | 0.25 |
| Diff Concentrate | 14.75 | < .001 | 0.27 | 19.83 | < .001 | 0.33 | 8.63 | .002 | 0.17 |
| Restless | 7.45 | .003 | 0.15 | 15.62 | < .001 | 0.28 | 10.68 | < .001 | 0.21 |
| Hunger | 0.09 | .87 | 0.00 | 20.43 | < .001 | 0.33 | 2.15 | .13 | 0.05 |
| Impatient | 20.45 | < .001 | 0.33 | 29.62 | < .001 | 0.42 | 18.47 | < .001 | 0.31 |
| Craving | 20.21 | < .001 | 0.33 | 52.86 | < .001 | 0.56 | 8.55 | .001 | 0.17 |
| Drowsy | 10.03 | .001 | 0.20 | 13.35 | .001 | 0.25 | 8.39 | .001 | 0.17 |
| Depressed | 13.97 | < .001 | 0.25 | 9.00 | .01 | 0.18 | 12.06 | < .001 | 0.23 |
| Desire Sweets | 4.48 | .01 | 0.10 | 12.98 | .001 | 0.24 | 6.05 | 0.01 | 0.13 |
| Tiffany Drobesb | |||||||||
| Factor 1 | 19.36 | < .001 | 0.32 | 44.81 | < .001 | 0.52 | 3.10 | .06 | 0.07 |
| Factor 2 | 10.21 | .001 | 0.20 | 30.49 | < .001 | 0.43 | 4.39 | .03 | 0.10 |
| DEPSb | |||||||||
| Satisfying | 9.73 | .001 | 0.19 | 62.38 | < .001 | 0.60 | 11.03 | < .001 | 0.21 |
| Pleasant | 11.13 | .001 | 0.21 | 38.34 | < .001 | 0.48 | 9.10 | .002 | 0.18 |
| Taste Good | 6.13 | .01 | 0.13 | 16.83 | < .001 | 0.29 | 0.81 | .41 | 0.02 |
| Dizzy | 0.73 | .45 | 0.02 | 16.30 | < .001 | 0.28 | 5.12 | .02 | 0.11 |
| Calm | 1.08 | .34 | 0.03 | 48.73 | < .001 | 0.54 | 21.66 | < .001 | 0.35 |
| Concentrate | 4.75 | .02 | 0.10 | 30.04 | < .001 | 0.44 | 6.79 | .003 | 0.14 |
| Awake | 2.01 | .15 | 0.05 | 40.24 | < .001 | 0.50 | 14.70 | < .001 | 0.26 |
| Reduce Hunger | 1.75 | .19 | 0.04 | 17.93 | < .001 | 0.30 | 4.73 | .02 | 0.10 |
| Sick | 0.38 | .64 | 0.01 | 3.52 | .07 | 0.08 | 0.26 | .083 | 0.01 |
Note.
MNWS = Minnesota Nicotine Withdrawal Scale DEPS = Direct Effects of Product Scale
dfgroup = (1,39); all other df = (2,78)
dfgroup = (1,41); all other df = (2,82)
A significant main effect of group was observed for Tiffany-Drobes Factor 1 [F(1,41) = 10.12, p < .01, ηp2 = 0.20] and Factor 2 [F(1,41) = 5.23, p < .05, ηp2 = 0.11]. For both, scores (collapsed across device, bout, and time) were higher for ECIG-naïve cigarette smokers than for ECIG-experienced users: 15.7 (SE = 1.8) versus 8.4 (SE = 1.5) respectively for Factor 1 (intention to vape), and 6.8 (SE = 1.1) versus 3.5 (SE = 0.9) respectively for Factor 2 (anticipation of relief from withdrawal). Factor 1 scores also revealed significant main effects of bout and time (F’s = 19.35 and 44.81, respectively, ps < .001). Factor 1 scores were higher at the directed bout 1 (M = 13.8, SE = 1.3) than ad lib bout 3 (M = 10.5, SE = 1.1; Tukey’s HSD, p < .05), but not ad lib bout 2 (M = 11.8, SE = 1.2). Additionally, Factor 1 scores were higher at pre-bout timepoints (M = 15.1, SE = 1.4) than post-bout timepoints (M = 9.0, SE = 1.1). Factor 2 was significant for a Bout by Time interaction [F(2,82) = 4.39, p < .05, ηp2 = 0.10], with mean scores of 7.8 (SE = 1.0) versus 4.2 (SE = 0.7) from pre- to post-directed bout 1, 3.8 (SE = 0.7) versus 5.5 (SE = 0.9) from pre- to post- ad lib bout 2, and 3.4 (SE = 0.6) versus 6.4 (SE = 0.9) from pre- to post-ad lib bout 3 (Tukey’s HSD, ps < .01 for all pre-post comparisons).
Direct Nicotine and Product Effects
Only 4 of a possible 190 interactions involving the Group and Device factors were observed to be significant for the DENS and DEPS measures (other Fs < 3.90, ps > .05, ηp2 < 0.09), and thus, these interactions are excluded from Tables 5 and 6. Also omitted from the tables are all other items for the DENS, with only 3 of 50 main/interaction effects revealed to be statistically significant. Specifically, “lightheaded” was significant for a Bout by Time interaction, F(2,82) = 3.76, p < .05, ηp2 = 0.08, with scores generally increasing from before to after each bout. However, these increases were reliable only for the directed bout 1: 8.1 (SE = 1.7) to 17.3 (SE = 3.3) from pre-post directed bout 1 (Tukey’s HSD, p < .01), 10.1 (SE = 2.4) to 15.6 (SE = 3.6) from pre-post ad lib bout 2, and 10.3 (SE = 2.6) to 13.5 (SE = 3.5) from pre-post ad lib bout 3. “Nervous” was significant for a main effect of bout, F(2,82) = 3.46, p = .045, ηp2 = 0.08. Scores were not significantly different between directed 1 (M = 7.5, SE = 1.7), ad lib 2 (M = 6.2, SE = 1.8), and ad lib 3 bouts (M = 5.3, SE = 1.5 (Tukey’s HSD, ns).
Most DEPS items were significant for Bout by Time (Fs > 4.73, ps < .05, ηp2 > 0.10). Generally, ratings increased from pre- to post-bouts, with the largest increase occurring for the directed bout. Ratings of “satisfying” increased from pre-directed (M = 8.0, SE = 2.5) to post-directed bout 1 (M = 46.8, SE = 4.5), from pre-ad lib (M = 23.4, SE = 4.2) to post-ad lib bout 2 (M = 48.5, SE = 5.0), and from pre-ad lib (M = 27.5, SE = 4.6) to post-ad lib bout 3 [M = 47.6, SE = 5.3, F(2,82) = 11.03, p < .001, ηp2 = 0.21; Tukey’s HSD, ps < .01 for pre-post comparisons at each bout]. Ratings of concentrate were lower at pre-directed (M = 6.4, SE = 2.3) than post-directed bout 1 (M = 30.7, SE = 4.4), pre-ad lib bout 2 (M = 20.0, SE = 4.1), post-ad lib bout 2 (M = 30.5, SE = 4.1), and post-ad lib bout 3 (M = 29.8, SE = 5.0, ps < .05), but not different from pre-ad lib bout 3 (M = 16.0, SE = 3.8), F(2,82) = 6.79, p < .01, ηp2 = 0.14 . Ratings at pre-ad lib 3 also were lower than post-directed, post-ad lib 2, and post-ad lib 3 (Tukey’s HSD, ps < .05). Similar patterns were observed for other DEPS items.
DISCUSSION
The purpose of this study was to compare ECIG puff topography as measured via mouthpiece-based computerized device (eTop) to video-based observation. For all topography outcomes, no significant differences between these 2 measurement methods were observed and correlations were moderate to high. This pattern suggests that the eTop is a valid method for measurement of ECIG topography. Both methods also were reliable, as demonstrated by significant correlations between bouts for topography outcomes. Moreover, high correlations between the eTop and video recordings of the eTop in use demonstrate accuracy of this measurement method. These correlations were expected to be higher than the moderate correlation between the eTop and video alone, because they are based on measurement of the same bout rather than bouts from different sessions. For example, average puff durations did not significantly differ when using eTop versus video alone, though individual puffs may not have varied consistently across bouts (eg, first puff longer than second for one bout, opposite pattern for second bout). Together, these data support the idea that use of a mouthpiece does not interfere with ECIG puffing behavior.
Unlike previous work,2,22 puff topography did not differ between groups. ECIG-experienced users have shown to take longer and larger puffs than ECIG-naïve cigarette smokers.2,22-24 The lack of group differences in the current study may be explained by several factors. First, our sample sizes (ECIG-experienced = 25; ECIG-naïve = 18) were smaller than other studies (eg, n ~ 30/group in Hiler et al),2 and thus, our study may have lacked power to detect between-group topography differences. Second, the ECIG device and unflavored liquid used in our study were different from those preferred by our ECIG experienced users (ie, tanks/mods with sweet/fruit flavored liquids). Other work has often permitted participants to use their own device/liquid during sessions or at least to choose between a few liquid flavor options.2,12 These differences may have altered the puffing behavior of the experienced users. Of course, the use of an unflavored liquid also may have affected puffing for ECIG-naïve cigarette smokers; among smokers who have tried ECIGs, fruit/sweet flavors are most preferred followed by mint/menthol and tobacco.25 Third, the required period of overnight nicotine/tobacco abstinence was able to be verified with samples of expired air CO, but not plasma nicotine.2 As most ECIGs do not produce significant CO, ECIG use in the hours immediately before session may have gone undetected for the ECIG-experienced users. In support of the idea that the ECIG-naïve cigarette smokers experienced a longer period of abstinence, baseline levels of tobacco/nicotine abstinence symptoms were significantly higher for this group relative to the ECIG-experienced users. Periods of biologically-confirmed overnight abstinence result in more intense puffing (eg, more, longer, larger puffs).7,26 Consequently, the expected group differences in puff topography may have been eliminated by a differential level of abstinence between groups.
The eTop and video-based methods also did not differ on subjective ratings of nicotine/tobacco abstinence or direct product effects, consistent with previous work.12 Importantly, however, these subjective ratings were influenced by ECIG use; nicotine/tobacco abstinence symptoms (eg, craving, difficulty concentrating) were suppressed, and product effects (eg, satisfying, taste good) were increased from pre- to post-ECIG bouts. A similar pattern was observed for heart rate, with beats/minute increasing significantly at all bouts independent of the measurement method used. Still, use of the eTop altered a few aspects of the ECIG puffing experience. Participants provided higher ratings of “alter behavior,” “reduce enjoyment,” and “make puffing more difficult” when using this device compared to video alone. This same finding has been reported when cigarette smoking topography measurement is compared between mouthpiece-based computerized devices and video alone. Specifically, use of a mouthpiece influenced perceptions of the smoking experience (eg, smoking enjoyment and difficulty) without changing cigarette puff topography.7 Similar discrepancies between subjective and behavioral responses have been observed for other drugs of abuse.27,28 For example, doses of opioids that produce no detectable subjective effects,29 or even aversive subjective effects (eg, “disliking”),30 are self-administered nonetheless by opioid users.
Whereas video-based topography measurement offers a valid tool that has minimal influence on ECIG puffing characteristics, its limitations include the amount of labor/time necessary for evaluating videos and the inability to measure puff volume and flow rate. In contrast, the eTop device measures data instantaneously and captures these important topography parameters. Mouthpiece-based measurement of topography is limited, however, by the fact that ECIGs come in a variety of shapes and sizes that sometimes require custom fabrication of the mouthpiece. Thus, computerized devices like the eTop require multiple mouthpieces that can be switched out as needed to fit a given ECIG, a challenge considering how rapidly new ECIG designs appear on the market. In addition, the eTop in this study was tethered to a laptop computer, and therefore prevents its deployment in studies of user topography in the natural environment, a limitation also of video recording. As has been demonstrated for cigarette topography, ECIG topography is likely affected by many factors in users’ natural environment (eg, other drug use; emotion).31,32 Nonetheless, the current laboratory-based version of the eTop provides a reliable and valid method for measuring ECIG puff topography in that setting.
IMPLICATIONS FOR TOBACCO REGULATION
The regulation of ECIG-delivered nicotine will be informed by systematic evaluation of users’ response to the many different combinations of product design and liquid features. Indeed, users may alter their puffing behavior to extract more or less nicotine from an ECIG as these features change. Therefore, it is vital to develop valid and reliable methods to measure ECIG puff topography. Such methods can then be used by the US Food and Drug Administration to predict how users might respond to a given set of product configurations, and whether that response may expose users to nicotine concentrations that are both safe and effective.
Human Subjects Approval Statement
All study procedures were approved by the West Virginia University Institutional Review Board (#1508790777). Informed consent was obtained from all participants. The treatment of human subjects was in compliance with the ethical standard outlined in the Helsinki Declaration of 1975 as revised in 2000.
Acknowledgements
This work was supported by the National Institutes of Health (NIH) and the Center for Tobacco Products of the US Food and Drug Administration (FDA) (3P50DA036105-03S1), as well as the Center for the Study of Tobacco Products Pilot Research Program (5P50DA036105-05). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or FDA. Support also provided by the National Institute of General Medical Sciences (T32GM081741). This paper is based on thesis research that was conducted, completed, and defended at West Virginia University by Nicholas Felicione under the guidance of Melissa Blank. Portions of this work were presented at the NIH Tobacco Regulatory Science Meeting (June 2018; Bethesda, MD, US) and at the 25th Annual Meeting of the Society for Research on Nicotine and Tobacco (February 2019; San Francisco, CA, US).
Footnotes
Conflict of Interest Disclosure Statement
Drs. Eissenberg and Shihadeh are paid consultants in litigation against the tobacco industry and are named on a patent application for a device that measures the puffing behavior of electronic cigarette users that is not described in this manuscript. Dr. Eissenberg also is a paid consultant in litigation against the electronic cigarette industry.
Contributor Information
Nicholas J. Felicione, Roswell Park Comprehensive Cancer Center, Department of Health Behavior, Buffalo, NY, United States..
Nareg Karaoghlanian, American University of Beirut, Department of Mechanical Engineering, Beirut, Lebanon..
Alan Shihadeh, American University of Beirut, Department of Mechanical Engineering, Beirut, Lebanon..
Thomas Eissenberg, Virginia Commonwealth University, Department of Psychology, Center for the Study of Tobacco Products, Richmond, VA, United States..
Melissa D. Blank, West Virginia University, Department of Psychology, WV Prevention Research Center, Morgantown, WV, United States..
References
- 1.Wagener TL, Floyd EL, Stepanov I, et al. Have combustible cigarettes met their match? The nicotine delivery profiles and harmful constituent exposures of second-generation and third-generation electronic cigarette users. Tob Control. 2017;26:e23–e28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hiler M, Breland A, Spindle T, et al. Electronic cigarette user plasma nicotine concentration, puff topography, heart rate, and subjective effects: influence of liquid nicotine concentration and user experience. Exp Clin Psychopharm. 2017;25(5):380–392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hsu G, Sun JY, Zhu SH. Evolution of electronic cigarette brands from 2013–2014 and 2016–2017: analysis of brand websites. J Med Internet Res. 2018;20(3):e80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Herning RI, Jones RT, Bachman J, Mines AH. Puff volume increases when low-nicotine cigarettes are smoked. Br Med J (Clin Res Ed). 1981;283(6285):187–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Koszowski B, Rosenberry ZR, Kanu A, et al. Nicotine and carbon monoxide exposure from inhalation of cigarillo smoke. Parmacol Biochem Behav. 2015;139(Pt A):7–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Cobb CO, Blank MD, Morlett A, et al. Comparison of puff topography, toxicant exposure, and subjective effects in low- and high-frequency waterpipe users: a double-blind, placebo-control study. Nicotine Tob Res. 2015;17(6):667–674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Blank MD, Disharoon S, Eissenberg T. Comparison of methods for measurement of smoking behavior: mouth-piece-based computerized devices versus direct observation. Nicotine Tob Res. 2009:11(7);896–903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Shihadeh A, Antonios C, Azar S. A portable, low-resistance puff topography instrument for pulsating, high-flow smoking devices. Behav Res Methods. 2005;37(1):186–191. [DOI] [PubMed] [Google Scholar]
- 9.Mikheev VB, Buehler SS, Brinkman MC, et al. The application of commercially available mobile cigarette topography devices for e-cigarette vaping behavior measurement. Nicotine Tob Res. 2020;22(5):681–688. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Behar RZ, Hua M, Talbot P. Puffing topography and nicotine intake of electronic cigarette users. PLoS One. 2015;10(2):e0117222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Lee YH, Gawron M, Goniewicz ML. Changes in puffing behavior among smokers who switched from tobacco to electronic cigarettes. Addict Behav. 2015;48:1–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Spindle TR, Hiler MM, Breland AB, et al. The influence of a mouthpiece-based topography measurement device on electronic cigarette user’s plasma nicotine concentration, heart rate, and subjective effects under directed and ad libitum use conditions. Nicotine Tob Res. 2017;19(4):469–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.St. Helen G, Ross KC, Dempsey DA, et al. Nicotine delivery and vaping behavior during ad libitum e-cigarette access. Tob Regul Sci. 2016;2(4):363–376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Robinson RJ, Hensel EC, Morabito PN, Roundtree KA. Electronic cigarette topography in the natural environment. PLoS One. 2015;10(6):e0129296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Koszowski B, Salazar E, Hill Thanner M, et al. Electronic nicotine delivery systems: comparative topography assessments. Tob Regul Sci. 2020;6(1):54–65. [Google Scholar]
- 16.Spindle TR, Talih S, Hiler MM, et al. Effects of electronic cigarette liquid solvents propylene glycol and vegetable glycerin on user nicotine delivery, heart rate, subjective effects, and puff topography. Drug Alcohol Depend. 2018;188:193–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hughes JR, Hatsukami D. Signs and symptoms of tobacco withdrawal. Arch Gen Psychiatry. 1986;43(3):289–294. [DOI] [PubMed] [Google Scholar]
- 18.Cox LS, Tiffany ST, Christen AG. Evaluation of the brief questionnaire of smoking urges (QSU-brief) in laboratory and clinical settings. Nicotine Tob Res. 2001;3(1):7–16. [DOI] [PubMed] [Google Scholar]
- 19.Evans SE, Blank M, Sams C, et al. Transdermal nicotine-induced tobacco abstinence symptom suppression: nicotine dose and smokers’ gender. Exp Clin Psychopharm. 2006;14(2):121–135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström test for nicotine dependence: a revision of the Fagerström tolerance questionnaire. Br J Addict. 1991;86(9):1119–1127. [DOI] [PubMed] [Google Scholar]
- 21.Foulds J, Veldheer S, Yingst J, et al. Development of a questionnaire for assessing dependence on electronic cigarettes among a large sample of ex-smoking e-cigarette users. Nicotine Tob Res. 2015;17(2):186–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Farsalinos KE, Spyrou A, Stefopoulos C, et al. Nicotine absorption from electronic cigarette use: Comparison between experienced consumers (vapers) and naïve users (smokers). Sci Rep. 2015;5:11269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Spindle TR, Breland AB, Karaoghlanian NV, et al. Preliminary results of an examination of electronic cigarette user puff topography: the effect of a mouthpiece-based topography measurement device on plasma nicotine and subjective effects. Nicotine Tob Res. 2015;17(2):142–149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hua M, Yip H, Talbot P. Mining data on usage of electronic nicotine delivery systems (ENDS) from YouTube videos. Tob Control. 2013;22(2):103–106. [DOI] [PubMed] [Google Scholar]
- 25.Berg CJ. Preferred flavors and reasons for e-cigarette use and discontinued use among never, current, and former smokers. Int J Public Health. 2016;61(2):225–236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Zacny JP, Stitzer ML. Effects of smoke deprivation interval on puff topography. Clin Pharmacol Ther. 1985;38(1):109–115. [DOI] [PubMed] [Google Scholar]
- 27.Dudish-Poulsen SA, Hatsukami DK. Dissociation between subjective and behavioral responses after cocaine stimuli presentations. Drug Alcohol Depend. 1997;47(1):1–9. [DOI] [PubMed] [Google Scholar]
- 28.Liguori A, D’Agostino RB, Sworkin SI, et al. Alcohol effects on mood, equilibrium, and simulated driving. Alcohol Clin Exp Res. 1999;23(5):815–821. [PubMed] [Google Scholar]
- 29.Lamb RJ, Preston KL, Schindler CW, et al. The reinforcing and subjective effects of morphine in post-addicts: a dose-response study. J Pharmacol Exp Ther. 1991;259(3):1165–1173. [PubMed] [Google Scholar]
- 30.Jones BE, Prada JA. Drug-seeking behavior during methadone maintenance. Psychopharmacologia. 1975;41(1):7–10. [DOI] [PubMed] [Google Scholar]
- 31.Farris SG, Aston ER, Leyro TM, et al. Distress intolerance and smoking topography in the context of a biological challenge. Nicotine Tob Res. 2019;21(5):568–575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Roche DJ, Bujarski S, Hartwell E, et al. Combined varenicline and naltrexone treatment reduces smoking topography intensity in heavy-drinking smokers. Pharmacol Biochem Behav. 2015;134:92–98. [DOI] [PMC free article] [PubMed] [Google Scholar]