Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Oct 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2019 Feb 18;27(5):443–454. doi: 10.1037/pha0000261

Abuse Liability Assessment of an Electronic Cigarette in Combustible Cigarette Smokers

Sarah F Maloney 1,2, Alison Breland 1,2, Eric K Soule 1,2,3,*, Marzena Hiler 1,2, Carolina Ramôa 4,*, Thokozeni Lipato 5, Thomas Eissenberg 1,2
PMCID: PMC6754311  NIHMSID: NIHMS1048901  PMID: 30777773

Abstract

Under certain conditions, electronic cigarettes (ECIGs) can deliver nicotine to and suppress tobacco abstinence symptoms in cigarette smokers. Growing popularity of ECIGs raises abuse liability concerns. This study’s purpose was to compare the abuse liability of an ECIG (1.5 Ohm, 3.3 Volt) filled with 36 mg/ml or 0 mg/ml nicotine to an FDA-approved nicotine inhaler and participants’ own brand of cigarettes. Smokers (N = 24) completed four sessions in which they completed to the Multiple-Choice Procedure (MCP) and plasma nicotine concentration and subjective effects were measured. Mean (SD) MCP crossover point was $0.87 (1.0) for the 36 mg/ml nicotine ECIG and $0.96 (1.2) for the 0 mg/ml ECIG, significantly higher than the nicotine inhaler mean of $0.32 (0.6) but significantly lower than the own brand cigarette mean of $1.42 (1.4). Ten puffs from an own brand cigarette increased mean plasma nicotine concentration from 3.55 (2.8) to 13.64 (9.8) ng/ml, as compared to an increase from 3.16 (1.8) to 8.51 (5.4) ng/ml for the 36 mg/ml ECIG. The 36 mg/ml ECIG reduced nicotine abstinence symptoms more than the 0 mg/ml ECIG, and both ECIGs were rated as more reinforcing than the inhaler but less reinforcing than participants’ own brand cigarettes (ps < .05). Results suggest that the ECIG examined had higher abuse liability than the nicotine inhaler but lower than combustible cigarettes. These data and methods may be useful for policymakers by revealing how ECIG abuse liability compares to tobacco/nicotine products with abuse liability profiles that are well established.

Keywords: electronic cigarettes, abuse liability, Multiple-Choice Procedure, combustible cigarettes, nicotine inhaler


Tobacco cigarette smoking is the leading cause of preventable death worldwide (World Health Organization, 2008). Combustible cigarettes now share the market with a variety of other tobacco products, including electronic cigarettes (ECIGs). ECIGs entered the U.S. market in 2007 (Regan, Promoff, Dube, & Arrazola, 2013) and have been advertised as cigarette alternatives for smokers (Pearson, Richardson, Niaura, Vallone, & Abrams, 2012). In 2017, 2.8% of U.S. adults were current (someday or everyday use) ECIG users (Wang et al., 2018). Higher rates of current ECIG use were reported among recent former cigarette smokers (one year or less; 22.0%) and current cigarette smokers (15.9%), than never smokers (0.4%; Schoenborn & Gindi, 2015). Some smokers report using ECIGs for smoking reduction and cessation (Breland et al., 2017; Dockrell et al., 2013; Etter, 2010). Notably, in 2017 current (past 30-day) ECIG use (11.7%) was more common than combustible cigarette use (7.6%) among U.S. high school students (Wang et al., 2018).

ECIGs are battery-powered devices that aerosolize a liquid that is often contained in a reservoir (i.e., tank, cartridge, pod). Users inhale the aerosol through a mouthpiece, mimicking some of the behaviors associated with combustible tobacco cigarettes. ECIG users can purchase ECIG liquids in thousands of different flavors (Zhu et al., 2014) and nicotine concentrations that often range from 0 mg/ml to 36 mg/ml (Breland et al., 2017); recently liquids with even higher nicotine concentrations have been reported (Gibson-Young, 2018). ECIGs vary in their ability to deliver nicotine, such that early products delivered little or no nicotine to users (e.g., Hajek, Przulj, Phillips, Anderson & McRobbie, 2017; Vansickel, Cobb, Weaver, & Eissenberg, 2010), but as the product class evolved, some newer and more advanced models of ECIGs are capable of delivering more nicotine to users than early models (e.g., Farsalinos et al., 2014; Hiler et al. 2017; Vansickel & Eissenberg, 2013; Wagener et al., 2017).

The growing popularity of ECIGs and the ability of these devices to deliver substantial amounts of nicotine to users have raised concerns among policymakers regarding their impact on public health (Green, Fielding, & Brownson, 2018; Palazzolo, 2013; Walton et al., 2015). Specifically, many individuals question the abuse liability of ECIGs, or the likelihood for these products to be used in excess and produce dependence and adverse consequences (Balster & Walsh, 2010; Jaffe & Jaffe, 1989; Carter et al., 2009). The likelihood for a drug or product to be abused is contingent on many factors such as the rate of drug delivery, sensory and subjective reinforcing effects, social acceptability, and product appeal and attractiveness (Balster & Walsh, 2010; Carter et al., 2009; Henningfield, Hatsukami, Zeller, & Peters, 2011; Jaffe & Jaffe, 1989). Generally, for common drugs of abuse, as the rate and dose of drug delivery increases, the rewarding effects (e.g., euphoric feelings) and the likelihood for that drug or drug product to be abused also increases (Jaffe & Jaffe, 1989; Benowitz, 1996). For example, combustible cigarettes deliver nicotine quickly to users and produce rewarding effects, and therefore have high abuse liability (Henningfield & Keenan, 1993; Benowitz, 1996); nicotine replacement therapies (NRT) deliver nicotine relatively slowly, produce weak rewarding effects, and therefore have lower abuse liability (Schuh, Schuh, Henningfield, & Stitzer, 1997).

Abuse liability can be evaluated using a variety of methods, including subjective effect measurements and behavioral tasks (Carter et al., 2009; Fischman & Foltin, 1991). With regard to subjective effects, the assessment of positive reinforcing effects (e.g., euphoria) and negative reinforcing effects (e.g., suppression of aversive drug abstinence symptoms) is important in determining abuse liability. Products that produce these reinforcing effects have an increased likelihood of being used repeatedly (e.g., Eissenberg, 2004). ECIGs can suppress nicotine abstinence symptoms in tobacco cigarette smokers and in experienced ECIG users (Dawkins & Corcoran, 2014; Hiler et al., 2017; Vansickel & Eissenberg, 2013; Vansickel, Weaver, & Eissenberg, 2012), suggesting negative reinforcement. Interestingly, these abstinence-suppressing effects have been observed with ECIGs that do not deliver nicotine to users (Vansickel, Cobb, Weaver & Eissenberg, 2010).

With regard to behavioral tasks, choice procedures have been used to assess the abuse liability of ECIGs, as well as other drugs and drug products. The Multiple-Choice Procedure (MCP; Griffiths, Troisi, Silverman, & Mumford, 1993) is a useful tool for assessing the abuse liability of nicotine and tobacco products because it allows comparisons across drug products (Carter et al., 2009). The MCP involves having participants make discrete choices between a drug/drug product and increasing amounts of money by circling their choice on a pen and paper task. A choice (i.e., money or product) is selected randomly and presented to the participant to keep or use. The highest monetary value that participants choose to receive product puffs over money is used to estimate the product’s reinforcing efficacy and is often called the crossover point. To our knowledge, four clinical laboratory studies have used the MCP to assess ECIG abuse liability (Barnes, Bono, Lester, Eissenberg, & Cobb, 2017; Krishnan-Sarin et al., 2017; McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012).

Several studies have investigated the abuse liability of nicotine containing ECIGs (i.e., 18 mg/ml and 24 mg/ml) in cigarette smokers using the MCP (McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012). In these studies, participants’ own brand cigarettes had higher crossover points on the MCP than the nicotine-containing ECIGs examined, suggesting that, in cigarette smokers, nicotine-containing ECIGs have a lower reinforcing efficacy than combustible cigarettes (McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012). The MCP has been used to assess the influence of ECIG liquid flavor (tobacco vs. menthol; cherry vs. unflavored) and ECIG reduced harm messaging (reduced harm message or reduced carcinogen exposure message vs. no message) in cigarette smokers (Barnes, Bono, Lester, Eissenberg, & Cobb, 2017). The average MCP crossover point in this study was lower for all ECIG conditions compared to participants’ own brand cigarettes, except for the tobacco flavored ECIGs regardless of the presence of a reduced harm message. Interestingly, the cherry flavored ECIGs had higher crossover points than the unflavored ECIGs, indicating greater abuse potential, and there were no differences observed between the ECIGs with and without a reduced harm/carcinogen message on the MCP regardless of flavor (Barnes, Bono, Lester, Eissenberg, & Cobb, 2017). A recent study used a modified version of the MCP to investigate the reinforcement value of different ECIG liquid menthol and nicotine concentrations in ECIG users, some of whom also used combustible cigarettes. There were no significant differences between the ECIG liquid menthol or nicotine concentrations on the modified MCP, however, the presence of menthol increased subjective ratings of wanting and liking in the ECIG conditions (Krishnan-Sarin et al., 2017). Three of these clinical studies suggest that the ECIGs examined have a lower abuse liability than combustible cigarettes in samples of cigarette smokers (Barnes, Bono, Lester, Eissenberg, & Cobb, 2017; McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012). Two studies suggest that other factors, such as flavor, may influence ECIG abuse liability as well (Barnes, Bono, Lester, Eissenberg, & Cobb, 2017; Krishnan-Sarin et al., 2017).

Examining ECIGs systematically to understand their potential harms to nicotine-naïve individuals’ and potential benefits to smokers can inform product regulations (Walton et al., 2015). In this context, regulatory action can be guided by determining the abuse liability of ECIGs and comparing it to a product with a known high abuse liability (i.e., combustible cigarettes; Schuh, Schuh, Henningfield, & Stitzer, 1997) and a product with a known low abuse liability (e.g., nicotine inhaler; West et al., 2000). Furthermore, comparing a nicotine-containing ECIG to a non-nicotine-containing ECIG will help to provide insight into the role that nicotine plays in ECIG abuse liability. To our knowledge, there are there are no published studies examining an ECIG with and without nicotine on measures of abuse liability in comparison to an FDA-approved nicotine inhaler and combustible cigarettes. Therefore, the current study aims to investigate the abuse liability of an ECIG with 0 mg/ml nicotine and 36 mg/ml nicotine in comparison to combustible cigarettes and the Nicotrol nicotine inhaler in cigarette smokers.

Method

Participants

This study was approved by Virginia Commonwealth University’s (VCU’s) institutional review board (IRB: HM20005746). Cigarette smokers were recruited by local advertisements and word of mouth. Participants were eligible if they were between the ages of 18 and 55, smoked ten or more cigarettes per day for at least a year, had an expired carbon monoxide (CO) ≥ 15 ppm at screening, and were willing to use an ECIG in the lab. Individuals were not eligible to participate if they reported any current disease or psychiatric conditions, history of chronic organ-related disease, high or low blood pressure, breast feeding, pregnancy (verified by urinalysis), regular use of prescription medications (excluding birth control or vitamins), past-month use of cocaine, opioids, benzodiazepines, methamphetamines, past 30-day use of marijuana >10 days, or past 30-day use of alcohol > 25 days. Individuals who had used an ECIG > 20 times in their lives were considered to be experienced with ECIGs and were not eligible to participate in this study. We limited the sample to ECIG-naïve smokers to ensure that the positive control (own brand cigarettes) would test unequivocally positive (Carter & Griffiths, 2009).

Procedures

Once screening was complete, eligible participants were scheduled for four separate lab sessions that were separated by a minimum of 48 hours. In each session, participants used one of four study products: their own brand of cigarettes (OB), an ECIG with nicotine (ECIG_36), an ECIG without nicotine (ECIG_0), and a nicotine inhaler (IN). These four sessions (see Table 1 for session procedure details) were each approximately five hours long, Latin-square ordered, and ECIG conditions were double blind (keeping participants and staff blind to the own brand cigarette and the inhaler conditions was not feasible). Participants were required to come into the lab at least 12 hours abstinent from all nicotine and tobacco products and were asked to provide an expired air CO sample at the beginning of each session to verify abstinence from combustible tobacco (≤ 10 ppm, as in Hiler et al., 2017). See Table 1 for a detailed description of the session procedures.

Table 1.

Table of session procedures.

Time Study Task

− 10 minutes Abstinence verification via expired CO sample; participant connected to physiological monitoring equipment.
0 minutes IV catheter inserted into participant’s forearm.
5 minutes Baseline subjective questionnaires completed.
30 minutes Baseline blood sample taken and subjective questionnaires completed.
35 minutes First directed 10-puff product bout.
40 minutes Blood sample #2 taken and subjective questionnaires completed.
55 minutes Subjective questionnaires completed.
60 minutes Second directed 10-puff bout.
65 minutes Blood sample #3 taken and subjective questionnaires completed.
 - 90 minute rest period -
155 minutes Blood sample #4 taken and subjective questionnaires completed. IV catheter removed from participant’s forearm.
165 minutes MCP administered and choice selected by participant randomly.
170 minutes Choice reinforcement: 10 minute, 10-puff product consumption period or participants receive payment in cash.
180 minutes Subjective questionnaires completed.
195 minutes Subjective questionnaires completed.
200 minutes MCP administered and choice selected by participant randomly.
205 minutes Choice reinforcement: 10 minute, 10-puff product consumption period or participants receive payment in cash.
215 minutes Subjective questionnaires completed.
230 minutes Subjective questionnaires completed.
235 minutes MCP administered and choice selected by participant randomly.
240 minutes Choice reinforcement: 10 minute, 10-puff product consumption period or participants receive payment in cash.
250 minutes Subjective questionnaires completed.
255 minutes Participants disconnected from the physiological monitoring equipment and compensated for their time.

Materials

During each of the four sessions, participants were provided with a study product; OB, ECIG_36, ECIG_0, and IN. In the OB condition, participants’ usual brand of cigarettes was provided to them. In the ECIG conditions, participants were provided with an “eGo” 3.3 Volt, 1000 mAh ECIG battery (11.2 cm high cylinder, 1.9 cm diameter at the base) paired with a 1.5 Ohm, dual-coil, 510-style “cartomizer” (5 cm high cylinder, 0.9 cm diameter at the base, black opaque color; both from SmokTech, Shenzhen, China). The cartomizer was preloaded with 1 ml of 70% propylene glycol/30% vegetable glycerin flavored liquid (tobacco or menthol, matched to participants’ OB cigarette flavor). The two ECIG study sessions differed by nicotine concentration of the liquid; 0 mg/ml or 36 mg/ml nicotine. ECIG liquid nicotine concentration was verified independently by VCU’s Bioanalytical Shared Resources Laboratory prior to administration. Finally, in the IN condition participants were provided the Nicotrol inhaler (10 mg nicotine, Pfizer).

Outcome Measures

Multiple-Choice Procedure.

The MCP is a pen and paper task that measures and allows for comparisons of abuse liability between different drugs and drug delivery platforms (Griffiths, Troisi, Silverman, & Mumford, 1993). In the current study, participants were asked to make eleven choices between increasing amounts of money (i.e., $0.01, $0.02, $0.04, $0.08, $0.16, $0.32...$10.24) or 10 puffs from the study product used in that session (OB, ECIG_36, ECIG_0, or IN); participants selected their choice by circling the preferred item in the pair (money versus 10 puffs). After completing the entire task, one of the pairs was drawn randomly by the participant. Participants were then presented immediately with the item they circled in that pair, either 10 puffs from the session product (that could be taken at the participant’s leisure during a 10 minute consumption period) or the corresponding monetary amount (in cash). The pair with the highest dollar amount at which participants chose to receive product puffs over the corresponding dollar value was the crossover point. The higher the crossover point, the greater the reinforcing efficacy of the product. For participants who chose money for all eleven choices, a crossover point of $0.00 was used for analysis and for participants who choose all study product a crossover point of $10.24 was used for analysis.

Physiological measures.

Immediately after blood collection, samples were centrifuged, and the serum was collected and stored at −70°C until it was sent to VCU’s Bioanalytical Analysis Core Laboratories and analyzed for plasma nicotine concentration. The limit of quantitation (LOQ) for the samples was 2 ng/ml (for additional detail, see Breland et al, 2006). A Criticare Systems model 507 monitored heart rate (every 20 seconds) and blood pressure (every four minutes). Expired CO was measured at the start of every session using the BreathCO monitor (Vitalograph, Lenexa, KS) that was fitted with a disposable mouth piece.

Subjective questionnaires.

Four subjective questionnaires were administered at different time points throughout the session: the Hughes and Hatsukami Tobacco Withdrawal Scale (Hughes & Hatsukami, 1986), the Direct Effects of Nicotine Questionnaire (Evans et al., 2006; Perkins et al., 1994), the Direct Effects of Product Use Questionnaire (Buchhalter et al., 2005; Foulds et al., 1992; Pickworth, Bunker, & Henningfield, 1994), and the Tiffany-Drobes Questionnaire of Smoking Urges-brief (Tiffany & Drobes, 1991). With the exception of the Tiffany-Drobes Questionnaire of Smoking Urges-brief, all items were administered using a computerized visual analog scale that had a horizontal line that was anchored with “not at all” on the far left and “extremely” on the far right. Participants were presented with a word or phrase in the center of the horizontal line and were asked to rate how they were currently feeling by using a mouse to click anywhere on the line. Scores were calculated as a percentage of the total line length that participants marked relative to the left anchor (0 – 100). Items from the Tiffany-Drobes Questionnaire of Smoking Urges-brief were rated on a seven-point Likert scale in which participants clicked seven discrete ratings from ‘not at all’ to ‘extremely’.

For the present study, an adapted version of the Hughes and Hatsukami Tobacco Withdrawal Scale (Hughes & Hatsukami, 1986) was used to measure tobacco abstinence symptoms (see Breland, Evans, et al., 2002; Buchhalter et al., 2005). This shortened version of the scale consisted of eleven items and was administered a total of 11 times in each session; baseline, before and immediately following the two directed bouts, and before and after each of the three MCP procedures/choice reinforcements (product puffs or money). The Direct Effects of Nicotine Questionnaire was modified from a previous study (Perkins et al., 1994), consisted of ten smoking related items, and was administered five times in each session; baseline and before and immediately following the first two bouts. The Direct Effects of Product Use Questionnaire was modified from the “Direct Effects of Tobacco” scale that was developed through previous studies evaluating the subjective effects of tobacco cigarette smoking (Buchhalter et al., 2005; Foulds et al., 1992; Pickworth, Bunker, & Henningfield, 1994). This modified scale was generalized for participants to rate the four different study products and was administered three times in each session: immediately following the first bout, and before and after the second bout. The TD was modified from the original Questionnaire of Smoking Urges (Tiffany & Drobes, 1991) and consisted of ten smoking-related items that were scored to create two factors: intention to smoke (Factor 1; 0–30) and anticipation of relief from smoking abstinence (Factor 2; 0–24; Cox, Tiffany, & Christen, 2001). This scale was administered five times in each session; baseline and before and immediately following the first two bouts.

Data Preparation

Plasma samples for which the analysis results were less than the LOQ (2 ng/ml) were replaced with 2 ng/ml (as in Vansickel et al., 2010). Each baseline sample was inspected to determine if participants were abstinent for at least 12 hours from all nicotine and tobacco products at the start of each study session; a value greater than 5.0 ng/ml was indicative of non-abstinence (see Hiler et al., 2017; Spindle et al., 2017; Spindle et al., 2018). Ten participants had at least one session with baseline plasma nicotine concentrations over the 5.0 ng/ml. Abstainers and non-abstainers were dichotomized and each outcome measure was analyzed as described below.

Heart rate data were averaged across time to produce a single value for baseline and the five minutes during each product bout. The data points during the rest period (1.5 hours) were divided into two groups (1st half and 2nd half of the rest period) and then averaged to produce a single data point for the first half and a single data point for the second half. Although heart rate was monitored throughout the remaining two hours of the session, data were not analyzed as different participants experienced different conditions based on their MCP results (i.e., puffs of a product or money). Although measured, blood pressure data are not included here.

Due to technical problems, some participants were not administered certain questionnaires at one or more time points, requiring their data to be excluded from analyses involving those measures. Thus, the Direct Effects of Nicotine Questionnaire and the Tiffany-Drobes Questionnaire of Smoking Urges-brief analysis included 21 participants and the Direct Effects of Product Use Questionnaire analysis included 22 participants. Other than the aforementioned subjective questionnaires, there were no other missing data.

Data Analysis Plan

The statistical analyses for the outcome measures were performed using IBM SPSS (Version 23.0). Participants were excluded from individual analyses if they had any missing data for the outcome variable of interest. For the MCP crossover points, a four (product) by three (time) within-subjects ANOVA was performed to test for differences in crossover point between the study products. For plasma nicotine levels, a four (product) by four (time) within-subjects ANOVA was performed to test for differences in nicotine delivery between the study products (α = .05). The subjective effects questionnaire items were analyzed using within-subjects, repeated measures ANOVAs. Analyses for subjective effects were conducted only with subjective data preceding the MCP due to differing choices and selections made by participants after MCP administration, making data interpretation difficult. Thus, items from the Hughes and Hatsukami Tobacco Withdrawal Scale were analyzed using a four (product) by six (time) repeated measures ANOVA (α = .05). Items from the Direct Effects of Nicotine Questionnaire and the two factors from the Tiffany-Drobes Questionnaire of Smoking Urges-brief were analyzed using a four (product) by five (time) repeated measures ANOVA (α = .05). Finally, items from the Direct Effects of Product Use Questionnaire were analyzed using a four (product) by three (time) repeated measures ANOVA (α = .05).

The Huynh-Feldt correction (Huynh & Feldt, 1976) was used to correct for violations of the sphericity assumption. Means from any outcome measure for which analyses revealed a significant interaction or main effect were analyzed using Bonferroni-corrected (α = .05/ number of comparisons), paired-samples t-tests (Bonferroni, 1936). These t-tests were conducted to examine differences from baseline for measures with a true baseline (i.e., plasma nicotine, heart rate, and all subjective measures except the Direct Effects of Product Use questionnaire). Bonferroni-corrected paired samples t-tests were used to compare OB to ECIG_36 and eG_0, and to compare IN to ECIG_36 and ECIG_0. Comparisons between OB and IN were not made due to previous research establishing that these products have high and low abuse liability (Henningfield & Keenan 1993; Hughes 1998; West et al., 2000). Additional analyses were conducted to assess differences between the ECIG_36 and the ECIG_0 conditions using Bonferroni-corrected paired samples t-tests.

The effect of abstinence status was examined before the aforementioned analyses using mixed factorial ANOVAs with abstinence status as a between-subjects factor (i.e., abstinent or non-abstinent) and condition (i.e., product) and time levels for the outcome measures (i.e., MCP, blood plasma nicotine concentrations, heart rate, and subjective measures) as within-subjects factors. Except where noted (see Subjective Effects, below) there were no significant main effects or interactions involving abstinence status. Post-hoc tests were conducted on any variables with a significant effect of abstinence status (α = .05).

Results

Twenty-four participants completed this study (See Table 2 for demographic information). Table 3 shows statistical analysis results for all outcome measures. Interactions of condition and time and main effects of condition were of greatest interest as they highlight differences in aspects of abuse liability between the study products.

Table 2.

Demographic Characteristics of the Sample (N=24).

Characteristic Mean or N (SD or %)

Number Female 6 (25.0%)
Number non-Hispanic White or Caucasian 6 (25.0%)
Number non-Hispanic Black or African American 11 (45.8%)
Age (years) 30.9 (9.5)
Screen CO 20.1 (5.0)
Cigarettes/day 16.3 (6.6)
Duration cigarettes use (years) 10.1 (9.1)
Fagerström TND a 5.2 (2.0)
Penn State Dependence b 13.6 (4.5)
Number menthol smokers 17 (70.8%)
Education (years) 13.3 (2.2)
a

The Fagerström Test for Cigarette Dependence (Heatherton et al., 1991).

b

Penn State Electronic Cigarette Dependence Index (Foulds et al., 2014).

Table 3.

Summary of Results for All Outcome Measures.

Outcome Measure Condition
Time
Condition × Time
F p ηp2 F p ηp2 F p ηp2

MCP 9.75   < .001 .30 1.96   ns .08 0.74   ns .03
Plasma Nicotine 21.51   < .001 .48 32.92   < .001 .59 10.85   < .001 .32
HR 6.95   < .001 .23 51.02   < .001 .69 12.36   < .001 .35
Systolic BP 8.44   < .001 .27 2.44   ns .10 1.26   ns .05
Diastolic BP 8.87   < .001 .28 6.42   < .001 .22 2.41   < .05 .10
Subjective Measures
 Hughes-Hatsukamia
  Anxious 1.55   ns .06 8.20   < .001 .26 2.90   < .05 .09
  Craving 11.02   < .001 .32 12.40   < .001 .35 6.80   < .001 .23
  Depression 0.91   ns .04 0.47   ns .02 1.59   ns .06
  Difficulty concentrating 1.88   ns .08 2.34   ns .09 0.73   ns .03
  Drowsy 1.18   ns .05 1.95   ns .08 1.87   ns .08
  Hunger 1.70   ns .07 5.83   < .01 .20 1.13   ns .05
  Impatient 2.00   ns .08 6.39   < .001 .22 1.91   < .05 .08
  Irritable 0.48   ns .02 6.21   < .01 .21 1.12   ns .05
  Restless 2.84   ns .11 2.97   < .05* .11 1.39   ns .06
  Sweets 0.90   ns .04 1.30   ns .05 1.28   ns .05
  Urge 7.51   < .001 .25 17.98   < .001 .44 3.37   < .01 .13
 Direct effects of nicotine b
  Confused 0.18   ns .01 0.83   ns .04 1.18   ns .06
  Dizzy 2.65   ns .12 3.41   < .05* .15 2.15   ns .10
  Headache 0.41   ns .02 0.73   ns .04 1.17   ns .06
  Heart pound 1.05   ns .05 1.84   ns .08 1.19   ns .06
  Light headed 5.62   < .01 .22 5.14   < .01 .21 2.61   < .05 .12
  Nausea 1.34   ns .06 0.12   ns .01 1.19   ns .06
  Nervous 0.19   ns .01 2.01   ns .09 0.80   ns .04
  Salivate 0.13   ns .01 1.98   ns .09 0.48   ns .02
  Sweaty 0.22   ns .01 5.44   < .01 .21 0.77   ns .04
  Weak 1.02   ns .05 0.34   .76 .02 0.63   ns .03
 Direct effects of product use c
  Awake 15.45   < .001 .42 6.71   < .01 .24 1.49   ns .07
  Calm 14.86   < .001 .41 11.43   < .001 .35 2.26   < .05 .10
  Concentrate 11.49   < .001 .35 0.07   ns .00 0.54   ns .03
  Dizzy 8.16   < .001 .28 2.76   ns .12 1.74   ns .08
  Pleasant 34.26   < .001 .62 4.59   < .05 .18 1.81   ns .08
  Reduced hunger 9.54   < .001 .31 1.20   ns .05 0.87   ns .04
  Right now 0.59   ns .03 5.70   < .01 .21 1.81   ns .08
  Satisfy 44.20   < .001 .68 2.54   ns .11 2.11   ns .09
  Sick 1.05   ns .05 0.23   ns .01 0.64   ns .03
  Taste good 40.48   < .001 .66 3.87   < .05 .16 1.96   ns .09
 Tiffany-Drobesd
  Factor 1 9.18   < .001 .32 16.90   < .001 .46 8.71   < .001 .30
  Factor 2 2.25   ns .10 6.31   < .05 .24 4.40   < .01 .18

Note.

a

Hughes and Hatsukami Tobacco Withdrawal Scale,

b

Direct Effects of Nicotine Use,

c

Direct Effects of Product Use,

d

Tiffany-Drobes Questionnaire of Smoking Urges Brief.

(*)

Asterisk denotes items that did not have a statistical difference between condition or time points with follow up post-hoc tests.

Multiple-Choice Procedure

A significant main effect of condition was observed for the MCP crossover point. Figure 1 (A)shows the mean crossover point for each condition. Collapsed across time, the mean MCP crossover point for the OB condition was significantly higher than the mean of the ECIG_36 condition [t (23) = 3.27, p < .01]. There was no significant difference between the mean crossover point in the OB condition and the ECIG_0 condition. The mean MCP crossover point for the IN was significantly lower than means for the ECIG_36 condition the ECIG_0 condition [ts (23) > 2.71, ps < .025; Bonferroni-corrected p-value].

Figure 1.

Figure 1.

Mean (+SEM) MCP crossover point (Panel A; n = 24), VAS score for the item “Did the product taste good?” (Panel B; n = 22), and VAS score for the item “Was the product satisfying?” (Panel C; n = 22) for ECIG-naive cigarette smokers after use of products. Asterisks (*) indicate significant difference from OB and plus sign (+) indicates a significant difference from IN (t-test, ps <.05).

Physiological Measures

Plasma nicotine.

A significant condition by time interaction was observed for plasma nicotine concentration. As seen in Figure 2 (A), mean plasma nicotine concentration increased in the OB condition and the ECIG_36 condition following the first and second bout [ts (23) > 4.62, ps < .001] and remained higher following the rest period [ts (23) > 3.25, ps < .01].

Figure 2.

Figure 2.

Mean (±SEM) plasma nicotine concentration (Panel A; n = 24), VAS score for the item assessing “Craving a cigarette/nicotine” (Panel B; n = 24), and Factor 1 score (Panel C; n = 21) for ECIG-naive cigarette smokers. Arrows represent the onset of each 10-puff product bouts (30-seconds between each puff). Filled symbols indicate a significant difference from baseline. Asterisks (*) indicates a significant difference from OB, plus sign (+) indicates significant difference from IN, and (#) pound symbol indicates significant difference from ECIG_36 at that time-point for that product (t-test).

Plasma nicotine concentration was significantly higher in the OB condition compared to ECIG_0 following the first and second bout and the rest period [ts (23) > 4.29, ps < .001]. OB did not differ significantly from ECIG_36 at any time point. Plasma nicotine concentration was higher in the ECIG_36 condition compared to ECIG_0 and IN after the first and second bouts [ts (23) > 3.25, ps < .01]. Plasma nicotine concentration in the IN condition was significantly higher than in the ECIG_0 condition following the second bout and the rest period [ts (23) > 2.62, ps < .025].

Heart rate.

A significant condition by time interaction was observed for heart rate. Participants’ mean heart rate in the OB condition increased significantly from a baseline mean of 67.08 bpm (SD = 8.2) to 79.16 bpm (SD = 8.2) during the first bout and to 77.73 bpm (SD = 7.9) during the second bout [ts (23) > 7.76, ps < .001]. Participants’ mean heart rate in the ECIG_36 condition increased significantly from a baseline mean heart rate of 67.54 bpm (SD = 9.6) to 72.59 bpm (SD = 8.6) during the first bout and remained higher at a mean heart rate of 71.37 bpm (SD = 8.3) during the second bout [ts (23) > 3.92, ps < .01]. Participants’ mean heart rate in the IN condition increased significantly from a baseline mean of 67.71 bpm (SD = 12.0) to 69.74 bpm (SD = 11.7) during the first bout [t (23) = 3.06, p < .01]. Participants’ mean heart rate was higher in the OB condition during both bouts and the first half of the rest period compared to ECIG_36 and ECIG_0 conditions during these time points [ts (23) > 2.72, ps < .025]. Participants’ heart rate was higher in the ECIG_36 condition during the second bout compared to the IN condition’s mean of 67.47 bpm (SD = 10.8) at this time point, [t (23) = 2.67, p < .025].

Subjective Measures

Abstinence status had an effect on four of the 33 subjective outcome items (i.e., “Craving a cigarette/nicotine”, “Urges to smoke”, “Did the product make you dizzy?”, and “Did the product taste good?”). These results are further described in the below.

Hughes and Hatsukami Tobacco Withdrawal Scale.

A significant condition by abstinence status interaction was observed for the items “Craving a cigarette/nicotine” and “Urges to smoke” [Fs > 3.08, ps < .05]. For the item “Craving”, post-hoc tests revealed that collapsed across time, in the OB condition, abstinent participants (n=14) had a higher mean rating of “Craving” of 64.68 (19.0) than non-abstinent participants (n = 10) who had a mean rating of 35.48 (SD = 25.8), [t (22) = 3.20, p < .01]. For the item “Urges to smoke”, post hoc tests revealed a significant difference between participants who abstained and participants who did not abstain in the OB condition [t (22) = 3.44,p < .01]. Collapsed across time, participants in the OB condition who abstained had a higher mean rating of 69.07 (SD = 18.1) on the item “Urges to smoke” relative to participants who did not abstain, who had an overall mean rating of 38.35 (SD = 25.7), [t (22) = 3.44, p < .01].

Collapsed across abstinence status, significant condition by time interactions were observed for the items “Anxious”, “Craving a cigarette/nicotine”, “Impatient”, and “Urges to smoke” (Table 3). In the OB condition, participants’ mean ratings of “Anxious”, “Craving”, and “Urges to smoke” were significantly reduced following the first bout and were reduced further following the second bout [ts (23) < 3.03, ps < .01]. As Figure 2 (B) shows, for “Craving”, participants’ mean rating was significantly reduced in the OB condition after the first and second bout [ts (23) < 4.75, ps < .001]. In the ECIG_36 condition, participants’ mean ratings of “Anxious”, “Craving”, and “Urges to smoke” were significantly reduced compared to baseline following the first and second bouts [ts (23) > 2.94, ps < .01]. For example, participant ratings of “Urges to smoke” decreased significantly in the ECIG_36 condition from a baseline mean rating of 80.42 (SD = 26.3) to a mean rating of 56.17 (SD = 36.8) after the first bout and remained lower than baseline with a mean rating of 56.21 (SD = 37.8) following the second bout [ts (23) > 3.04, ps < .01].

Direct Effects of Product Scale.

A significant effect of abstinence status was observed for the items “Did the product make you dizzy?” and “Did the product taste good?” [Fs > 4.58, ps < .05]. For the item “Dizzy?”, abstinent participants had a higher mean rating of 20.46 (SD = 24.3) in the IN condition following the second bout relative to non-abstinent participants who had a mean rating of 1.44 (SD = 4.0), [t (20) = 2.31,p < .05]. Also, for the item “Taste good?”, post hoc tests revealed that there was a significant difference between participants who did and did not abstain in the ECIG_0 condition before and after the second bout [ts (20) > 2.31, ps < .05]. More specifically, prior to the second bout, participants who abstained had a mean rating of 59.77 (SD = 30.3) for the item “Taste good” in the ECIG_0 condition whereas participants who did not abstain had a mean rating of 22.56 (SD = 28.7), [t (20) = 2.93, p < .01]. Similar results were observed following the second bout.

Collapsed across abstinence status, a significant condition by time interaction was observed for the item “Did the product calm you down?” and significant main effects of condition were observed for the items “Did the product make you feel more awake?”, “Did the product help you concentrate?”, “Did the product make you dizzy?”, “Was the product pleasant?”, “Did the product reduce your hunger for food?”, “Was the product satisfying?”, and “Did the product taste good?”. Collapsed across time, participants had significantly higher mean ratings for the items “Awake”, “Concentrate”, “Dizzy”, “Pleasant”, “Hunger”, “Satisfy”, and “Taste Good” in the OB condition compared to the ECIG_36 condition and the ECIG_0 condition [ts (21) > 2.71, ps < .05]. Participants had significantly higher mean ratings for the items “Awake”, “Concentrate”, “Pleasant”, “Satisfy”, and “Taste good” in the ECIG_36 condition compared to the IN condition [ts (21) > 2.56, ps < .05]. Participants had significantly higher ratings for the items “Pleasant”, “Satisfy”, and “Taste Good” in the ECIG_0 condition compared to the IN condition [ts (21) > 4.51, ps < .001]. Also, participants had a significantly higher rating for the items “Pleasant” and “Taste good” in the ECIG_0 condition compared to the ECIG_36 condition, but only at the first time point [ts (21) > 2.19, ps < .05]. Figure 1 (B and C) depicts participants’ mean subjective ratings by condition for the items “Taste good” and “Satisfy”.

Tiffany-Drobes Questionnaire of Smoking Urges-Brief.

Significant condition by time interactions were observed for Factor 1 (desire and intention to smoke) and Factor 2 (anticipation of relief from smoking abstinence). As shown in Figure 2 (C), following the first bout, participants’ mean Factor 1 score was reduced significantly in the OB condition following the first bout, remained lower before the second bout, and was reduced further following the second bout [ts (20) > 3.46, ps < .01]. Participants’ mean Factor 1 score was reduced significantly in the ECIG_36 condition following the first and second bout [ts (20) > 2.90, ps < .01]. Participants’ mean Factor 1 score was reduced significantly in the ECIG_0 condition following the first bout [t (20) = 2.94, p < .01]. Following the first bout, participants’ mean Factor 2 score was reduced significantly in the OB condition from a mean baseline score of 11.24 (SD = 8.1) to 6.48 (SD = 7.4) following the first bout, remained lower prior to the second bout at 7.71 (SD = 8.3), and was reduced further to 4.71 (SD = 5.6) following the second bout [ts > 2.80, ps (20) < .0125].

Discussion

This study compared the abuse liability of an ECIG with and without nicotine (ECIG_36 and ECIG_0) to combustible cigarettes (OB) and an FDA-approved nicotine inhaler (IN). Abuse liability was assessed via the Multiple-Choice Procedure (MCP), plasma nicotine delivery, and subjective effects (i.e., nicotine abstinence suppression and direct effects of product and nicotine). Results from the MCP revealed that the mean MCP crossover point in the OB condition was $1.42, significantly higher than the mean crossover point in the ECIG_36 condition that was $0.87. These results are similar to previous studies that have found that some nicotine-containing ECIGs have a lower reinforcing efficacy compared to combustible cigarettes on the MCP in cigarette smokers (McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012). However, another previous study that examined the impact of flavor and harm messaging using the MCP found that participants’ MCP crossover point for the tobacco flavored ECIG (containing 36 mg/ml nicotine) did not differ significantly from participants’ own brand of cigarette, although significant differences were observed between participants’ own brand of cigarettes and the unflavored, menthol, and cherry flavored ECIGs (also containing 36 mg/ml nicotine; Barnes, Bono, Lester, Eissenberg, & Cobb, 2017). Similarly, in the present study, the OB crossover point of $1.42 and the ECIG_0 crossover point of $0.96 did not differ significantly. Furthermore, the mean ECIG_36 crossover point of $0.87 and the mean ECIG_0 MCP crossover point of $0.96 were significantly higher than the mean IN crossover point of $0.32, indicating that this particular ECIG paired with either the 0 or the 36 mg/ml liquid had a higher reinforcing efficacy than the IN.

Plasma nicotine results revealed that the OB and ECIG_36 conditions delivered nicotine reliably to participants while the IN condition and ECIG_0 condition did not. This result suggests that the ECIG_36 could support nicotine dependence and therefore has the potential for abuse. In the current study, increases in plasma nicotine observed with OB use were lower than what has been reported previously (Hajek, Przulj, Phillips, Anderson & McRobbie, 2017; Lopez et al., 2016; Vansickel et al., 2010), while ECIG_36 nicotine increases were similar to what has been reported previously with this device (Lopez et al., 2016; Hiler et al., 2017). However, higher wattage ECIG devices may be able to deliver more nicotine (e.g., Wagener et al., 2017), especially when paired with liquid nicotine similar to the concentration reported here. Nicotine delivery for the IN was similar to previous studies that demonstrated limited nicotine delivery after product use, even with additional puffs (i.e., 120 puffs; Schuh, Schuh, Henningfield, & Stitzer, 1997). Furthermore, significant increases in heart rate were observed for OB, ECIG_36, and IN following the first bout. The largest increases in heart rate following the first bout were seen after OB use (mean increase of 12.1 bpm) followed by ECIG_36 use (mean increase of 5.0 bpm), and finally IN use (mean increase of 2.0 bpm). Increases in heart rate following ECIG use have also been reported in previous studies (Lopez et al., 2016; Hiler et al., 2017).

Results from subjective ratings of tobacco abstinence suppression demonstrated that the greatest symptom reduction was observed after OB product use. Following the first and second OB bout, significant reductions were observed on measures assessing cigarette craving, urge to smoke, intention to smoke (Factor 1), and anticipation of relief from abstinence symptoms (Factor 2). Similar to previous studies (Hiler et al., 2017; Lopez et al., 2016; Vansickel, Cobb, Weaver & Eissenberg, 2010; Vansickel & Eissenberg, 2013), ECIG_36 reduced cigarette cravings, urge to smoke, and intention to smoke significantly. These findings suggest that this product is able to ameliorate some of the negative side effects associated with acute nicotine abstinence in smokers, possibly warranting further investigation into this product’s utility for suppressing nicotine abstinence symptoms for longer periods of time and perhaps for smoking reduction and/or cessation. Interestingly, ECIG_0 suppressed intention to smoke significantly following the first bout. This finding supports previous research that suggests that ECIG factors beyond nicotine delivery likely play a role in nicotine abstinence suppression (Buchhalter, Acosta, Evans, Breland, & Eissenberg, 2005). A previous study investigating non-nicotine containing ECIGs demonstrated that participants’ expectation of nicotine while using the non-nicotine containing ECIG was capable of suppressing nicotine abstinence symptoms (Copp et al., 2015); however, other studies have found that ECIGs without nicotine do not reduce nicotine abstinence symptoms in smokers significantly (Hiler et al., 2017). Further, IN did not reduce nicotine abstinence symptoms significantly. Previous studies comparing a nicotine inhaler to an ECIG (Steinberg et al., 2014) and to a heat-not-burn product (Fagerström, Hughes, Rasmussen & Callas, 2000), found that the inhaler was less satisfying than the ECIG examined (Steinberg et al., 2014) and did not reduce abstinence symptoms as well as the heat-not-burn product (Fagerström, Hughes, Rasmussen & Callas, 2000). When comparing products to each other, OB reduced nicotine abstinence symptoms significantly more than the ECIG_36 and ECIG_0. These findings are consistent with previous literature that has demonstrated greater reductions in nicotine abstinence symptoms in smokers following OB consumption compared to ECIG consumption (Lopez et al., 2016; Stiles et al., 2017; Vansickel, Weaver, & Eissenberg, 2012). Finally, ECIG_36 reduced cigarette cravings and intentions to smoke significantly more than IN, suggesting that ECIG_36 is more reinforcing than IN.

In addition to suppressing tobacco abstinence symptoms more effectively, OB was rated as more calming, pleasant, satisfying, and better tasting compared to ECIG_36 and ECIG_0. Furthermore, participants had higher ratings of feeling awake, dizzy, and light-headed in the OB relative to the ECIG_36 and the ECIG_0 conditions. These results are similar to that of previous studies that have shown that OB produces greater levels of subjective reinforcement than ECIGs (Lopez et al., 2016; McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012). Also, participants rated ECIG_36 and ECIG_0 as more pleasant, satisfying, and better tasting than IN. In addition, in the ECIG_36 condition, participants felt more awake and calm than in the IN condition. These results indicate that both nicotine and non-nicotine containing ECIGs are more reinforcing than the IN in smokers, which suggest an elevated abuse liability compared to IN. Finally, ECIG_0 was rated as more pleasant and better tasting than ECIG_36. ECIG users have reported that ECIG liquids with higher nicotine concentrations produce a stronger throat hit (Etter, 2016), therefore, the current results may indicate that the 36 mg/ml nicotine concentration was too harsh for some users.

The present study had several limitations. First, not all participants complied with pre-study abstinence criteria. Specifically, 42% of the sample had baseline plasma nicotine concentrations greater than the 5.0 ng/ml cutoff (see Hiler et al., 2017; Spindle et al., 2017; 2018). When data were analyzed to examine the effect of abstinence status on the outcome measures, only four of 33 subjective items were affected by abstinence status significantly. Future studies may benefit from including a 1-hour waiting period prior to the start of the session during which participants are observed closely in order to assure some level of compliance with pre-study tobacco product abstinence conditions (as in Spindle et al., 2018). Second, this study only examined a single ECIG model, one power setting, two liquid flavors, and two nicotine concentrations, making generalizations across the ECIG product class difficult. While this design allowed for more experimental control, future studies would benefit from examining different device styles, power settings, flavors, and nicotine concentrations to understand better the factors that impact ECIG abuse liability. Third, following 1.5-hour rest period, significantly lower ratings of “Craving a cigarette/nicotine” in the OB condition were observed, relative to the ECIG_0 and the ECIG_36 condition, suggesting that a longer interval between directed bout administration and MCP assessments may be required. Lower craving levels in the OB condition may have influenced MCP results by lowering the crossover point in the OB condition. In previous studies using the MCP, participants sampled the products on a separate day (Barnes et al., 2017; McPherson et al., 2016; Vansickel, Weaver, & Eissenberg, 2012), a procedure that is less likely to lead to the potential for product sampling to influence MCP response. Fourth, MCP results in the ECIG_36 condition may have been influenced by negative sensory effects (e.g., throat hit; Etter, 2016), potentially lowering the crossover point in that condition. Finally, participants’ inexperience with ECIGs may have influenced the choices on the MCP via the novelty effect (e.g., increased interest in new technology) or the lack of knowledge about product cost; more-experienced users who are more familiar with ECIG may value the products more highly. Moreover, inexperienced ECIG users obtain less nicotine than experienced ECIG users (Hiler et al., 2017), thus changing their experiences with the product and possibly their choices and ratings on the current study’s outcome measures.

Collectively, the results of this study demonstrated that the ECIG device and liquids examined had moderate levels of abuse liability: on average lower than combustible cigarettes, but higher than an FDA-approved nicotine replacement therapy (i.e., nicotine inhaler). These results are similar to those reported in an abuse liability study that compared three nicotine containing ECIGs to combustible cigarettes and nicotine gum (Stiles et al., 2017). Taken together with recent studies showing that youth are using ECIGs (e.g., Shang, Huang, Chaloupka, & Emery, 2017; Soneji et al., 2017; Wang et al., 2018) and the results from this study demonstrating moderate abuse liability of ECIGs suggest the need for policy that prevents the initiation and use of these products in vulnerable populations such as nicotine-naïve individuals and youth. The current results also suggest that these products are able to suppress nicotine abstinence symptoms to a greater degree than the nicotine inhaler and, therefore, may have more utility for smokers who have been unsuccessful in quitting smoking through traditional means (though the health effects of longer-term ECIG use is uncertain). More research is needed to understand the efficacy of these products to suppress tobacco abstinence symptoms over longer periods and to find optimal device/liquid combinations that are associated with long-term cigarette abstinence and a positive safety profile in treatment seeking smokers. Future studies would benefit from comparing the abuse liability of these products across different populations such as experienced ECIG users, as the abuse liability of ECIGs may differ in this population. In addition, further research is needed in order to understand how other ECIG features such as flavors, sensory cues, device settings, etc. affect the abuse liability of these products. Finally, the methods described in this study extended previous work by demonstrating how the MCP can be used to determine the relative abuse liability across three product classes: ECIGs, combustible cigarettes, and a pharmaceutical nicotine inhaler. These methods are likely generalizable to many products and populations and thus can be used as an efficient tool for determining the relative abuse liability of novel tobacco products under controlled conditions.

Significance statement.

The current study examined the abuse liability of an ECIG device (i.e., eGo; 3.3 Volt; 1.5 Ohm), with 36 mg/ml and 0 mg/ml nicotine liquid, using various abuse liability assessments. Results suggest that the ECIG examined in this study had moderate levels of abuse liability: higher than an FDA-approved nicotine inhaler, but on average lower than participants’ own brand of cigarettes. The product evaluation methods described in the present study can be an efficient tool for determining the relative abuse liability of novel tobacco products under controlled conditions.

Acknowledgements

This study was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number P50DA036105 and U54DA036105 and the Center for Tobacco Products of the U.S. Food and Drug Administration. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Food and Drug Administration.

Dr. Eissenberg is a paid consultant in litigation against the tobacco industry and is named on a patent for a device that measures the puffing behavior of ECIG users.

The success of this project would have not been possible without the support and contributions of many individuals. The authors would like to thank and recognize the contributions of Barbara Kilgalen, Janet Austin, Hannah Mayberry, Caroline Smith, and Melanie Crabtree who assisted with data collection and management.

This work has been presented previously at The Society for Research on Nicotine and Tobacco (SRNT) 2017 (Florence, Italy), SRNT 2018 (Baltimore, Maryland), NIH Tobacco Regulatory Science Conference (TCORS) October 2017 (Bethesda, Maryland), TCORS June 2018 (Bethesda, Maryland), and the Virginia Conference on Youth Tobacco Use 6th Triennial Conference, March 2018 (Richmond, Virginia).

Footnotes

Disclosures

All of the authors have made significant contributions to this manuscript and have read and approved the final manuscript.

References

  1. Balster RL, & Walsh SL (2010). Abuse Liability Evaluation. Encyclopedia of Psychopharmacology, 2–7. [Google Scholar]
  2. Barnes AJ, Bono RS, Lester RC, Eissenberg TE, & Cobb CO (2017). Effect of Flavors and Modified Risk Messages on E-cigarette Abuse Liability. Tobacco Regulatory Science, 3(4), 374–387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Benowitz NL (1996). Pharmacology of nicotine: Addiction and therapeutics. Annual Review of Pharmacology and Toxicology, 36(1), 597–613. [DOI] [PubMed] [Google Scholar]
  4. Bonferroni CE (1936). Statistical theory of classification and calculation of the probability.Publication of the R Institute of the Science of Economy and Commerce of Florence, 8, 3–62. [Google Scholar]
  5. Breland AB, Evans SE, Buchhalter AR, & Eissenberg T (2002). Acute effects of Advance™: A potential reduced exposure product for smokers. Tobacco Control, 11(4), 376–378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Breland AB, Kleykamp BA, & Eissenberg T (2006). Clinical laboratory evaluation of potential reduced exposure products for smokers. Nicotine and Tobacco Research, 8(6), 727–738. doi: 10.1080/14622200600789585 [DOI] [PubMed] [Google Scholar]
  7. Breland A, Soule E, Lopez A, Ramôa C, El-Hellani A & Eissenberg T (2017), Electronic cigarettes: What are they and what do they do? Annals of the New York Academy of Sciences, 1–26. doi: 10.01111/nyas.12977. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Buchhalter AR, Acosta MC, Evans SE, Breland AB, & Eissenberg T (2005) Tobacco abstinence symptom suppression: The role played by the smoking-related stimuli that are delivered by denicotinized cigarettes. Addiction, 100(4), 550–559. [DOI] [PubMed] [Google Scholar]
  9. Carter LP, & Griffiths RR (2009). Principles of laboratory assessment of drug abuse liability and implications for clinical development. Drug and Alcohol Dependence, 105, S14–S25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carter LP, Stitzer ML, Henningfield JE, O’Connor RJ, Cummings KM, &Hatsukami DK (2009). Abuse Liability Assessment of Tobacco Products Including Potential Reduced Exposure Products (PREPs). Cancer Epidemiology, Biomarkers & Prevention: A Publication of the American Association for Cancer Research, Cosponsored by the American Society of Preventive Oncology, 18(12), 3241–3262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Copp SR, Collins JL, Dar R, & Barrett SP (2015). The effects of nicotine stimulus and response expectancies on male and female smokers’ responses to nicotine-free electronic cigarettes. Addictive Behaviors, 40, 144–147. [DOI] [PubMed] [Google Scholar]
  12. Cox LS, Tiffany ST, & Christen AG (2001). Evaluation of the brief questionnaire of smoking urges (QSU-brief) in laboratory and clinical settings. Nicotine & Tobacco Research, 3(1), 7–16. [DOI] [PubMed] [Google Scholar]
  13. Dawkins L, & Corcoran O (2014). Acute electronic cigarette use: Nicotine delivery and subjective effects in regular users. Psychopharmacology, 231(2), 401–407. [DOI] [PubMed] [Google Scholar]
  14. Eissenberg T (2004). Measuring the emergence of tobacco dependence: The contribution of negative reinforcement models. Addiction, 99(s1), 5–29. [DOI] [PubMed] [Google Scholar]
  15. Etter JF (2016). Throat hit in users of the electronic cigarette: An exploratory study. Psychology of Addictive Behaviors, 30(1), 93. [DOI] [PubMed] [Google Scholar]
  16. Evans SE, Blank M, Sams C, Weaver MF, & Eissenberg T (2006). Transdermal nicotine-induced tobacco abstinence symptom suppression: Nicotine dose and smokers’ gender. Experimental and Clinical Psychopharmacology, 14(2), 121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Fagerström KO, Hughes JR, Rasmussen T, & Callas PW (2000). Randomised trial investigating effect of a novel nicotine delivery device (Eclipse) and a nicotine oral inhaler on smoking behaviour, nicotine and carbon monoxide exposure, and motivation to quit. Tobacco Control, 9(3), 327–333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Farsalinos KE, Spyrou A, Tsimopoulou K, Stefopoulous C, Romagna G, & Voudris V (2014). Nicotine absorption from electronic cigarette use: Comparison between first and new-generation devices. Scientific Reports, 4(4133), 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fischman MW, & Foltin RW (1991). Utility of subjective-effects measurements in assessing abuse liability of drugs in humans. British Journal of Addiction, 86(12), 1563–1570. [DOI] [PubMed] [Google Scholar]
  20. Foulds J, Stapleton J, Feyerabend C, Vesey C, Jarvis M, & Russell MA (1992). Effect of transdermal nicotine patches on cigarette smoking: A double-blind crossover study. Psychopharmacology, 106(3), 421–427. [DOI] [PubMed] [Google Scholar]
  21. Gibson-Young LM (2018). JUULING: What kids don’t know will hurt them. Pediatrics, 35(6). [Google Scholar]
  22. Green LW, Fielding JE, & Brownson RC (2018). The debate about electronic cigarettes: Harm minimization or the precautionary principle. Annual Review of Public Health, 39, 189–191. [DOI] [PubMed] [Google Scholar]
  23. Griffiths RR, Troisi JR, Silverman K, & Mumford GK (1993). Multiple-choice procedure: An efficient approach for investigating drug reinforcement in humans. Behavioural Pharmacology, 4(1), 3–14. [PubMed] [Google Scholar]
  24. Hajek P, Przulj D, Phillips A, Anderson R, & McRobbie H (2017). Nicotine delivery to users from cigarettes and from different types of e-cigarettes. Psychopharmacology, 234(5), 773–779. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Heatherton TF, Kozlowski LT, Frecker RC, & Fagerstrom K (1991). The Fagerstrom test for nicotine dependence: A revision of the Fagerstrom Tolerance Questionnaire. British Journal of Addiction, 86(9), 1119–1127. [DOI] [PubMed] [Google Scholar]
  26. Henningfield JE, & Keenan RM (1993). Nicotine delivery kinetics and abuse liability. Journal of Consulting and Clinical Psychology, 61(5), 743–750. [DOI] [PubMed] [Google Scholar]
  27. Hiler M, Breland A, Spindle T, Maloney S, Lipato T, Karaoghlanian N, ... & Eissenberg T (2017). Electronic cigarette user plasma nicotine concentration, puff topography, heart rate, and subjective effects: Influence of liquid nicotine concentration and user experience. Experimental and Clinical Psychopharmacology, 25(5), 380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hughes JR (1998). Dependence on and abuse of nicotine replacement medications: An update. Nicotine Safety and Toxicity, 147–157. [Google Scholar]
  29. Hughes JR, & Hatsukami D (1986). Signs and symptoms of tobacco withdrawal. Archives of General Psychiatry, 43(3), 289–294. [DOI] [PubMed] [Google Scholar]
  30. Huynh H, & Feldt LS (1976). Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics, 1(1), 69–82. [Google Scholar]
  31. Jaffe JH, & Jaffe FK (1989). Historical perspectives on the use of subjective effects measures in assessing the abuse potential of drugs Testing for abuse liability of drugs in humans (pp. 43). NIDA Research Monograph, Number 92. [PubMed] [Google Scholar]
  32. Krishnan-Sarin S, Green BG, Kong G, Cavallo DA, Jatlow P, Gueorguieva R, ... & O’Malley SS(2017). Studying the interactive effects of menthol and nicotine among youth: An examination using e-cigarettes. Drug and Alcohol Dependence, 180, 193–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lee YH, Gawron M, & Goniewicz ML (2015). Changes in puffing behavior among smokers who switched from tobacco to electronic cigarettes. Addictive Behaviors, 48, 1–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lopez AA, Hiler M, Maloney S, Eissenberg T, & Breland AB (2016b). Expanding clinical laboratory tobacco product evaluation methods to loose-leaf tobacco vaporizers. Drug & Alcohol Dependence, 169, 33–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mauchly JW (1940). Significance test for sphericity of a normal n-variate distribution. The Annals of Mathematical Statistics, 11(2), 204–209. [Google Scholar]
  36. McPherson S, Howell D, Lewis J, Barbosa-Leiker C, Metoyer PB, & Roll J (2016).Self-reported smoking effects and comparative value between cigarettes and high dose e-cigarettes in nicotine-dependent cigarette smokers. Behavioural Pharmacology, 27(2 and 3-Special Issue), 301–307. [DOI] [PubMed] [Google Scholar]
  37. Palazzolo DL (2013). Electronic cigarettes and vaping: a new challenge in clinical medicine and public health. A literature review. Frontiers in Public Health, 1, 56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Pearson JL, Richardson A, Niaura RS, Vallone DM, & Abrams DB (2012). E-cigarette awareness, use, and harm perceptions in US adults. American Journal of Public Health, 102(9), 1758–1766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Perkins KA, Sexton JE, Reynolds WA, Grobe JE, Fonte C, & Stiller RL (1994).Comparison of acute subjective and heart rate effects of nicotine intake via tobacco smoking versus nasal spray. Pharmacology Biochemistry and Behavior, 47(2), 295–299. [DOI] [PubMed] [Google Scholar]
  40. Phillips E , Wang TW., Husten CG, Corey GC, Apelberg BJ, Jamal A, Homa DM, & King BA (2017). Tobacco product use among adults –United States, 2015. Centers for Disease Control and Prevention: Morbidity and Mortality Weekly Report, 66 (44), 1209–1215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Pickworth WB, Bunker EB, & Henningfield JE (1994). Transdermal nicotine: Reduction of smoking with minimal abuse liability. Psychopharmacology, 115(1), 9–14. [DOI] [PubMed] [Google Scholar]
  42. Regan AK, Promoff G, Dube SR, & Arrazola R (2013). Electronic nicotine delivery systems: Adult use and awareness of the ‘e-cigarette’ in the USA. Tobacco Control, 22(1), 19–23. [DOI] [PubMed] [Google Scholar]
  43. Schoenborn CA, & Gindi RM (2015). Electronic cigarette use among adults: United States, 2014. NCHS data brief, 217, 1–8. [PubMed] [Google Scholar]
  44. Schuh KJ, Schuh LM, Henningfield JE, & Stitzer ML (1997). Nicotine nasal spray and vapor inhaler: Abuse liability assessment. Psychopharmacology, 130(4), 352–361. [DOI] [PubMed] [Google Scholar]
  45. Shang C, Huang J, Chaloupka FJ, & Emery SL (2017). The impact of flavour, device type and warning messages on youth preferences for electronic nicotine delivery systems: Evidence from an online discrete choice experiment. Tobacco Control, tobaccocontrol-2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Spindle TR, Hiler MM, Breland AB, Karaoghlanian NV, Shihadeh AL, & Eissenberg T (2016). The influence of a mouthpiece-based topography measurement device on electronic cigarette user’s plasma nicotine concentration, heart rate, and subjective effects under directed and ad libitum use conditions. Nicotine & Tobacco Research, 19(4), 469–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Spindle TR, Talih S, Hiler MM, Karaoghlanian N, Halquist MS, Breland AB, ... & Eissenberg T (2018). Effects of electronic cigarette liquid solvents propylene glycol and vegetable glycerin on user nicotine delivery, heart rate, subjective effects, and puff topography. Drug and Alcohol Dependence, 188, 193–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Steinberg MB, Zimmermann MH, Delnevo CD, Lewis MJ, Shukla P, Coups EJ, & Foulds J (2014). E-cigarette versus nicotine inhaler: comparing the perceptions and experiences of inhaled nicotine devices. Journal of General Internal Medicine, 29(11), 1444–1450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Stiles MF, Campbell LR, Graff DW, Jones BA, Fant RV, & Henningfield JE (2017). Pharmacodynamic and pharmacokinetic assessment of electronic cigarettes, combustible cigarettes, and nicotine gum: implications for abuse liability. Psychopharmacology, 234(17), 2643–2655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Soneji S, Barrington-Trimis JL, Wills TA, Leventhal AM, Unger JB, Gibson LA, ... & Sargent JD (2017). Association between initial use of e-cigarettes and subsequent cigarette smoking among adolescents and young adults: a systematic review and meta-analysis. JAMA Pediatrics, 171(8), 788–797. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Tiffany ST, & Drobes DJ (1991). The development and initial validation of a questionnaire on smoking urges. British Journal of Addiction, 86(11), 1467–1476. [DOI] [PubMed] [Google Scholar]
  52. Vansickel AR, Cobb CO, Weaver MF, & Eissenberg TE (2010). A clinical laboratory model for evaluating the acute effects of electronic “cigarettes”: Nicotine delivery profile and cardiovascular and subjective effects. Cancer Epidemiology, Biomarkers & Prevention : A Publication of the American Association for Cancer Research, Cosponsored by the American Society of Preventive Oncology, 19(8), 1945–1953. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Vansickel AR, & Eissenberg T (2013). Electronic cigarettes: Effective nicotine delivery after acute administration. Nicotine & Tobacco Research, 15(1), 267–270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Vansickel AR, Weaver MF, & Eissenberg T (2012). Clinical laboratory assessment of the abuse liability of an electronic cigarette. Addiction, 107(8), 1493–1500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Wagener TL, Floyd EL, Stepanov I, Driskill LM, Frank SG, Meier E, ... & Queimado L (2017). Have combustible cigarettes met their match? The nicotine delivery profiles and harmful constituent exposures of second-generation and third-generation electronic cigarette users. Tobacco Control, 26, e23–e28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Walton KM, Abrams DB, Bailey WC, Clark D, Connolly GN, Djordjevic MV, ...& Hatsukami DK(2015). NIH electronic cigarette workshop: Developing a research agenda. Nicotine & Tobacco Research, 17(2), 259–269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Wang TW, Asman K, Gentzke AS, Cullen KA, Holder-Hayes E, Reyes-Guzman C,Jamal A, Neff L, & King B (2018). Tobacco product use among adults-United States, 2017. Centers for Disease Control and Prevention: Morbidity and Mortality Weekly Report, 67(44); 1225–1232. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Wang TW, Gentzke A, Sharapova S, Cullen KA, Ambrose BK, & Jamal A (2018). Tobacco Product Use Among Middle and High School Students—United States, 2011–2017. Morbidity and Mortality Weekly Report, 67(22), 629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. West R, Hajek P, Foulds J, Nilsson F, May S, & Meadows A (2000). A comparison of the abuse liability and dependence potential of nicotine patch, gum, spray and inhaler. Psychopharmacology, 149(3), 198–202. [DOI] [PubMed] [Google Scholar]
  60. World Health Organization (2008). WHO report on the global tobacco epidemic, 2008: The MPOWER package. World Health Organization; Geneva. [Google Scholar]
  61. Zhu SH, Sun JY, Bonnevie E, Cummins SE, Gamst A, Yin L, & Lee M (2014). Four hundred and sixty brands of e-cigarettes and counting: Implications for product regulation. Tobacco Control, 23(3), iii3–iii9. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES