Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Apr 1.
Published in final edited form as: Exp Clin Psychopharmacol. 2019 Jul 1;28(2):235–244. doi: 10.1037/pha0000305

Abuse liability of electronic cigarettes (ECIGs) in men who are experienced ECIG users

Alison Breland 1,2, Sarah F Maloney 1,2, Eric K Soule 1,2,3, Carolina Ramôa 1,2, Andrew Barnes 2,4, Thokozeni Lipato 2,5, Thomas Eissenberg 1,2
PMCID: PMC6938579  NIHMSID: NIHMS1035758  PMID: 31259592

Abstract

Most electronic cigarettes (ECIGs) aerosolize a nicotine-containing liquid that users inhale. Few experimental studies have examined ECIG abuse liability (the extent to which use of these products may lead to persistent and/or problematic use). In this study, 24 male experienced ECIG users completed four sessions that differed by product used: own ECIG (OWN), an eGo ECIG filled with participants’ own brand/flavor liquid in 0 mg/ml nicotine (ECIG_0), an eGo ECIG filled with the highest nicotine concentration available in participants’ own brand/flavor (ECIG_highest), and an FDA-approved nicotine inhaler (IN). Outcome measures included crossover point on the multiple-choice procedure (MCP), plasma nicotine delivery, and subjective effect profile. After ten puffs, a significantly higher mean crossover point was observed for OWN at $1.35 (SD=0.90) compared to ECIG_highest at $0.88 (SD=0.89), ECIG_0 at $0.83 (SD=0.79) and IN at $0.72 (SD=0.84). Significant increases in mean plasma nicotine concentration were observed for OWN at 7.94 ng/ml (SD=6.19) and ECIG_highest at 7.51 ng/ml (SD=5.39). Significant reductions in abstinence symptom suppression, and higher ratings of satisfaction were observed for OWN and ECIG_highest, with significantly less suppression and lower ratings of satisfaction for ECIG_0 and IN. These findings suggest that human laboratory methods can be used to assess ECIG abuse liability, and that nicotine-containing ECIGs have greater abuse liability than nicotine-free ECIGs and the nicotine inhaler. Potential regulations intended to limit ECIG abuse liability should be tested using these or similar procedures.

Keywords: electronic cigarettes, abuse liability

Introduction

Electronic cigarettes (ECIGs) are a class of tobacco products that vary considerably in terms of materials, design, construction, electrical power, flavors, and nicotine concentration of the liquid placed inside them. In general, they consist of a device that aerosolizes a nicotine-containing liquid that users inhale (Breland et al., 2017). These products are increasing in popularity among adults and youth (e.g., Cullen et al., 2018; Jamal et al., 2017; King, Patel, Nguyen, & Dube, 2015; McMillan, Gottleib, Shefer, Winickoff, & Klein, 2015). ECIG use among youth is a risk factor for progression to combustible cigarettes (e.g., Leventhal et al., 2015; Soneji et al., 2017; Spindle et al., 2017; Watkins, Glantz, & Chaffee, 2018). Further, from 2017 to 2018, significantly more youth reported using an ECIG on ≥ 20 days/month (20.0% to 27.7%; Cullen et al., 2018). The rapid increase in ECIG use prevalence among adults and youth as well as the increases in the frequency of use suggest that ECIG use is associated with abuse; the potential abuse of these products warrants systematic investigation into their abuse liability.

Abuse liability can be defined as “the likelihood that individuals will engage in persistent or problematic use of a drug and the likelihood that individuals will experience undesirable consequences as a result of its use” (Carter et al., 2009, p. 2). Abuse liability for drugs and tobacco products can be measured in a variety of ways, as reviewed in several other publications (Maloney et al., 2019 as well as Balster & Walsh, 2010; Carter et al., 2009; Henningfield, Hatsukami, Zeller, & Peters, 2011; Jaffe & Jaffee, 1989). Some previous studies have assessed aspects of ECIG abuse liability, such as nicotine delivery and tobacco/nicotine abstinence symptom suppression. Overall, ECIG nicotine delivery varies considerably as a function of device design and power, liquid nicotine concentration, and user behavior (e.g., Hiler et al., 2017; Vansickel, Cobb, Weaver, & Eissenberg, 2010; Yan & D’Ruiz, 2015; Farsalinos et al., 2014; Wagener et al., 2017). In some studies, ECIG nicotine delivery can be similar to or exceed that associated with combustible cigarettes (e.g., Hiler et al., 2017; Ramôa et al., 2016; Wagener et al., 2017). In terms of abstinence symptom suppression, ECIGs can reduce nicotine/tobacco abstinence symptoms in cigarette smokers and also in established ECIG users (e.g., Dawkins & Corcoran, 2014; Hiler et al., 2017, Vansickel et al., 2010; Vansickel & Eissenberg, 2013); the magnitude of symptom suppression is related to the nicotine concentration of the liquid, suggesting that drug delivery is important (Hiler et al., 2017) though other factors are also relevant (Vansickel et al., 2010).

Few studies have assessed the abuse liability of ECIGs with behavioral tasks, in part due to complexity in measurement (Cassidy, Tidey, Colby, Long, & Higgins, 2017). In four studies that have been completed with cigarette smokers (Barnes et al., 2017; Maloney et al., 2019; McPherson et al., 2016, Vansickel et al., 2012), using a task called the multiple-choice procedure (MCP, Griffiths, Troisi, Silverman, Mumford, 1993), all showed that cigarette smokers found the nicotine-containing ECIGs used less reinforcing than their own brand tobacco cigarette, suggesting a lower abuse potential for ECIGs. In another study of cigarette smokers, a progressive ratio task was used, and results indicated that smokers would work harder for flavored ECIGs compared to unflavored ones (Audrain-McGovern, Strasser, & Wileyto, 2016). One other study was conducted with both dual ECIG/cigarette users and exclusive ECIG users (adolescents and young adults), in which a tank-style ECIG and no menthol, low menthol, or high menthol flavor was administered. No differences for the MCP were observed across conditions, although mean rating of wanting/liking was higher in the high menthol conditions compared to the no menthol conditions (Krishnan-Sarin et al., 2017). Overall, these studies indicate that in cigarette smokers, ECIGs may have a lower abuse liability than cigarettes, and that flavors play a role in abuse liability.

Other work has shown that when dual cigarette/ECIG users shop for their tobacco products in an experimental marketplace in which cigarette prices increase, they will purchase more alternative products such as their usual brand of ECIGs over other nicotine/tobacco products (Quisenberry, Koffarnus, Epstein, & Bickel, 2017). In another experimental marketplace study, cigarette smokers valued a nicotine-containing tank-style ECIG less than cigarettes (Stein, Koffarus, Stepanov, Hatsukami, & Bickel, 2018). Finally, another study of cigarette smokers showed that a cartridge-style ECIG filled with nicotine-containing, menthol-flavored liquid had a lower abuse liability (as measured by subjective effects and nicotine delivery) than cigarettes, but greater than 4 mg nicotine gum (Stiles, Campbell, Jin, Graff, Fant, & Henningfield, 2018).

Thus, with little published work assessing the abuse liability of ECIGs among experienced ECIG users, this study’s purpose was to assess the abuse liability of an eGo ECIG, loaded with the highest concentration of nicotine available in participants’ own brand/own flavor. Control conditions include the same ECIG loaded with 0 mg/ml nicotine in participants’ own brand/flavor (negative control), participants’ own ECIG and liquid (positive control), as well as the Nicotrol inhaler (a product with a known low abuse liability). Outcome measures included the reinforcing efficacy of each product (as assessed by MCP), plasma nicotine delivery, and subjective effect measures. A secondary purpose was to explore how biological and behavioral measures of ECIG abuse liability are related.

Method

Participants

This study was approved by Virginia Commonwealth University’s (VCU’s) institutional review board (“Choice assessment of nicotine-containing products in cigarette smokers and electronic cigarette users”; HM20005746). ECIG users (reported open-style ECIG use ≥ 3 months, > one cartridge or 1 ml of ECIG liquid/day of > than 6 mg/ml nicotine, or, less than 6 mg/ml if they used ≥ 10 ml of liquid per day) were recruited by advertisements and word of mouth. ECIG users were also defined as individuals who reported using fewer than five cigarettes/day (Vansickel & Eissenberg, 2013), and using other tobacco products (e.g., cigar, waterpipe) no more than 3 times a week. Other eligibility criteria included reporting being healthy and aged 18–55. Individuals were excluded from participation if they reported a history of chronic disease or psychiatric condition, regular prescription medication use (aside from birth control), marijuana use > 10 days and alcohol use > 25 days in the past 30, and any illicit drug use (e.g. cocaine, opioids, benzodiazepines, and methamphetamine) in the past 30 days. Women were excluded if they tested positive for pregnancy (by urinalysis) at screening or reported breastfeeding.

Procedure

The study procedure was similar to that described elsewhere (Maloney et al., 2019). Briefly, participants completed each of the four, Latin-square ordered, ~4.5-hour sessions at Virginia Commonwealth University’s (VCU) Clinical Behavioral Pharmacology Laboratory. Sessions were separated by a minimum of 48 hours and differed by the product used: own ECIG (OWN), an experimenter-supplied ECIG (~3.3 V, 1000 mAh battery attached to a 1.5 Ohm, dual coil, 510-style cartomizer; produced by SmokTech; Shenzhen, China) that was filled with 0 mg/ml nicotine (ECIG_0) in the participant’s preferred brand/flavor, the identical ECIG filled with the highest mg/ml nicotine available the participant’s preferred brand/flavor (ECIG_highest) or the FDA-approved 10 mg Nicotrol inhaler (IN). Participants were instructed to abstain from nicotine/tobacco for ≥ 12 hours prior to each session. Abstinence from combustible tobacco was verified via participants’ expired air CO concentration prior to the start of the session (≤ 10 ppm) and later analysis of plasma nicotine prior to the start of the session (≤ 5 ng/ml was considered abstinent, as in Hiler et al., 2017). In each session, participants completed two, 10-puff product use bouts (with ~30 second inter-puff-intervals and 20 minutes in between bouts), followed by a 90-minute rest period, and then three administrations of the multiple-choice procedure (MCP), each separated by 20 minutes. After each MCP administration, participants’ choices were reinforced with a random drawing of one of their choices (puffs or money). If puffs were selected from the drawing, participants were given the opportunity to use the session product at that time. If money was selected from the drawing, it was given to participants immediately. An intravenous catheter was used to sample blood (~7 ml) 4 times in each session (at baseline, after the first use bout, after the second use bout, and before MCP administration). Subjective questionnaires were administered at baseline and following each of the two use bouts. Physiological monitoring (heart rate, blood pressure) occurred throughout each session, although these data are not reported.

Materials

Participants’ preferred brand/flavor liquid (in their preferred nicotine concentration, in 0 mg/ml concentration, and in the highest nicotine concentration available in that brand/flavor) was purchased by study staff and study staff loaded the liquid into the ECIG tank or cartomizer for the appropriate condition. All liquids were analyzed independently for nicotine content (See Table 1).

Table 1.

Participants’ own brand liquid and labeled vs. tested concentrations: preferred concentration (OWN), highest concentration available in own brand and flavor (ECIG_highest), and 0 mg concentration (ECIG_0).

Participant Flavor Category1 Device
power
(in
Watts)2
OWN labeled
concentration
(mg)
OWN tested
concentration
(mg)
ECIG_highest
labeled
concentration
(mg)
ECIG_highest
tested
concentration
(mg)
ECIG_0 tested
concentration
(mg)3
1 Fruit 12* 9.75 12 9.75 0.19
2 Beverage 12* 7.92 12 7.92 0.14
3 Fruit 12* 6.16 12 6.16 0.87
4 Tobacco 18* 12.30 18 12.30 0.39
5 Dessert/Sweets 50 6* 2.76 6 2.76 0
6 Fruit 12* 10.43 12 10.43 0.10
7 Unknown 29 6 4.64 18 13.38 0.32
8 Dessert/Sweets 75 18 0.98 36 1.88 0.11
9 Beverage 100 6* 3.67 6 3.67 0
10 Dessert/Sweets 15 6 4.53 18 13.94 0.1
11 Dessert/Sweets 35 12* 10.21 12 10.21 0.08
12 Dessert/Sweets 3 2.76 6 4.89 0
13 Dessert/Sweets 12 10.53 18 13.68 0.22
14 Fruit 32.5 12* 11.11 12 11.11 0
15 Fruit 49 6 4.35 12 10.97 0
16 Unknown 80 6 5.55 12 11.01 0
17 Dessert/Sweets 60 6* 4.89 6 4.89 0
18 Unknown 50 6* 5.79 6 5.79 0
19 Fruit 50 12 8.67 18 12.74 0
20 Fruit 85 6 4.83 6 4.83 0
21 Beverage 40 6 5.69 12 11.11 0
22 Dessert/Sweets 29.4 12* 10.36 12 10.36 0
23 Fruit 8 6 5.65 6 5.65 0
24 Fruit 12* 11.16 12 11.16 0
Mean 49.24 9.38 6.86 12.50 8.77 0.11
SD 25.60 4.08 3.22 6.60 3.71 0.20
2

Data incomplete for this question (question not asked for first four participants; other participants did not know their device power settings)

3

All 0 mg liquids were labeled as 0 mg

*

denotes that OWN and ECIG_highest used the same concentration liquid

Measures

Multiple-Choice Procedure.

The Multiple-Choice Procedure (MCP) is a pen and paper task that measures and allows for comparisons of abuse liability between different drugs and drug delivery platforms (Griffiths, Troisi, Silverman, & Mumford, 1993). In the current study, participants were asked to make eleven separate choices between increasing amounts of money ($0.01 to $10.24) or ten puffs from the study products. After participants made their choices, one was drawn at random by the participant, and the choice (either money or puffs) was delivered immediately. The maximum dollar value that participants chose puffs of the study product over money was defined as the crossover point. The higher the crossover value, the greater the reinforcing efficacy of the product. For participants who chose money for all eleven choices, a crossover point of $0.00 was used for analysis and for participants who chose all study product a crossover point of $10.24 was used for analysis.

Subjective questionnaires.

Four subjective questionnaires were administered before and after each directed bout: the Hughes and Hatsukami Tobacco Withdrawal Scale (modified from Hughes & Hatsukami, 1986; see Breland, et al., 2002, Buchhalter, Acosta, Evans, Breland, & Eissenberg, 2005), the Direct Effects of Nicotine Questionnaire (modified from Perkins et al., 1994; see Evans, Blank, Sams, Weaver, & Eissenberg, 2006), and a modified version of The Tiffany-Drobes Questionnaire of Smoking Urges (QSU-Brief; modified so that “cigarette” was replaced with “e-cig”; and “smoke” replaced with “vape”; Cox, Tiffany, & Christen, 2001). The Direct Effects of Product Use Questionnaire (modified from Buchhalter et al., 2005; Foulds et al., 1992; Pickworth, Bunker, & Henningfield, 1994) was administered immediately following the first bout, before the second bout, and immediately following the second bout. With the exception of the QSU-Brief, all subjective measures were administered using a computerized visual analog scale such that a horizontal line anchored with “not at all” on the left and “extremely” on the right was presented beneath a word/phrase and participants clicked a mouse anywhere along the line (see Table 2 for abbreviated versions of the word/phrase). Scores were calculated by the percentage of the total line length from the left anchor where participants clicked. Items from the QSU-Brief were rated on a seven-point Likert scale in which participants could click seven discrete ratings from ‘not at all’ to ‘extremely’. Scores were calculated to create Factor 1 (Anticipation of Relief from Withdrawal), and Factor 2 (Desire and Intention to Use ECIGs).

Table 2.

Summary of Results for All Outcome Measures.

Condition Time Condition × Time
Outcome Measure F p ηp2 F p ηp2 F p ηp2

MCPa 5.34 <.01 0.20
Plasma Nicotineb 15.14 <.01 0.40 19.78 <.01 0.46 12.29 <.01 0.35
Subjective Measures
  Hughes-Hatsukamib
    Anxious 3.41 n.s. 0.13 7.15 <.05 0.24 0.84 n.s. 0.04
    Craving 8.61 <.01 0.27 15.08 <.01 0.40 5.35 <.01 0.19
    Depression 1.27 n.s. 0.05 1.84 n.s. 0.07 1.48 n.s. 0.06
    Difficulty concentrating 2.50 n.s. 0.10 4.20 <.05 0.16 0.97 n.s. 0.04
    Drowsy 2.01 n.s. 0.08 3.26 <.05 0.12 1.14 n.s. 0.05
    Hunger 2.65 n.s. 0.10 2.70 n.s. 0.11 0.78 n.s. 0.03
    Impatient 1.91 n.s. 0.08 3.27 n.s. 0.12 0.65 n.s. 0.03
    Irritable 3.21 n.s. 0.12 3.24 n.s. 0.12 1.25 n.s. 0.05
    Restless 1.13 n.s. 0.05 5.64 <.05 0.20 2.78 n.s. 0.11
    Sweets 1.17 n.s. 0.05 2.12 n.s. 0.09 1.17 n.s. 0.05
    Urge 8.19 <.01 0.26 16.19 <.01 0.41 7.99 <.01 0.26
  Tiffany-Drobesc
    Factor 1 10.50 <.01 0.32 25.75 <.01 0.54 5.85 <.01 0.21
    Factor 2 3.43 <.05 0.14 7.02 <.05 0.24 2.40 <.05 0.10
    Direct effects of nicotineb
    Confused 1.08 n.s. 0.05 1.71 n.s. 0.07 0.73 n.s. 0.03
    Dizzy 2.98 n.s. 0.12 6.61 <.05 0.22 3.94 <.05 0.15
    Headache 0.09 n.s. 0.00 0.23 n.s. 0.01 0.90 n.s. 0.04
    * Heart pound 1.74 n.s. 0.07 5.43 <.01 0.19 1.69 n.s. 0.07
    Light headed 0.32 n.s. 0.12 9.74 <.01 0.30 4.89 <.01 0.18
    Nausea 1.27 n.s. 0.05 2.02 n.s. 0.08 1.46 n.s. 0.06
    Nervous 0.28 n.s. 0.01 2.32 n.s. 0.09 0.40 n.s. 0.02
    Salivate 2.38 n.s. 0.09 2.24 n.s. 0.09 2.33 n.s. 0.09
    Sweaty 1.75 n.s. 0.07 1.21 n.s. 0.05 0.31 n.s. 0.01
    Weak 0.85 n.s. 0.04 1.52 n.s. 0.06 0.99 n.s. 0.04
  Direct effects of product usea
    Awake 3.37 <.05 0.13 5.29 <.05 0.19 1.36 n.s. 0.06
    Calm 7.27 <.01 0.25 13.45 <.01 0.38 0.50 n.s. 0.02
    Concentrate 7.93 <.01 0.27 4.32 n.s. 0.16 0.53 n.s. 0.02
    Dizzy 4.88 <.05 0.18 2.74 n.s. 0.11 0.77 n.s. 0.03
    * Pleasant 25.21 <.01 0.55 0.01 n.s. 0.01 3.08 n.s. 0.13
    Reduced hunger 2.32 n.s. 0.10 8.67 <.05 0.28 0.51 n.s. 0.02
    Right now 6.02 <.01 0.22 15.72 <.01 0.42 1.30 n.s. 0.06
    Satisfy 19.74 <.01 0.47 0.58 n.s. 0.03 1.21 n.s. 0.05
    Sick 2.33 n.s. 0.10 2.03 n.s. 0.08 2.16 n.s. 0.09
    * Taste good 47.20 <.01 0.69 0.05 n.s. 0.00 2.13 n.s. 0.09
a

adf= 3,66

b

df = 3, 207

c

df = 3, 198

*

Effect of abstinence status observed, but not noted in this table (see text).

Data Preparation and Analysis

For the MCP, data were prepared as described above. For plasma nicotine delivery, as in previous work, values below the LOQ were replaced with 2 ng/ml as this is a more conservative approach than assuming that values below the LOQ are zero (Lopez, Hiler, Maloney, Eissenberg, & Breland, 2016; Vansickel et al., 2010).

For the MCP and two subjective questionnaires, several participants had missing data. For the MCP, one participant failed to complete the task properly in one session and this participants’ data were not included in the analysis of this measure. For the QSU-Brief, one participant had missing data for one time point, thus, this participant was not included in the analysis for these factors. For the Direct Effects of Product, one participant had missing data for two time points, thus, this participant was not included in the analysis for these items.

Because a lack of 12-hour abstinence might have influenced study results, prior to analysis, plasma nicotine data were examined for values above 5 ng/ml (indicating that participants may have failed to abstain from nicotine for at least 12 hours pre-session; see Hiler et al., 2017; Spindle et al., 2016; Spindle et al., 2018). Using abstinence status as a between-subjects factor, nicotine data were analyzed to determine if abstinence/lack of abstinence affected results. Eight participants had at least one baseline plasma sample that was above 5 ng/ml nicotine. For all other outcome measures, abstinence status was assessed similarly (as in Maloney et al., 2019). Any significant main effects or interactions observed for abstinence status are noted in the results section.

For the MCP, a condition by time (three levels of time) ANOVA was conducted. Condition by time repeated measures analysis of variance (ANOVA) was used to examine plasma nicotine (four levels of time) and heart rate (three levels of time). For subjective measures, timepoints analyzed were: directly before and directly after product use (four timepoints) for the Hughes-Hatsukami, Tiffany-Drobes, and Direct Effects of Nicotine items, and directly after product use (two timepoints) for the Direct Effects of Products items. For any analysis with a significant interaction or main effect, paired-samples t-tests with a Bonferroni correction (α = .05/number of comparisons) were used to compare means within and across conditions (as in Maloney et al., 2019). Within conditions, baseline values were compared to subsequent values over time. Across conditions, OWN and IN were compared to all other conditions (ECIG_0, ECIG_highest, IN; similar to Maloney et al., 2019). Finally, for bivariate analyses examining associations between biological (plasma nicotine) and behavioral (subjective effects, crossover point), Pearson’s correlations were used.

Results

Participant Characteristics

Twenty-four participants (all male; 71% white or Caucasian) completed all four sessions and thus their data were included in the analyses. These 24 participants were, on average, 28.6 years old (SD = 7.4), had been using ECIGs for 18.0 months (SD = 13.1), used liquid with a concentration of 9.4 mg/ml (SD = 4.1), and reported using 6.1 ml (SD = 4.5) per day. Participants’ mean CO concentration at screening was 3.3 ppm (SD = 3.5) and on average, they reported smoking 0.04 (SD = 0.20) cigarettes per day (one participant reported smoking one cigarette/day). Most participants were former smokers (21 of 24), and one participant was a current smoker. Two participants reported never smoking. Participants who were former smokers reported being abstinent from smoking for 2.0 years (SD = 3.27). Participants’ mean CO concentration pre-session was 2.83 ppm (SD = 1.94).

Multiple-Choice Procedure (MCP)

For the MCP, a significant interaction of condition by time was observed [F(6,132) = 2.78; p = 0.03]. Because of this interaction of time, means could not be averaged, as in previous work (Vansickel, Weaver, & Eissenberg, 2012). Due to the non-uniform administration of reinforcers (puffs or money) after each MCP administration, which may account for the interaction of time, only data from the first administration of the MCP was used in subsequent analyses. Thus, an ANOVA with four levels of condition was conducted, and a significant main effect of condition was observed, as shown in Table 2. Post-hoc tests revealed that, compared to the OWN mean crossover point of $1.35 (SD = 0.90), a significantly lower mean crossover point was observed for ECIG_highest ($0.88; SD = 0.89), for IN ($0.72; SD = 0.84), and for ECIG_0 ($0.83; SD = 0.79; [ts(22) > −3.9, ps <.016; Bonferroni-corrected p-value]). No significant difference was observed between the mean IN crossover point and the mean for ECIG_highest or ECIG_0.

Plasma Nicotine

A significant main effect of abstinence status was observed [F(1,22) = 7.03, p < .05). Overall, higher mean nicotine concentration was observed across all conditions and times for non-abstinent participants (mean = 7.31 ng/ml, SD = 6.93) compared to abstinent participants (mean = 5.41 ng/ml, SD = 5.89). Subsequently, difference scores were calculated (by subtracting the baseline data point from all subsequent data points) and these data were analyzed using the same strategy and results revealed no significant main effects or interactions involving the abstinence status factor; these difference score data are reported here. Significant main effects of condition and time as well as a significant interaction of condition by time were observed, as shown in Table 1 (ps < .001). As shown in Figure 1, post hoc tests revealed that mean plasma nicotine concentration increased significantly after product use in the OB and ECIG_highest conditions, and after the second product use in the IN condition, but not at any time points in the ECIG_0 condition. More specifically, after the first use, increase for OWN was 7.94 ng/ml (SD = 6.19) and for ECIG_highest was 7.51 ng/ml (SD = 5.39), OWN was significantly higher than ECIG_0 at 0.00 ng/ml (SD = 1.61) and IN at 0.71 ng/ml (SD = 1.48). Compared to IN, the mean ECIG_highest concentration was higher after the first product use, and before the second product use. Also compared to IN, mean ECIG_0 concentration was lower before the second product use ([ts (23) > −5.8, all ps <.016 or .025; Bonferroni-corrected p-values).

Figure 1.

Figure 1.

Mean plasma nicotine increase (+/− SEM; calculated using difference from baseline) for 24 ECIG-naïve smokers. Filled symbols indicate a significant difference from baseline. Asterisk indicates a significant difference from OB (*), number sign indicates (#) significant differences from IN, (Tukey’s post-hoc tests; ps <.05).

Subjective Questionnaires

Hughes and Hatsukami Tobacco Withdrawal Scale.

As shown in Table 2, significant condition by time interactions and significant main effects of condition and time were observed for “Craving an e-cigarette/nicotine” and “Urge to use an e-cigarette”. For the item “Craving an e-cigarette/nicotine”, as shown in Figure 2, post hoc tests revealed that mean scores on this item decreased significantly in each condition, except for ECIG_0. Compared to OWN, mean score in the ECIG_0 condition after baseline was significantly higher at all time points, and mean score after the first product use in the IN condition was also significantly higher. Compared to IN, mean score in the ECIG_0 condition was higher before and after the second product use ([ts (23) > −4.6, ps < 0.16; Bonferroni-corrected p-value). A similar pattern was observed for the item “Urge to use an e-cigarette”, post-hoc tests revealed that mean score on this item also decreased significantly after each condition, except in ECIG_0. Compared to OB, mean score after the first product use was higher in IN and ECIG_0, and higher in ECIG_0 before and after the second product use. Compared to IN, mean score after product use in ECIG_0 was higher before and after the second product use only ([ts (23) < −5.0, all ps <.016 or .025; Bonferroni-corrected p-values).

Figure 2. Left panel.

Figure 2.

Mean scores for the item, “Craving a cigarette/nicotine” (+/− SEM) for 24 ECIG-naïve smokers. Filled symbols indicate a significant difference from baseline. Asterisk indicates a significant difference from OB (*), number sign indicates (#) significant differences from IN, (Tukey’s post-hoc tests; ps <.05). Right panel. Mean scores for the item, “Lightheaded?” (+/− SEM) for 24 ECIG-naïve smokers. In all other respects the right panel is identical to the left.

Significant effects of time (collapsed across condition) were observed for the items “Anxious”, “Difficulty concentrating”, “Drowsy”, and “Restless”. Post hoc tests revealed that for “Anxious”, compared to the baseline mean score of 15.60 (SD = 19.17), the mean score was significantly reduced after the first product use at 7.42 (SD = 10.05) and second product uses, at 7.70 (SD = 9.34; [ts (23) > 2.5, ps <0.016; Bonferroni-corrected p-values). For “Difficulty concentrating” and “Restless”, mean scores were significantly reduced after the first product use only [ts (23) > 2.4, ps < 0.016; Bonferroni-corrected p-values).

Tiffany-Drobes Questionnaire of E-cigarette Urges Brief

Significant condition by time interactions and significant main effects of condition and time were observed for both factors. Post-hoc tests revealed that for Factor 1, mean score decreased significantly from baseline to after product use in all conditions except for ECIG_0. For example, in the OWN condition, mean score at baseline was 20.74 (SD = 9.43) and after the first product use decreased to 10.43 (SD = 8.29), and after the second product use, to 9.91 (SD = 8.34). In the ECIG_highest condition, mean score at baseline was 20.61 (SD = 7.63), and after the first product use decreased to 14.17 (SD = 8.85), and after the second product use, to 13.35 (SD = 9.56). In the IN condition, mean score at baseline was 21.13 (SD = 6.94) and decreased after the first product use to 15.91 (SD = 8.55), and after the second product use, to 15.83 (SD = 9.27). After the first product use, compared to the OWN mean score of 10.43 (SD = 8.29), the mean score after the first ECIG_highest use of 14.17 (SD = 8.85), mean score after the first IN use of 15.91 (SD = 8.55), and mean score after the first ECIG_0 use of 19.96 (SD = 8.03) were all significantly higher. Compared to OWN mean score before the second use of 13.96 (SD = 9.22), mean score before the second ECIG_0 use of 20.57 (SD = 7.84) was significantly higher. Compared to OWN mean score of 9.91 (SD = 8.34) after the second use, mean score after the second ECIG_0 use was also significantly higher at 19.61 (SD = 8.04) as well as after the second IN use at 15.83 (SD = 9.27; [ts (22) > −4.9, ps < .016]; Bonferroni-corrected p-values). Compared to IN across condition, mean score after the first IN use was lower than mean score after ECIG_0 use ([ts (22) = −2.62, p < .025]; Bonferroni-corrected p-value). For Factor 2, mean score decreased significantly only in the OWN condition, from 7.52 (SD = 7.93) at baseline to 2.61 (SD = 3.30) after the first use, and to 2.87 (SD = 3.83) after the second use ([ts (22) > 7.60, ps < .016]). Across conditions, compared to OWN, mean score after the first product use was higher for ECIG_0 at 5.78 (SD = 6.37; [t (22) = −2.66, p <.016], Bonferroni-corrected p-value).

Direct Effects of Nicotine Scale

A significant interaction of time by abstinence was observed for the item “Heart pound”, [F(3, 66) > 3.60, ps < .05] with overall higher scores across time for participants who had not abstained compared to those who had abstained.

Significant interactions of condition by time and significant main effects of time were observed for “Dizzy” and “Lightheaded”. A significant main effect of time was also observed for “Heart pound”. For “Dizzy”, post hoc tests revealed a significant increase on the rating for this item in the OWN, IN, and ECIG_highest conditions, but not ECIG_0. For OWN, mean score increased from 3.13 (SD = 7.39) before the first use, to 17.46 (SD = 24.69) after the first product use. In IN, mean score increased from 3.00 (SD = 7.68) to 10.83 (SD = 17.48) after the first product use. For ECIG_highest, mean score increased from 1.75 (SD = 3.50) to 8.17 (SD = 13.59) after the first product use ([ts (23) > −2.9, ps < .016]; Bonferroni-corrected p-values). Across conditions, compared to OWN mean score of 17.46 (SD = 24.69) after the first product use, mean score in the ECIG_0 condition was significantly lower, at 3.17 (SD = 8.95). Across conditions, compared to the IN mean score of 10.83 (SD = 17.48) after the first product use, mean score in the ECIG_0 condition was also significantly lower. Last, mean score in the IN condition after the second product use of 7.83 (SD = 13.57), was significantly higher than mean score in the ECIG_0 condition of 3.96 (SD = 10.21; [ts (23) > 2.60, ps < .016 or 0.025]; Bonferroni-corrected p-values).

For “Lightheaded”, as shown in Figure 2, post hoc tests revealed a similar pattern, with a significant increase on the rating for this item after the first product use in the OB, IN and ECIG_highest conditions ([ts (23) > −3.7,ps < .016]; Bonferroni-corrected p-values), but not for ECIG_0. Across conditions, mean score in OWN was significantly higher than in ECIG_highest after the first product use. Compared to IN, mean score was higher after the first product use than for ECIG_0 ([ts (23) > −3.05, ps < .016 or 0.025]; Bonferroni-corrected p-values). For “Heart Pound”, mean scores after the first product use were higher in OWN compared to ECIG_0 ([t(23) = 2.72, p < .025]; Bonferroni-corrected p-value).

Direct Effects of Product Scale

Significant main effects of abstinence status were observed for two items: “Was the product pleasant?” and “Did the product taste good?” [F(1,21) > 7.90, ps <.05). For both items, overall mean ratings were lower for those who had not abstained, than for those who had abstained. For example, for the item “Did the product taste good?” the mean rating (collapsed across condition and time) for participants who abstained was 71.38 (SD = 33.29), compared to a mean of 52.66 (SD = 39.15) for participants who had not abstained. A similar pattern was observed for the item “Was the product pleasant?”.

Significant main effects of condition (collapsed across time) were observed for the items “Did the product make you feel more awake”, “Did the product calm you down?”, “Did the product help you concentrate?”, “Did the product make you dizzy”, “Was the product pleasant?”, “Would you like to use another product right now?”, “Was the product satisfying?”, and “Did the product taste good?”. Across all items, no differences were observed between OWN and ECIG_highest, except for the item “Would you like to use another product right now?”, in which scores for ECIG_highest at 55.09 (SD = 30.34) were significantly higher than scores for OWN at 34.54 (SD = 30.68; [t (22) = 3.63, p < .016]; Bonferroni-corrected p-value). For many items (“Pleasant”, “Taste good”, “Satisfy”, “Calm”, and “Concentrate”), IN was rated lower than OWN ([ts (22) > −6.48, ps < .016; Bonferroni-corrected p-values) . OWN was often rated higher than ECIG_0 on the items “Calm”, “Satisfy”, “Awake”, “Concentrate”, “Pleasant”, and “Dizzy”; ([ts (22) > 2.80; ps < .016]; Bonferroni-corrected p-values). Compared to IN, scores were significantly lower than ECIG_highest for the items “Satisfy”, “Taste good”, “Pleasant”, and “Right now” ([ts (22) > 2.94, ps < .025]; Bonferroni-corrected p-values). Compared to IN, scores were significantly higher for ECIG_0 for “Right now” “Pleasant” and “Taste good”. For “Dizzy”, scores for IN were significantly higher than for ECIG_0 ([ts (22) > −3.00, ps < .025]).

Correlations among Abuse Liability Measures

After examining condition effects individually across plasma nicotine and behavioral measures of abuse liability, correlations among these measures were examined as an exploratory analysis to understand whether patterns of responses for these individual measures were similar. Associations among nicotine plasma increase, differences in self-reported responses to the item “Craving a cigarette/nicotine” from the Hughes and Hatsukami Tobacco Withdrawal Scale, and crossover point from the MCP were tested. Broadly, these three measures were found to be correlated with each other, suggesting they each may be related indicators of a larger ECIG abuse liability construct. In particular, plasma nicotine increase was associated with the change in craving (from before product use to after product use), such that higher plasma increases were correlated with larger reductions in craving (Pearson’s r = −0.30, p <0.01). Further, larger reductions in craving were positively associated with higher MCP crossover points (Pearson’s r = −0.33, p <0.01) suggesting participants were willing to pay more or work harder for products that reduced their craving more. Plasma nicotine increase and crossover point were not correlated (Pearson’s r = 0.03, p <0.79).

Discussion

The purpose of this study was to assess the abuse liability of a ~3.3V, 1.5 ohm, dual-coil ECIG, loaded with the highest concentration of nicotine available in participants’ own brand/own flavor. Control conditions included the same ECIG loaded with 0 mg/ml nicotine (negative control), participants’ own ECIG and liquid (positive control), as well as the Nicotrol inhaler (a product with a known low abuse liability). Overall, the findings from this study suggest that nicotine-containing ECIGs have more abuse liability than a nicotine-free ECIG and the Nicotrol inhaler. The crossover point on the MCP (a measure of the reinforcing efficacy of a drug or product) was highest for OWN ECIGs compared to all other conditions. Plasma nicotine concentrations were significantly increased after the first product use in the OB and ECIG_highest conditions, but not after the first use of ECIG_0 or IN. Abstinence symptoms were reduced more in the OWN and ECIG_highest conditions than with ECIG_0 or IN. Other subjective measures such as feeling lightheaded and feeling satisfied were higher in the OWN and ECIG_highest conditions compared to ECIG_0 and IN. Further, factors beyond nicotine delivery and subjective effects may influence abuse potential (as OWN had a higher crossover point than ECIG_highest). Results from this study contribute to the existing literature regarding ECIG abuse liability while also highlighting laboratory methods that can be used to assess ECIG abuse liability as compared to other tobacco/nicotine products.

Significantly higher crossover values in the OWN condition compared to the ECIG_highest condition, the ECIG_0, condition, and the IN condition suggest a greater reinforcing efficacy of own brand ECIGs, and thus, greater abuse potential. In this study, OWN ECIG was a device set to 49 W (on average, based on available data) and filled with 6.86 mg/ml nicotine (tested nicotine concentration; see Table 1). In comparison, ECIG_highest was a ~7.3 W device filled with 8.7 mg/ml nicotine (tested nicotine concentration). The greater power output for OWN relative to ECIG_highest may have affected participants’ evaluation of the ECIG’s reinforcing effects as increased power is associated with increased nicotine yield (Talih et al., 2017). In addition, participants may have been able to give a better estimate of the value of OWN compared to an abstraction of a product they use regularly (ECIG_highest), and which they had only briefly sampled. The finding of OWN being valued the highest is similar to previous studies where own brand cigarettes were compared to other products (e.g., Barnes et al., 2017; Maloney et al., 2019; McPherson et al., 2016, Vansickel et al., 2012).

Second, similar plasma nicotine increases in the OWN and the ECIG_highest conditions, and little or no nicotine delivery in the ECIG_0 and IN conditions indicates that a variety of ECIG types filled with nicotine-containing liquid of a range of concentrations can deliver the dependence-producing drug nicotine, thus increasing the potential for abuse. These plasma nicotine concentrations are somewhat similar to those observed elsewhere using cigarette smokers as participants (i.e., Maloney et al., 2019; ~5.35 ng/ml after 10 puffs from a ~7.3 W ECIG filled with 36 mg/ml nicotine). In addition, compared to other work with ECIG-experienced individuals using the same ECIG device filled with 8 mg/ml nicotine (Hiler et al., 2017; 8.2 ng/ml, SD = 7.8), the mean plasma nicotine concentration in the ECIG_highest condition (filled with 8.7 mg/ml nicotine, on average) reported in the current study was also similar. However, ECIG nicotine delivery in experienced ECIG users can also be much higher: in individuals using the same ECIG device filled with 36 mg/ml nicotine, average plasma nicotine concentrations were 17.9 ng/ml (SD = 17.2; Hiler et al., 2017). Overall, studies of ECIGs in which plasma nicotine delivery is measured highlight the variability in ECIG nicotine delivery across product type and nicotine concentration. Interestingly, although nicotine delivery was similar for OWN and ECIG_highest, crossover values were different, suggesting that participants’ own brand ECIG devices have qualities beyond nicotine delivery that are reinforcing, including potentially device design features and learned patterns of use specific to participants’ own brand device. Further research is needed to determine what other qualities may contribute to ECIG abuse liability beyond nicotine delivery.

In this study, the mean labeled nicotine concentration for participants’ own brand liquid was higher than the mean tested nicotine concentration, both for their preferred (own) nicotine concentration and for the highest nicotine concentration available in their own brand and own flavor. Several other studies have noted discrepancies between labeled and tested nicotine concentration in ECIG liquids. In general, some tested samples contain less nicotine than the labeled concentrations and some contain more (e.g., Girvalaki et al., 2018; Kim, Goniewicz, Yu, Kim, & Gupta, 2015; Raymond, Collette-Merrill, Harrison, Jarvis, & Rasmussen, 2018).

Third, similar reductions in abstinence symptom suppression for the OWN and ECIG_highest conditions (and less for ECIG_0 and IN) indicates that a variety of nicotine-containing ECIG types can suppress abstinence symptoms, which could increase the use of these products, thus also increasing the potential for abuse. Results for ECIG_highest, OWN, and ECIG_0 followed a similar pattern to those previously observed for other nicotine-containing and nicotine-free ECIGs in experienced users (Hiler et al., 2017). Results are in contrast to those observed for denicotinized cigarettes, which show that these products can suppress abstinence symptoms without delivering nicotine (e.g., Breland, Buchhalter, Evans, Eissenberg, 2002; Butschsky, Bailey, Henningfield, Pickworth, 1995). Presumably, denicotinized cigarettes differ in some way from an ECIG without nicotine, in that denicotinized cigarettes are subjectively reinforcing for cigarette smokers, while in comparison, an ECIG without nicotine is less reinforcing for experienced ECIG users (and thus has a lower abuse potential than an ECIG that delivers nicotine).

Further, significantly higher mean ratings for items such as lightheaded in the OWN condition compared to other conditions indicate that OWN EICGs have a greater potential for abuse. Finally, significantly higher ratings of satisfaction and other items for OWN and ECIG_highest also suggest higher abuse potential for these products, compared to ECIG_0 and IN. Other work has also shown that experienced ECIG users rate nicotine-containing ECIGs higher than nicotine-free ECIGs, in terms of satisfaction (Hiler et al., 2017).

Exploratory results showing a correlation between plasma nicotine increase and craving reduction, as well as a correlation between craving reduction and MCP crossover point suggest that ratings of subjective ECIG effects and measures of willingness to pay for an ECIG may be related. These results are supported by evidence reported by Wilson, Franck, Koffarnus, & Bickel (2016) who found that higher scores on the Questionnaire of Smoking Urges were associated with willingness to pay higher prices for ECIGs.

This study had several limitations. First, immediate verification of pre-session nicotine abstinence in exclusive ECIG users currently is not possible, as the usual test in cigarette smokers is by measuring expired air carbon monoxide (CO) concentrations, and ECIGs do not deliver CO. This problem has been noted by other researchers (e.g., Blank et al., 2016; Hiler et al., 2017), and until a test that reveals recent ECIG use is available, a waiting period before session onset may be required (as in Spindle et al., 2018), to assure that each participant has been abstinent from nicotine for at least the time of that waiting period. Second, this study compared participants’ own ECIG/liquid to the highest nicotine concentration available in their own liquid and although the nicotine concentrations were significantly different (6.86 mg/ml for OWN vs 8.77 mg/ml for ECIG_highest; t(23) = −3.09; p < .05); testing with a higher nicotine concentration may reveal greater differences between products. Third, ECIG flavor was not manipulated in this study, but flavor is likely a factor in ECIG abuse liability (Barnes et al., 2017; Audrain-McGovern, Strasser & Wileyto, 2016). Fourth, this study included no women participants and thus is generalizable to men only. Fifth, the sample size was relatively small, although effect sizes were large enough to detect effects for the MCP, plasma nicotine, and some subjective effects. Last, the laboratory setting may not generalize to the real world, although laboratory methods such as those used in this study can be good predictors of population level drug abuse (e.g., Rush, Collins, & Pazzaglia, 1988, Schuh, Schuster, Hopper, & Mendel, 2000).

The results of this study also show the value of a comprehensive abuse liability assessment, including behavioral tasks, assessment of drug delivery, and subjective effects, as well as adequate control conditions. Additional work must be done to assess fully the abuse potential of ECIGs, including the effects of flavors and other device and liquid characteristics, in experienced ECIG users as well as in cigarette smokers (as in Maloney et al., 2019). Laboratory studies assessing the abuse potential of products can help inform the Food and Drug Administration (FDA), which has regulatory authority over ECIGs. FDA. Specifically, FDA could use the techniques described here to assess the effects of potential future regulation on product abuse potential.

Public health significance:

The purpose of this study was to assess the abuse liability of a ~3.3V, 1.5 ohm, dual-coil ECIG (~7.3 W), loaded with the highest nicotine concentration available in participants’ own brand/own flavor, using a variety of abuse liability measures. Results suggest that nicotine-containing ECIGs have greater abuse liability than a nicotine-free ECIG and an FDA-approved nicotine inhaler. The results of this study show the value of a comprehensive abuse liability assessment, including behavioral tasks, assessment of drug delivery, and subjective effects, as well as adequate control conditions.

Acknowledgements

This research was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number P50DA036105 and U54DA036105 and the Center for Tobacco Products of the U.S. Food and Drug Administration. The authors would also like to thank Barbara Kilgalen, Hannah Mayberry, Caroline Smith, Melanie Crabtree, and Janet Austin for their help with data collection and data management.

This work has been presented previously at The Society for Research on Nicotine and Tobacco (SRNT) 2017 (Florence, Italy), SRNT 2018 (Baltimore, Maryland), the Reduce Tobacco Use Conference 2018 (Norfolk, VA), and the NIH Tobacco Regulatory Science Conference (TCORS) October 2017 (Bethesda, Maryland).

Footnotes

Disclosures

All authors contributed significantly to the study and all authors have read and approved the final manuscript. The content is solely the responsibility of the authors and does not necessarily represent the views of the NIH or the FDA. Dr. Eissenberg is a paid consultant in litigation against the tobacco industry and is named on a patent for a device that measures the puffing behavior of ECIG users.

References

  1. Audrain-McGovern J, Strasser AA, & Wileyto EP (2016). The impact of flavoring on the rewarding and reinforcing value of e-cigarettes with nicotine among young adult smokers. Drug and alcohol dependence, 166, 263–267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Barnes AJ, Bono RS, Lester RC, Eissenberg TE, & Cobb CO (2017). Effect of flavors and modified risk messages on e-cigarette abuse liability. Tobacco Regulatory Science, 3(4), 374–387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Balster RL, & Bigelow GE (2003). Guidelines and methodological reviews concerning drug abuse liability assessment. Drug and Alcohol Dependence, 70(3), S13–S40. [DOI] [PubMed] [Google Scholar]
  4. Balster RL, & Walsh SL (2010). Abuse Liability Evaluation. Encyclopedia of Psychopharmacology, 2–7. [Google Scholar]
  5. Blank MD, Breland AB, Cobb CO, Spindle T, Ramôa C, & Eissenberg T (2016). Clinical laboratory evaluation of electronic cigarettes: methodological challenges. Tobacco regulatory science, 2(4), 426–439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Breland AB, Buchhalter AR, Evans SE, & Eissenberg T (2002). Evaluating acute effects of potential reduced-exposure products for smokers: clinical laboratory methodology. Nicotine & Tobacco Research, 4(Suppl_2), S131–S140. [DOI] [PubMed] [Google Scholar]
  7. Breland AB, Evans SE, Buchhalter AR, & Eissenberg T (2002). Acute effects of AdvanceTM: a potential reduced exposure product for smokers. Tobacco Control, 11(4), 376–378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Breland A, Soule E, Lopez A, Ramôa C, El-Hellani A, & Eissenberg T (2017). Electronic cigarettes: what are they and what do they do? Annals of the New York Academy of Sciences, 1394(1), 5–30. DOI: 10.1111/nyas.12977 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Buchhalter AR, Acosta MC, Evans SE, Breland AB, & Eissenberg T (2005) Tobacco abstinence symptom suppression: the role played by the smoking-related stimuli that are delivered by denicotinized cigarettes. Addiction, 100(4), 550–559. [DOI] [PubMed] [Google Scholar]
  10. Butschky MF, Bailey D, Henningfield JE, & Pickworth WB (1995). Smoking without nicotine delivery decreases withdrawal in 12-hr abstinent smokers. Pharmacology, Biochemistry and Behavior, 50, 91–96. [DOI] [PubMed] [Google Scholar]
  11. Carter LP, Stitzer ML, Henningfield JE, O’Connor RJ, Cummings KM, & Hatsukami DK (2009). Abuse Liability Assessment of Tobacco Products Including Potential Reduced Exposure Products (PREPs). Cancer Epidemiology, Biomarkers & Prevention: A Publication of the American Association for Cancer Research, Cosponsored by the American Society of Preventive Oncology, 18(12), 3241–3262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cassidy RN, Tidey JW, Colby SM, Long V, & Higgins ST (2017). Initial development of an e-cigarette purchase task: a mixed methods study. Tobacco regulatory science, 3(2), 139–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Cox LS, Tiffany ST, & Christen AG (2001). Evaluation of the brief questionnaire of smoking urges (QSU-brief) in laboratory and clinical settings. Nicotine & Tobacco Research, 3(1), 7–16. [DOI] [PubMed] [Google Scholar]
  14. Cullen KA, Ambrose BK, Gentzke AS, Apelberg BJ, Jamal A, King BA (2018). Notes from the field: Use of electronic cigarettes and any tobacco product among middle and high school students—United States, 2011–2018. MMWR. Morbidity and Mortality Weekly Report, 67(45), 1276–1277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dawkins L, & Corcoran O (2014). Acute electronic cigarette use: nicotine delivery and subjective effects in regular users. Psychopharmacology, 231(2), 401–407. DOI: 10.1007/s00213-013-3249-8 [DOI] [PubMed] [Google Scholar]
  16. Evans SE, Blank M, Sams C, Weaver MF, & Eissenberg T (2006). Transdermal nicotine-induced tobacco abstinence symptom suppression: nicotine dose and smokers’ gender. Experimental and Clinical Psychopharmacology, 14(2), 121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Farsalinos KE, Spyrou A, Tsimopoulou K, Stefopoulous C, Romagna G, & Voudris V (2014). Nicotine absorption from electronic cigarette use: comparison between first and new-generation devices. Scientific Reports, 4(4133), 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Foulds J, Stapleton J, Feyerabend C, Vesey C, Jarvis M, & Russell MA (1992). Effect of transdermal nicotine patches on cigarette smoking: a double-blind crossover study. Psychopharmacology, 106(3), 421–427. [DOI] [PubMed] [Google Scholar]
  19. Girvalaki C, Tzatzarakis M, Kyriakos CN, Vardavas AI, Stivaktakis PD, Kavvalakis M, … & Vardavas C (2018). Composition and chemical health hazards of the most common electronic cigarette liquids in nine European countries. Inhalation Toxicology, 30(9–10), 361–369. [DOI] [PubMed] [Google Scholar]
  20. Griffiths RR, Troisi JR, Silverman K, & Miumford GK (1993). Multiple-choice procedure: an efficient approach for investigating drug reinforcement in humans. Behavioural Pharmacology, 4(1), 3–14. [PubMed] [Google Scholar]
  21. Goldenson NI, Kirkpatrick MG, Barrington-Trimis JL, Pang RD, McBeth JF, Pentz, … & Leventhal AM (2016). Effects of sweet flavorings and nicotine on the appeal and sensory properties of e-cigarettes among young adult vapers: Application of a novel methodology. Drug and Alcohol Dependence, 168, 176–180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Henningfield JE, Hatsukami DK, Zeller M, & Peters E (2011). Conference on abuse liability and appeal of tobacco products: Conclusions and recommendations. Drug and Alcohol Dependence, 116(1–3), 1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hiler M, Breland A, Spindle T, Maloney S, Lipato T, Karaoghlanian N & Eissenberg T (2017). Electronic cigarette user plasma nicotine concentration, puff topography, heart rate, and subjective effects: Influence of liquid nicotine concentration and user experience. Experimental and Clinical Psychopharmacology, 25(5), 380–392. DOI: 10.1037/pha0000140 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hughes JR, & Hatsukami D (1986). Signs and symptoms of tobacco withdrawal. Archives of General Psychiatry, 43(3), 289–294. [DOI] [PubMed] [Google Scholar]
  25. Jaffe JH, & Jaffe FK (1989). Historical perspectives on the use of subjective effects measures in assessing the abuse potential of drugs Testing for abuse liability of drugs in humans (pp. 43). NIDA Research Monograph, Number 92. [PubMed] [Google Scholar]
  26. Jamal A, Gentzke A, Hu SS, Cullen KA, Apelberg BJ, Homa DM, & King BA (2017). Tobacco use among middle and high school students – United States, 2011–2016. Centers for Disease Control and Prevention MMWR Morbidity and Mortality Weekly Report, 66 (23), 597–603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kim S, Goniewicz M, Yu S, Kim B, & Gupta R (2015). Variations in label information and nicotine levels in electronic cigarette refill liquids in South Korea: regulation challenges. International journal of environmental research and public health, 12(5), 4859–4868. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. King BA, Patel R, Nguyen KH, & Dube SR (2015). Trends in awareness and use of electronic cigarettes among US adults, 2010–2013. Nicotine & Tobacco Research, 17(2), 219–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Krishnan-Sarin S, Green BG, Kong G, Cavallo DA, Jatlow P, Gueorguieva R, … & O’Malley SS (2017). Studying the interactive effects of menthol and nicotine among youth: An examination using e-cigarettes. Drug and Alcohol Dependence, 180, 193–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Leventhal AM, Strong DR, Kirkpatrick MG, Unger JB, Sussman S, Riggs NR, … & Audrain-McGovern J (2015). Association of electronic cigarette use with initiation of combustible tobacco product smoking in early adolescence. JAMA, 314(7), 700–707. doi: 10.1001/jama.2015.8950. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Lopez AA, Hiler M, Maloney S, Eissenberg T, & Breland AB (2016). Expanding clinical laboratory tobacco product evaluation methods to loose-leaf tobacco vaporizers. Drug & Alcohol Dependence, 169, 33–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Maloney SF, Breland A, Soule EK, Hiler M, Ramôa C, Lipato T, & Eissenberg T (2019). Abuse liability assessment of an electronic cigarette in combustible cigarette smokers. Experimental and Clinical Psychopharmacology. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. McMillen RC, Gottlieb MA, Shaefer RMW, Winickoff JP, & Klein JD (2015). Trends in Electronic Cigarette Use Among U.S. Adults: Use is Increasing in Both Smokers and Nonsmokers. Nicotine & Tobacco Research : Official Journal of the Society for Research on Nicotine and Tobacco, 17(10), 1195–202. 10.1093/ntr/ntu213 [DOI] [PubMed] [Google Scholar]
  34. McPherson S, Howell D, Lewis J, Barbosa-Leiker C, Metoyer PB, & Roll J (2016). Self-reported smoking effects and comparative value between cigarettes and high dose e-cigarettes in nicotine-dependent cigarette smokers. Behavioural Pharmacology, 27(2 and 3-Special Issue), 301–307. [DOI] [PubMed] [Google Scholar]
  35. Perkins KA, Sexton JE, Reynolds WA, Grobe JE, Fonte C, & Stiller RL (1994). Comparison of acute subjective and heart rate effects of nicotine intake via tobacco smoking versus nasal spray. Pharmacology Biochemistry and Behavior, 47(2), 295–299. [DOI] [PubMed] [Google Scholar]
  36. Pickworth WB, Bunker EB, & Henningfield JE (1994). Transdermal nicotine: reduction of smoking with minimal abuse liability. Psychopharmacology, 115(1), 9–14. [DOI] [PubMed] [Google Scholar]
  37. Quisenberry AJ, Koffarnus MN, Epstein LH, & Bickel WK (2017). The Experimental Tobacco Marketplace II: Substitutability and sex effects in dual electronic cigarette and conventional cigarette users. Drug and alcohol dependence, 178, 551–555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Ramôa CP, Hiler MM, Spindle TR, Lopez AA, Lipato T, Karaoughlanian N, Shihadeh A, Breland AB, Eissenberg T (2016). Electronic cigarette nicotine delivery can exceed that of combustible cigarettes: A preliminary report. Tobacco Control, 25(e1), e6–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Raymond BH, Collette-Merrill K, Harrison RG, Jarvis S, & Rasmussen RJ (2018). The nicotine content of a sample of e-cigarette liquid manufactured in the United States. Journal of addiction medicine, 12(2), 127–131. [DOI] [PubMed] [Google Scholar]
  40. Rush CR, Kollins SH, & Pazzaglia PJ (1998). Discriminative-stimulus and participantrated effects of methylphenidate, bupropion, and triazolam in d-amphetamine-trained humans. Experimental and clinical psychopharmacology, 6(1), 32. [DOI] [PubMed] [Google Scholar]
  41. Schoenborn CA, & Gindi RM (2015). Electronic Cigarette Use Among Adults: United States, 2014. NCHS Data Brief, 217, 1–8. [PubMed] [Google Scholar]
  42. Schuh LM, Schuster CR, Hopper JA, & Mendel CM (2000). Abuse liability assessment of sibutramine, a novel weight control agent. Psychopharmacology, 147(4), 339–346. [DOI] [PubMed] [Google Scholar]
  43. Soneji S, Barrington-Trimis JL, Wills TA, Leventhal AM, Unger JB, Gibson LA, …& Sargent JD (2017). Association between initial use of e-cigarettes and subsequent cigarette smoking among adolescents and young adults: A systematic review and meta-analysis. JAMA Pediatrics, 171(8), 788–797. DOI: 10.1001/jamapediatrics.2017.1488 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Spindle TR, Hiler MM, Cooke ME, Eissenberg T, Kendler KS & Dick DM (2017). Electronic cigarette use and uptake of cigarette smoking: a longitudinal examination of US college students. Addictive Behaviors, 67, 66–72. DOI: 10.1016/j.addbeh.2016.12.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Spindle TR, Hiler MM, Breland AB, Karaoghlanian NV, Shihadeh AL, & Eissenberg T (2016). The influence of a mouthpiece-based topography measurement device on electronic cigarette user’s plasma nicotine concentration, heart rate, and subjective effects under directed and ad libitum use conditions. Nicotine & Tobacco Research, 19(4), 469–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Spindle TR, Talih S, Hiler MM, Karaoghlanian N, Halquist MS, Breland AB, … & Eissenberg T (2018). Effects of electronic cigarette liquid solvents propylene glycol and vegetable glycerin on user nicotine delivery, heart rate, subjective effects, and puff topography. Drug and alcohol dependence, 188, 193–199 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Stein JS, Koffarnus MN, Stepanov I, Hatsukami DK, & Bickel WK (2018). Cigarette and e-liquid demand and substitution in e-cigarette-naïve smokers. Experimental and clinical psychopharmacology, 26(3), 233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Stiles MF, Campbell LR, Jin T, Graff DW, Fant RV, & Henningfield JE (2018). Assessment of the abuse liability of three menthol Vuse Solo electronic cigarettes relative to combustible cigarettes and nicotine gum. Psychopharmacology, 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Talih S, Balhas Z, Salman R, El-Hage R, Karaoghlanian N, El-Hellani A, Baassiri M, Jaroudi E, Eissenberg T, Saliba N & Shihadeh A (2017). Transport phenomena governing nicotine emissions from electronic cigarettes: Model formulation and experimental investigation. Aerosol Science and Technology, 51(1), 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Vansickel AR, Cobb CO, Weaver MF, & Eissenberg TE (2010). A clinical laboratory model for evaluating the acute effects of electronic “cigarettes”: nicotine delivery profile and cardiovascular and subjective effects. Cancer Epidemiology, Biomarkers & Prevention : A Publication of the American Association for Cancer Research, Cosponsored by the American Society of Preventive Oncology, 19(8), 1945–1953. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Vansickel AR, Weaver MF, & Eissenberg T (2012). Clinical laboratory assessment of the abuse liability of an electronic cigarette. Addiction, 107(8), 1493–1500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Vansickel AR, & Eissenberg T (2013). Electronic cigarettes: effective nicotine delivery after acute administration. Nicotine & Tobacco Research, 15(1), 267–270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Wagener TL, Floyd EL, Stepanov I, Driskill LM, Frank SG, Meier E, … & Queimado L (2017). Have combustible cigarettes met their match? The nicotine delivery profiles and harmful constituent exposures of second-generation and third-generation electronic cigarette users. Tobacco Control, 26 e23–e28. DOI: 10.1136/tobaccocontrol-2016-05304 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Watkins SL, Glantz SA, & Chaffee BW (2018). Association of noncigarette tobacco product use with future cigarette smoking among youth in the population assessment of tobacco and health (PATH) study, 2013–2015. JAMA Pediatrics, 172(2), 181–187. doi: 10.1001/jamapediatrics.2017.4173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Wilson AG, Franck CT, Koffarnus MN, & Bickel WK (2015). Behavioral economics of cigarette purchase tasks: within-subject comparison of real, potentially real, and hypothetical cigarettes. Nicotine & Tobacco Research, 18(5), 524–530. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Yan XS, & D’Ruiz C (2015). Effects of using electronic cigarettes on nicotine delivery and cardiovascular function in comparison with regular cigarettes. Regulatory Toxicology and Pharmacology, 71(1), 24–34. [DOI] [PubMed] [Google Scholar]
  57. Yingst JM, Veldheer S, Hammett E, Hrabovsky S, & Foulds J (2017). A method for classifying user-reported electronic cigarette liquid flavors. Nicotine & Tobacco Research, 19(11), 1381–1385. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES