Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Apr 1.
Published in final edited form as: Addict Behav. 2017 Nov 16;79:203–212. doi: 10.1016/j.addbeh.2017.11.016

Establishing Consensus on Survey Measures for Electronic Nicotine and Non-Nicotine Delivery System Use: Current Challenges and Considerations for Researchers

Scott R Weaver a, Hyoshin Kim b, Allison M Glasser c, Erin L Sutfin d, Jessica Barrington-Trimis e, Thomas J Payne f, Megan Saddleson g, Alexandra Loukas h
PMCID: PMC5807235  NIHMSID: NIHMS922259  PMID: 29173942

Abstract

The development and validation of survey measures for electronic nicotine and non-nicotine delivery system (ENDS) use has not kept pace with the burgeoning research on them. This, along with the diverse and evolving nature of ENDS, presents several unique measurement challenges and hampers surveillance and tobacco regulatory research efforts. In this commentary, we identify four important areas related to ENDS use (describing ENDS products; defining current use; evaluating frequency and quantity of use; and characterizing devices and e-liquids) and summarize a selective review of the measurement and definitions of these constructs across prominent national tobacco use surveys and 30 projects within the 14 federally-funded Tobacco Centers of Regulatory Science. Across these national, regional, and local studies, there was considerable variability and relatively little consensus in ENDS use measures – thus highlighting the need for caution when comparing findings across studies or over time until more research is available to evaluate the sensitivity of findings to differing measures. Drawing from the nascent ENDS use measurement research literature and our experiences, we conclude with general considerations for measuring ENDS use for tobacco researchers as an initial step towards the development of consensus measures.

Keywords: electronic nicotine device systems, electronic cigarette, e-cigarette, measurement, consensus measures

1. Introduction

Despite a large and growing federal investment in research on electronic nicotine and non-nicotine delivery systems (ENDS/ENNDS),1 interpretations of findings from this research and their application to inform policy has been hampered by a lack of validated measurement tools for ENDS. The recent market entry of ENDS, along with their diversity and rapidly evolving nature, has presented unique measurement challenges. As a result, few standards and data on the psychometric properties of ENDS use measures exist to guide researchers on measurement development, selection, and interpretation. Furthermore, the validity and reliability of measures may change as the nature of ENDS products and their use continue to evolve. Thus, there is an ongoing, critical need for guidance in selecting and developing measures to assess ENDS use. This commentary article identifies four important areas related to the evaluation of ENDS use in population surveys: (a) describing ENDS products (terminology, images); (b) defining current use; (c) evaluating frequency and quantity of use; (d) characterizing devices and e-liquids (device type, flavors, and nicotine content). We provide a summary (conducted August-October 2015) of the measurement of these constructs across national tobacco use surveys and 30 projects within the 14 federally-funded Tobacco Centers of Regulatory Science (TCORS; see Tables 1 & 2), with emphasis on areas where there is greater consistency and possibly emerging consensus. Additionally, we provide considerations for selecting ENDS use measures for future studies.

Table 1.

Electronic nicotine and non-nicotine delivery system (ENDS) items from U.S. national surveys

Construct NYTS (2014) NATS (2014) HealthStyles (2014) MTF (2015) PATH (2015-2016)a TUS-CPS (2014)
Preamble/Terminology The next six questions are about the use of electronic cigarettes or e-cigarettes such as Blu, 21st Century Smoke or NJOY. Electronic cigarettes, or e-cigarettes as they are often called, are battery-operated devices that simulate smoking a cigarette, but do not involve the burning of tobacco. The heated vapor produced by an e-cigarette often contains nicotine. Electronic vaporizers make a mist that is inhaled and have the feel of cigarette smoking. Examples include e-cigarettes and e-pens. (Y/A) Some [EPRODTYPE1]s can be bought as one-time, disposable products, while others can be bought as reusable kits with a cartridge or tank system. Some people refill their own [EPRODTYPE1]s with nicotine fluid, sometimes called “e-liquid” or “e-juice”. Disposable [EPRODTYPE1]s, [EPRODTYPE1] cartridges and e-liquid come in many different flavors and nicotine concentrations. Some common brands include Fin, NJOY, Blu, e-Go and Vuse. Please think only about [EPRODTYPE1]s as you answer these questions. [There will be separate questions later about other kinds of electronic nicotine products you use in addition to [EPRODTYPE1].

(Y/A) The next questions are about electronic nicotine products, such as e-cigarettes, vape pens, hookah pens, personal vaporizers and mods, e-cigars, e-pipes, and e-hookahs. These products are battery-powered and produce vapor instead of smoke. They typically use a nicotine liquid, although the amount of nicotine can vary and some may not contain any nicotine at all. Some common brands include Fin, NJOY, Blu, e-Go and Vuse.		 The next question is about electronic or e-cigarettes. You may also know them as vape-pens, hookah-pens, e-hookahs, or e-vaporizers. Some look like cigarettes, and others look like pens or small pipes. These are battery-powered, usually contain liquid nicotine, and produce vapor instead of smoke.(br)(Information for the field worker and not routinely read to the respondent) E-cigarettes and similar products can be bought as one-time, disposable products, as re-usable kits with a cartridge, or with refillable chambers. These items contain a nicotine and/or flavored liquid, often called “e-liquids” or “e-juice.” Some of these products look like regular cigarettes or pens, and some more closely resemble a small pipe or cigar. Popular brands include “NJOY,” “Blu,” “Logic,” and “Vuse.”
Lifetime/Ever Use Have you ever tried an electronic cigarette or e-cigarette such as Blu, 21st Century Smoke or NJOY? A. Yes, B. No Have you ever used an electronic cigarette, even just one time in your entire life? 1. Yes, 2. No Have you ever tried any of the following products, even just one time? 5. Electronic Cigarettes or E-cigarettes, such as Ruyan or NJOY Have you ever used an electronic vaporizer such as an e-cigarette? 1. Never, 2. Once or twice, 3. Occasionally but not regularly, 4. Regularly in the past, 5. Regularly now (Y) Have you ever used an electronic nicotine product, even one or two times? (Electronic nicotine products include e-cigarettes, vape pens, hookah pens, personal vaporizers and mods, e-cigars, e-pipes, and ehookahs.) 1. Yes, 2. No

(Y/A) Which of the following electronic nicotine products have you used? Choose all that apply. 1. E-cigarette including vape pens, hookah pens, personal vaporizers, and mods 2. E-cigar, 3. E-pipe, 4. E-hookah, 5. Something else (SPECIFY)		 Have you EVER used E-cigarettes EVEN ONE TIME? 1. Yes, 2. No
How many times in total do you think you have used an electronic-cigarette during your lifetime? 1. 1–10, 2. 11–20, 3. 21–50, 4. Over 50 times How many days have you used an electronic vapor product, such as an electronic cigarette (e-cigarette), electronic hookah (e-hookah), or vape pen in your entire life? 1. 1 day, 2. 2–10 days, 3. 11–20 days, 4. 21–50 days, 5. 51–100 days, 6. Over 100 days (Y/A) How many times have you used an [EPRODTYPE1] in your entire life? 1. 1 time, even just a few puffs, 2. 2 to 10 times, 3. 11 to 20 times, 4. 21 to 50 times, 5. 51 to 99 times, 6. 100 or more times
Frequency/Quantity During the past 30 days, on how many days did you use electronic cigarettes or e-cigarettes such as Blu, 21st Century Smoke, or NJOY? A. 0 days, B. 1 or 2 days, C. 3 to 5 days, D. 6 to 9 days, E. 10 to 19 days, F. 20 to 29 days, G. All 30 days Do you now use electronic cigarettes every day, some days, rarely, or not at all? 1. Every day, 2. Some days, 3. Rarely, 4. Not at all In the past 30 days, which of the following products have you used at least once? 5. Electronic Cigarettes or E-cigarettes, such as Blu, NJOY, or LOGIC During the last 30 days, on how many days (if any) have you used an electronic vaporizer such as an e-cigarette? 1. None, 2. 1–2 days, 3. 3–5 days, 4. 6–9 days, 5. 10–19 days, 6. 20–30 days (A) In the past 30 days, have you used an electronic nicotine product, even one or two times? (Electronic nicotine products include e-cigarettes, vape pens, hookah pens, personal vaporizers and mods, e-cigars, e-pipes, and e-hookahs.) 1. Yes, 2. No

(Y) When was the last time you used an [EPRODTYPE1], even one or two times? 1. Earlier today, 2. Not today but sometime in the past 7 days, 3. Not in the past 7 days but sometime in the past 30 days, 4. Not in the past 30 days, but sometime in the past 6 months, 5. No in the past 6 months but sometime in the past year, 6. 1 to 4 years ago, 7. 5 or more years ago		 [Someday users] On how many of the past 30 days did you use E-cigarettes?
(A) On how many of the past 30 days did you use an [EPRODTYPE1]?

(Y) In the past 30 days, on how many days did you use an [EPRODTYPE1]?
Ever users: (Y/A) Have you ever used [EPRODTYPE1]s fairly regularly? 1. Yes, 2. No

Current users: (A) Do you now use [EPRODTYPE]s …1. Every day, 2. Some days, 3. Not at all

Product Regularly Used: (Y/A) Which electronic nicotine product do you use most often? 1. E-cigarette including vape pens, hookah pens, personal vaporizers, and mods, 2. E-cigar, 3. E-pipe, 4. E-hookah
(A) On average, about how many [EPRODFILL1] do you now use each day? 1. Less than 1 each day, 2. [Fill in the blank]

(Y) In the past 30 days, on the days you used an [EPRODTYPE1], how many [EPRODFILL1] did you use per day? 1. Less than 1 per day, 2. [Fill in the blank]
Flavors Which of the following tobacco products that you used in the past 30 days were flavored to taste like menthol (mint), alcohol (wine, cognac), candy, fruit, chocolate or other sweets? (CHOOSE ALL THAT APPLY) C. Electronic cigarettes or e-cigarettes Were any of the electronic cigarettes that you used in the past 30 days flavored to taste like menthol, mint, clove, spice, candy, fruit, chocolate, or other sweets? 1. Yes, 2. No (Y) When you first used an [EPRODTYPE1], which flavor did you use? Choose all that apply. 1. Tobacco-flavored, 2. Menthol or mint, 3. Clove or spice, 4. Fruit, 5. Chocolate, 6. An alcoholic drink (such as wine, cognac, margarita or other cocktails), 7. A non-alcoholic drink (such as coffee, soda, energy drinks, or other beverages), 8. Candy, desserts or other sweets, 9. Some other flavor (SPECIFY)

(A) When you first started using [EPRODTYPE1]s, which flavor did you use? Choose all that apply. 1. Tobacco-flavored, 2. Menthol or mint, 3. Clove or spice, 4. Fruit, 5. Chocolate, 6. An alcoholic drink (such as wine, cognac, margarita or other cocktails), 7. A non-alcoholic drink (such as coffee, soda, energy drinks, or other beverages), 8. Candy, desserts or other sweets, 9. Some other flavor (SPECIFY)		 Some tobacco products come in flavors such as menthol or mint, clove, spice, candy, fruit, chocolate, alcohol, or other flavors. When you use an E-cigarette is it usually flavored?
(Y/A) When you first started using [EPRODTYPE1]s, which flavor did you use? Choose all that apply. 1. Tobacco-flavored, 2. Menthol or mint, 3. Clove or spice, 4. Fruit, 5. Chocolate, 6. An alcoholic drink (such as wine, cognac, margarita or other cocktails), 7. A non-alcoholic drink (such as coffee, soda, energy drinks, or other beverages), 3. Candy, desserts or other sweets, 9. Some other flavor
(Y) Which flavors of [EPRODFILL2] have you used in the past 30 days? Choose all that apply. 1. Tobacco-flavored, 2. Menthol or mint, 3. Clove or spice, 4. Fruit, 5. Chocolate, 6. An alcoholic drink (such as wine, cognac, margarita or other cocktails), 6. A non-alcoholic drink (such as coffee, soda, energy drinks, or other beverages), 8. Candy, desserts or other sweets, 9. Some other flavor

(A) In the past 30 days, which flavors of [EPRODFILL2] have you used? 1. Tobacco-flavored, 2. Menthol or mint, 3. Clove or spice, 4. Fruit, 5. Chocolate, 6. An alcoholic drink (such as wine, cognac, margarita or other cocktails), 6. A non-alcoholic drink (such as coffee, soda, energy drinks, or other beverages), 8. Candy, desserts or other sweets, 9. Some other flavor
(A) What flavor [is/was] [your regular brand/the brand you last used]? Choose all that apply. 1. Tobacco-flavored, 2. Menthol or mint, 3. Clove or spice, 4. Fruit, 5. Chocolate, 6. An alcoholic drink (such as wine, cognac, margarita or other cocktails), 7. A nonalcoholic drink (such as coffee, soda, energy drinks, or other beverages), 8. Candy, desserts or other sweets, 9. Some other flavor (SPECIFY)
Device Type Rechargeable vs. Disposable: (Y/A) [Is/Was] your [EPRODTYPE1] rechargeable? 1. Yes, 2. No

Cartridges: (Y/A) [Does/Did] your [EPRODTYPE1] use cartridges? 1. Yes, 2. No

Tank System: (Y/A) [Does/Did] your [EPRODTYPE1] use a tank system? 1. Yes, 2. No

Refillable:

  • (Y/A) [Can/Could] you refill your [[EPRODTYPE1]/[EPRODTYPE1] cartridges] with “e-liquid”? 1. Yes, 2. No

  • (Y/A) Who [refills/refilled] your [[EPRODTYPE1]/[EPRODTYPE1] cartridge] with e-liquid? 1. I refill it myself, 2. I get it refilled at a vape shop or vapor lounge, 3. I buy it already filled, 4. I get it refilled some other way

  • (A) How often [do/did] you refill your [[EPRODTYPE1]/tank/cartridg es] with e-liquid? 1. Never, 2. At least once a day, 3. Once every couple of days, 4. Once a week, 5. Once a month, 6. Once a year

Voltage:

  • (Y/A) [Can/Could] you change the voltage on your [EPRODTYPE1]? 1. Yes, 2. No

  • (A) [Did/Do] you change the voltage on your [EPRODTYPE1]? 1. Yes, 2. No
Nicotine content The LAST TIME you used an electronic vaporizer such as an e-cigarette, what was in the mist you inhaled? (Mark only one circle.) 1. Nicotine, 2. Marijuana or hash oil, 3. Just flavoring, 4. Other, 8. Don’t know (A) Does the [EPRODTYPE2] you usually use contain nicotine? 1. Yes, 2. No

(Y) Please think about the [EPRODFILL3] you use most of the time. Does the [EPRODFILL3] you use contain nicotine? 1. Yes, 2. No

(A) What concentration of nicotine do you usually use? 1. I don’t know the concentration, 2. 0mg or 0%, 3. 1–12mg or 0.1–1.2%, 4. 13–17mg or 1.3–1.7%, 5. 18–24mg or 1.8–2.4%, 6. 25+mg or 2.5+%
Open in a new tab

NYTS: National Youth Tobacco Survey; NATS: National Adult Tobacco Survey; MTF: Monitoring the Future Survey; PATH: Population Assessment of Tobacco and Health Study Survey (Wave 3 – 2015–2016); TUS-CPS: Tobacco Use Supplement-Current Population Survey; Y: Youth PATH Survey; A: Adult PATH Survey

a

[EPRODTYPE] was shown as one of the following terms that the respondent based on their response to a previous question asking them which electronic nicotine product they used most often: e-cigarette, e-cigar, e-pipe, e-hookah, or electronic nicotine product.

*

This table has been adapted from the “2016 Surgeon General’s Report: E-Cigarette Use among Youth and Young Adults”, Chapter 2, Appendix 2.2 (Key Measures of Use). (U.S. Department of Health and Human Services, 2016)

Table 2.

Summary of electronic nicotine and non-nicotine delivery system (ENDS) use measurement across the Tobacco Centers of Regulatory Science (TCORS)

Construct Summary of Measures
Product Description (preamble, terminology, use of images) Product Description:
Most studies included a brief description of ENDS products, including various terms used to describe the products, such as vape pens, vaping devices, vapor pens, electronic vapor products, personal vaporizer, and vape. Some studies provided sample brands, and many mentioned the use of flavors, with fewer mentioning nicotine. Different product types (disposable, rechargeable, refillable tanks) were included in several descriptions, with some comparing the appearance of these products to cigarettes and pens.
Terminology in Items:
In the actual survey items, most studies used some form of electronic cigarettes, e-cigarettes, or e-cigs.
Use of Images:
Approximately one-half of TCORS surveys used product images in the product description.
Definitions of Use Lifetime Use:
The most common measure and definition used was “Have you ever used…” with some variation in the verb (“tried”), nature of use (“as intended”) or minimum amount of use (“used even once”, “even 1 or 2 puffs”). Less frequent were measures that determined lifetime use by asking the age of first use with a never used option or asking the total number of times used.
Current Use:
The majority of studies, including nearly all youth studies, defined current use by a measure of recent past use, where recent was most typically specified as the past 30 days, or less often the past 7 days. Response options were most often “yes/no,” number of days used, or time since last use (even if longer than 30 days). Determining the line between past use and current use is not well-established at this point, nor are the criteria for classifying one as a past-/ex-user. A minority of studies (all of which involved adult samples) defined current use by self-endorsement of current use with either a yes/no response option or, more often, every day/some days/not at all options. One study included a rarely response option. Two studies used multiple criteria to define current use. One study defined current use based either on a minimum number of days used in the past 4 weeks or any use in the past week. The other study ascertained current use and then assessed time of last use, though a specific definition was not provided.
Regular/Established Use:
Criteria for regular use appear to be more variable. Investigators of about half of the projects either did not define regular use or were uncertain how they would define it. Among those that did define regular use, most used some threshold of number of days of use, ranging from any use in the past 30 days to more than half of the days in the past month. A few studies either asked the respondent directly if they “used fairly regularly” or “on a regular basis.” Similarly, most investigators either did not define regular/established use or were uncertain how to define this construct. Among those few studies that did define regular/established use, there was considerable variability, with most using a specific time-based constraint (e.g., used for 2 months or more) or some combination of duration with either frequency, condition (i.e., use with nicotine), or recent use.
Frequency and Quantity Frequency of Use:
The most commonly used frequency questions were, “During the [past xx days], have you used [ENDS term]?” and “During the [past xx days], how many days have you used [ENDS term]?” Most studies reported the use of the “past 30 days” time frame. A few studies reported the use of “past 7 days.” Response options included Yes/No, or an open-ended format or categorical format for the number of days used. Most studies included one or more follow-up questions asking the number of times used per day if respondents affirmed use during the time frame. A few studies used the definition of “one time” as being “about 15 puffs” or “lasting about 10 minutes.” Some studies collected “daily use” data by asking the number of uses per day or by utilizing a daily diary. Other studies collected frequency measures, such as querying the number of occasions in a specific time period or intervention period, querying the use with categorical responses (e.g., every day/some days/not at all, weekly or more/monthly or more/not now/never used regularly), or utilizing a daily diary for 7 days.
Quantity of Use:
The most common question to assess quantity referred to the number of disposable cigarettes, cartridges, or refills used on those days. A few studies used the number of puffs per occasion; only one study asked about the number of occasions as well as the number of puffs per occasion. Other studies asked about how long it takes to use a cartridge/tank; and one study collected the number of puffs recorded by the ENDS device. Note that not all studies asking frequency-related questions included quantity-related questions.
Device Type and Characteristics All items addressing device types and characteristics differed across studies, but some had only slight variations in wording. Commonly used descriptors included disposable, rechargeable, mod, and refillable. Other studies also referred to the presence of a device cartridge or tank. In reference to the liquid used to refill cartridges, studies used terms such as e-liquid and e-juice. Most studies used photos in conjunction with the device type item, but very few assessed device brand.
Flavors Out of the categories of flavor descriptors used across studies, fruit and candy/dessert were the most common categories, followed by mint/menthol, tobacco, alcohol, coffee, spice, and other (including not flavored). The parameters of flavored use differed across studies, but the most frequent measure was past 30-day use.
Nicotine Content Most studies did not measure nicotine content; those that did assessed typical concentration (mg/mL), usually providing “zero” and “don’t know” options. One study assessed nicotine content by frequency (always/mostly/sometimes/rarely/never).
Open in a new tab

Note. ENDS use measures in TCORS surveys was assessed via an online Qualtrics survey sent to all TCORS and completed August-September 2015 (n=30 TCORS projects).

2. Product Description

Despite the widespread awareness of ENDS (Weaver et al., 2016), terminology varies among both consumers and researchers, and continues to evolve over time. Accordingly, many national tobacco use surveys and TCORS studies provide preamble text to describe ENDS prior to question administration, often by using multiple terms and identifying their common characteristics (e.g., flavored e-liquids, battery powered) and brand names. Consistent across preambles was the use of some variant of electronic cigarette, e-cigarette or e-cig when referring to ENDS. Less consistent was the inclusion of additional terminology (e.g., vape pens, vaping devices, mod, personal vaporizer), length and detail of the preamble text, and reference to specific characteristics (e.g., nicotine, flavors, disposable, refillable). Images of ENDS products were included in roughly half of the TCORS surveys, most of which were online surveys, as well as in the Population Assessment of Tobacco and Health (PATH) Study (Wave 3). Research is needed to evaluate the sensitivity of participant responses with respect to how ENDS are described (Walton et al., 2015). Findings from a recent study suggest a shift is occurring away from “e-cigarette” terminology towards “vapor” terminology, prompting the authors of that study to advise defaulting to “vapor” terminology (e.g., describing use as “vaping” and users as “vapers”), although another study found that “e-cigarette” and “vape” are widely understood among smokers participating in a web-based smoking intervention (Ayers et al., 2016; Pearson et al., 2016). Nonetheless, researchers should consider providing a preamble with sufficient detail and, when feasible, include images depicting the wide variety of ENDS device types, improving the likelihood that the target population has a clear understanding regarding the products to which the researcher is referring (Alexander et al., 2016).

3. Definitions of Current Use

Establishing meaningful definitions of ENDS use is key to understanding use patterns and the population health impact of ENDS. Whereas definitions and measures of lifetime use of ENDS are generally consistent across national and TCORS studies, there are important differences and scholarly debate in defining “current use.” Most commonly, particularly with youth studies, current use refers to any past month use. However, the value of this definition has been challenged for its inclusion of recent experimenters or recent former users, and for lumping infrequent users with frequent, established users (Amato, Boyle, & Levy, 2015, 2016; Kozlowski & Giovino, 2014). In studies of adults, current users are often defined by self-identification of current use through reported frequency of use (e.g., “some days”, “every day”, or “rarely”). The PATH Study survey programming (Wave 3; see Table 1) distinguishes regular current (some days or every day) adult users from experimental current users by whether they reported having ever “fairly regularly” used the product. A few studies have defined current use as ≥5 days of use during the past 30 days, suggesting this threshold might exclude infrequent, potentially experimental, users (Amato et al., 2015, 2016). The potential relevance of such a distinction is underscored by a recent study indicating that daily, but not non-daily, ENDS use was predictive of greater cigarette cessation attempts and reduced smoking among UK adults (Brose, Hitchman, Brown, West, & McNeill, 2015). Definitions of current ENDS use should incorporate this important distinction; however, broad consensus is lacking on the measures and the criteria for defining user groups, particularly across different study populations (e.g., youth and adults) where differential criteria for current use may be warranted.

Prevalence rates generated by different definitions can vary considerably, compromising comparisons across studies. Until there is consensus on how to define ENDS use status, researchers should consider including sufficient questions to permit application and reporting of multiple definitions to facilitate comparisons with other studies. At a minimum, this could entail assessing frequency of use (e.g., number of days used in past 30 days) and whether the individual self-identifies as a “current” user of ENDS. If distinguishing regular from experimental use, a measure of lifetime frequency or quantity of use would be necessary, although we are not aware of any validated measure or demarcation.

4. Frequency and Quantity

Compared to measuring cigarettes, assessing the frequency and quantity of ENDS use has been more challenging. ENDS use is less finite, entailing anything from a puff or two at a time, a longer session, or relatively continuous use throughout the day. For this reason, ENDS use is substantially more variable and more difficult to quantify than cigarette use (Kim, Davis, Dohack, & Clark, 2016). Most TCORS studies and national surveys have focused on measuring frequency, rather than quantity, of use. Most commonly, frequency was measured by asking the number of days respondents used ENDS in the past 30 days. Many TCORS studies and the PATH study (among daily users only) included one or more follow-up questions asking the number of times or occasions used per day. Few studies assessed quantity of ENDS use, likely due to the difficulties in consistently and accurately capturing these data from respondents across the wide range of devices and use patterns (e.g., number of puffs, disposables, cartridges, or refills; size of cartridge/tank; volume of e-liquid used) (Cooper, Harrell, & Perry, 2016; Hinds et al., 2016; Kim et al., 2016). A shortcoming of questions assessing the quantity of disposables or cartridges is that they do not permit comparable responses from those using products that include a refillable e-liquid tank; similarly, questions assessing volume of e-liquid are both difficult for participants to answer and not applicable to disposable or cartridge ENDS users (Cooper et al., 2016).

Beyond general frequency estimations of the number of days used, it may not be possible to obtain reliable and detailed frequency or quantity data from users via standard, single-wave or infrequent longitudinal surveys. Diary and other methods such as ecological momentary assessment (EMA) or electronic measurements recorded by the device may comprise more valid and reliable approaches, although these procedures may be more burdensome, or alter the user’s behavior. In sum, besides measuring number of days used in the past 30 days, there currently exists little consensus on frequency/quantity measures.

5. Device and e-Liquid Characteristics

5.1. Device Type

The recently issued FDA Deeming Rule has explicitly acknowledged the relevance of ENDS for health and safety considerations by including device components and parts in the final rule and requiring a separate premarket application for each device (Food and Drug Administration, 2016). Assessment of the device type used is also critical to the study of use patterns, with some literature suggesting that some device types may be more or less conducive to smoking cessation or reduction due to nicotine delivery efficiency (Vansickel & Eissenberg, 2013). As the ENDS market continues to evolve, it is becoming increasingly difficult to characterize the wide range of products available. Several characteristics of ENDS products have been frequently used to classify the type of device, including whether the device is an open or closed system, is rechargeable, or refillable vs. used with prefilled cartridges, though terminology for each type and the extent of further classification varies across studies. Studies (e.g., PATH) have increasingly used photographs in conjunction with terms in order to clarify the type of device used, though this practice remains uncommon (Hinds et al., 2016). Some studies have queried users on specific brand names that might be later coded for device characteristics of interest, but the vast number of ENDS product brands and rapid evolution of the market makes collection of these data difficult.

Measures of device elements (i.e., disposable, rechargeable, modifiable, refillable) will generally be imprecise proxy measures for the technological features and capabilities of the products that directly influence nicotine delivery, comparability to combusted cigarettes, convenience of use, exposure to toxic compounds, and satisfaction with the device. However, these measures have shown predictive validity in terms of patterns and frequency of use, user satisfaction, and cessation outcomes (Hitchman, Brose, Brown, Robson, & McNeill, 2015; Yingst et al., 2015). It may be helpful to provide illustrative photographs for the different device types and components being assessed (Hinds et al., 2016).

5.2. Flavors

The availability of a wide array of flavors in ENDS plays an important role in the maintenance of tobacco use and in the appeal to new users, particularly youth, who are exploring this “attractive” feature in ENDS (Hoffman, Salgado, Dresler, Faller, & Bartlett, 2016; Pepper, Ribisl, & Brewer, 2016). There is concern about the potential for exposure to flavoring chemicals that are known respiratory toxins (Barrington-Trimis, Samet, & McConnell, 2014; Behar et al., 2016; Farsalinos et al., 2013; Hutzler et al., 2014; Kosmider et al., 2016; Leigh, Lawton, Hershberger, & Goniewicz, 2016; Tierney, Karpinski, Brown, Luo, & Pankow, 2015). The challenge is in the assessment of a wide array of flavor categories and heterogeneous user behaviors, including the use of various flavors simultaneously.

Surveys generally use broad categories, which may include fruit and candy/dessert (most common), mint/menthol, tobacco, alcohol, coffee, spice, or other (including unflavored). These flavor categories may capture some relevant dimensions, particularly those related to product appeal. However, more research is needed to ascertain how reliably the 7,000+ flavors currently available in the market can be categorized, particularly when flavors may not neatly fit into a predefined category (Zhu et al., 2014). Depending on the research question(s), it may be important to assess the flavor of the respondents’ first ENDS product, currently and regularly used flavors, and whether and how multiple flavors are used by the same user. The availability and range of flavored ENDS may change as a result of certain provisions within FDA’s final Deeming Rule that will require registration and product listing for each distinct e-liquid product (Food and Drug Administration, 2016), which could have implications for measurement.

5.3. Nicotine Content

Assessment of nicotine content is also important to the characterization of ENDS use. The FDA does not consider e-liquids absent of tobacco or tobacco-derived nicotine to be a covered tobacco product under its Deeming Regulation (Food and Drug Administration, 2016). However, its measurement is complicated by the diversity of available products and the lack of current regulation of e-liquids. Prior research has found that many youth, often classified as experimental users, were unaware whether the e-liquid they used contained nicotine (Hinds et al., 2016). Several studies examining nicotine concentrations have noted inaccurate description of some “nicotine-free” e-liquids (Hutzler et al., 2014), and that the labeled nicotine concentration is often incorrect by 10% or more (Davis, Dang, Kim, & Talbot, 2015; Goniewicz, Kuma, Gawron, Knysak, & Kosmider, 2013; Lisko, Tran, Stanfill, Blount, & Watson, 2015).

Few national and TCORS studies assessed nicotine concentration, but those that did asked for the level of nicotine concentration in mg/mL. However, this measurement can further be affected by puffing behaviors and type of device used, thus rendering assessment of nicotine delivery (a more relevant measure than nicotine concentration) challenging. Compared to first generation devices – such as disposable ENDS – advanced open-tank delivery systems with higher voltage and temperature capabilities are far more efficient and thus require far lower nicotine concentrations to achieve the same level of nicotine delivery (Farsalinos et al., 2014). The level of nicotine may be relevant to studies of the abuse liability and addiction potential of ENDS, which has not yet been comprehensively evaluated.

Despite these considerations and limitations in self-report measures of nicotine content, they may provide useful approximations of relative nicotine exposure (particularly if the device type is known) and relevant information for perceptions and intentions of use. At a minimum, we recommend that researchers assess whether the respondent usually uses ENDS containing nicotine or without nicotine, with a “don’t know” option.

6. Conclusions and General Considerations

Our review of prominent national studies and the national, regional, and local studies conducted by the TCORS reveals considerable variability in ENDS use measures. This may partially reflect differing study populations and aims, different periods of data collection, or changing ENDS technologies, but we suspect that much of this variability is due to the absence of consensus on optimal, validated measures. There is a need for consensus measures and efforts (e.g., PhenX-https://www.phenxtoolkit.org/; DHHS harmonization across surveys) to facilitate comparisons across studies and over time, and to generate valid data to guide policy and regulatory actions. With the FDA now regulating ENDS, continued monitoring of the potential harms and benefits of ENDS use (i.e., the likelihood that ENDS will attract non-users and prevent/delay/accelerate other tobacco use) is crucial, requiring fine-tuned measurement of patterns of use and the types of devices and e-liquids used (Cobb et al., 2015; Villanti et al., 2011). In the absence of consensus measures, there is a need to consider the potential sensitivity of research findings to the choice of measures, and the strengths and limitations of the measures for a particular population and the research questions of interest. Further, measures will require continual re-evaluation in light of the evolving and uncertain future landscape of the ENDS market. For instance, changes in terminology and user behaviors, fast-paced technology development and evolution, and regulatory action have implications for ENDS use measures. Designing measures to withstand the test of time by paying close attention to market trends and industry developments, as well as employing multiple variants of a measure may be helpful, particularly for longitudinal studies.

Until more data on the sensitivity of research findings across different measures of ENDS use are available, caution is warranted in interpreting differences in prevalence rates and statistical associations across studies that employ different measures. This caution also extends to longitudinal comparisons within a study where measures have changed. Those conducting longitudinal research are in a difficult situation when deciding whether to modify measures as the product and market landscapes change. Either decision may complicate interpretation of longitudinal changes; however, we would generally recommend modification of measures, such as to fully capture an expanding and diversifying ENDS product category, if doing so seems likely to yield more useful, valid data, even if comparisons with earlier measures are compromised.

Given the present state of ENDS use measurement, reliance on a single measure for key constructs is not advisable. Diversity in measurement can be a useful tool to guard against measurement artifacts and drawing conclusions that are sensitive to a particular measure. Robustness of findings, or lack thereof, across different measures may not only inform the development of better measures, but also facilitate a better understanding of ENDS use phenomena. Opportunities may already exist to exploit variability in extant measurement with existing data to study the sensitivity of prevalence and other use estimates to choice of measure, assuming we can overcome challenges stemming from the many ways in which studies and measures differ, including designs and methodologies employed; data accessing and sharing issues; and the ongoing, evolving nature of ENDS products and how they are perceived and used by consumers. We have identified several areas where more measurement development and validation research is needed. There is an urgent need for research utilizing various designs (qualitative, natural experiments), approaches (EMA studies, lab-based studies), and technologies (e.g., objective monitoring devices).

HIGHLIGHTS.

  • Few standards and data on the psychometric properties of ENDS use measures exist.

  • This paper reviews and discusses measurement of ENDS use measures.

  • There is large variability and relatively little consensus in ENDS use measures.

  • This paper provides some considerations for measuring the use of ENDS.

  • Areas in need of measurement development and validation are identified.

Acknowledgments

The authors thank Blair Coleman, Anne Hartman, and other members of the TCORS Measurement Workgroup for thoughtful comments during working group calls, helpful resources and details for HHS national surveys and studies, and their critical review of the manuscript.

Role of Funding Sources

Research reported in this publication was supported by grant numbers P50DA036128, P50CA180523, P50CA180905, P50CA179546, P50CA180906, P50CA180907, 1U54CA189222, and 1P50HL120163 from the National Institutes of Health (NIH) and the FDA Center for Tobacco Products (CTP). NIH and FDA had no official role in the study design, collection, analysis or interpretation of the data, writing the manuscript, or the decision to submit the paper for publication with the exception of the review of the manuscript by the individuals named in the acknowledgements. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributors

SW wrote the first draft of the abstract, introduction, and conclusions sections and performed editing of all sections. HK, AG, and SW co-wrote the introduction section. ES and AL co-wrote the Product Description section. HK and MS co-wrote the Measuring Frequency and Quantity of Use sections. TP and SW co-wrote the Definitions of Current Use section. JBT and AG co-wrote the Device and e-Liquid Characteristics section. All authors contributed to the planning and writing of this manuscript and have approved the final manuscript.

Conflict of Interest

The authors report no actual or potential conflict of interest.

1

We acknowledge that there is neither consensus nor consistency in terminology for referring to these devices in the scientifi c literature. Multiple terms are used and each is arguably imperfect. In this paper, we use “electronic nicotine and non-nicotine delivery systems” as a variation of a commonly used term in acknowledgement that many youth and some adult users report using the devices without nicotine (Miech, Patrick, O’Malley, & Johnston, 2017; Weaver, Kemp, Heath, Pechacek, & Eriksen, 2017). For brevity, we abbreviate as “ENDS,” hereinafter.

References

  1. Alexander JP, Coleman BN, Johnson SE, Tessman GK, Tworek C, Dickinson DM. Smoke and Vapor: Exploring the Terminology Landscape among Electronic Cigarette Users. Tobacco Regulatory Science. 2016;2(3):204–213. doi: 10.18001/TRS.2.3.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Amato MS, Boyle RG, Levy D. How to define e-cigarette prevalence? Finding clues in the use frequency distribution. Tobacco Control. 2015 doi: 10.1136/tobaccocontrol-2015-052236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Amato MS, Boyle RG, Levy D. E-cigarette use 1 year later in a population-based prospective cohort. Tobacco Control. 2016 doi: 10.1136/tobaccocontrol-2016-053177. [DOI] [PubMed] [Google Scholar]
  4. Ayers JW, Althouse BM, Allem JP, Leas EC, Dredze M, Williams RS. Revisiting the rise of electronic nicotine delivery systems using search query surveillance. American Journal of Preventive Medicine. 2016;50(6):E173–E181. doi: 10.1016/j.amepre.2015.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Barrington-Trimis JL, Samet JM, McConnell R. Flavorings in electronic cigarettes: An unrecognized respiratory health hazard? JAMA. 2014;312(23):2493–2494. doi: 10.1001/jama.2014.14830. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Behar RZ, Luo W, Lin SC, Wang Y, Valle J, Pankow JF, Talbot P. Distribution, quantification and toxicity of cinnamaldehyde in electronic cigarette refill fluids and aerosols. Tobacco Control. 2016 doi: 10.1136/tobaccocontrol-2016-053224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Brose LS, Hitchman SC, Brown J, West R, McNeill A. Is the use of electronic cigarettes while smoking associated with smoking cessation attempts, cessation and reduced cigarette consumption? A survey with a 1-year follow-up. Addiction. 2015;110(7):1160–1168. doi: 10.1111/add.12917. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cobb CO, Villanti AC, Graham AL, Pearson JL, Glasser AM, Rath JM, Niaura R. Markov modeling to estimate the population impact of emerging tobacco products: A proof-of-concept study. Tobacco Regulatory Science. 2015;1(2):129–141. doi: 10.18001/TRS.1.2.3. [DOI] [Google Scholar]
  9. Cooper M, Harrell MB, Perry CL. A qualitative approach to understanding real-world electronic cigarette use: Implications for measurement and regulation. Preventing Chronic Disease. 2016;13:E07. doi: 10.5888/pcd13.150502. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Davis B, Dang M, Kim J, Talbot P. Nicotine concentrations in electronic cigarette refill and do-it-yourself fluids. Nicotine & Tobacco Research. 2015;17(2):134–141. doi: 10.1093/ntr/ntu080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Farsalinos KE, Romagna G, Tsiapras D, Kyrzopoulos S, Spyrou A, Voudris V. Impact of flavour variability on electronic cigarette use experience: an internet survey. International journal of environmental research and public health. 2013;10(12):7272–7282. doi: 10.3390/ijerph10127272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Farsalinos KE, Spyrou A, Tsimopoulou K, Stefopoulos C, Romagna G, Voudris V. Nicotine absorption from electronic cigarette use: Comparison between first and new-generation devices. Scientific Reports. 2014;4:4133. doi: 10.1038/srep04133. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Food and Drug Administration. Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act; Restrictions on the Sale and Distribution of Tobacco Products and Required Warning Statements for Tobacco Products; Final Rule (21 CFR Parts 1100, 1140, and 1143) Fed Reg. 2016;81(90):28974–29106. [PubMed] [Google Scholar]
  14. Goniewicz ML, Kuma T, Gawron M, Knysak J, Kosmider L. Nicotine levels in electronic cigarettes. Nicotine & Tobacco Research. 2013;15(1):158–166. doi: 10.1093/ntr/nts103. [DOI] [PubMed] [Google Scholar]
  15. Hinds JT, Loukas A, Chow S, Pasch KE, Harrell MB, Perry CL, Wackowski OA. Using cognitive interviewing to better assess young adult e-cigarette use. Nicotine & Tobacco Research. 2016;18(10):1998–2005. doi: 10.1093/ntr/ntw096. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hitchman SC, Brose LS, Brown J, Robson D, McNeill A. Associations between e-cigarette type, frequency of use, and quitting smoking: Findings from a longitudinal online panel survey in Great Britain. Nicotine & Tobacco Research. 2015;17(10):1–8. doi: 10.1093/ntr/ntv078. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hoffman AC, Salgado RV, Dresler C, Faller RW, Bartlett C. Flavour preferences in youth versus adults: A review. Tobacco Control. 2016;25(Suppl 2):ii32–ii39. doi: 10.1136/tobaccocontrol-2016-053192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hutzler C, Paschke M, Kruschinski S, Henkler F, Hahn J, Luch A. Chemical hazards present in liquids and vapors of electronic cigarettes. Archives of Toxicology. 2014;88(7):1295–1308. doi: 10.1007/s00204-014-1294-7. [DOI] [PubMed] [Google Scholar]
  19. Kim H, Davis AH, Dohack JL, Clark PI. E-cigarettes use behavior and experience of adults: Qualitative research findings to inform e-cigarette use measure development. Nicotine & Tobacco Research. 2016;19(2):ntw175. doi: 10.1093/ntr/ntw175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kosmider L, Sobczak A, Prokopowicz A, Kurek J, Zaciera M, Knysak J, Goniewicz ML. Cherry-flavoured electronic cigarettes expose users to the inhalation irritant, benzaldehyde. Thorax. 2016;71(4):376–377. doi: 10.1136/thoraxjnl-2015-207895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kozlowski LT, Giovino GA. Softening of monthly cigarette use in youth and the need to harden measures in surveillance. Preventive Medicine Reports. 2014;1:53–55. doi: 10.1016/j.pmedr.2014.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Leigh NJ, Lawton RI, Hershberger PA, Goniewicz ML. Flavourings significantly affect inhalation toxicity of aerosol generated from electronic nicotine delivery systems (ENDS) Tobacco Control. 2016;25(Suppl 2):ii81–ii87. doi: 10.1136/tobaccocontrol-2016-053205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lisko JG, Tran H, Stanfill SB, Blount BC, Watson CH. Chemical composition and evaluation of nicotine, tobacco alkaloids, pH, and selected flavors in e-cigarette cartridges and refill solutions. Nicotine & Tobacco Research. 2015;17(10):1270–1278. doi: 10.1093/ntr/ntu279. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Miech R, Patrick ME, O’Malley PM, Johnston LD. What are kids vaping? Results from a national survey of US adolescents. Tobacco Control. 2017;112(1):1–7. doi: 10.1136/tobaccocontrol-2016-053014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Pearson JL, Amato MS, Wang X, Zhao K, Cha S, Cohn AM, Graham AL. How U.S. smokers refer to e-cigarettes: An examination of user-generated posts from a web-based smoking cessation intervention, 2008-2015. Nicotine & Tobacco Research. 2016;19(2):253–257. doi: 10.1093/ntr/ntw206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Pepper JK, Ribisl KM, Brewer NT. Adolescents’ interest in trying flavoured e-cigarettes. Tobacco Control. 2016;25(Suppl 2):ii62–ii66. doi: 10.1136/tobaccocontrol-2016-053174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Tierney PA, Karpinski CD, Brown JE, Luo W, Pankow JF. Flavour chemicals in electronic cigarette fluids. Tobacco Control. 2015;25(e1):e10–15. doi: 10.1136/tobaccocontrol-2014-052175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. U.S. Department of Health and Human Services. E-Cigarette Use Among Youth and Young Adults A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2016. [Google Scholar]
  29. Vansickel AR, Eissenberg T. Electronic cigarettes: Effective nicotine delivery after acute administration. Nicotine & Tobacco Research. 2013;15(1):267–270. doi: 10.1093/ntr/ntr316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Villanti AC, Vargyas EJ, Niaura RS, Beck SE, Pearson JL, Abrams DB. Food and Drug Administration regulation of tobacco: Integrating science, law, policy, and advocacy. American Journal of Public Health. 2011;101(7):1160–1162. doi: 10.2105/AJPH.2011.300229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Walton KM, Abrams DB, Bailey WC, Clark D, Connolly GN, Djordjevic MV, Hatsukami DK. NIH electronic cigarette workshop: Developing a research agenda. Nicotine & Tobacco Research. 2015;17(2):259–269. doi: 10.1093/ntr/ntu214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Weaver SR, Kemp CB, Heath JW, Pechacek TF, Eriksen MP. Use of Nicotine in Electronic Nicotine Delivery Systems by US Adults, 2015. Public Health Reports. 2017;132:545–548. doi: 10.1177/0033354917723597. https://doi.org/10.1177/0033354917723597. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Weaver SR, Majeed BA, Pechacek TF, Nyman AL, Gregory KR, Eriksen MP. Use of electronic nicotine delivery systems and other tobacco products among USA adults, 2014: Results from a national survey. Int J Public Health. 2016;61(2):177–188. doi: 10.1007/s00038-015-0761-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Yingst JM, Veldheer S, Hrabovsky S, Nichols TT, Wilson SJ, Foulds J. Factors associated with electronic cigarette users’ device preferences and transition from first generation to advanced generation devices. Nicotine & Tobacco Research. 2015;17(10):1242–1246. doi: 10.1093/ntr/ntv052. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Zhu SH, Sun JY, Bonnevie E, Cummins SE, Gamst A, Yin L, Lee M. Four hundred and sixty brands of e-cigarettes and counting: Implications for product regulation. Tobacco Control. 2014;23(suppl 3):iii3–iii9. doi: 10.1136/tobaccocontrol-2014-051670. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES