Characterizing Different-Flavored E-Cigarette Solutions From User-Reported Sensory Attributes and Appeal

Abstract

Scientists and regulators need parsimonious methods of characterizing flavored e-cigarettes which may vary widely in both chemical flavoring constituents and marketing descriptors. This laboratory experiment characterized user-reported appeal and experience of five cross-cutting sensory attributes (sweetness, bitterness, smoothness, harshness, coolness) of 10 common e-cigarette flavors. In a within-subject double-blind single-visit protocol, current nicotine/tobacco product users (N = 119) self-administered a single puff of each e-liquid flavor via a pod-style device and rated its appeal and sensory attributes on 0–100 scales. Custom-manufactured e-liquids, nicotine concentration: M (SD) = 23.4 (0.9) mg/mL, representative of commonly marketed fruit (green apple, strawberry), dessert (dark chocolate, vanilla), mint (peppermint, spearmint), nonmint cooling (menthol, koolada), and tobacco (subtle tobacco, full-flavored tobacco) flavor descriptors were used and their constituents were independently analyzed. Results largely demonstrated that a flavor’s sensory attributes concorded with its marketed flavor descriptor. Among the 10 flavors, vanilla was rated sweetest (B[difference vs. mean of 9 other flavors] = 14.44, 95% CI [10.84, 18.03]), full-flavored tobacco was most bitter, B = 8.34, 95% CI [4.73, 11.96], subtle tobacco was most harsh, B = 5.69, 95% CI [1.70, 9.68], and peppermint scored highest in both smoothness, B = 6.98, 95% CI [3.13, 10.82], and coolness, B = 29.25, 95% CI [25.50, 33.01]. Flavors with higher appeal ratings tended to be sweeter, smoother, cooler, and less bitter and harsh. Chemical analysis found numerous flavoring constituents among study products without any clear differentiation of chemicals being present in particular flavor categories, which underscores the utility of using sensory ratings to characterize different-flavored e-cigarettes over and above constituent analyses. Characterizing e-cigarette flavors by subjective sensory attributes may be useful in future research and regulatory activities.

In the United States in 2018, 75% of all U.S. adult e-cigarette users reported exclusively using non-tobacco-flavored e-cigarette solutions in various flavor categories including fruit, candy, sweets, chocolate, clove, spice, herb, alcohol, menthol, and mint (Leventhal & Dai, 2021). In addition to external influences including marketing, social influences, and cultural trends, the subjective experience of inhaling aerosol from non-tobacco-flavored e-cigarette solutions is an important factor that drives e-cigarette use (Barnes et al., 2017; Leventhal, Goldenson, et al., 2019; Leventhal, Mason, et al., 2020; St Helen et al., 2018). Controlled administration of different-flavored e-cigarette products under double-blind conditions can be used to isolate the effects of flavor on the key markers of the subjective user experience of vaping including the ratings of appeal and sensory qualities of e-cigarette solutions. Prior studies have demonstrated that e-cigarette solutions in nontobacco flavors are generally perceived as more appealing (Leventhal, Goldenson, et al., 2019), more rewarding (Audrain-McGovern et al., 2016), and less aversive (DeVito et al., 2020; Krishnan-Sarin et al., 2017; Leventhal, Mason, et al., 2020) than tobacco-flavored e-cigarettes.

There exists a great deal of product diversity in e-cigarette flavors with over 15,000 flavors commercially available representing fruit, dessert, mint, nonmint cooling, tobacco, and other characterizing flavor categories (Krüsemann et al., 2019). To date, most classification and characterization of e-cigarette flavors are based firstly upon the e-cigarette solution’s characterizing name or descriptor, and to a lesser extent on the presence of certain chemical flavoring constituents (e.g., menthol; Tierney et al., 2016). However, e-cigarette products can be marketed with names or descriptors that do not clearly denote their flavor (e.g., “unicorn puke” flavor; Jackler & Ramamurthi, 2017). Additionally, two distinct characterizing flavors can share many common constituents (e.g., vanillin constituents are commonly used in both fruit and tobacco flavors; Krüsemann et al., 2021). Furthermore, for many of the thousands of flavoring constituents that are found in e-cigarettes, there is little scientific data on the impact of these constituents on the overall user experience and appeal of e-cigarettes. There is also the possibility that different concentration levels of a particular constituent have linear or nonlinear effects on the appeal of e-cigarette aerosol. A parallel can be drawn to the psychopharmacology and abuse liability assessment of nonnicotine drugs, which found that some compounds have dose-dependent psychoactive effects (e.g., some drugs can act as sedatives at higher doses but a stimulant at lower doses). There is already some evidence indicating that increasing concentration levels of some flavorants in e-cigarette solutions might have linear or nonlinear effects on the product’s sensory attributes and appeal (Pullicin et al., 2020; Rosbrook & Green, 2016).

Given the abovementioned complexities, knowing the presence (or absence) of a particular chemical constituent in an e-cigarette solution may not be sufficient to make predictions about a flavor’s overall sensory profile or its appeal. This is problematic because regulators must make decisions whether new flavored e-cigarette products can be marketed to the public. Merely knowing the flavoring ingredients and marketing descriptor does not provide sufficient information for regulators wishing to characterize the possible appeal of novel flavored e-cigarette products. Regulators need evidence-based methods capable of systematically and parsimoniously characterizing various types of flavored e-cigarettes that can be considered alongside a product’s flavoring constituents and marketing descriptor. Methods for characterizing flavored e-cigarette products should ideally provide insight into the likely appeal of the product and consumer’s direct experience using the product.

The subjective sensory qualities of inhaling the aerosol emitted from flavored e-cigarette products may provide information that can be used to qualify different types of flavors irrespective of its constituents or characterizing flavor name (Davis et al., 2021). Previous evidence indicates that e-cigarette solutions marketed as menthol-flavored products increase ratings of coolness (Jackson et al., 2021), fruit-flavored products increase ratings of sweetness (Jackson et al., 2021; Leventhal, Cho, et al., 2020), and tobacco-flavored products increase ratings of bitterness or harshness (Leventhal, Cho, et al., 2020). Moreover, previous research has shown that sensory attribute ratings of an e-cigarette product, such as bitterness and smoothness, are associated with measures of user-reported appeal, such as willingness to use the product again (Leventhal, Cho, et al., 2020).

While existing studies support the premise that sensory attributes are germane to characterizing different-flavored products, to comprehensively determine the utility of this method, it is important to apply it to a wide variety of different flavors. To date, most studies on the sensory attributes of flavored e-cigarettes have examined a small number of flavors, investigated flavors within a single flavor category, or focused on one or two types of sensory ratings (e.g., only coolness). There are at least five key overarching flavor categories (fruit, mint, nonmint cooling such as menthol, dessert, and tobacco). Within each of these flavor categories, there are multiple different flavor profiles that may have distinct impacts (e.g., vanilla and chocolate may generate distinct user experiences despite both being dessert flavors).

The present study is a secondary analysis of a laboratory experiment that tested differences in the appeal and sensory attributes of e-cigarette solutions in nicotine-salt versus free-base formulations (Leventhal et al., 2021). The main effects of flavor were not reported in the primary outcomes article, but nonetheless provide a unique opportunity to apply the sensory attribute characterization methodology because the experiment included 10 different-flavored solutions. In this article, we describe the relative appeal and sensory attribute ratings of each of the 10 different-flavored e-cigarette solutions used in the study. We also examine the constituents of the 10 flavored solutions to examine how they compare to the flavor’s characterizing names and their appeal and sensory qualities.

Method

Transparency and Openness

We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study. All data, analysis code, and research materials are available by emailing the corresponding author. Data were analyzed using Stata Version 16 (StataCorp, 2019). This study was registered under ClinicalTrials.gov Identifier: NCT04399031 under the title “Effects of e-Cigarettes on Perceptions and Behavior.”

Participants

Adults aged 21+ from the Los Angeles region who reported past 30-day tobacco product use in the form of e-cigarettes and/or combustible cigarettes were recruited from November 2019 to March 2020 (N = 119). Participant enrollment was discontinued prematurely at the study institution due to the discontinuation of all nontreatment clinical research because of the coronavirus disease (COVID-19) pandemic. Inclusion criteria for e-cigarette users were: (a) e-cigarette use ≥3 days/week over the past 30 days, (b) lifetime duration of vaping for ≥2 months, and (c) nicotine-containing e-cigarette use. Inclusion criteria for combustible cigarette users were as follows: (a) cigarette use ≥3 days/week for ≥2 years and (b) interest in trying e-cigarettes if not also currently using e-cigarettes (current smokers that currently used e-cigarettes were also eligible). Exclusion criteria for both groups were as follows: (a) impending plan to quit nicotine/tobacco product use, (b) currently or planning to become pregnant or breastfeeding, (c) current daily use of combustible tobacco products other than combustible cigarettes or e-cigarettes, and (d) positive breath alcohol at the study visit. Subjects provided written informed consent. The University of Southern California Institutional Review Board approved the protocol (protocol is available by emailing the corresponding author).

Design and Materials

A within-participant randomized order double-blind factorial design was used. The procedure included e-cigarette solutions custom manufactured to be representative of commonly marketed flavor categories, including two fruit (green apple, strawberry), two dessert (dark chocolate, vanilla), two mint (peppermint, spearmint), two nonmint cooling (menthol, koolada), and two tobacco (subtle tobacco, full-flavored tobacco) flavored solutions (solutions from Avail Vapors; Richmond, VA). Koolada is a term used for flavors with cooling effects that may contain synthetic coolants. Each of the 10 flavors was administered twice: once in a nicotine-salt formulation (benzoic acid added) and once in a free-base formulation (no benzoic acid) solution. Thus, there were 20 total solutions, each of which contained roughly equivalent nicotine concentrations, M (SD) nicotine concentration: 23.4 (0.9) mg/mL, range: 21.4–27.9, in a 50/50 propylene glycol/vegetable glycerin vehicle. Solutions were administered via a Suorin iShare Pod System Device (7 W; 2.0 Ω resistance; 130 mAh built-in battery), which was selected for its similarities to JUUL and other best-selling e-cigarette devices and capability to be refilled with custom flavored solutions.

Procedure

After a phone screen to determine study eligibility, participants were invited to an in-person lab visit. Participants were asked to abstain from using any nicotine or tobacco products for at least 2 hr prior to arrival in order to prevent nicotine saturation during the procedure. At the beginning of the visit, participants provided informed consent and were instructed that they would administer products with varying nicotine levels and flavorings. After consent, participants provided carbon monoxide (BreathCO™, Vitalograph®, Lenexa, KS) and alcohol breath samples (BACtrack S80, BACtrack, San Francisco, CA) as well as saliva and urine samples for use with NicAlert™ test strips (LiveWellTesting.com, San Diego, CA) which provide a semiquantitative index of salivary and urine cotinine. Female participants provided urine samples for pregnancy tests.

The standardized e-cigarette appeal and sensory rating procedure used in the study has been previously validated and involved one practice trial using flavorless 0% nicotine solution followed by 20 experimental trials (10 flavors presented in two nicotine formulations [free-base and salt]; Leventhal, Goldenson, et al., 2019; Leventhal et al., 2021). Each participant received a randomized ordering of the 20 experimental trials (i.e., e-liquids) which was generated using a random sequence generator. Prior to the study visit, e-cigarette solutions were preprepared in their randomized order using sequentially numbered containers. Participants and data collection staff were blind to the solutions administered at each trial. In a ventilated testing room, participants self-administered each solution following a tutorial video with instructions for a guided puffing procedure, which involved a 1-puff cycle (10-s preparation, 4-s inhalation, 1-s hold, and 2-s exhale interval) followed by appeal and sensory attribute ratings. After each product administration, participants were provided with water and instructed to rinse their mouths to minimize intertrial sensory carryover. The 20 experimental trials were separated into four 5-trial blocks with 10-min breaks separating each block. In between blocks, questionnaires measuring participant characteristics were completed and, for safety monitoring, blood pressure and heart rate were taken.

Measures

Flavoring Constituent Analysis

Screenings of flavoring compounds in the 20 e-cigarette solutions were independently assayed by the Roswell Park Comprehensive Cancer Center Nicotine and Tobacco Product Assessment Core. As in prior work (Leventhal et al., 2021), solutions were blinded and analyzed via gas chromatography and mass spectrometry. From these analyses, we identified cases in which a flavor screened positive for one of the 25 most commonly used flavoring constituents reported in a previous comprehensive assay of 16,839 e-cigarette solutions (Krüsemann et al., 2021). Descriptors of flavoring constituents were reported and based on data from The Good Scents Company Information System (The Good Scents Company, 2021).

Appeal and Sensory Attribute Outcomes

Participants provided appeal ratings following each 1-puff trial on visual analogue scales (VAS; 0–100 range): (a) “How much did you like the e-cigarette?” (b) “How much did you dislike the e-cigarette?” (c) “Would you use this e-cigarette again?” Rating anchors were “Not at all” and “Extremely” for each measure except for (c) “use again” which had anchors of “Not at all” and “Definitely.” For each trial, a composite appeal mean score based on the average of “Liking,” “Disliking” (reverse-scored),” and “Willingness to use again” ratings was calculated (Cronbach’s α = .93) as in prior research (Leventhal et al., 2021). Participants also rated five sensory attributes on each trial (VAS; 0–100 range): (a) “How sweet was the e-cigarette?” (b) “How bitter was the e-cigarette?” (c) “How smooth was the e-cigarette?” (d) “How harsh was the e-cigarette?” (e) “How cool or cooling was the e-cigarette?” with rating anchors of “Not at all” and “Extremely” for each measure. Each sensory rating was analyzed as separate outcome.

Smoking Status and Other Participant Characteristics

In between trial blocks where e-cigarette administration occurred, participants completed demographic and tobacco product use history questionnaires to characterize the sample. Ever- versus never-smoking status was defined as lifetime smoking ≥100 versus ≤99 cigarettes. Participants who were current e-cigarette users completed the Penn State Electronic Cigarette Dependence Index (PSECD), an e-cigarette dependence measure (range: 0–20; Foulds et al., 2015). Participants who were current combustible cigarette users completed the Fagerström Test for Cigarette Dependence (FTCD), a cigarette dependence questionnaire (Heatherton et al., 1991). Biomarkers of exhaled carbon monoxide and urine and salivary cotinine were collected to access exposure to nicotine and combusted tobacco, respectively.

Data Analysis

All analyses were conducted in Stata Version 16 (StataCorp, 2019). Descriptive statistics were conducted on sample characteristics. Due to the nesting of data in participants (each participant completed 20 total observations consisting of 10 flavors administered in each nicotine formulation condition), multilevel models (MLMs) were conducted to allow for modeling of sensory outcomes from each trial including participant random effects. Because the effects of flavor on any outcomes were not significantly moderated by salt versus free-base nicotine formulation status (with the sole exception of harshness which was driven solely by the dark chocolate flavor indicating effects largely generalized across flavors, see Table S1), results are presented aggregated across free-base and salt solutions of each respective flavor. We first tested descriptive MLMs including a 10-level categorical flavor variable to identify whether there were omnibus differences across flavors on each of the outcomes using Type-III fixed effects. Next, primary MLMs tested the fixed effects of each flavor binary variable (e.g., strawberry vs. all nonstrawberry flavors) on outcomes, with separate models tested for each flavor binary variable and outcome combination (appeal, sweetness, bitterness, harshness, smoothness, and coolness). Results were presented as unstandardized effect estimates with standard errors (B ± SE) representing the difference in means of each flavor relative to the average of the other nine flavors (range: −100 to 100). Significance was based on p values that were two-tailed and corrected for multiple tests using the Benjamini–Hochberg procedure to maintain the study-wise false-discovery rate of .05 (Benjamini & Hochberg, 1995). As smoothness ratings were introduced after the first eight participants, five participants of the 119 included in the analytic sample were missing ratings for between 4 to 16 trials but were included in analyses since equal observations across participants are not a requirement of MLMs. In addition, smoothness was not added as an outcome of interest to the study protocol initially, thus resulting in a sample of 111 participants.

Results

Participant Characteristics

The analytic sample (N = 119) was 32.8% female, had a M (SD) age of 42.1 (14.4) years, and was racially/ethnically diverse (Table 1; see study flow chart in Figure 1). All participants were current e-cigarette and/or combustible cigarette users, with 30.2% (N = 36) current exclusive e-cigarette users, 25.2% (N = 30) current dual users of e-cigarettes and cigarettes, and 44.5% (N = 53) current exclusive cigarette users. Regardless of current user status, 88.2% (N = 105) participants were ever cigarette smokers who had smoked >100 combustible cigarettes in their lifetime. In the overall sample collapsing across e-cigarette and cigarette use status, the mean semiquantitative cotinine level, 4.0 (SD = 1.6), fell into the 200–500 ng/mL range and the average breath CO was 6.4 (SD = 5.9).

Table 1.

Descriptive Statistics of Participant Characteristics

Variable	M (SD) or N (%)
Demographics
Female, N (%)	39 (32.8)
Age, M (SD), years	42.1 (14.4)
Race, N (%)
White	35 (30.2)
Black	46 (39.7)
Asian	9 (7.8)
Multiracial	12 (10.3)
Othera	14 (12.1)
Hispanic ethnicity, N (%)	23 (19.8)
Tobacco product use characteristics
Ever combustible cigarette smokers, N (%)b	105 (88.2)
Current e-cigarette vaping status, N (%)c	66 (55.5)
Current other tobacco product use, N (%)d	27 (25.0)
Biomarkers, M (SD)
Carbon monoxide, ppm	6.4 (5.9)
Cotinine semiquantitative levele	4.0 (1.6)
Combustible cigarettes
Age started smoking regularly, M (SD), yearsf	18.7 (6.9)
Current cigarettes/day, M (SD)g	11.0 (6.8)
Cigarettes/day when smoking heaviest, M (SD)f	17.8 (10.9)
Days smoked in past 30g	23.4 (10.7)
FTCD, M (SD)g	4.4 (2.2)
Usually smoke(d) menthol cigarettes, N (%)f	47 (53.4)
E-cigarettes
PSECD, M (SD)h	10.1 (4.9)
Puffs per day, M (SD)h	85.5 (90.4)
Nicotine concentration typically used, M (SD), mg/mL,h	29.3 (20.6)
Duration of e-cigarette use, M (SD), monthsi	21 (16.2)
Days vaped in past 30h	23.4 (8.6)
E-cigarette device type typically used, N (%)i
Cig-a-like	2 (3.3)
Tank/pen	5 (8.2)
Advanced personal vaporizer/mod	11 (18.0)
Pod-based	34 (55.7)
Other	9 (14.8)
Preferred e-cigarette flavor, N (%)i
Fruit	28 (45.9)
Dessert	7(11.5)
Mint	6 (9.8)
Menthol	9 (14.8)
Tobacco	9 (14.8)
Other	2 (3.3)

Note. FTCD = Fagerstrom test for cigarette dependence; PSECD = Penn state electronic cigarette dependence index; ppm = parts per million. Overall N = 119. Sample size ranges from 110 to 119 across variables due to differential patterns of missing data across variables.

^aIncludes “American Indian or Alaskan Native,” “Middle Eastern,” “Pacific Islander (including Hawaii),” and “Other.”

^bSmoked ≥100 cigarettes lifetime and in the past 30 days (n = 22 were former smokers who did not smoke at all in past month, n = 83 were current [past-month] smokers).

^cVapes ≥3 days per week for past ≥2 months.

^dPast 30-day use of “chewing tobacco, snuff or dip,” “dissolvable tobacco product,” “bidis,” “kreteks,” “regular pipe tobacco,” “snus,” “big cigars,” “little cigars or cigarillos,” or “hookah water pipe.”

^eNicAlert Strip (range: 1–6; 0 = 0–10, 1 = 10–30, 2 = 30–100, 3 = 100–200, 4 = 200–500, 5 = 500–1,000, 6 ≥ 1,000 ng/mL).

^fFormer or current smokers only (n = 105). Ns range 83105 due to missing data across variables.

^gCurrent smokers only (n = 83). Ns range 80–83 due to missing data across variables.

^hCurrent vapers only (n = 66). Ns range 54–66 due to missing data across variables.

ⁱEver vapers only (n = 64). Ns range 54–64 due to missing data across variable.

Figure 1. Study Flowchart.

^a Smoothness measure introduced into the study after the first eight participants had already completed the protocol.

The subsample of current e-cigarette users exhibited moderate e-cigarette dependence levels, M (SD) PSECD score = 10.1 (4.9), and used e-cigarettes for a M (SD) duration of 21 (16.2) months. Similar to the devices used in the study protocol, the majority of current e-cigarette users (55.7%, N = 34) reported mainly using pod-based e-cigarettes. Mean (SD) nicotine concentration used by participants who were e-cigarette users was 29.3 (20.6) mg/mL. Most participants (45.9%, N = 28) typically preferred fruit e-cigarette flavors.

The subsample of current combustible cigarette users exhibited moderate cigarette dependence levels, M (SD) FTCD score = 4.4 (2.2). The M (SD) number of cigarettes smoked by participants who were cigarette users was 11.0 (6.8) per day and 53.8% (N = 47) of current smokers typically smoked menthol cigarettes.

Flavoring Constituent Analysis of the Study E-Cigarette Solutions

Of the 25 most common e-liquid flavorants identified in a comprehensive analysis of flavoring constituents (Krüsemann et al., 2021), all but one was found in the ≥1 of the study e-cigarette solutions (Table 2). The flavors with dessert or fruit characterizing names generally contained many of the common flavorants: The strawberry and vanilla study e-cigarette solutions contained 13 and 10 of the top 25 flavorants, respectively. Flavors with nonmint cooling and tobacco characterizing names, however, contained fewer common flavorants. Koolada, for example, did not contain any of the 25 most common flavorants. Some flavoring constituents were only found in study e-cigarette solutions with characterizing names belonging to a single flavor category: ethyl acetate, isoamyl acetate, ethyl 2-methyl butyrate, and hexyl acetate were found only in the study e-cigarette solutions with fruit characterizing names (i.e., green apple, strawberry) while maltol and butyric acid were found only in the study e-cigarette solutions with dessert characterizing names (i.e., vanilla, dark chocolate). However, other flavoring constituents were found in study e-cigarette solutions from different flavor categories (e.g., vanillin and methyl cyclopentenolone were found in study e-cigarette solutions with fruit, dessert, and tobacco characterizing names).

Table 2.

Commonly Used Flavoring Constituents That Were Present in Study E-Cigarette Solutions

Flavor	Sweet/Vanilla/Creamy/Chocolate	Sweet/Cotton Candy/Jammy/Strawberry	Fruity/Sweet/Apple	Fruity/Sweet/Grape	Sweet/Cotton Candy/Caramellic/Jammy	Sweet/Creamy/Vanilla/Caramellic	Sweet/Caramellic/Fruity	Sweet/Maple/Bready	Fruity/Creamy/Peach	Fresh/Green/Fruity	Sweet/Fruity/Banana	Fruity/Fresh/Berry	Pungent/Sour/Fruit	Acidic/Sour/Creamy	Citrus/Orange/Lemon/Floral/Woody	Chemical/Fruity/Cherry	Sweet/Pineapple/Fruity	Sweet/Almond/Cherry/Nutty	Cooling/Mentholic/Minty	Green/Fruity/Apple	Cheesy/Fruity/Fatty	Fruity/Sweet/Winey	Coconut/Creamy/Vanilla/Nutty	Coconut/Creamy/Milky	Fruity/Green/Fresh
Green Apple			B	B						B	B	B									B	B			B
Strawberry	S	B	B	B				B		B	B	B				B	B				S		B		B
Dark Chocolate	B				B	B		B						B				B
Vanilla	B	F	B		B	B		B	S					B						B		B
Menthol																			B
Koolada
Peppermint						B									B				B
Spearmint										B									B
Subtle Tobacco	B					B		B
Full Flavored Tobacco	B	B						B														B
NIST Name	Vanillin	Ethyl maltol	Ethyl Butyrate	Ethyl Acetate	Maltol	Ethyl Vanillin	Furaneol	Methyl cyclopentenolone	γ-Decalactone	Cis-3-hexenol	Isoamyl acetate	Ethyl 2-methyl butyrate	Acetic Acid	Butyric Acid	Linalool	Benzyl Alcohol	Ethyl hexanoate	Benzaldehyde	Menthol	Isoamyl isovalerate	Hexanoic Acid	Ethyl propionate	γ-Undecalactone	8-Decalactone	Hexyl acetate

Note. F = present in only free-base solution; S = present in only salt solution; B = present in both free-base and salt solutions.

We report the 25 most common e-cigaratte flavoring constitutents as identified in a comprehensive overview of e-liquid ingredients (Krüsemann et al., 2021). Flavor descriptors are based on data from The Good Scents Company Information System (The Good Scents Company, 2021).

Exploratory analyses examining the effect of constituent presence versus absence on the product’s respective appeal and sensory attributes yielded unstable effect estimates due to collinearity and were therefore not presented. For example, substantially discordant outcomes were found across univariable models (examining one constituent as the sole predictor) and multivariable models (examining multiple constituents in the same model as simultaneous predictors) including effect estimates switching directions and providing very wide confidence intervals in multivariable models.

Primary Results

Differences in Appeal Across the 10 Characterizing Flavors

The omnibus 10-level categorical variable of flavor had a significant effect on appeal (p < 0.001). The mean (SE) rating for each flavor is depicted in Table S2. Analyses of the relative difference in mean appeal rating for each flavor in comparison to the collapsed average across the nine other flavors are reported in Figure 2 and the corresponding p values of each contrast are depicted in Table S3. These analyses found that strawberry was most appealing, yielding ratings that were 7.07, 95% CI [3.44, 10.69], higher than the mean of the nine other flavors, followed by, in order of relative appeal, vanilla, B = 6.29, 95% CI [2.66, 9.93], and then peppermint, B = 5.57, 95% CI [1.93, 9.21], which were both significantly higher in appeal than the collapsed average across all the flavors. Then, menthol, B = 2.24, 95% CI [−1.93, 5.87], green apple, B = 1.71, 95% CI [−1.91, 5.33], spearmint, B = −1.07, 95% CI [−4.68, 2.55], and koolada, B =−2.03, 95% CI [−5.68, 1.63], were in the middle range of appeal and did not significantly differ from the collapsed average across all flavors. Finally, subtle tobacco, B =−4.59, 95% CI [−8.21, −0.97], full-flavored tobacco, B =−6.16, 95% CI [−9.80, −2.52], and dark chocolate, B =−9.00, 95% CI [−12.61, −5.39], were successively the least appealing, each of which was significantly lower in appeal than the collapsed average across all flavors.

Figure 2. Relative Difference in Mean Appeal of Each Flavor Compared to the Combined Mean Appeal of the Other Flavors, Rank Ordered.

Note. See the online article for the color version of this figure. * Rating significantly different from the mean ratings of the nine other flavors (p < .05).

Flavor Effects on Sensory Attributes

The omnibus 10-level flavor categorical variable significantly affected each of the attribute outcomes (ps ≤ 0.001; see Table S2, for mean ratings of each flavor). Relative differences in mean sensory attribute ratings between each flavor and the collapsed average of the nine other flavors are depicted in Figure 3 and the corresponding p values of each contrast are in Table S3.

Figure 3. Relative Difference in Mean Sensory Attribute Ratings From the Collapsed Average of the Nine Other Flavors, Rank Ordered.

Note. See the online article for the color version of this figure.

* Rating significantly different from the mean ratings of the nine other flavors (p < .05)

Flavors that were significantly different from the collapsed average across all flavors were as follows. Sweetness: Dessert and fruit flavors yielded the highest sweetness ratings, with vanilla, B = 14.44, 95% CI [10.84, 18.03], strawberry, B = 13.21, 95% CI [9.62, 16.80], and green apple, B = 5.19, 95% CI [5.19, 1.58], completed the protocol. ranking the highest in sweetness. Koolada, B =−11.53, 95% CI [−15.16, −7.91], menthol, B =−8.67, 95% CI [−12.29, −5.06], full-flavored tobacco, B =−8.57, 95% CI [−12.20, −4.98], and spearmint, B =−6.16, 95% CI [−9.79, −2.56], were significantly lower than average in sweetness. Bitterness: Full-flavored tobacco, B = 8.34, 95% CI [4.73, 11.96], and dark chocolate yielded the highest bitterness ratings, B = 5.42, 95% CI [1.81, 9.03], while peppermint, B =−7.65, 95% CI [−11.26, −4.04], strawberry, B = −5.87, 95% CI [−9.48, −2.26], and vanilla, B =−4.24, 95% CI [−7.87, −0.60], yielded the lowest bitterness ratings. Smoothness: Peppermint produced the highest smoothness ratings, B = 6.98, 95% CI [3.13, 10.82]. Smoothness was lowest in full-flavored tobacco, B =−6.21, 95% CI [−10.08, −2.35], dark chocolate, B =−6.11, 95% CI [−9.96, −2.27], and subtle tobacco, B =−4.93, 95% CI [−8.77, −1.10]. Harshness: Harshness was the highest in the tobacco flavor, subtle tobacco, which produced ratings that were 5.69, 95% CI [1.70, 9.68], higher than the mean. Strawberry had the lowest harshness rating of −4.74, 95% CI [08.74, −0.74], below the mean. Coolness: Peppermint, B = 29.25, 95% CI [25.50, 33.01], menthol, B = 15.37, 95% CI [11.48, 19.26], and spearmint, B = 5.00, 95% CI [1.07, 8.92], were significantly higher than the average of coolness. Dark chocolate, B =−13.68, 95% CI [−17.60, −9.77], subtle tobacco, B =−10.08, 95% CI [−14.00, −6.16], full-flavored tobacco, B =−9.21, 95% CI [−13.17, −5.25], vanilla, B =−6.20, 95% CI [−10.17, −2.23], strawberry, B =−5.15, 95% CI [−9.11, −1.20], and green apple, B =−4.74, 95% CI [−8.68, −0.80], were significantly lower than average in coolness.

Discussion

The 10 different-flavored e-cigarette solutions tested in this study produced a diversity of sensory attribute profiles that were fairly consistent with expectations based on the characterizing flavor descriptor. Additionally, e-liquid flavors rated as sweeter, smoother, cooler, and less bitter or harsh tended to be more appealing. Key findings include, first, the study e-cigarette solutions with the characterizing names strawberry and vanilla (flavors in the fruit and dessert categories, respectively) were the two flavors that ranked highest in appeal relative to the other e-cigarette flavors. These two e-cigarette solutions also ranked as the highest in sweetness and the lowest in harshness, concording with prior evidence that e-liquids rated higher in sweetness and lower in harshness also tend to be perceived as more appealing (Baker et al., 2021; Kroemer et al., 2018; Leventhal, Cho, et al., 2020). Second, the two flavors with characterizing names in the tobacco flavor category (i.e., subtle tobacco, full-flavored tobacco) and the dessert flavor with the characterizing name dark chocolate ranked the lowest in appeal. These flavors additionally ranked the highest in both bitterness and harshness and lowest in smoothness and coolness. Third, e-cigarette solutions in the mint flavor category (i.e., peppermint and spearmint) and in the nonmint cooling flavor category (i.e., menthol and koolada) generally ranked high in coolness and smoothness. Finally, the flavoring constituent analysis indicated a complex distribution of flavoring chemicals across the 10 different-flavored solutions tested in this study, with some constituents present in many types of products and others in only one or a few. This precluded formal analyses attempting to link constituents with particular sensory profiles because of the collinearity in constituent statuses across different flavors yielded unstable effect estimates.

Prior work has demonstrated that fruit and menthol flavors have been shown to yield higher appeal ratings compared to tobacco flavors (Leventhal, Goldenson, et al., 2019), a finding replicated by the results in the present study. Results in the present study also demonstrate that flavors within a single-category type can exhibit large differences in appeal and sensory qualities (e.g., the flavor vanilla ranked as the second highest in appeal, while the flavor dark chocolate ranked the lowest in appeal, despite these two flavors belonging to the same dessert flavor category). Therefore, our findings reveal nuances in flavor effects over a wider range of both flavors and flavor categories, highlighting the utility of sensory attributes as an additional source of information over and above the marketing name of a flavored product.

Mint is a popular flavor category (Ali et al., 2020), especially among youth (Leventhal, Miech, et al., 2019), and was a best-selling JUUL flavor (Morean et al., 2020) until it was removed from the market in November 2019 (Diaz et al., 2020). The present study is among the first to test the subjective effects of mint flavors, distinct from other cooling flavors such as menthol. While three of the four mint and nonmint cooling solutions ranked high in the cooling sensory category, these flavor types were distinct in their appeal and other sensory attributes. Specifically, the two mint flavors (peppermint and spearmint) were moderate in sweetness but low in bitterness, whereas menthol and koolada were high in bitterness. Also, peppermint was uniquely high in appeal and smoothness, yet spearmint and the two nonmint cooling flavors were in the midrange on these outcomes. More research is needed to determine the sensory profiles of cooling flavors not represented in the present study (e.g., wintergreen, mint chocolate, strawberry ice) and how the specific menthol and nonmenthol cooling agents identified in certain e-cigarette e-liquids, such as WS-3 and WS-23 (Erythropel et al., 2020; Omaiye et al., 2021) contribute to unique user experiences.

Constituents alone may not reveal important heterogeneity across products with different flavor descriptors. The constituent vanillin, for example, was found in both vanilla, the flavor that ranked highest in sweetness, and full-flavored tobacco, the flavor that ranked highest in bitterness and significantly below the average in sweetness. It is possible that differences in the sensory attributes of dessert and tobacco-flavored e-cigarette solutions are driven by differences in their respective levels of vanillin concentration or perhaps other constituents that compliment or interact with vanillin. Therefore, the presence (or absence) of a particular chemical constituent may not be sufficient to make predictions about a flavor’s overall sensory profile or its appeal. There are thousands of different chemical constituents used in e-cigarette solutions, with hundreds of chemical constituents that are commonly used in 10% or more of all e-cigarette solutions available on the market (Krüsemann et al., 2021). The sheer number of e-cigarette constituents used may complicate a constituent-based method of analyzing and regulating different-flavored e-cigarettes. Moreover, analyses in the present study found some common constituents were used in e-liquid solutions of different flavor categories (e.g., methyl cyclopentenolone was present in solutions with fruit, dessert, and tobacco characterizing names), and therefore their presence in an e-liquid does not in and of itself help categorize the e-liquid into any one particular flavor category.

Variation in the concentration level of a particular constituent could also be a key source of heterogeneity that further complicates methods of investigating and regulating e-cigarette flavors based on their constituents. In some cases, there may be nonlinear dose-dependent effects of varying concentration levels on the user experience for a particular constituent. For instance, the menthol literature indicates that low or moderate concentrations of menthol improves e-cigarette appeal, but high menthol concentrations can cause sensory irritation and strong odors that can be perceived as aversive (Bono et al., 2019; Fan et al., 2016; Leventhal, Cho, et al., 2020; Leventhal, Goldenson, et al., 2019; Rosbrook & Green, 2016). A limitation of this study is that the constituent assay detected only the presence versus absence of each flavorant. For certain commonly used constituents, such as menthol, detailed research unpacking the possible linear or nonlinear effects of constituent concentrations on e-cigarette product appeal may be fruitful but would require considerable resources.

Different manufacturers could use characterizing names for products with constituents that may traditionally be used in one of the main flavor category descriptors (e.g., fruit, mint, dessert) that obscure the general flavor (e.g., “smash,” “unicorn puke”; Jackler & Ramamurthi, 2017; Williams, 2020). Therefore, the sensory rating methodology in the present study provides an additional means of classification. This sensory-based strategy of categorizing different flavors could also be adapted for use in observational survey studies as an additional means of classifying the types of flavors used in the population. Indeed, a previous youth survey found that self-reported use of e-cigarettes that delivered cooling sensations was common, reported among some respondents who did not use products marketed as having mint or menthol characterizing flavors, and associated with more frequent vaping (Davis et al., 2021). Determining the appeal and sensory attributes of e-cigarette solutions could be used as a method for screening the potential appeal and user experience of new products undergoing premarket review. E-cigarette manufacturers could include premarket research on the sensory attributes of their products in their marketing applications along with other information regulators typically require (e.g., information on surveillance of use, toxicity, ingredients, abuse liability, health effects, marketing practices, and perceptions). Such information could be useful to make decisions about whether to authorize companies to legally market new flavored products to the public.

This study had limitations. First, the single-day study design could suppress differences in sensory attributes across flavors due to sensory habituation; however, participants drank water in between each vaping trial, and previous work using this methodology with 40 different products found no evidence of attenuation or amplification of interflavor differences with more trials (Pang et al., 2020). Future research should compare the reliability and validity of ratings based on a single puff versus other more extensive puffing parameters. Second, the present study analyzed five sensory qualities (i.e., sweetness, bitterness, smoothness, harshness, and coolness); however, there may be other sensory domains (e.g., sourness, fruitiness, mintiness) not analyzed in this study and should be addressed in the future work. Third, there are many possible flavoring constituents not assessed in this study’s constituent analysis, which focused on commonly used constituents. Future research that provides a more in-depth analysis of a larger variety of constituents used in e-cigarette solutions is needed. Fourth, interindividual differences in tobacco product use status (exclusive e-cigarette use, exclusive cigarette use, dual use) or flavor (e.g., menthol vs. nonmenthol cigarette) could moderate the effects of flavor variation on appeal and therefore merit further research. Fifth, we did not biochemically verify prestudy nicotine exposure as a study eligibility criterion or verify compliance with the 2-hr minimum tobacco product abstinence period. It is possible that some participants did not comply with these instructions or did not accurately report tobacco product use, which could have inflated between-person variance and in turn reduced statistical power to detect the primary within-person effects of flavor on study outcomes. Finally, the study could not accurately test associations linking the presence of constituents within a product to its respective appeal or sensory ratings. Exploratory analyses found that there was too much collinearity in the presence of constituents across the different-flavored e-cigarette products which yielded unstable estimates when analyzing them concomitantly. This problem of collinearity reflects the reality that many products include various combinations of constituents, which makes linking particular chemicals to particular outcomes difficult and highlights the utility of sensory attribute ratings in characterizing e-cigarette flavors.

Despite these limitations, this work provides data on the effects of e-cigarette flavors on appeal and sensory attributes at a level of detail not previously analyzed. The take-home message is that the presence of constituents, the flavor name that a product is marketed by, and the sensory attributes may all be important methods to classify flavors, with the sensory attributes providing unique explanatory values. Given this premise, and that sensory attributes of a flavored e-cigarette solution generally correspond with its respective appeal, regulatory agencies might benefit from examining user reports of sensory attribute data about a product. Such data may provide additional information about the characteristics of a flavored product to compliment information about its constituents, how a product is marketed, and other data that guide regulatory decisions. This may be especially useful for regulatory review and decisions regarding manufacturer’s applications to market a new e-cigarette product. Furthermore, characterizing sensory attributes may also be useful for science to enhance the ability to compare different studies on a product with the same characterizing flavor name from different manufacturers.

Public Health Significance.

E-cigarette products can be marketed with names that do not clearly denote their flavor (e.g., “unicorn puke” flavor) and contain numerous types of unstudied flavoring chemicals, which create difficulties for regulators wishing to characterize the possible appeal of novel flavored e-cigarette products. The results of this study provide initial evidence that user-reported sensory attributes can be used to parsimoniously characterize various flavored e-cigarettes. Regulators could use this method to characterize the sensory attributes of new flavored e-cigarette products prior to authorizing the marketing of new products to the public.