Skip to main content
PMC home page
International Journal of Environmental Research and Public Health logoLink to International Journal of Environmental Research and Public Health
. 2020 Dec 30;18(1):207. doi: 10.3390/ijerph18010207

Response to Flavored Cartridge/Pod-Based Product Ban among Adult JUUL Users: “You Get Nicotine However You Can Get It”

Jessica M Yingst 1, Candace R Bordner 1,2,*, Andréa L Hobkirk 2, Brianna Hoglen 2, Sophia I Allen 1, Nicolle M Krebs 1, Kenneth R Houser 2, Craig Livelsberger 1, Jonathan Foulds 1
PMCID: PMC7795757  PMID: 33396598

Abstract

In order to curb increasing youth electronic cigarette (e-cig) use, the United States Food and Drug Administration (FDA) banned the sale of flavored cartridge/pod-based products in February 2020. This mixed-methods study aimed to evaluate the impact of the FDA ban on adult JUUL users. The samples of current adult JUUL users were surveyed via Amazon Mechanical Turk at three time-points n = 76 (Sample 1); n = 128 (Sample 2); n = 86 (Sample 3) before and after the FDA flavored/pod ban. The participants were asked to report the JUUL flavored pod used most often and answer questions on purchasing generic pods or refilling (Quantitative). JUUL users were then interviewed in order to explore their perceptions and behaviors related to the FDA ban of flavored cartridge/pod-based products (Qualitative; n = 16). Quantitative data analysis evaluated the differences in variables by time-point. Qualitative data were coded into themes while using the constant comparative method. We found a significant decrease in the use of mint pods (43.4%, 22.7%, 16.3%) (p < 0.01), while there was a significant increase in the use of menthol pods (6.6%, 26.6%, 37.2%) (p < 0.01). Themes that emerged from the qualitative data included switching from mint to menthol pods, refilling pods, and switching to other products that are available in the desired flavors, such as generic pods or disposable e-cigs. Future research is needed in order to evaluate the impact of these behaviors on public health.

Keywords: electronic cigarette, perceptions, mixed methods research, flavored tobacco products, food and drug administration, disposable e-cigarettes, tobacco control policy

1. Introduction

Electronic cigarettes (e-cigs) are a diverse class of tobacco products that aerosolize a nicotine-containing liquid that is directly inhaled by the user. E-cig devices are available in many variations and they have rapidly evolved over time. The first e-cig devices were simple, small devices shaped like a combustible cigarette (cigalikes) that housed the nicotine-containing liquid in cartridges. Advances in technology lead to more complex, larger pen and box-shaped devices that utilized refillable tanks, or reservoirs, to house the nicotine liquid (advanced/mods) [1]. The most recent evolution of e-cigs, which are referred to as pod-mods, are small, discrete devices that utilize interchangeable pods to house the nicotine-containing liquid, typically in a salt form, which is less harsh [2]. These devices have a sleek design, user-friendly functionality, and an array of flavor options [3]. They are also known to deliver nicotine in doses that are higher than many earlier devices [4,5,6]. This type of device was made popular by the market leader called JUUL [7].

In 2018, 14.9% of adults in the United States reported ever using an e-cig, and 3.2% reported current use [8]. Many adults report initiating e-cig use as a way to quit cigarette smoking [9,10]; however, the impact of e-cig use on quitting smoking has been controversial in the literature [11,12,13]. Some studies have reported that e-cig use is not associated with increased success in quitting [14,15], while other studies have found that e-cigs can be effective aids in quitting [16,17,18]. A recent, large randomized controlled trial that was conducted in the United Kingdom found that those using e-cigs were almost twice as likely to remain abstinent from cigarettes at one year, when compared with those using nicotine replacement therapy [17]. Of importance, studies have reported that flavored products are used by the majority of adult users [19,20,21], and could be associated with a successful transition from cigarettes to e-cigs [22,23,24,25].

While the use of e-cigs may help adult smokers to reduce or quit smoking, the wide availability of e-cigs poses a risk to youth. In 2020, 19.6% of high school students and 4.7% of middle school students in the United States reported using an e-cig in the past 30 days, which makes e-cigs the most widely used tobacco product in this age group [26]. Most youth e-cig users report using pod-based products [26,27], like JUUL, which are capable of delivering nicotine in high doses that are similar to a cigarette [5,6]. In addition, the majority of youth e-cig users reported initiating e-cig use because of the appealing flavor options [28]. One study reported that 77.9% of adolescent and 90.3% of young adult users in the United States reported appealing flavors as a reason for e-cig use [28]. Together, these findings suggest that flavored products available with devices that are capable of delivering large amounts of nicotine may be driving youth e-cig use.

In an effort to curb youth uptake, several steps were taken to reduce youth access to flavored products in the United States. First, JUUL Labs Inc. voluntarily removed mango, crème, fruit, and cucumber flavored products from retail stores in November 2018 and later online in October 2019. Soon after, in November 2019, JUUL also completely removed mint flavored products from the market [29]. These steps were taken in response to criticism that JUUL received for their marketing practices, which were seen to promote youth use through the use of young models, brightly colored designs, multitude of flavored products, and use of the social media platform for promotions [7]. Most recently (February 2020), the United States Food and Drug Administration (FDA) banned the sale of all flavored cartridge/pod-based products. This ban included “any small, enclosed unit (sealed or unsealed) designed to fit within or operate as part of an ENDS product”. Of importance, the ban did not include tobacco or menthol flavored pod-based products or flavored liquid products that were used in tank-based devices or flavored disposable products [27]. Although similar, mint was distinguished from menthol, because it is not a traditional cigarette flavor and it was popular among youth [30].

Because flavored cartridge/pod-based products that are widely-used by both adults and youth are no longer available on the market, it is important to evaluate the impact on users. This explanatory sequential mixed-methods study aimed to evaluate the impact of the flavored cartridge/pod-based ban among adult JUUL users specifically. First, the samples of current adult JUUL users were surveyed at varying time points (before and after the FDA flavor ban) in order to measure changes in the proportion of users reporting the use of flavored pods. In addition, this study also aimed to explore use behaviors that could result from the changes in flavored pod availability. In order to further explain the quantitative results, qualitative interviews were conducted with adult JUUL users to explore their perceptions of the flavored cartridge/pod-based ban and understand how it affected their use of these products.

2. Methods

This explanatory sequential mixed-methods study of current JUUL users included a quantitative online survey at three time points and qualitative interviews. The results from the quantitative and qualitative data were separately analyzed and the results were integrated for interpretation. Figure 1 depicts a procedural diagram for the study.

Figure 1.

Figure 1

Open in a new tab

Procedural Diagram.

2.1. Quantitative Data Collection

2.1.1. Participants

Cross-sectional samples of current JUUL users (aged ≥ 18 years and reporting JUUL use in the past 30 days) completed online surveys regarding their JUUL use. Survey data were collected at three time points. The first sample of participants (Sample 1) was surveyed prior to JUUL flavored pod restrictions, when flavored pods were only available online (July 2019). The second sample of participants (Sample 2) was surveyed in January 2020, after the removal of JUUL flavored pods, including mint, from the market. The third sample of participants (Sample 3) was surveyed in April 2020, after the FDA enacted the ban of all flavored cartridge/pod-based products.

All participants were recruited via Amazon Mechanical Turk (MTurk), an online labor market that uses crowdsourcing in order to recruit diverse adult (18 years of age of older) samples for online tasks that require human intelligence. MTurk is an efficient and convenient method for recruiting smokers and e-cigarette users for survey research, including JUUL users [31,32], and studies have shown that the data that were collected via MTurk for social science research are valid and internally consistent, with good test-retest reliability [33,34,35]. In order to improve the reliability and validity, the respondents were restricted to those in the United States with job approval ratings on MTurk of at least 98%. All of responses were briefly reviewed by the researchers before approval and payment was provided to the user. Duplicate participants at each time-point were identified by the MTurk Worker ID and the duplicate record was removed. In order to verify that participants were actual users of JUUL, responses to open-ended questions and time of use questions were examined during data cleaning. In addition, the users were asked to upload a photograph of their actual JUUL device. The participants who provided data that appeared falsified (i.e., repeated number use, logically impossible responses) and/or who did not provide a picture of their device were excluded from the data.

All of the participants gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Penn State College of Medicine Institutional Review Board (IRB #12315).

2.1.2. Measures

The participants were asked to report demographic characteristics, including age and gender. They were also asked questions regarding their smoking history, including whether they have ever smoked cigarettes and the number of days/months/years since they last smoked cigarettes. Those reporting cigarette smoking in the past 30 days were considered to be current smokers.

Questions relating to JUUL use included the number of days used in the past 30 days and the number of times used per day (“time” defined as use lasting either 10 min. or 15 puffs). In addition, questions were included to measure the use of flavored pods and the location of their purchase (in stores vs. online). The participants were asked, “During the past 30 days, which flavor JUUL refill pod have you used most frequently?” and “Where do you purchase your JUUL pods most often?” Finally, the participants were asked whether they ever refill their single-use JUUL pods with nicotine liquid and if they purchase generic pods, defined as pods which are compatible with the JUUL device, but not manufactured by JUUL. Those who purchased generic pods were asked to select the reason they chose to purchase generic pods from a list of options (less expensive, additional flavor options, easier to purchase, are refillable, hold more liquid, available in 0% nicotine, and other).

For the last sample of survey participants (S3), we asked the participants to retrospectively report the JUUL flavored pod that they used most often prior to JUUL removing flavors from the market, in addition to the flavor of the JUUL pod that they used at the time of survey completion. The participants were asked, “Prior to JUUL stopping the sales of flavored pods, which JUUL flavor did you use most often?”.

2.1.3. Data Analysis

The participants who reported their basic demographic information and their preferred flavor were included in the analysis. Means and frequencies were used in order to describe the participants and their use characteristics. Chi-square analysis and one-way ANOVA were used to determine the differences in variables of interest by survey wave (between-subject). Fisher’s Exact Test, with the Freeman–Halton extension [36], was used in order to evaluate differences in proportions when the expected frequency in any cell was less than 5 [37]. McNemar’s test was used to look at the within-subject changes in flavor for the S3 retrospective and current flavored pod usage.

2.2. Qualitative Data Collection

2.2.1. Participants

The participants were adult (age 21+) JUUL users who reported JUUL use in the past 90 days. An inclusion period of 90 days was selected to ensure that the sample was not biased towards participants who continued JUUL use after the FDA ban. The participants were recruited from a database of potential research participants, maintained by the Penn State Center for Research on Tobacco and Health, who previously indicated JUUL use and indicated interest in participating in future research studies. This database included participants from the quantitative study discussed above. Those who indicated interest in future research were sent an email describing the study and instructing them to call the research center if they were interested in participating. Those that were interested were screened and those eligible were sent an email with their scheduled interview time, a summary explanation of the research, and a link to join the researcher via ZOOM for Healthcare, a HIPAA compliant online videoconferencing service, at the scheduled visit time.

All of the participants gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the Penn State College of Medicine Institutional Review Board approved the protocol (IRB #14507).

2.2.2. Measures

The researcher started the interview by asking the participant to report on basic demographic characteristics and to show their JUUL device in order to confirm use. The researcher then used the interview guide to ask the participant a series of questions regarding their JUUL use, including how they were impacted by the ban on flavored cartridge/pod-based based products. All of the interview visits were audio and video recorded while using ZOOM for Healthcare.

2.2.3. Data Analysis

The means and frequencies were used to describe the demographic data. First, all of the interviews were transcribed verbatim into transcripts, and then imported to NVIVO 10 software (QSR International, Burlington, MA, USA) for coding and analysis. Researchers then generated a codebook containing 10 a priori themes that were based on the literature. Transcripts were then reviewed while using the constant comparative method [38] and the codebook was refined through comparison and discussion. Next, each researcher coded the same four transcripts (25% of the data) while using the finalized codebook and inter-rater reliability (IRR) was calculated between the researchers (J.M.Y and C.R.B.) with a reliability standard of k > 0.8 in order to ensure consistency in coding. All of the data presented in this study were coded into the theme, FDA flavor ban. Sub-themes for the FDA flavor ban theme included no impact, switching to other available pods, switching to other available products, and continuing use of flavored pod-based products illicitly.

2.2.4. Data Integration

Quantitative and qualitative data were integrated using a joint display. A joint display integrates the quantitative and qualitative data in a table, allowing further insight into the data that could not be understood when separately analyzing the data [39].

3. Results

3.1. Quantitative Study

Two hundred and ninety participants met the study inclusion criteria for the quantitative study (n = 76 (S1); n = 128 (S2); n = 86 (S3)). The participant characteristics can be found in Supplemental Table S1. There were no significant differences for these demographic variables between the survey time-points.

At survey time-point 1 (S1 July 2019), mint flavored pods were the most commonly used (43.4%). There was a significant decrease in the use of mint pods from S1 to S3 (April 2020) (43.4%, 22.7%, 16.3%) (p < 0.01) (Figure 2). In addition, there was a significant increase in the use of menthol pods from S1 to S3 (6.6%, 26.6%, 37.2%) (p < 0.01), which makes menthol the most commonly used flavor by S3. While the use of other flavored pods slightly decreased over time, these differences were not significant. Of interest, there was not a significant decrease in use of the popular flavor mango that should not have been legally available at S3 (14.5%, 8.6%, 11.6%, p = 0.42). Similar trends were found in a subsample of young adult users (18–25 years) (n = 63). Mint use decreased (52.9%, 30.4%, 21.7%) (p = 0.13), however, not significantly, while menthol use increased (5.9%, 26.1%, 43.5%) (p = 0.03).

Figure 2.

Figure 2

Open in a new tab

Most frequently used pod flavor by survey time-point.

At S3 only, the participants were asked to report their current flavored pod used and the flavored pod used most often prior to the FDA ban. With this within-subject analysis, there were significant decreases in the use of mint, mango, and fruit pods, and significant increases in the use of Virginia tobacco, classic tobacco, and menthol pods (Table 1). There were no changes in the use of crème and cucumber pods.

Table 1.

Within-subject changes in flavor usage after FDA regulation from data collected at survey time-point 3 (S3) (April 2020).

Flavor S3 Previous Flavor Used (n = 86) S3 Current Flavor Used (n = 86) p-Value
Virginia Tobacco, % yes (n) 14.0 (12) 17.4 (15) <0.01
Classic Tobacco, % yes (n) 4.7 (4) 10.5 (9) <0.01
Menthol, % yes (n) 11.6 (10) 37.2 (32) <0.01
Mint, % yes (n) 26.7 (23) 16.3 (14) <0.01
Mango, % yes (n) 29.1 (25) 11.6 (10) <0.01
Fruit, % yes (n) 8.1 (7) 4.7 (4) <0.01
Crème, % yes (n) 3.5 (3) 1.2 (1) 1.0
Cucumber, % yes (n) 2.3 (2) 1.2 (1) 1.0
Open in a new tab

In addition to changes in flavored pod usage, this study also measured the impacts on use behaviors. There was a significant difference over time in the proportion of participants who reported refilling their pods (19.7%, 34.4%, 11.6%) (p < 0.01) (Table 2). There were no differences in the proportion of users who reported using generic pods (p = 0.15) or their reason for purchasing generic pods (p = 0.29). In addition, there was not a difference in the proportion of users who bought their pods online; however, at S2, when flavored pods were only available online, almost half of the participants (46.1%, n= 66) reported purchasing pods online.

Table 2.

Participant behaviors by survey time-point.

Participant Behaviors Time-Point 1
July 2019
(n = 76)		 Time-Point 2
January 2020
(n = 128)		 Time-Point 3
April 2020
(n = 86)		 p-Value
Current location to purchase JUUL pods is online, % yes (n) 23.7 (18) 20.3 (26) 15.1 (13) 0.38
Ever refilled pods, % yes (n) 19.7 (15) 34.4 (44) 11.6 (10) <0.01
Ever experienced issues with purchasing genuine pods, % yes (n) 14.5 (11) 18.8 (24) 18.6 (16) 0.71
Ever intentionally purchased generic pods, % yes (n) 7.9 (6) 17.2 (22) 11.6 (10) 0.12
Reason purchased generic pods because of more flavor options, % yes (n) 50.0 (3/6) 40.9 (9/22) 70.0 (7/10) 0.39
Open in a new tab

3.2. Qualitative Results

The characteristics of the sample (n = 16) can be found in Supplemental Table S1.

Many of the participants reported that the recent ban of flavored pods did not affect their use. One participant said, “It hasn’t really affected me because I just smoke the tobacco flavor pod” (female, age 24, 2 CPD). Others noted that they were not affected because they do not prefer the fruit flavors. One said, “I’ve always only preferred the menthol and that’s the only one that I’ve used. I mean I’ve tried [flavors] in the beginning cause it came with them, but never been a fan of the fruity flavors” (male, age 33, 20 CPD). Another participant said, “I know some people really get hung up on those flavors. It’s never been a huge draw for me” (male, age 43, 1 CPD). Others were not affected, because they continued use of the banned flavors. These users reported stockpiling their pods, saying “so it made me kind of hoard them” (male, age 38, former smoker). Others reported continuing to buy the banned products from stores illegally selling them. One participant said, “there were like a few stores I had around here that were like sketchy and like secretly had them” (female, age 26, former smoker).

Other participants noted that, while they were affected by the ban of flavored pods, they were able to quickly adapt to the available pods, mainly because the products still contained nicotine. One said, “I think the mint flavor was like my favorite one… and then when they banned that, um, I don’t know, you just adapt. Like we all hated the other flavors that they left us, but like we’ve adapted... Like, at first I didn’t like the menthol flavor at all. And now I prefer it because, well, [it’s] one of two flavors that are around. So you get nicotine however you can get it” (female, age 29, 2 CPD). Other participants said, “I did really like the mint ones. That was my go to, but the menthols are just as good” (female, age 24, former smoker) and, “I mean, once you start, you try the JUUL and you’re like, okay mango’s good. This is amazing… but the point is you’re addicted to nicotine. It doesn’t matter at that point. I’m just like oh… I don’t have my crème B. It’s just like you tell an alcoholic, you know, sorry we don’t have your [drink]. We only got the peach or something. It’s like I don’t care. Give me the stuff with alcohol or nicotine” (male, age 34, never smoker). Finally, several participants noted how the ban of flavored pods has lead them to use other flavored products, including disposable devices. One participant said, “They still sell all sorts of flavors and disposable devices, which is even easier to access than buying a JUUL kit” (female, age 59, never smoker). Another participant noted that they use the available JUUL pod, but treat themselves with flavored disposable products. They said, “When I feel like I want to reward myself, I tell myself that I’ll get like a flavored Puff Bar, something like that. But they’re a little bit more expensive” (male, age 23, 3 CPD). Another participant reported the use of generic flavored pods. They said, “It has completely changed my buying habits. I always a hundred percent of the time, if it’s available, choose a generic that has a better flavor over the actual JUUL pod” (female, age 28, 10 CPD). These participants expressed confusion over differences in which flavored products were allowed to be sold. These participants said, “I’m kind of confused on what happened with that situation. Like, how come other vape companies can use like different flavors but JUUL can’t” (male, age 23, 3 CPD) and “So that just makes no sense to me. I go and they’ve told JUUL, you can’t sell your customers flavors, but I can buy flavors in all these other brands that are disposable. So that’s just kind of punitive if you ask me” (female, age 59, never smoker).

Table 3 displays the merged results for the quantitative and the qualitative findings.

Table 3.

Joint Display of Quantitative and Qualitative Findings.

Overall Finding
(Impact of FDA Ban)		 Quantitative Finding Qualitative Finding
Users were not impacted Prior to any flavor restrictions, almost a quarter (23.7%) of participants already used tobacco or menthol products. Theme: Users were not affected because they already used tobacco or menthol flavors.

  • “I know some people really get hung up on those flavors. It’s never been a huge draw for me”

    “I’ve always only preferred the menthol and that’s the only one that I’ve used”
Users switched to available products There was a significant reduction in mint use and a significant increase in menthol after the FDA ban. Theme: Users easily transitioned from mint to menthol because both contained nicotine.

  • “I did really like the mint ones. That was my go to, but the menthols are just as good”

    “You just adapt… so you get nicotine however you can get it”
Users started using products available in their preferred flavor After the FDA ban, more than 10% of participants reported using generic products. Theme: Users sought generic pods (or disposable products) when their preferred flavors were not available.

  • “I always a hundred percent of the time, if it’s available, choose a generic that has a better flavor over the actual JUUL pod”

    “When I feel like I want to reward myself, I tell myself that I’ll get like a flavored Puff Bar.”
Users continued use of banned flavors After the FDA ban, about 35% of participants reported continuing to use banned flavors. Theme: Participants stock-piled banned flavors or found locations that continued to sell the products.

  • “So it made me kind of hoard them.”

    “There were like a few stores I had around here that were like sketchy and like secretly had them.”
Open in a new tab

4. Discussion

Among adult JUUL users, we found that the use of flavored JUUL pods was significantly impacted by the removal of flavored JUUL pods from the market, and the later FDA ban on the sales of all brands of flavored cartridge/pod-based products. Prior to JUUL removing their flavored products from the market, mint was the most popular flavored pod, being used by more than 43% of users. Following the FDA ban, the data from the three time-points showed a significant decrease in the use of mint pods and a significant increase in the use of menthol pods, which makes it the most popular flavored pod (34.8%). These findings were consistent with the within-subject data that were collected from participants at time-point 3. In addition, the findings from our small study were also supported by retail market data, which showed a dramatic increase in menthol flavored cartridge/pod-based product sales following JUUL’s removal of mint flavored pods from the market [29,40]. One study reported that, by May 2020, menthol dominated the market, with 61.8% of prefilled cartridge/pod-based sales [29]. Many participants perceived that the shift from mint, a flavor removed from the market, to menthol, an available flavor, was easy, because both products contained nicotine, the real driver behind their continued use. This is an important finding, because it suggests the willingness of users to switch to the flavors that are available [40].

While the use of some flavors significantly changed after the FDA ban, we found that other flavors, including some that were banned, did not change. This same pattern was also found in a study that evaluated flavor use after the ban of flavored products in San Francisco [41]. This suggests that users are still able to access banned flavored pods in some way, especially since the retail data suggest that users are not purchasing these products in stores [29]. First, it is possible that users stockpiled JUUL pods in anticipation of the ban, as indicated in the qualitative data, and they were still using pods from their own supply at the time of the survey. Additionally, it is possible that users were able to purchase JUUL pods from stores who continued to sell them illegally or obtain pods from other countries that are able to legally sell JUUL flavored pods. In addition, it is also possible that users could have purchased counterfeit pods that are marketed as JUUL [42]. Finally, it is possible that users misinterpreted the question and instead of reporting use of JUUL brand pods, they were reporting the flavor of generic or refilled JUUL-compatible pods. It will be important for future studies to evaluate how users are able to access the banned flavors.

We found a slight increase in the number of users reporting generic pods use from S1 to S3, with approximately 12% of users reporting the use of generic brand flavored cartridge/pod-based products after the FDA ban. This upward trend in generic pod use could be due to a lack of compliance with the FDA regulations by the companies that produce these generic products. While companies not in compliance risk enforcement action by the FDA [27], enforcement could be delayed due to the onset of the COVID-19 pandemic, leaving these products to remain on the market. This shift from brand name products, like JUUL, with known quality control methods to generic products is concerning. Recently, there was an outbreak of a serious lung illness, called EVALI (e-cigarette or vaping product use-associated lung injury), which was later identified as being caused by black market THC cartridges that contained Vitamin E acetate [43,44]. While EVALI was not linked with the use of nicotine e-cig pods, it is possible that a similar outbreak could occur among nicotine e-cig users if the use of generic products or illicit products continues outside of quality controls that are required for FDA Premarket Tobacco Product Application (or PMTA) approval.

In addition, we found a significant difference over time in the proportion of users who reported refilling their single-use JUUL pods, a behavior that is warned against for safety and quality reasons [45]. This involves purchasing an e-cig liquid that is not intended for use in JUUL products and adding that liquid to a JUUL pod. JUUL pods are designed for one-time use, so this is not easy, but consumers can find instructional videos and posts online. Our data showed that, when flavored JUUL pods were no longer on the market, there was an increase in refilling. Subsequently, with the FDA ban, there was a sharp decrease in reported refilling. This decrease in refilling could be attributed to participants learning to find other sources of flavored products, such as purchasing generic brand flavored pods, reducing the need to refill their JUUL pods with the flavors that they like. It is also possible that users switched to other flavored product types. While flavored cartridge/pod-based products were banned by the FDA, flavored disposable e-cig products were not. A recent study examining retail data showed an increase in the use of mint flavored disposable products from 2017 to 2020 [29], and the National Youth Tobacco Survey reported a 1000% increase in the use of disposable products from 2019 to 2020 [26]. In addition, a study of internet searches reported a sharp increase of searches pertaining to disposable products, like Puff Bar [26,46]. Switching to products that are available with flavors was discussed by participants during the qualitative interviews, and many perceived the differences in what flavored products could and could not be sold to be confusing. This loophole in regulation is particularly concerning for youth populations who may still have access to flavored products.

The current study was not able to assess whether the FDA ban on flavored products led adult users to return to cigarette smoking. Future longitudinal studies are needed in order to further evaluate the impact of flavored product bans on returning to smoking. Other limitations include the use of a small convenience sample for the quantitative portion of the study. We conducted post-hoc power analysis to address this limitation, and determined that the study had sufficient power to detect small to medium effects (phi ≥ 0.2), while it was under powered for detecting small effects sizes (phi < 0.2). Given the effect sizes that were found for our chi-square analyses assessing change in flavor proportion across the three time-points, we had sufficient power to detect small to medium effects in comparisons of mint, menthol, and mango (phi ≥ 0.2, estimated power ≥ 87%) and the study was underpowered to detect small effects in classic tobacco, Virginia tobacco, fruit, crème, and cucumber (phi < 0.2, estimated power < 45%). Future research with larger sample sizes will help to elucidate the more nuanced effects that regulations and market changes might have on flavor use over time.

Additionally, the study was limited by the recruitment of the sample while using an online platform, called MTurk. The MTurk participants are not representative of the general population; therefore, we are not able to make any population-based inferences that are based on these results [47]. However, MTurk is an ideal platform to recruit JUUL users, since those completing tasks tend to be younger, with an over-representation of substance users [33,48]. In addition, this study focused on adult JUUL users. This was due to the age restrictions of the MTurk platform, which only allows for those 18 and older to have an account. Because the youth population is of interest, we completed a sub-subsample analysis to compare the impacts on young adults to the entire sample. These results were similar, which suggested that young adult and adult users respond similarly to flavor restrictions. Future studies are needed to evaluate the impact of these restrictions among a youth and young adult population.

Finally, the data that were collected for the quantitative part of the study was cross-sectional, collected from unique samples at three time points. The data collected were not collected longitudinally from the same participants over time, which would have yielded richer data and could have provided insight on whether users continued JUUL use or returned to smoking after the FDA ban. Finally, while some of the participants completed both the quantitative and qualitative parts of the study (n = 7, 43.8% of the qualitative interviews completed), the majority of qualitative interviews were completed by participants who were not included in the quantitative part of the study. However, the samples that were recruited for both parts of the study were very similar in regards to demographics and JUUL use characteristics, likely minimizing any issues.

5. Conclusions

Availability restrictions, such as the removal from shelves or outright bans, of specific e-cig liquid flavored cartridge/pod-based products led to only partial reductions in the use of banned flavors. Where loopholes or a lack of enforcement exists, some manufacturers seek to exploit these loopholes, and users adjust their behavior in order to acquire either banned flavors or the closest available e-cig liquid (e.g., refilling pods with flavored liquids no longer sold by JUUL or switching from mint to menthol flavored JUUL pods). Our quantitative and qualitative user data correspond with literature on retail sales data and online search engine trends that followed the flavor bans. Future research is needed in order to evaluate black market/bootleg sales and the potential impacts on users and public health.

Supplementary Materials

The following are available online at https://www.mdpi.com/1660-4601/18/1/207/s1, Table S1: Participant Characteristics.

Click here for additional data file. (87.7KB, pdf)

Author Contributions

Conceptualization, J.M.Y., J.F. and A.L.H.; Methodology, J.M.Y., J.F.; Formal Analysis, J.M.Y. and C.R.B.; Investigation, J.M.Y., C.R.B., B.H., S.I.A., N.M.K., K.R.H., C.L., and J.F.; Writing—Original Draft Preparation, J.M.Y. and C.R.B.; Writing—Review & Editing, J.M.Y., C.R.B., S.I.A., N.M.K., K.R.H., and J.F.; Supervision, J.M.Y. and C.R.B.; Project Administration, J.M.Y. and C.R.B.; Funding Acquisition, J.F. and A.L.H. All authors have read and agreed to the published version of the manuscript.

Funding

The research described was supported by the Penn State Hershey Cancer Institute, Penn State Social Science Research Institute, the National Center for Advancing Translational Sciences, National Institutes of Health (through Grant UL1 TR002014), and internal funds provided by the Penn State College of Medicine Department of Psychiatry.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Penn State College of Medicine Institutional Review Board (protocol code 12315, approved 4/26/2019 and protocol code 14507, approved 4/3/2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to participant consent.

Conflicts of Interest

J.F. has done paid consulting for pharmaceutical companies involved in producing smoking cessation medications, including GSK, Pfizer, Novartis, J&J, and Cypress Bioscience. The other authors have no disclosures to report related to this publication. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Breland A., Soule E., Lopez A., Ramôa C., El-Hellani A., Eissenberg T. Electronic cigarettes: What are they and what do they do? Ann. N. Y. Acad. Sci. 2017;1394:5–30. doi: 10.1111/nyas.12977. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Eissenberg T., Soule E., Saliba N. National Institutes of Health (NIH) Tobacco Regulatory Science Meeting. Bethesda; Rockville, MD, USA: 2018. JUUL: The Prototypical “Pod Mod”: Design Characteristics, Toxicant Yield, and Preliminary Nicotine Delivery and Subjective Effect Profile. [Google Scholar]
  • 3.Fadus M.C., Smith T.T., Squeglia L.M. The rise of e-cigarettes, pod mod devices, and JUUL among youth: Factors influencing use, health implications, and downstream effects. Drug Alcohol Depend. 2019;201:85–93. doi: 10.1016/j.drugalcdep.2019.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Yingst J.M., Foulds J., Veldheer S., Hrabovsky S., Trushin N., Eissenberg T.T., Williams J., Richie J.R., Nichols T.T., Wilson S.J., et al. Nicotine absorption during electronic cigarette use among regular users. PLoS ONE. 2019;14:e0220300. doi: 10.1371/journal.pone.0220300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Yingst J., Hrabovsky S., Hobkirk A., Trushin N., Richie J.P., Foulds J. Nicotine Absorption Profile Among Regular Users of a Pod-Based Electronic Nicotine Delivery System. JAMA Netw. Open. 2019;2:e1915494. doi: 10.1001/jamanetworkopen.2019.15494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hajek P., Pittaccio K., Pesola F., Smith K.M., Phillips-Waller A., Przulj D. Nicotine delivery and users’ reactions to Juul compared with cigarettes and other e-cigarette products. Addiction. 2020;115:1141–1148. doi: 10.1111/add.14936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Huang J., Duan Z., Kwok J., Binns S., Vera L.E., Kim Y., Szczypka G., Emery S.L. Vaping versus JUULing: How the extraordinary growth and marketing of JUUL transformed the US retail e-cigarette market. Tob. Control. 2019;28:146–151. doi: 10.1136/tobaccocontrol-2018-054382. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Creamer M.R., Wang T.W., Babb S., Cullen K.A., Day H., Willis G., Jamal A., Neff L. Tobacco Product Use and Cessation Indicators Among Adults—United States, 2018. Morb. Mortal. Wkly. Rep. 2019;68:1013–1019. doi: 10.15585/mmwr.mm6845a2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Patel D., Davis K.C., Cox S., Bradfield B., King B.A., Shafer P., Caraballo R., Bunnell R. Reasons for current E-cigarette use among U. S. adults. Prev. Med. 2016;93:14–20. doi: 10.1016/j.ypmed.2016.09.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Nicksic N.E., Snell L.M., Barnes A.J. Reasons to use e-cigarettes among adults and youth in the Population Assessment of Tobacco and Health (PATH) study. Addict. Behav. 2019;93:93–99. doi: 10.1016/j.addbeh.2019.01.037. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Warner K.E., Mendez D. E-cigarettes: Comparing the Possible Risks of Increasing Smoking Initiation with the Potential Benefits of Increasing Smoking Cessation. Nicotine Tob. Res. 2019;21:41–47. doi: 10.1093/ntr/nty062. [DOI] [PubMed] [Google Scholar]
  • 12.Eaton D.L., Kwan L.Y., Stratton K. National Academies of Science, Public Health Consequences of E-Cigarettes. The National Acadamies Press; Washington, DC, USA: 2018. [PubMed] [Google Scholar]
  • 13.Patil S., Arakeri G., Patil S., Ali Baeshen H., Raj T., Sarode S.C., Sarode G.S., Awan K.H., Gomez R., Brennan P.A. Are electronic nicotine delivery systems (ENDs) helping cigarette smokers quit?—Current evidence. J. Oral Pathol. Med. 2020;49:181–189. doi: 10.1111/jop.12966. [DOI] [PubMed] [Google Scholar]
  • 14.Jackson S.E., Shahab L., West R., Brown J. Associations between dual use of e-cigarettes and smoking cessation: A prospective study of smokers in England. Addict. Behav. 2020;103:106230. doi: 10.1016/j.addbeh.2019.106230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Pierce J.P., Benmarhnia T., Chen R., White M., Abrams D.B., Ambrose B.K., Blanco C., Borek N., Choi K., Coleman B., et al. Role of e-cigarettes and pharmacotherapy during attempts to quit cigarette smoking: The PATH Study 2013–2016. PLoS ONE. 2020;15:e0237938. doi: 10.1371/journal.pone.0237938. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Bullen C., Howe C., Laugesen M., McRobbie H., Parag V., Williman J., Walker N. Electronic cigarettes for smoking cessation: A randomised controlled trial. Lancet. 2013;382:1629–1637. doi: 10.1016/S0140-6736(13)61842-5. [DOI] [PubMed] [Google Scholar]
  • 17.Hajek P., Phillips-Waller A., Przulj D., Pesola F., Smith K.M., Bisal N., Li J., Parrott S., Sasieni P., Dawkins L., et al. A Randomized Trial of E-Cigarettes versus Nicotine-Replacement Therapy. N. Engl. J. Med. 2019;380:629–637. doi: 10.1056/NEJMoa1808779. [DOI] [PubMed] [Google Scholar]
  • 18.McRobbie H., Bullen C., Hartmann-Boyce J., Hajek P. Electronic cigarettes for smoking cessation and reduction. Cochrane Database Syst. Rev. 2014;2020:CD010216. doi: 10.1002/14651858.cd010216.pub2. [DOI] [PubMed] [Google Scholar]
  • 19.Harrell M.B., Weaver S., Loukas A., Creamer M., Marti C., Jackson C., Heath J., Nayak P., Perry C., Pechacek T., et al. Flavored e-cigarette use: Characterizing youth, young adult, and adult users. Prev. Med. Rep. 2017;5:33–40. doi: 10.1016/j.pmedr.2016.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Yingst J.M., Veldheer S., Hammett E., Hrabovsky S., Foulds J. A Method for Classifying User-Reported Electronic Cigarette Liquid Flavors. Nicotine Tob. Res. 2017;19:1381–1385. doi: 10.1093/ntr/ntw383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Du P., Bascom R., Fan T., Sinharoy A., Yingst J., Mondal P., Foulds J. Changes in Flavor Preference in a Cohort of Long-Term Electronic Cigarette Users. Ann. Am. Thorac. Soc. 2020;17:573–581. doi: 10.1513/AnnalsATS.201906-472OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Tackett A.P., Lechner W.V., Meier E., Grant D.M., Driskill L.M., Tahirkheli N.N., Wagener T.L. Biochemically verified smoking cessation and vaping beliefs among vape store customers. Addiction. 2015;110:868–874. doi: 10.1111/add.12878. [DOI] [PubMed] [Google Scholar]
  • 23.Russell C., McKeganey N., Dickson T., Nides M. Changing patterns of first e-cigarette flavor used and current flavors used by 20,836 adult frequent e-cigarette users in the USA. Harm Reduct. J. 2018;15:33. doi: 10.1186/s12954-018-0238-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Gendall P.J., Hoek J. Role of flavours in vaping uptake and cessation among New Zealand smokers and non-smokers: A cross-sectional study. Tob. Control. 2020;30 doi: 10.1136/tobaccocontrol-2019-055469. [DOI] [PubMed] [Google Scholar]
  • 25.Russell C., Haseen F., McKeganey N. Factors associated with past 30-day abstinence from cigarette smoking in adult established smokers who used a JUUL vaporizer for 6 months. Harm Reduct. J. 2019;16:59. doi: 10.1186/s12954-019-0331-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Wang T.W., Neff L.J., Park-Lee E., Ren C., Cullen K.A., King B.A. E-cigarette Use Among Middle and High School Students—United States, 2020. Morb. Mortal. Wkly. Rep. 2020;69:1310–1312. doi: 10.15585/mmwr.mm6937e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.United States Food and Drug Administration FDA Finalizes Enforcement Policy on Unauthorized Flavored Cartridge-Based E-Cigarettes that Appeal to Children, Including Fruit and Mint. [(accessed on 30 October 2020)];2020 Available online: https://www.fda.gov/news-events/press-announcements/fda-finalizes-enforcement-policy-unauthorized-flavored-cartridge-based-e-cigarettes-appeal-children.
  • 28.Soneji S., Knutzen K.E., Villanti A.C. Use of Flavored E-Cigarettes among Adolescents, Young Adults, and Older Adults: Findings from the Population Assessment for Tobacco and Health Study. Public Health Rep. 2019;134:282–292. doi: 10.1177/0033354919830967. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Ali F.R.M., Diaz M.C., Vallone D., Tynan M.A., Cordova J., Seaman E.L., Trivers K.F., Schillo B.A., Talley B., King B.A. E-cigarette Unit Sales, by Product and Flavor Type—United States, 2014–2020. Morb. Mortal. Wkly. Rep. 2020;69:1313–1318. doi: 10.15585/mmwr.mm6937e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Leventhal A.M., Miech R., Barrington-Trimis J., Johnston L.D., O’Malley P.M., Patrick M.E. Flavors of e-Cigarettes Used by Youths in the United States. JAMA. 2019;322:2132–2134. doi: 10.1001/jama.2019.17968. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Leavens E.L., Stevens E.M., Brett E.I., Hébert E.T., Villanti A.C., Pearson J.L., Wagener T.L. JUUL electronic cigarette use patterns, other tobacco product use, and reasons for use among ever users: Results from a convenience sample. Addict. Behav. 2019;95:178–183. doi: 10.1016/j.addbeh.2019.02.011. [DOI] [PubMed] [Google Scholar]
  • 32.Bauhoff S., Montero A., Scharf D. Perceptions of e-cigarettes: A comparison of adult smokers and non-smokers in a Mechanical Turk sample. Am. J. Drug Alcohol Abus. 2017;43:311–323. doi: 10.1080/00952990.2016.1207654. [DOI] [PubMed] [Google Scholar]
  • 33.Strickland J.C., Stoops W.W. The use of crowdsourcing in addiction science research: Amazon Mechanical Turk. Exp. Clin. Psychopharmacol. 2019;27:1. doi: 10.1037/pha0000235. [DOI] [PubMed] [Google Scholar]
  • 34.Kim H.S., Hodgins D.C. Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon’s Mechanical Turk. Psychol. Addict. Behav. 2017;31:85–94. doi: 10.1037/adb0000219. [DOI] [PubMed] [Google Scholar]
  • 35.Thomas K.A., Clifford S. Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments. Comput. Hum. Behav. 2017;77:184–197. doi: 10.1016/j.chb.2017.08.038. [DOI] [Google Scholar]
  • 36.Freeman G.H., Halton J.H. Halton, note on an exact treatment of contingency, goodness of fit and other problems of significance. Biometrika. 1951;38:141–149. doi: 10.1093/biomet/38.1-2.141. [DOI] [PubMed] [Google Scholar]
  • 37.Kim H.-Y. Statistical notes for clinical researchers: Chi-squared test and Fisher’s exact test. Restor. Dent. Endod. 2017;42:152–155. doi: 10.5395/rde.2017.42.2.152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Creswell J.W. A Concise Introduction to Mixed Methods Research. SAGE Publications; New York, NY, USA: 2014. p. 132. [Google Scholar]
  • 39.Guetterman T.C., Fetters M.D., Creswell J.W. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research through Joint Displays. Ann. Fam. Med. 2015;13:554–561. doi: 10.1370/afm.1865. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Diaz M.C., Donovan E.M., A Schillo B., Vallone D. Menthol e-cigarette sales rise following 2020 FDA guidance. Tob. Control. :2020. doi: 10.1136/tobaccocontrol-2020-056053. [DOI] [PubMed] [Google Scholar]
  • 41.Yang Y., Lindblom E.N., Salloum R.G., Ward K.D. The impact of a comprehensive tobacco product flavor ban in San Francisco among young adults. Addict. Behav. Rep. 2020;11:100273. doi: 10.1016/j.abrep.2020.100273. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.LaVito A. Fake Juul Pods Line Store Shelves, Worrying Users and Posing Another Threat to the Embattled Company. [(accessed on 30 October 2020)];2019 Available online: https://www.cnbc.com/2019/08/26/fake-juul-pods-fill-shelves-after-vaping-giant-pulled-fruity-flavors.html.
  • 43.Blount B.C., Karwowski M.P., Shields P.G., Morel-Espinosa M., Valentin-Blasini L., Gardner M., Braselton M., Brosius C.R., Caron K.T., Chambers D., et al. Vitamin E Acetate in Bronchoalveolar-Lavage Fluid Associated with EVALI. N. Engl. J. Med. 2020;382:697–705. doi: 10.1056/NEJMoa1916433. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Hall W., Gartner C., Bonevski B. Lessons from the public health responses to the US outbreak of vaping-related lung injury. Addiction. :2020. doi: 10.1111/add.15108. [DOI] [PubMed] [Google Scholar]
  • 45.Ramamurthi D., Chau C., Jackler R.K. JUUL and other stealth vaporisers: Hiding the habit from parents and teachers. Tob. Control. 2019;28:610–616. doi: 10.1136/tobaccocontrol-2018-054455. [DOI] [PubMed] [Google Scholar]
  • 46.Dai H., Hao J. Online popularity of JUUL and Puff Bars in the USA: 2019–2020. Tob. Control. :2020. doi: 10.1136/tobaccocontrol-2020-055727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Huff C., Tingley D. “Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Res. Politics. 2015;2 doi: 10.1177/2053168015604648. [DOI] [Google Scholar]
  • 48.Mellis A.M., Bickel W.K. Mechanical Turk Data Collection in Addiction Research: Utility, Concerns and Best Practices. Addiction. :2020. doi: 10.1111/add.15032. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Click here for additional data file. (87.7KB, pdf)

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to participant consent.

Articles from International Journal of Environmental Research and Public Health are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES