Characterization of Electronic Cigarette Warning Statements Portrayed in YouTube Videos

Abstract

Introduction

In 2018, the United States Food and Drug Administration (FDA) required that electronic cigarette (e-cigarette) manufacturers, packagers, importers, distributors, and retailers display an addictive or alternate warning statement on e-cigarette visual advertisements. Few studies have investigated the FDA-mandated and other warnings on social media. This study examined the prevalence and content of warning statements in e-cigarette-related YouTube videos.

Methods

In 2019, The Virginia Commonwealth University Center for the Study of Tobacco Products conducted bi-monthly (February-June) YouTube searches by relevance and view count to identify e-cigarette-related videos. Overall, 178 videos met the inclusion criteria. Staff coded each video for the presence of a visual/verbal warning statement, warning statement type (eg, FDA-mandated, addiction/tobacco, safety/toxic exposure, health effects), sponsorship, and tobacco product characteristics. A data extraction tool collected the video URL, title, upload date, and number of views, likes/dislikes, and comments.

Results

Only 5.1% of videos contained FDA-mandated and 21.9% contained non-mandated warnings. All videos with FDA-mandated and 46.2% of non-mandated warnings were represented visually. Only 13.1% of industry-sponsored videos uploaded after the mandate effective date had an FDA-mandated warning statement and videos with FDA-mandated and non-mandated (v. no) warnings had significantly fewer views, likes, dislikes, and comments. Among all non-mandated warnings, 31.3% featured an addiction/tobacco, 18.8% a safety/toxic exposure, and 37.5% a health effects warning.

Conclusions

The prevalence of FDA-mandated warning statements in e-cigarette related YouTube videos was low. FDA enforcement of the warning statement mandate on YouTube could increase the public’s understanding of the addictive nature of nicotine in e-cigarettes.

Implications

The FDA has the authority to regulate the advertisement and promotion of e-cigarettes on the Internet. These data can inform future FDA requirements related to the language content and visual representation of addiction/tobacco, safety/exposure, and health effects warning statements that appear in YouTube videos and other visual social media popular among young people. Such data would help consumers make informed decisions about purchasing e-cigarette products, using e-cigarettes, and avoiding unintentional harm related to e-cigarettes. In addition, these data may help social media platforms make decisions on whether they will prohibit advertisements that promote or facilitate the sale of tobacco products.

Introduction

Electronic cigarette (e-cigarette) use is increasing among young people^1–4 who often perceive e-cigarettes as having little to no harm.^5–7 However, e-cigarettes contain harmful chemicals like acetaldehyde⁸ and can emit^9,10 nicotine, a highly addictive, toxic chemical.¹¹ To increase public awareness about tobacco harms, the United States Food and Drug Administration (FDA) requires that tobacco products, including e-cigarettes, display warning statements that characterize their harm.¹² However, few studies have investigated if manufacturers, importers, packagers, and distributors who advertise and promote e-cigarettes have complied with the FDA warning statement mandate on packages and advertisements with visual components.

On May 10, 2016, the FDA extended its regulatory authority to e-cigarettes, cigars, hookah, pipes, dissolvable tobacco, and any future tobacco products (known as the “deeming rule”).¹² The deeming rule states that products made or derived from tobacco must include the statement, “WARNING: This product contains nicotine. Nicotine is an addictive chemical,” or the statement, “This product is made from tobacco,” if the product does not contain nicotine, but is made from tobacco.¹² FDA’s e-cigarette warning statement mandate was implemented on August 10, 2018. Prior to the mandate’s implementation, voluntary efforts by manufacturers to educate consumers about nicotine harms were limited. We previously found that while 97.9% of e-cigarette liquid bottles in a national sample had a warning statement, only 22.4% had a statement that the product “contained nicotine” and none stated that “nicotine is addictive.”  ¹³ Prior studies suggest that warnings modify consumers’ e-cigarette health risk beliefs,^14–16 but overall, there is limited research on the impact of the FDA-mandated or non-mandated warnings found on social media platforms like YouTube.

FDA stated that print and other advertisements with a visual component are required to show the mandated warning statement and provided examples, but not a comprehensive list of visual advertisements subject to its authority (ie, signs, shelf-talkers, webpages, e-mails).¹⁷ Other common means used to advertise/promote e-cigarettes include social media platforms such as YouTube, Instagram, and Twitter. In 2019, the FDA sent warning letters to companies that manufacture, advertise, sell, or distribute e-cigarette liquids for failing to include the mandated warning statement on Facebook, Instagram, and Twitter posts.^18,19 Thus, visual e-cigarette promotions/advertisements on social media, specifically those posted or sponsored by manufacturers, packagers, importers, distributors, and retailers of e-cigarette products are subject to the FDA warning statement mandate.

Although consumers do not purchase products directly from YouTube, it is the most widely used peer-to-peer advertisement platform globally, and young people are the largest consumers of YouTube content.²⁰ Unlike Facebook, which prohibits advertisements that promote or facilitate the sales or consumption of drugs, tobacco products (including e-cigarettes), and related paraphernalia,²¹ YouTube does not have these restrictions. Although the prevalence of YouTube e-cigarette advertising/promotion is unclear, limited research suggests that many e-cigarette-related YouTube videos are sponsored by the tobacco/e-cigarette industry.²² E-cigarette-related YouTube content may be generated by non-industry affiliated consumers (ie, e-cigarette enthusiasts) or consumers affiliated with manufacturers, packagers, importers, distributors, and retailers of e-cigarette products. Actors who post videos may be paid by a tobacco industry entity, but it is often difficult to determine tobacco industry connections with indirect advertisements.²³ It is possible that videos that are directly and indirectly sponsored (ie, video actors are paid to review and discuss the product) are under FDA’s regulatory authority and must comply with warning statement requirements (ie, occupy 20% of the visual advertisement/warning area, etc.).^12,24 Investigating whether YouTube videos comply with the mandate will provide valuable information to FDA who can determine whether/how it will enforce its warning statement requirement in social media visuals.

Of studies that examined e-cigarette-related YouTube videos, none were designed to examine the prevalence of warning statements.^22,25–33 Many prior studies focused on characterizing the portrayal of e-cigarettes/e-cigarette use^{25,26,30–32} and utilized data collected before the deeming rule went into effect.^{22,25–27,29–32} For example, studies conducted in 2013 found that 10.7% of e-cigarette related YouTube videos (n = 21) included any warning-related content,³⁰ and 13.6% and 10.1% of video tags, titles, or descriptions referenced health and safety, respectively.²⁹ A study conducted in 2014 found that 13% of YouTube videos with e-cigarette-related marketing claims included a health-related warning.³² A study that conducted a content analysis of e-cigarette in YouTube music videos from 2013–2017 did not identify any warnings.³⁴ A 2017 study found that consumers’ e-cigarette health risk beliefs were modified by an addiction warning in an e-cigarette advertisement and in turn, were less willing to try e-cigarettes.¹⁵ It is important that the prevalence of warning statements, especially FDA-mandated warnings, be documented on YouTube to inform potential FDA action. Similarly, several experimental and observational studies examined the effect of FDA-mandated and non-mandated warnings on tobacco use or cessation.^35–40 These studies highlight the potential impact of warnings on e-cigarette-related risk perceptions and the need for studies on e-cigarette warnings on social media where consumers may be exposed to e-cigarette promotions.

Consequently, both the prevalence and content of warning statements, especially FDA-mandated warnings, as they actually appear in YouTube videos are understudied. The purposes of this study were to systematically identify the (1) prevalence of FDA-mandated e-cigarette warning statements and (2) frequency and content of warning statements in e-cigarette-related YouTube videos given that FDA does not preempt or preclude e-cigarette manufacturers from using their own warnings. Data from this study can be used to determine whether YouTube postings are compliant with the FDA warning mandate and help determine if existing non-FDA mandated statements in visuals should remain voluntary, mandated, or prohibited. Moreover, data from this study could inform future regulation that provides more detailed guidance for visuals where e-cigarettes are promoted and/or advertised.

Methods

Sampling

In our prior study,⁴¹ we conducted preliminary YouTube searches using pre-identified search terms that refer to e-cigarette products.^42,43 Two search terms were added for data collection of a larger study. After pilot testing to maximize retrieval of e-cigarette-related videos, we identified the following search terms which yielded the most relevant and frequently viewed videos: “e-cigarettes,” “vaping,” “juul,” “e-juice,” and “e-liquid.” To limit the influence of cookies on results we utilized a web browser extension and customized internet data extraction tool (ie, scraper). We retrieved YouTube videos every two months/bimonthly over six months (February, April, June) in 2019, on the first business day of each month. To replicate consumer behaviors, we entered each search term and retrieved the first 20 videos sorted by 1) relevance, the relatedness to the search term, and 2) view count, the number of consumer views since the video was posted. During each search period, 10 YouTube searches were completed for the five search terms by relevance and view count. Overall, 600 videos were extracted.

After videos were entered into our database, we used the scraper to collect static and dynamic video characteristics. Static data included the video URL, title, and uploader alias while dynamic data included the number of views, likes, dislikes, and comments. We reviewed the first minute of each video for our exclusion criteria based on research that consumers spend an average of one minute on a website.⁴⁴ Videos were excluded if e-cigarettes, their parts, and/or components were not displayed or mentioned in the first minute. Videos were excluded if they were not in English; live-streamed; mentioned marijuana, demonstrated marijuana use within the first minute, or included marijuana in the title (n = 296). Duplicates (n = 122) and 4 videos were excluded because they were unavailable/no longer on YouTube during coding. After duplicate, unavailable, and irrelevant videos were removed (n = 422), 178 videos were retained for analysis.

Data Collection/Coding

Coding/codebook development procedures were informed by our prior study.⁴¹ Before coding, two senior researchers developed a four-hour training protocol and trained all staff responsible for data collection and coding. Two phases of coding were used. Phase I included team members who developed the codebook and completed coding for all variables. In phase I, two coders independently viewed and coded all videos using a codebook developed a priori. When there was disagreement, a third coder reviewed discrepancies, viewed, and coded the video. The team resolved conflicting codes and variables were recoded based on consensus weekly. All new codes identified were discussed weekly, defined, and added to the codebook until the final coding was completed. Phase II of coding was completed to categorize warning statements identified during data collection. Four coders reviewed and coded each warning in one of the following categories which were developed a priori: health effects, safety/toxic exposure, addiction/tobacco, or no warning statement. Weekly, the team resolved conflicting category codes, and warnings were recoded based on consensus.

Measures

General Video Characteristics

Videos were coded for the number of views, likes, dislikes, and comments to determine the potential appeal, novelty, and interest in the content and/or behaviors depicted.^45,46 We coded video upload date during three time periods: pre-deeming (prior to May 10, 2016), post-deeming to the warning statement mandate (May 10, 2016 to August 9, 2018); and warning statement mandate to data collection (August 10, 2018 to present).

Sponsorship

Sponsors were defined as a person/organization that provided funds for a project/activity carried out by another (ie, providing money/gifts). Team members recorded the presence and name of potential sponsors using the video/description box content. For example, sponsors were identified from the actors in the video stating that the video/displayed product was sponsored/provided by an e-cigarette manufacturer, packager, importer, distributor, or retailer; if the actors were members of the e-cigarette industry; and by reviewing video description box text and uploader alias/name. We classified sponsorships as 1) e-cigarette manufacturers, packagers, importers, distributors, or retailers (ie, SMOK, Suorin), 2) non-tobacco industry source (ie, Mayo Clinic, CBS), or 3) not sponsored/no sponsor identified. Intercoder reliability for initial coding of sponsorship was low (Krippendorff’s α =.17).

E-cigarette Measures

Team members recorded the type and brand of e-cigarette/tobacco products⁴⁷ (e-cigarette device, liquid, or component/part; loose tobacco) featured in the video (ie, visually shown or verbally mentioned/discussed at any time in the video). We also recorded the promotion of flavored liquids (yes/no); the presence of coils (yes/no); mention of FDA (yes/no); and mention of marijuana (yes/no).⁴¹ Intercoder reliability for initial coding of e-cigarette measures showed moderate to adequate agreement (Krippendorff’s α: product type, 0.38, number of liquid brands, 0.69, flavor promotion, 0.75, coils, 0.49, mention FDA, 0.47, or marijuana, 0.59). As stated, consensus coding and recoding were done to resolve conflicts.

Warning Statements

Team members coded whether a video presented warnings visually and/or verbally and documented all warnings, verbatim, each time it was presented in any video. Warning statements matching the FDA-mandated warning, verbatim, were classified as FDA-mandated while all other warnings were considered non-mandated. Intercoder reliability for initial team consensus coding of warning statements showed adequate agreement (Krippendorff’s α =.69).

To be classified as a warning statement, the descriptive information must have met our definitions developed a priori, been factual, and specific to e-cigarette products. Health effects warnings specified that the product (ie, device/ingredients) has some impact on disease/health conditions, reduces a health risk like cancer, or explicitly stated that the “product can be harmful to your health.” Health effects statements were explicit rather than implied (ie, the product causes cancer; this product is harmful to your health). We included the health effects of nicotine (ie, raises the heart rate, blood pressure) but not the immediate toxic/poisoning effects of nicotine. For example, a statement that too much nicotine at one time will make you sick was classified as a safety/toxic exposure statement because it addresses nicotine’s toxicity versus nicotine’s general health effects.

Safety/toxic exposure warnings specified that the product has some impact on safety or that exposure to the product, parts, or component have toxic effects (ie, batteries can explode; lock product away; toxic if contact with the skin). We differentiated between the immediate toxic effects of nicotine from other health effects similar to food allergy warnings (ie, this product contains peanuts). For example, the immediate toxic effects of nicotine include vomiting, diarrhea, dizziness, convulsions, etc.⁴⁸ Addiction/tobacco warnings specified that the product has addictive qualities, addictive chemicals, or contains tobacco. These statements were separate from the FDA-mandated warning category.

Analysis

Summary and bivariate (χ  ², Fisher’s exact, and Mann-Whitney U) statistics were generated in SAS 9.4 to assess warning statement prevalence and category, video characteristics, sponsorship, and e-cigarette measures by FDA-mandated warning statement status.

Results

General Characteristics of E-cigarette-Related YouTube Videos by FDA-mandated Warning Statement Status

Tables 1 and 21 display general characteristics of e-cigarette-related YouTube videos in our sample (n = 178) and videos uploaded after the mandate effective date (n = 61), respectively) by FDA-mandated warning statement status. Overall, 27.0% included any warning statement (5.1% FDA-mandated and 21.9% non-mandated). The median number of views was 827,408.50, 7,700 likes, 544.5 dislikes, 994 comments, and length of 8 minutes and 30 seconds. Around 15% of videos included a visually-represented warning, 36.0% were industry-sponsored, and 11.8% were non-industry-sponsored.

Table 1.

General Characteristics of E-cigarette Related YouTube Videos by FDA-Mandated Warning Statement Status (N = 178)

General characteristic	Total (N = 178)	Included FDA-mandated warning Statement n = 9 (5.1%)	Included non-mandated warning statement n = 39 (21.9%)	No warning statement n = 130 (73.0%)
Number of views1
Total number	544,657,180	3,470,564	37,885,730	503,300,886
Median	827,408.50	99,809.00a	439,749.00a	1,105,493.50b
Interquartile range	2,707,190.00	621,700.00	1,118,538.00	3,516,502.00
Range	2,157–39,436,103	15,639–1,086,091	3,416–5,792,032	2,157–39,436,103
Number of likes1
Total number	6,398,166	84,528	433,991	5,879,647
Median	7,700.00	1461.00ab	3900.00a	10184.00b
Interquartile range	24,000.00	8,700.00	8,264.00	36,500.00
Range	23–671,000	259–45,395	23–85,000	88–671,000
Number of dislikes1
Total number	429,365	3,929	46,746	378,690
Median	544.50	65.00a	352.00a	603.50b
Interquartile range	2,906.00	905.00	1,240.00	3,595.00
Range	1–40,000	14–1,533	2–9,700	1–40,000
Number of comments13
Total number	646,292	17,213	58,266	570,813
Median	994.00	293.00ab	687.50a	1120.50b
Interquartile range	2,909.00	1,295.00	1,319.00	3,985.00
Range	4–161,777	61–12,223	4–11,884	16–161,777
Duration in minutes and seconds2
Total number	26 h 45 min 15 sec	1 h 40 min 48 sec	7 h 7 min 2 sec	17 h 57 min 25 sec
Median	8:30	11:02a	10:02ab	7:43b
Interquartile range	6:50	1:31	7:11	6:25
Range	0:18–51:29	5:55–15:13	0:56–51:29	0:18–42:20
	% (n)	% (n)	% (n)	% (n)
Warning statement representation3
Visual only	12.9 (23)	100.0 (9)	35.9 (14)	—
Verbal only	11.8 (21)	0	53.9 (21)	—
Visual and verbal	2.2 (4)	0	10.3 (4)	—
No warning statement	73.0 (130)	—	—	100 (130)
Sponsored video3
Yes	47.8 (85)	55.6 (5)a	43.6 (17)a	48.5 (63)a
No	52.3 (93)	44.4 (4)	56.4 (22)	51.5 (67)
Sponsor type3
E-cigarette industry	36.0 (64)	44.4 (4)a	15.4 (6)b	41.5 (54)a
Non-industry	11.8 (21)	11.1 (1)	28.2 (11)	6.9 (9)
No sponsor	52.2 (93)	44.4 (4)	56.4 (22)	51.5 (67)
	% (n)	% (n)	% (n)	% (n)
Total number of videos uploaded3
Pre-deeming (prior to May 10, 2016)	16.9 (30)	0a	25.6 (10)b	15.4 (20)c
Post-deeming to mandate (May 10, 2016—August 9, 2018)	48.9 (87)	11.1 (1)	28.2 (11)	57.7 (75)
Post-warning statement mandate (August 10, 2018—present)	34.3 (61)	88.9 (8)	46.2 (18)	26.9 (35)

Warning statements categories include FDA-mandated warning statements and author-defined, non-mandated warning statements. Two videos with FDA-mandated warning statements also displayed 1 non-mandated warning statement. Values with different superscripts in each FDA-mandated warning statement category indicate statistically significant differences for that variable. Column percents are reported. Comparisons of videos with FDA-mandated, non-mandated, and no warning statements were based on the following statistical tests¹: Mann-Whitney U,² χ  ² or Fisher’s exact.³The number of comments for 5 videos were missing and the sample sizes are as follows: n = 38 for videos with warnings and n = 126 for videos without any warning statements. H, hours, min, minute, sec, seconds.

Table 2.

General Characteristics of E-cigarette Related YouTube Videos Uploaded After the Mandate Effective Date by FDA-Mandated Warning Statement Status (N = 61)

General characteristic	Total (N = 61)	Included FDA-Mandated Warning Statement n = 8 (13.1%)	Included Non-mandated Warning Statement n = 18 (29.5%)	No Warning Statement n = 35 (57.4%)
Number of views1
Total number	102,918,585	2,818,757	14,024,924	86,074,904
Median	483,467.0	65,065.0a	201,260.0a	1,067,784.0b
Interquartile range	1,786,139.0	734,956.5	541,538.0	2,733,999.0
Range	4,705–12,941,092	15,639–1,086,091	4,705–4,858,834	19,305–12,941,092
Number of likes1
Total number	1,797,349	66,528	217,754	271,879
Median	8,200.0	1,402.0a	3,600.0a	17,000.0b
Interquartile range	28,264.0	7,365.0	6,464.0	39,400.0
Range	26–272,000	259–45,395	26–83,000	121–272,000
Number of dislikes1
Total number	106,086	2,829	25,218	78,039
Median	501.0	53.5a	163.0a	640.0b
Interquartile range	2,932.0	540.5	1,662.0	3,566.0
Range	8–9700	14–1,533	8–9,700	28–8,900
Number of Comments13
Total number	162,654	15,728	26,053	120,873
Median	1,100.0	254.0a	526.0a	1,665.5b
Interquartile range	4,751.0	1,153.0	983.0	4999.0
Range	4–15,911	61–12,223	4–9,654	30–15,911
Duration in minutes and seconds2
Total number	10 h 24 min 57 sec	1 h 30 min 26 sec	3 h 33 min 48 sec	5 h 20 min 43 sec
Median	10:09	11:16a	10:25ab	9:32b
Interquartile range	6:17	2:25	2:24	11:37
Range	0:18–51:29	5:55–15:13	1:53–51:29	0:18–19:54
	% (n)	% (n)	% (n)	% (n)
Warning statement representation3
Visual only	27.9 (17)	100.0 (9)	50.0 (9)	—
Verbal only	11.5 (7)	0	38.9 (7)	—
Visual and verbal	3.3 (2)	0	11.1 (2)	—
No warning statement	57.4 (35)	—	—	100 (35)
Sponsored video3
Yes	54.1 (33)	62.5 (5)a	66.7 (12)a	45.7 (16)a
No	45.9 (28)	37.5 (3)	33.3 6)	54.3 (19)
Sponsor type2
E-cigarette industry	36.1 (22)	50.0 (4)a	33.3 (6)a	34.3 (12)a
Non-industry	18.0 (11)	12.5 (1)	33.3 (6)	11.4 (4)
No sponsor	45.9 (28)	37.50 (3)	33.3 (6)	54.3 (19)

h, hour, min, minute, sec, second. Warning statements categories include FDA-mandated warning statements and author-defined, non-mandated warning statements. Two videos with FDA-mandated warning statements also displayed 1 non-mandated warning statement. Values with different superscripts in each FDA-mandated warning statement category indicate statistically significant differences for that variable. Column percents are reported. Comparisons of videos with FDA-mandated, non-mandated, and no warning statements were based on the following statistical tests¹: Mann-Whitney U,² χ  ² or Fisher’s exact.³The number of comments for 2 videos were missing and the sample sizes are as follows: n = 17 for videos with non-mandated warnings and n = 34 for videos without any warning statements. H, hours, min, minute, sec, seconds.

Videos with FDA-mandated and non-mandated (v. no) warnings had fewer views and dislikes. Videos with FDA-mandated (v. no) warnings were longer and videos with non-mandated (v. no) warnings had fewer likes and comments (Table 1). These patterns persisted among videos uploaded after the mandate effective date (Table 2), although all videos with (v. without) warnings also had fewer likes and comments. All FDA-mandated warnings were presented visually, while 35.9% of non-mandated warnings were only presented visually and 10.3% included both visual and verbal presentations. Videos with FDA-mandated and no (v. non-mandated) warning statements were significantly more likely to be sponsored by e-cigarette/tobacco (44.4% v. 41.5% v. 15.4%, ps < .05). Overall, 16.9% of videos in our sample were uploaded before the deeming rule was issued (prior to May 10, 2016), 48.9% were uploaded between the issuance of the deeming rule and the mandate effective date (May 10, 2016 to August 9, 2018), and 34.3% were uploaded after the warning statement mandate went into effect (August 10, 2018 to present). Additionally, 13.1% and 29.5% of videos uploaded after the mandate effective date included FDA-mandated and non-mandated warnings, respectively. Videos with FDA-mandated and non-mandated (v. no) warnings were significantly more likely to be uploaded after the mandate effective date (August 10, 2018 to present) (Table 1). Among all industry-sponsored videos (n = 64), only 6.3% and 9.4% were uploaded after the mandate effective date and included FDA-mandated and non-mandated warnings, respectively (data not shown). Although no difference was found in sponsorship by FDA-mandated warning statement status, 50.0% of videos uploaded after the mandate effective date with FDA-mandated and 33.3% with non-mandated warnings were industry-sponsored, were represented visually, and as such were subject to the mandate. Conversely, among industry-sponsored videos uploaded after the mandate went into effect (n = 22), only 18.2% had an FDA-mandated and 27.3% had a non-mandated warning (Table 2).

Products Promoted in E-cigarette-Related YouTube Videos Uploaded After the Mandate Effective Date by FDA-mandated Warning Statement Status

Table 3 displays products promoted in videos uploaded after the mandate effective date by FDA-mandated warning statement status. At least 75% of videos uploaded after the mandate effective date, irrespective of warning statement status, featured e-cigarette devices. Fourty-five (no warning) to 87% ( FDA-mandated warning) of videos featured liquids and less than one-eighth featured loose tobacco. Eleven e-cigarette liquid brands were displayed in videos with FDA-mandated warnings and 23 brands were displayed in videos with non-FDA mandated compared to 35 brands in videos without warnings. At least 40% of videos with an FDA mandated, or non-mandated warning promoted a flavor compared to 20% of videos without warnings. No videos with FDA-mandated warnings featured coils or mentioned marijuana, but around 20% of other videos featured coils and less than 10% mentioned marijuana. Lastly, at least 25% of videos with warnings mentioned the FDA compared to only 14.3% of videos without warnings.

Table 3.

Products Promoted in E-cigarette Related YouTube Videos Uploaded After the Mandate Effective Date by FDA-Mandated Warning Statement Status (N = 61)

	Included FDA-Mandated Warning Statement N = 8 (13.1%)		Included Non-mandated Warning Statement N = 18 (29.5%)		No Warning Statement N = 35 (57.4%)
	n	%	n	%	n	%
Type of E-cigarette product1
E-cigarette device	6	75.0	18	100	33	94.3
E-cigarette liquid	7	87.5	15	83.3	16	45.7
Loose tobacco	0	0	2	11.1	2	5.7
Total # of e-cigarette liquid brands displayed in all videos	11	—	23	—	35	—
1+ flavor promoted (yes)	4	50.0	8	44.4	7	20.0
Promoted coils (yes)	0	0	4	22.2	6	17.1
Mention of FDA (yes)	2	25.0	5	27.8	5	14.3
Mention of marijuana (yes)	0	0	1	5.6	3	8.6

FDA, U.S. Food and Drug Administration.¹ Videos could promote more than 1 e-cigarette related product. E-cigarette liquid category included nicotine/tobacco-based liquids. Column percents are reported.

Number of Warning Statement Types in YouTube Videos

Figure 1 depicts the number of warnings by category and visual/verbal representation. We identified 64 warning statements, 43 were unique, and 5 were presented more than once across the 48 videos. Specifically, 31.3% of warnings were classified as health effects, 14.1% as safety/toxic exposure, 40.6% as addiction/tobacco, and 14.1% included an FDA-mandated warning. Coded warning statements are presented verbatim by representation, sponsorship, and type in Supplementary Table S1.

Figure 1.

Number of warning statements identified in E-cigarette relevant YouTube videos by warning statement category and visual/verbal representation (n = 64).

This figure shows the number of warning statements that were coded as health effects, safety/toxicity, addiction/tobacco, or FDA-mandated warning statements. The bar colors/patterns represent how the warning statement appeared in the video: visually, verbally, or visually and verbally. Videos could have multiple warning statements and several warning statements appeared in multiple videos. However, each individual warning statement was assigned to a single warning statement category and we reported its visual/verbal representation.

Discussion

This study is the first to primarily focus on the prevalence and content of e-cigarette-related warning statements on YouTube. Only 5.1% of YouTube videos in our sample depicted the FDA-mandated and 21.9% depicted non-mandated warnings. All FDA-mandated and 46.2% of non-mandated warnings were presented visually. Only 15.6% of e-cigarette/tobacco industry-sponsored videos included any warning statement and 18.2% of industry-sponsored videos uploaded after the mandate went into effect included an FDA-mandated warning. These findings suggest that there was low compliance before and after the FDA mandate went into effect. Our data provide the FDA with information to increase its oversight of internet-based social media advertisements and enforce compliance with the e-cigarette warning statement mandate.

Our data on the prevalence and type of warning statements displayed before and after the FDA mandate are consistent with FDA’s research priority to examine their potential impact on regulatory actions. Videos with (v. without) warnings uploaded after the mandate effective date had fewer views, likes, dislikes, and comments. This is consistent with a 2014 study where less than 5% of “pro” e-cigarette-related YouTube videos, but all (n = 3) “anti” e-cigarette videos included warning-related content. “Pro” e-cigarette videos had more views, favorites, and likes than “anti” e-cigarette videos which had more dislikes and comments.³⁰ YouTube video popularity and engagement changes rapidly and other differences, aside from warning statements, may contribute to these differences. Research is needed to clarify the extent to which warning statements decrease consumer appeal of e-cigarette-related YouTube videos.

We found that 40% of identified warning statements were classified as an addiction/tobacco statement and less than 15% were FDA-mandated regardless of upload date. The lack of addiction warnings could leave viewers, especially young people, vulnerable to using e-cigarettes and/or misperceptions of e-cigarette-related harms and addictiveness. Although few studies have examined the effects of the FDA-mandated warning after the mandate effective date, research indicates that warning statements influence consumers’ risk perceptions and possible e-cigarette use. Those who viewed tweets from a fictious e-cigarette brand with (v. without) FDA-mandated warnings were less likely to view the brand as healthy and 60–70% later recalled the warning statement.³⁵ The FDA-mandated warning (vs. an industry-generated warning) increased college student’s perceived risks of e-cigarette use and in turn reduced their intentions to use e-cigarettes.³⁷ Those who viewed FDA-mandated warnings on e-cigarette packaging, especially non-smokers, reported higher risk perceptions.³⁶ Similarly, health warnings (including nicotine/addiction statements) were more influential than e-cigarette flavor, price, and nicotine content in predicting participants’ intention to try e-cigarettes and perceptions of reduced harm,³⁸ while modified FDA-mandated warning statements elicited high intentions to quit vaping.⁴⁰ These studies suggest that requiring FDA-mandated warnings in YouTube videos could increase the public’s exposure to the warning statement and prevent misperceptions regarding the risks of e-cigarette products.

Similar to our previous study,¹³ we identified safety/toxic exposure statements (14%). Previous studies found that safety/toxic warnings warned people about possible e-cigarette risks, made them think twice about using e-cigarettes,⁴⁹ raised interest in quitting e-cigarettes, and increased agreement that e-cigarettes contain dangerous chemicals and could be dangerous to health.⁵⁰ The Deeming Rule did not mandate e-cigarette exposure warning statements,¹² but examining safety/exposure statements will help identify statements that can protect the public from toxic nicotine exposures. E-cigarette manufacturers are not preempted or precluded from using their own warnings on e-cigarette products provided they comply with FDA requirements. Our examination of non-mandated warnings helps fill a critical gap regarding warning statements not included in the 2016 deeming rule. Future research should continue to examine the effects of real-life and hypothetical non-mandated warning statements to inform mandates FDA could issue in the future.

Our findings also revealed insights into mandate compliance. Theoretically, YouTube content creators could have been aware of the warning statement mandate and eventual need for compliance, but our findings indicate most did not voluntarily comply. Only 50% and 33.3% of industry-sponsored videos uploaded after the mandate effective date (August 10, 2018 to present) had FDA-mandated or nonmandated warning statements, respectively. Moreover, many videos with warnings uploaded during this period featured e-cigarette liquids/liquid packaging or promoted flavors. These findings suggest that the mandate effective date influenced the number of videos uploaded with a warning, although most videos did not comply. Thus, if the FDA explicitly applies and enforces the warning statement mandate on YouTube, especially videos promoting e-cigarette liquids, more videos may display mandated and non-mandated warnings. However, there are unique challenges in determining how to enforce the mandate on YouTube.

Although FDA’s regulatory authority includes industry-sponsored content, influencers and consumers can practice free-speech that might otherwise violate the law and/or perpetuate misperceptions regarding the addictiveness of nicotine in e-cigarette products. FDA will need to determine whether e-cigarette-related content on YouTube is directly/indirectly sponsored by the e-cigarette/tobacco industry or if consumers are acting as “influencers” on behalf of the industry. FDA could require YouTube to remove industry-sponsored content that do not comply with FDA regulations to protect the public, especially young people. Alternatively, FDA could require YouTube to enforce required disclosure statements and/or increase the visibility of disclosure statements on sponsored content. Investigations of YouTube compliance with the warning statement mandate will provide valuable information to FDA who can determine if and/or how it will apply and enforce its warning statement requirement on social media. Studies are needed to determine the potential impact of the FDA warning statement mandate on social media platforms, including YouTube.

Limitations

Analyses did not adjust for the video upload date. Thus, videos with (v. without) warnings, which were more likely to have been uploaded on/after the mandate effective date, may have had less time to accrue measures of consumer engagement. However, among videos without warnings uploaded after the deeming rule was issued, 100% were uploaded within one year and 74.3% were uploaded within 6 months of the mandate effective date. Our sample was not limited to industry or U.S-produced content as data are collected for a larger study. Although the FDA does not have jurisdiction over non-industry/non-U.S. content, consumers encounter a range of content. We coded for sponsorship, but some sponsorships may not have been disclosed. We did not review the content of comments, which may provide additional insights into the role of warnings on video appeal. Future research should consider such examinations, potential confounding, and interaction effects with time/upload date. Although warnings can change over time, this study is the first to identify and examine the content of e-cigarette-related YouTube videos for warning statements and analyze data collected over 6 months.

Conclusions

In summary, most e-cigarette-related YouTube videos did not show an FDA-mandated warning statement. This is problematic because young people are the largest consumers of YouTube and use e-cigarettes in epidemic proportions. FDA has expressed an interest in understanding the potential impact of FDA regulatory actions. FDA has the authority to regulate online advertisements/visuals of parts/components under their jurisdiction. In warning letters, FDA expressed that social media posts (made on behalf of e-cigarette companies) with labeling and/or advertising for e-cigarette products were misbranded because they failed to include the FDA-mandated warning statement.¹⁸ However, these violations were for posts on Facebook, Instagram, and Twitter.¹⁸ FDA must determine how to enforce the mandate on industry-posted/sponsored content on YouTube, if at all. Studies are needed to examine how consumers perceive warning statements and if visible, what the impact of non-mandated warnings are on risk perceptions and e-cigarette use. Data from this study are available to the FDA and the field to expeditiously inform modifications to mandates or create new regulations to protect the public’s health.

Supplementary Material

A Contributorship Form detailing each author’s specific involvement with this content, as well as any supplementary data, are available online at https://academic.oup.com/ntr.

Funding

This research is supported by grant number U54DA036105 from the National Institute on Drug Abuse of the National Institutes of Health and the Center for Tobacco Products of the U.S. Food and Drug Administration. The content is solely the responsibility of the authors and does not necessarily represent the views of the NIH or the FDA.

Declaration of Interests

Thomas Eissenberg is a paid consultant in litigation against the tobacco industry and the electronic cigarette industry and is named on a patent describing a method for measuring e-cigarette user puff topography. Thomas Eissenberg and Eric Soule are named on a patent for a smartphone app that determines electronic cigarette device and liquid characteristics. The other authors declare they have no conflicts of interest.