Managing a Policy Paradox? Responses to Textual Warning Labels on E-Cigarette Advertisements Among U.S. National Samples of Youth Overall and Adults Who Smoke or Vape

Abstract

Context:

Current use and potential future uptake of e-cigarettes among youth remain public health concerns in the U.S., even as people who smoke combustible cigarettes could benefit from switching completely to e-cigarettes. The U.S. Food and Drug Administration (FDA) is considering alternative warning messages, but warnings that discourage youth from use may also deter people who smoke from switching. This study tests ten pre-registered hypotheses on effects of warning messages with national samples of youth overall and adults who smoke and/or vape.

Methods:

NORC recruited 1,639 adults (ages 18+) who smoke, vape, or use both products, from their probability-sampled AmeriSpeak Panel and augmented their AmeriSpeak Teen Panel with Lucid’s nonprobability opt-in panel to recruit 1,217 youth (ages 14-17) to participate in a web-based survey experiment. We randomly assigned respondents to view one of five warning label conditions and respond to measures of their e-cigarette risk beliefs, willingness to use e-cigarettes, and (among people who smoke or vape) considerations to quit these products.

Findings:

Relative to the current FDA warning about nicotine, warning messages about the harms of e-cigarette use for youth brain development did not influence risk beliefs or reduce willingness to use these products among youth. Brain development warning messages did increase beliefs about these harms among adults but did not increase quit considerations among people who vape, relative to the FDA warning. Warning messages with information about chemical constituents of vaping products and the harm of these chemicals produced higher e-cigarette quit considerations than did the FDA warning among adults who vape.

Conclusion:

Potential alternative warning label messages were largely ineffective relative to the current FDA warning about nicotine, though limited evidence suggests some potential for chemical + harm messaging to encourage people who use both e-cigarettes and cigarettes to consider quitting both.

INTRODUCTION

Trial and use of e-cigarettes among youth who do not smoke combustible cigarettes remain major public health concerns in the United States. E-cigarettes are the most commonly used tobacco product among both high-school and middle-school students, and both the US Food and Drug Administration (FDA) and US Surgeon General consider e-cigarette use among youth to be at troubling levels.^1-2 Though levels of use have declined since their peak in 2019, 12.2% of high school students and 5.6% of middle school students in 2023 reported using e-cigarettes in the past 30 days, totaling over 2.1 million youth who currently vape.³ The FDA, Surgeon General, and US Centers for Disease Control and Prevention (CDC) continue to conclude that widespread use of e-cigarettes by youth poses a variety of risks to health and well-being that include nicotine addiction, harm to the developing brain (which develops until about age 25), and exposure to harmful chemicals that include carcinogens and heavy metals.^4-6

While evidence is still emerging, analyses suggest there would be large public health benefits if adults who smoke combustible cigarettes switched completely to e-cigarettes.⁷ While the FDA has not approved e-cigarettes as an evidence-based treatment for smoking cessation, the CDC describes e-cigarettes as less harmful than regular cigarettes and acknowledges their potential benefit as a substitute for smoking.⁸ The FDA authorized the marketing of some vaping products based on evidence that these products could benefit people who switch completely or significantly reduce their cigarette smoking.⁹ Other health authorities like the UK National Health Service go further in stating that e-cigarettes can help people who smoke quit for good.¹⁰ Despite the potential benefits of switching to e-cigarettes, 11.5% of US adults (28.3 million people) smoke cigarettes every day or some days, compared to 4.5% of adults who regularly vape e-cigarettes.¹¹ While many adults who smoke have tried e-cigarettes, and a substantial proportion of people who recently quit smoking report regular use of e-cigarettes,¹² many adults who continue to smoke hold false beliefs that e-cigarettes are more or equally harmful as combustible cigarettes,¹³ beliefs which may prevent efforts to switch.

Exposure to e-cigarette product marketing is widespread among middle and high school youth.^14-15 E-cigarette advertising (ad) exposure may play a significant role in promoting e-cigarette trial and use.^16-17 Increasing the strength of warning labels on ads or packaging for other tobacco products can reduce the appeal of these products among young people,^18-19 suggesting that they may also be effective at curbing e-cigarette use. However, placing stronger health warnings on e-cigarette ads to discourage youth from their use may also deter people who smoke from switching to e-cigarettes, thereby limiting overall population health gains. Warnings thus must communicate the risks of initiating e-cigarettes while also protecting beliefs (or changing misperceptions) about the benefits of switching to e-cigarettes among people who smoke. These competing considerations are a challenge for regulators and health communicators.

Studies have gauged responses to different e-cigarette warning or prevention message themes among youth and young adults that include risks of nicotine addiction,^20-25,27-28 harms to the developing brain,^20,22,24 toxic/chemical content of e-cigarette liquid and vapor,^22-26 health consequences of vaping,^22-25,27-28 and/or similarity to cigarettes/cigarette smoke.^23,25 While at least one of these studies suggests promise for each theme in deterring youth, most rely on convenience samples.^{20,22-24,26-28} A parallel body of work has found mixed patterns of response among adults who smoke and/or vape to e-cigarette risk or harm messaging featuring themes of nicotine addiction,^29-30,32-35 toxic/chemical content of e-cigarettes,^30-31,35 health consequences of vaping,^30,32,35 and/or reduced harm relative to cigarettes.^33-34 Each of these studies focusing on responses among adults who vape or smoke also relied on convenience samples.^29-35

The current study assesses responses, among youth overall and among adults who smoke and/or vape, to five different e-cigarette warning message themes placed in the context of ads for vaping products. We compare the current standard, the FDA’s mandated warning on nicotine addiction, to two other classes of warning message themes – those describing potential harms of nicotine exposure to youth cognitive development and those describing chemical constituents of e-cigarette aerosols – that are prominent foci of public messaging about e-cigarette risks by the US Surgeon General, CDC, and FDA.^36-38 While both themes have been studied previously, most studies suggesting their utility in preventing youth uptake or promoting adult cessation of vaping (a) rely on perceived message effectiveness [self-rated impact]^{22-23,25,32,35} versus actual effectiveness [comparing outcomes after randomized exposure to different messages]^24,28,30 and/or (b) test their impact in isolation [either a textual warning message^22,25,30,35 or textual messages combined with anti-vaping imagery^{23,28,30,32,35}] versus embedded within e-cigarette advertising messages and imagery [in which they must compete for attention and impact]²⁵. Within the chemical constituents theme, we further explore the impact of three variations of messaging: messages describing chemical constituents only,^22-23,25,30 those describing chemicals + their harms,^{23-24,28,30,32,35} and those describing chemicals + their harms with an additional statement suggesting their equivalence to smoking,²³ the latter out of concern that implied equivalency messaging could be detrimental for efforts to convince people who smoke or dual-use to switch completely to vaping.¹³ In doing so, we address limitations of prior work in which no studies have directly compared responses to brain development and chemical constituent message themes between youth overall and adults who smoke and/or vape, a critical omission given a need to balance youth vaping prevention with adult smoking cessation.

We build on previous research by testing ten pre-registered hypotheses using national samples of youth (ages 14-17, combining probability and non-probability sampling) and national probability samples of adults (ages 18+) who smoke, vape, or do both, to offer new evidence on the potential promise and pitfalls of various e-cigarette warning messages for public health promotion. We predict that warning themes focused on cognitive development, chemical constituents, and health harms (whether or not making a comparison to cigarettes) will shift beliefs about the harms of e-cigarettes relative to the current FDA warning focused on nicotine addiction (H1 through H3); warning messages about youth cognitive development will outperform other themes among youth (H4 through H6); cigarette equivalence messaging will undermine efforts to promote switching among people who smoke (H7 and H8); chemical and harm message themes will encourage quit considerations among people who vape (H9); and messages about harms to cognitive development among adolescents will resonate more strongly among those ages ≤25 compared to older respondents (H10; see Table 1 for all hypotheses; see pre-registration here: https://osf.io/2bcfw/?view_only=30d2809856ba47539858a32eb30ed40b).

Table 1.

Pre-Registered Hypotheses and Summary of Findings

Hypothesis	Full Wording	Supported?
H1	Respondents exposed to warning label messages describing harmful effects of e-cigarettes on youth cognitive development (condition 2) will report greater risk of e-cigarettes on youth than respondents exposed to the FDA approved warning label on nicotine addiction (condition 1).	Yes, but only in adults who smoke and/or vape
H2	Respondents exposed to warning label messages describing chemical constituents of e-cigarettes (conditions 3-5) will report greater absolute risks of e-cigarettes than respondents exposed to the FDA approved warning label on nicotine addiction (condition 1).	No
H3	Respondents exposed to warning label messages describing chemical constituents of e-cigarettes, their health effects, and the equivalence to cigarette smoke (condition 5) will report greater relative risks of e-cigarettes compared to combustible cigarettes than respondents exposed to the FDA approved warning label on nicotine addiction (condition 1)	No
H4	Youth exposed to warning label messages describing harmful effects of e-cigarettes on youth cognitive development (condition 2) will report lower willingness to use e-cigarettes in the future than youth exposed to the FDA approved warning label on nicotine addiction (condition 1).	No
H5	Youth exposed to warning label messages describing chemical constituents of e-cigarettes (conditions 3-5) will report lower willingness to use e-cigarettes in the future than youth exposed to the FDA approved warning label on nicotine addiction (condition 1).	No
H6	Youth exposed to warning label messages describing harmful effects of e-cigarettes on youth cognitive development (condition 2) will report lower willingness to use e-cigarettes in the future than youth exposed to warning label messages describing chemical constituents of e-cigarettes (conditions 3-5).	No
H7	Adults who smoke who are exposed to warning label messages describing chemical constituents of e-cigarettes, their health effects, and the equivalence to cigarette smoke (condition 5) will report (a) greater relative risks of e-cigarettes compared to combustible cigarettes, and (b) lower perceived health benefits to switching from combustible cigarettes to e-cigarettes, than adults who smoke who are exposed to the FDA approved warning label on nicotine addiction (condition 1).	No
H8	Adults who smoke who are exposed to warning label messages describing chemical constituents of e-cigarettes, their health effects, and the equivalence to cigarette smoke (condition 5) will be less likely to report wanting to use e-cigarettes to help them quit smoking than adults who smoke who are exposed to the FDA approved warning label on nicotine addiction (condition 1).	No
H9	Adults who vape who are exposed to warning label messages describing chemical constituents of e-cigarettes (conditions 3-5) will report greater intentions to quit using e-cigarettes than adults who vape who are exposed to the FDA approved warning label on nicotine addiction (condition 1).	Yes, but only for chemicals + harm
H10	Among people who vape, effects of warning label messages describing harmful effects of e-cigarettes on youth cognitive development (condition 2 versus condition 1) on intentions to quit using e-cigarettes will be stronger among youth and young adults (ages 25 and younger) than older adults (ages 26 and older).	No

METHODS

Data Sources

We contracted with NORC to recruit samples of approximately 1,200 youth (ages 14-17) without regard to current tobacco product use and 1,500 adults (ages 18 and older) in three categories: people who only smoke cigarettes (target N = 500), people who only vape e-cigarettes (target N = 500), and people who smoke and vape (target N = 500). For youth, NORC combined data from their AmeriSpeak Panel (N = 423) and Lucid’s nonprobability online opt-in panel (N = 794). For adults who smoke and/or vape, we sampled only from the AmeriSpeak Panel (N = 1,639). AmeriSpeak is a probability-based panel designed to represent the US household population. Randomly selected US households are sampled using area probability and address-based sampling. Sampled households (covering 97% of US households) are then contacted by US mail, telephone, and field interviewers in person. Most AmeriSpeak households participate in online surveys via computer or smartphone, and we restricted our study to those with internet access since it involved visual images (ads). Lucid maintains a non-probability panel of adults and offers them invitations to participate in online surveys. NORC’s Institutional Review Board (IRB) reviewed and approved the studies and Cornell University’s IRB deemed both studies exempt by accepting NORC’s protocols and the transfer of de-identified data.

Youth sample.

NORC invited members of the AmeriSpeak Teen Panel to participate in the study after first obtaining parental consent to initiate contact. NORC also invited all parents in the AmeriSpeak Panel who have a child between ages 14-17 (regardless of whether the child was already part of the Teen Panel) to consent for NORC to contact their teen. NORC randomly sampled within a household if more than one teen was eligible. Lucid also pre-screened its panelists to identify parents of 14–17-year-olds and asked for consent to invite their teen to the survey. Quota-based invitations to eligible Lucid panel teens aimed to establish an overall sample (AmeriSpeak and Lucid combined) approximating US Census proportions by age, race, ethnicity, geography, and parental education. Respondents received $10 for completing the survey, which took 13 minutes (median). Prior to seeing the message stimuli, teens were encouraged to take the survey when they had privacy to reduce the possibility of parental surveillance of sensitive responses. Data collection began on September 10^th, 2021 and the study closed on October 19^th, 2021, yielding a final sample of N=1,217. The analytic sample excluded respondents who completed the interview in less than 1/3 of the median duration (speeders), those who skipped more than half the questions (skippers), and those who answered the same for every grid question shown (straight-liners). The weighted cumulative response rate (AAPOR RR3) for the AmeriSpeak sample component was 3.5%, combining the household panel recruitment rate (19.6%), household panel retention rate (78.8%), parent consent completion rate (38.3%), and teen survey completion rate (58.4%). Supplemental Appendix Table A1 shows ns and weighted/unweighted percentages for youth sample demographics and tobacco product use.

Adult sample.

NORC used previously collected screening information on smoking and vaping status (those who reported smoking and/or vaping every day or some days) to identify eligible AmeriSpeak panelists and set a quota for each tobacco user group toward a target of about 500 participants per group. We confirmed smoking/vaping status at the start of the survey. Respondents received the cash equivalent of $3 for completing the survey, which took 17 minutes (median). Data collection ran from September 10^th, 2021, to October 18^th, 2021, and yielded final samples of N = 565 people who only smoke, N = 514 people who only vape, and N = 560 people who smoke and vape (overall N = 1,639). This sample excluded speeders, skippers, and straight-liners as described above. The weighted cumulative response rate (AAPOR RR3) for the adult sample was 2.8%, combining the household panel recruitment rate (19.1%), household panel retention rate (75.1%), tobacco use screener completion rate (37.2%), and survey completion rate (53.0%). Supplemental Appendix Table A2 shows ns and weighted/unweighted percentages for adult sample demographics and tobacco use behavior.

Experimental Conditions

All respondents were randomly assigned to view one of five message theme conditions, each of which featured three different warning label messages placed on an ad for an e-cigarette product (see Table 2 for full text). Condition 1 (FDA warning) showed respondents three different ads, each including the current FDA-mandated warning message on nicotine addiction. The other four conditions also began with an ad that included the FDA warning (reasoning that new warnings would likely accompany rather than replace the current warning) followed by two different ads with condition-specific messages. Messages were based on an extensive review of public statements about e-cigarette and vaping risks from US federal authorities (FDA, CDC, Surgeon General) to ensure that at least one US governmental authority had established each claim as having a reasonable evidentiary basis. Condition 2 (youth brain development) emphasized the potential harms of exposure to nicotine during youth and young adulthood on the developing brain. Condition 3 (chemical constituents) emphasized that vaping can expose users to toxic or harmful chemicals. Condition 4 (chemical constituents + harms) described the same chemicals as Condition 3 but added a sentence on health effects of these chemicals on the body. Condition 5 (chemical constituents + harms + cigarette equivalence) featured the same content as condition 4 warnings but added that these chemicals are “also found in cigarette smoke.”

Table 2.

Full Warning Text for Each Randomized Condition

Condition 1: FDA	This product contains nicotine. Nicotine is an addictive chemical. This product contains nicotine. Nicotine is an addictive chemical. This product contains nicotine. Nicotine is an addictive chemical.
Condition 2: FDA +Youth Brain Development	This product contains nicotine. Nicotine is an addictive chemical. This product contains nicotine. Exposure to nicotine during youth and young adulthood can harm the developing brain. This damage can have long-term effects on memory, attention, and mood. This product contains nicotine. Exposure to nicotine during youth and young adulthood can alter cell activity in the developing brain. This change can worsen anxiety and irritability.
Condition 3: FDA + Chemical Constituents	This product contains nicotine. Nicotine is an addictive chemical. Vaping e-cigarettes can expose you to toxic metal particles like chromium and lead. Vaping e-cigarettes can expose you to harmful chemicals like formaldehyde and acrolein.
Condition 4: FDA + Chemical Constituents + Health Harms	This product contains nicotine. Nicotine is an addictive chemical. Vaping e-cigarettes can expose you to chromium and lead particles. These metals are known to be toxic to your heart, lungs, and nervous system. Vaping e-cigarettes can expose you to formaldehyde and acrolein. These chemicals are known to cause cancer and lung damage.
Condition 5: FDA + Chemical Constituents + Health Harms + Cigarette Equivalence	This product contains nicotine. Nicotine is an addictive chemical. Vaping e-cigarettes can expose you to chromium and lead particles. These metals are also found in cigarette smoke and are known to be toxic to your heart, lungs, and nervous system. Vaping e-cigarettes can expose you to formaldehyde and acrolein. These chemicals are also in found cigarette smoke and are known to cause cancer and lung damage.

The warning messages appeared in black text on a white background across the top 20% of e-cigarette ads for four e-cigarette brands (Blu, Logic, Loon, and JUUL). We chose the ads to reflect a breadth of marketing approaches and product characteristics (e.g., flavors and menthol, social appeals, use of color, sleek design). Since each respondent saw only three ads from four candidates, we randomly rotated the appearance and order of ads within each condition, and we randomly rotated the pairing of different warnings with different ads across all conditions. We did not allow respondents to progress until each ad was on their screen for at least 10 seconds. Youth (m = 74.5 seconds; SD = 108) and adults (m = 76.0 seconds; SD = 89) spent a comparable amount of time on the three pages in which the ads and warning label messages appeared.

Images of each condition and ad pairing and complete survey instruments can be found here: https://osf.io/dwtch/?view_only=b7e6e52a99364f17b3a1a8bfa23c1187.

Dependent Variables

Overview.

After exposure to all three ads with warnings, all respondents answered a series of questions about their absolute risk beliefs about vaping, relative risk beliefs about vaping and smoking, willingness to vape and smoke cigarettes in the future, and a variety of other items not involved in the pre-registered hypotheses reported here. People who smoke reported on their perceived reduction of risk associated with switching to e-cigarettes, quit intentions, and likelihood of using e-cigarettes to quit smoking. People who vape reported on their intentions to quit using e-cigarettes. All respondents provided demographic information, were debriefed on the study, and were given factsheets on (where applicable) the harms of vaping for youth or resources for smoking/vaping cessation. The full surveys for youth and adults can be found on the study pre-registration site.

Absolute risk beliefs.

Respondents indicated their agreement (from “strongly disagree”, 1, to “neither agree nor disagree”, 3, to “strongly agree”, 5) with 26 statements that describe various beliefs about vaping that we derived from risk belief items and message themes explored in previous work.^20-35 We deemed a subset of these items to be directly relevant to a warning label message theme and created averaged scales for each grouping (see Supplemental Appendix Tables A3 and A4 for item wording): 8 items for brain development (Cronbach’s alpha (α) = .91_y, .88_a; M = 3.69_y, 3.17_a; SD = 0.87_y, 0.72_a), 7 items for chemical constituents and harm (α = .91_y, .91_a; M = 4.07_y, 3.71_a; SD = 0.84_y, 0.76_a), and 3 items for nicotine (α = .73_y,, .67_a; M = 4.28_y, 4.02_a; SD = 0.76_y, 0.72_a).

Relative risk beliefs.

Respondents indicated whether a series of 14 attributes (again derived from previous work)^20-35,39 applied more to smoking or to vaping (scaled from “vaping much more than smoking”, 1, to “about the same to both”, 3, to “smoking much more than vaping”, 5). We deemed a subset of these items to be directly relevant to a warning message theme and created averaged scales for each grouping (see Tables 5 and 6 for item wording): 2 for youth brain development (r = .52_y, .51_a; p < .001_ya, M = 3.10_y, 3.09_a; SD = 0.88_y, 0.93_a), 5 for chemical constituents and harm (α = .84_y, .86_a; M = 3.31_y, 3.42_a; SD = 0.75_y, 0.80_a), 3 for nicotine (α = .75_y, .73_a; M = 3.30_y, 3.44_a; SD = 0.82_y, 0.82_a), and 1 for overall risk (M = 3.38_y, 3.51_a; SD = 0.98_y, 1.07_a).

Table 5.

Relative Risk Beliefs by Study Condition, Youth (Overall) and Adults Who Smoke and/or Vape (Post-Stratification Weighted)

	FDA	Youth Brain Development	Chemicals	Chemicals + Harm	Chemicals + Harm + Equivalence
	Mean	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)
Youth Ages 14-17 (n = 1,217)	n = 247	n = 244	n = 255	n = 253	n = 218
Youth brain development (2-item scale; results for H1 in grey)	M = 2.97	M = 2.99 F(1, 487) = 0.04 p = .835	M = 3.00 F(1, 500) = 0.15 p = .703	M = 3.07 F(1, 498) = 0.89 p = .347	M = 2.90 F(1, 462) = 0.41 p = .521
Chemical constituents and harm (5-item scale; results for H3 in grey)	M = 3.19	M = 3.24 F(1, 488) = 0.48 p = .489	M = 3.20 F(1, 500) = 0.00 p = .983	M = 3.28 F(1, 498) = 1.04 p = .309	M = 3.19 F(1,462) = 0.01 p = .915
Nicotine constituents and harm (3-item scale; not in hypotheses)	M = 3.24	M = 3.15 F(1, 489) = 0.94 p = .334	M = 3.25 F(1, 500) = 0.01 p = .912	M = 3.34 F(1, 498) = 1.76 p = .185	M = 3.21 F(1, 462) = 0.10 p = .756
Adults Who Smoke and/or Vape (n = 1,639)	n = 354	n = 311	n = 359	n = 296	n = 319
Youth brain development (2-item scale; results for H1 in grey)	M = 3.05	M = 3.11 F(1, 655) = 0.37 p = .543	M = 3.06 F(1, 700) = 0.00 p = .957	M = 2.98 F(1, 641) = 0.63 p = .427	M = 3.12 F(1, 663) = 0.57 p = .451
Chemical constituents and harm (5-item scale; results for H3 in grey)	M = 3.33	M = 3.39 F(1, 659) = 0.51 p = .476	M = 3.30 F(1, 705) = 0.13 p = .722	M = 3.23 F(1, 644) = 1.68 p = .195	M = 3.38 F(1,667) = 0.48 p = .487
Nicotine constituents and harm (3-item scale; not in hypotheses)	M = 3.31	M = 3.36 F(1, 659) = 0.30 p = .585	M = 3.34 F(1, 705) = 0.10 p = .752	M = 3.31 F(1, 644) = 0.00 p = .959	M = 3.43 F(1, 667) = 1.62 p = .203

Note: None of the F-tests reported in this table were statistically different from the null hypothesis at p < .05.

Table 6.

Relative Risk Beliefs and Intentions to Switch to E-Cigarettes (among Adults Who Smoke) and Intentions to Quit E-Cigarettes (among Adults Who Vape) by Experimental Condition (Post-Stratification Weighted)

	FDA	Youth Brain Development	Chemicals	Chemicals + Harm	Chemicals + Harm + Equivalence
	Mean	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)
Adults who Smoke (n = 1,125)	n = 243	n = 217	n = 234	n = 222	n = 209
Youth brain development (2-item scale; not in hypotheses)	M = 2.97	M = 3.08 F(1, 453) = 0.95 p = .330	M = 2.92 F(1, 472) = 0.15 p = .698	M = 2.85 F(1, 459) = 1.24 p = .266	M = 3.02 F(1, 447) = 0.28 p = .596
Chemical constituents and harm (5-item scale; results for H7a in grey)	M = 3.25	M = 3.37 F(1, 455) = 1.45 p = .230	M = 3.11 F(1, 472) = 2.52 p = .113	M = 3.10* F(1, 461) = 4.60 p = .032	M = 3.30 F(1,449) = 0.38 p = .537
Nicotine constituents and harm (3-item scale; not in hypotheses)	M = 3.37	M = 3.36 F(1, 455) = 0.00 p = .969	M = 3.21 F(1, 473) = 2.49 p = .116	M = 3.23 F(1, 460) = 3.17 p = .076	M = 3.49 F(1, 449) = 1.35 p = .246
Overall harm (single item; not in hypotheses)	M = 3.20	M = 3.52* F(1, 454) = 5.13 p = .024	M = 3.13 F(1, 471) = 0.28 p = .597	M = 3.10 F(1, 458) = 0.58 p = .448	M = 3.27 F(1, 447) = 0.23 p = .632
Decreased health risk by switching to e-cigarettes (single item; results for H7b in grey)	M = 2.71	M = 2.69 F(1, 456) = 0.02 p = .896	M = 2.48 F(1, 472) = 2.40 p = .122	M = 2.54 F(1, 460) = 1.43 p = .232	M = 2.56 F(1,448) = 1.42 p = .234
Likelihood of using e-cigarettes to quit smokinga (single item; results for H8 in grey)	M = 2.51	M = 2.54 F(1, 327) = 0.02 p = .884	M = 2.37 F(1, 342) = 0.40 p = .525	M = 2.45 F(1, 344) = 0.06 p = .804	M = 2.26 F(1, 316) = 1.27 p = .262
	% Yes	% Yes (χ2 vs. FDA)	% Yes (χ2 vs. FDA)	% Yes (χ2 vs. FDA)	% Yes (χ2 vs. FDA)
Adults who Vape (n = 1,074)	n = 242	n = 203	n = 238	n = 187	n = 204
Considering quitting e-cigarettes within 6 months (single dichotomous item; % who answered “yes”; results for H9 in grey)	64.9	66.0 χ2 (1, 445) = 0.07 p = .873	65.7 χ2 (1, 480) = 0.04 p = .907	79.8* χ2 (1, 429) = 11.89 p = .035	70.4 χ2 (1, 446) = 1.51 p = .429

Notes: ^aResults for this outcome are among the n = 824 adults who smoke who reported “considering quitting smoking completely” in the next 30 days or 6 months. *denotes p < .05.

Willingness to vape in the future.

We asked youth 5 questions about their willingness to vape in the next 7 days, next 30 days, next 6 months, 5 years from now, and if a friend offered a puff (scaled from “definitely will not”, 1, to “definitely will”, 4).^40-41 We averaged responses to these items into a scale (α = .97_y; M = 1.71_y; SD = 0.93_y).

Perceived reduction of risk to people who smoke by switching to e-cigarettes.

We asked adults who smoke (both exclusively and those who also vape) to respond to a single item (on the same Likert scale used for absolute risk beliefs) that stated, “If I were to switch completely from smoking to vaping, my health risks would decrease substantially” (M = 2.63_a; SD = 1.10_a).

Likelihood of using e-cigarettes to quit smoking.

We asked adults who smoke and who indicated at least some consideration of quitting smoking in the next six months (N = 1,124) how likely (from “very unlikely”, 1, to “uncertain”, 3, to “very likely”, 5) they are “to use a vaping product or e-cigarettes to help you quit?” (M = 2.53_a; SD = 1.40_a).

Considering quitting e-cigarettes.

We asked both youth and adults who vape whether they were “considering quitting vaping completely sometime soon” with response options that described plans to stop within 30 days, 6 months but not 30 days, or not in the next 6 months. We dichotomized the item by quit plans in the next 6 months or not.

Analytic Approach

We conducted all analyses using SPSS v.29. We tested hypotheses using multi-item scaled outcomes where available. We used individual mean imputation for missing data if respondents answered more than 50% of the items within a scale.⁴² Following this step, no more than 1.1% of values were missing from any item, so we used listwise deletion in all analyses.⁴³

NORC provided post-stratification population weights for both samples to approximate US Census distributions of key demographics (age, gender, region, race/ethnicity, education) and, for the youth sample, to adjust for the combination of probability and non-probability samples. We used the SPSS Complex Samples (CS) module to create analysis plan files for each dataset with the three tobacco use categories and sample source (probability, non-probability) as the strata variables for the adult and youth datasets, respectively, and the NORC-provided weight variables. All statistical tests used F-tests through the CS General Linear Model function except chi-square tests (via the CS Crosstabs function) performed on a single dichotomous outcome (considering quitting e-cigarettes). We used p > .05 as the preregistered criterion for hypothesis tests. We report on dichotomized versions of outcome variables in Supplemental Appendix Tables A3-A6 based on percentages of agreement (agreement versus neutral or disagreement), relative risk assessments (cigarettes are riskier versus “about the same” or vaping is riskier), willingness to vape/smoke (definitely will not versus any other category), and likelihood to use e-cigarettes to quit smoking (likely versus uncertain or unlikely) for more intuitive interpretation.

RESULTS

Testing H1, youth in the brain development condition were no more likely than those in the current FDA label condition to agree with beliefs about the risks of vaping for brain development (Table 3). Adults who smoke and/or vape in the brain development condition, however, were more likely than those in the FDA label condition to agree with beliefs about the risks of vaping for brain development (Table 4). H1 was thus partially supported.

Table 3.

Absolute Risk Beliefs and Willingness to Vape by Study Condition, Youth Ages 14-17 (Post-Stratification Weighted)

	FDA n = 247	Youth Brain Development n = 244	Chemicals n = 255	Chemicals + Harm n = 253	Chemicals + Harm + Equivalence n = 218
	Mean	Mean (Mean vs. FDA)	Mean (Mean vs. Anther Condition)	Mean (Mean vs. Anther Condition)	Mean (Mean vs. Anther Condition)
Youth brain development (8-item scale; results for H1 in grey)	M = 3.77	M = 3.87 F(1, 489) = 1.28 p = .258	M = 3.77 F(1, 500) = 0.01 p = .940	M = 3.79 F(1, 498) = 0.06 p = .808	M = 3.81 F(1, 463) = 0.18 p = .670
Chemical constituents and harm (7-item scale; results for H2 in grey)	M = 4.13	M = 4.20 F(1, 489) = 0.64 p = .423	M = 4.17 F(1, 500) = 0.20 p = .651	M = 4.28 F(1, 498) = 2.86 p = .091	M = 4.19 F(1, 463) = 0.39 p = .535
Nicotine constituents and harm (3-item scale; not involved in hypotheses)	M = 4.32	M = 4.42 F(1, 489) = 1.20 p = .273	M = 4.29 F(1, 500) = 0.07 p = .789	M = 4.35 F(1, 498) = 0.14 p = .709	M = 4.35 F(1, 462) = 0.11 p = .743
Willingness to use e-cigarettes (5-item scale; shaded results H4 = brain v. FDA; H5 = chemicals v. FDA; H6 = chemicals v. brain)	M = 1.53	M = 1.48 H4 F(1,489) = 0.43 H4 p = .513	M = 1.56 H5 F(1,500) = 0.09 H5p = .768 H6 F(1,497) = 0.89 H6 p = .346	M = 1.48 H5 F(1,498) = 0.79 H5 p = .374 H6 F(1,495) = 0.08 H6 p = .785	M = 1.48 H5 F(1,463) = 0.30 H5 p = .587 H6 F(1 460) = 0.00 H6 p = .958

Notes: None of the F-tests reported in this table were statistically different from the null hypothesis at p < .05.

Table 4.

Absolute Risk Beliefs by Study Condition, Adults Ages 18+ Who Smoke and/or Vape (Post-Stratification Weighted)

	FDA n = 354	Youth Brain Development n = 311	Chemicals n = 359	Chemicals + Harm n = 296	Chemicals + Harm + Equivalence n = 319
	Mean	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)	Mean (Mean vs. FDA)
Youth brain development (8-item scale; results for H1 in grey)	M = 3.19	M = 3.34* F(1, 660) = 3.93 p = .048	M = 3.11 F(1, 710) = 0.88 p = .348	M = 3.27 F(1, 647) = 0.97 p = .325	M = 3.27 F(1, 670) = 1.17 p = .279
Chemical constituents and harm (7-item scale; results for H2 in grey)	M = 3.67	M = 3.68 F(1, 661) = 0.02 p = .898	M = 3.66 F(1, 710) = 0.01 p = .926	M = 3.76 F(1, 647) = 1.09 p = .296	M = 3.72 F(1, 669) = 0.41 p = .523
Nicotine constituents and harm (3-item scale; not in hypotheses)	M = 4.01	M = 4.01 F(1, 657) = 0.00 p = .949	M = 3.90 F(1, 708) = 1.48 p = .225	M = 3.90 F(1, 644) = 1.16 p = .282	M = 3.91 F(1, 667) = 1.19 p = .275

Notes: * denotes p < .05.

Rejecting H2, there were no differences in beliefs about the chemical constituents and harms of vaping products between the three chemical constituent conditions and the FDA label condition for either youth (Table 3) or adults who smoke and/or vape (Table 4).

Rejecting H3, there were no differences in relative risk beliefs between the three chemical constituent conditions and the current FDA warning (Table 5).

Rejecting H4-H6 (bottom of Table 3), there were no differences in youth willingness to vape between the brain development condition (H4) or the three chemical constituent conditions (H5) and the FDA warning condition, nor between the brain development and three chemical constituent conditions (H6).

Rejecting H7a and H7b, compared to adults who smoke who were exposed to the FDA warning, those who were exposed to messaging about chemical constituents + harm + cigarette equivalence did not report greater risks of e-cigarettes relative to combustible cigarettes (H7a) nor lower levels of agreement that switching from combustible cigarettes to e-cigarettes would decrease their risks (H7b).

Rejecting H8, adults planning to quit smoking who were exposed to the chemical + harm + cigarette equivalence messaging were no different in their reported likelihood of using use e-cigarettes to quit than those exposed to the FDA warning.

Testing H9, quit considerations were significantly higher when adults who vape had been exposed to the chemicals + harm messaging (79.8%) rather than the FDA warning (64.9%; Table 6). Neither the chemical constituents-only messaging (condition 3; 65.7%) nor the chemicals + harm + cigarette equivalence warning (70.4%) influenced quit considerations relative to the current FDA warning. H9 was thus partially supported.

Rejecting H10 (not shown in tables), among people who vape, messages describing the harmful effects of e-cigarettes on brain development did not produce greater e-cigarette quit considerations among youth (14-17) or young adults (18-25) compared to older adults (26 or older). In fact, quit considerations trended lower (though not significantly) among youth and young adults exposed to brain development messages (46.3% and 57.6%, respectively) than those exposed to the FDA warning (53.9% and 64.6%, respectively). The brain development messages also did not increase quit considerations among older adults (69.0%) relative to the FDA warning (65.0%).

Post-Hoc Exploratory Analysis

We conducted post-hoc exploratory analyses to better understand the implications of analyses testing H9. We stratified the sample of adults who vape between those who only vape (for whom quitting would be uniformly beneficial for heath) and those who both smoke and vape (for whom quitting only vaping but continuing to smoke may limit health gains). Among adults who only vape, there were no differences in considerations to quit vaping between conditions (based on a chi-square test comparing all 5 conditions simultaneously to control for multiple exploratory comparisons). Among adults who vape and smoke, however, those in the chemicals + harm condition had higher rates of considering quitting vaping (88.9%) than all other conditions (ranging from 62.1% to 72.2%, p < .05 for a chi-square test comparing all conditions). We then considered rates of considering quitting both vaping and cigarettes among this subset of adult respondents. Adults who vape and smoke who were in the chemicals + harm condition also had higher rates of considering quitting both vaping and smoking (85.1%) than in other conditions (ranging from 52.7% to 63.4%, p < .05 for a chi-square test across conditions).

DISCUSSION

We found no evidence that warning messages focused on the harms of e-cigarette use for youth brain development outperformed the current FDA warning in shifting risk beliefs or willingness to vape in the future. At first glance, this would appear counter to previous work suggesting a potential persuasive advantage for brain development harm messages relative to nicotine addiction messages among youth and young adults^22-24,28. Several differences between the current study and past work are noteworthy, however.

First, most prior work tested the impact of warning messages either in isolation or in combination with anti-vaping visual imagery.^22-23,28 In contrast, we developed our tests to approximate likely real world implementations using text-only warning messages that occupied only 20% of an e-cigarette ad with the other 80% consisting of both visual and textual cues designed to entice audiences to consider vaping. The current study also exposed all respondents to only two thematically similar messages, combined with the current FDA warning, under the assumption that any new warnings entering the tobacco market would be rotated in with the existing one. We thus tested for an increase in relative effectiveness in shifting risk beliefs and willingness to vape in the future for only two alternative messages compared to three repeated exposures to the current FDA warning. It is possible that the current FDA nicotine warning may itself be influential and, when placed in the context of vaping ads, of comparable magnitude to any potential effects of alternative messaging. ^20-25,27-28 While the current FDA warning is not new, most of the youth in the sample did not regularly vape and thus may have had limited exposure to it prior to the study, which may have rendered it as impactful as other themes among youth.

Second, most prior work relied on observed differences in perceived message effectiveness (PME), not differences in beliefs or intentions (actual message effectiveness, AME).^22-23 It is possible that PME for warning messages viewed in isolation may simply not translate to AME of warning messages paired with ads for vaping products, at least in response to limited exposures. Researchers have questioned the predictive validity of PME for message design writ large,⁴⁴ and the only other published study of e-cigarette warning label messages that were paired with vaping ads found that youth brain development messages produced higher PME ratings but no differences in e-cigarette beliefs or e-cigarette use intentions.²⁵ This suggests that shifting PME may occur at a much lower threshold of exposure or impact than for indicators of belief change or shifts in willingness to vape. Youth also had relatively high baseline levels of agreement with most vaping risk beliefs (including brain development). In years just prior to this study, the FDA launched youth-directed media campaigns (“The Real Cost”) targeting many of these beliefs, so the impact of these messages may be lower today than it was even a few years ago due to waning novelty from prior campaign exposure.⁴⁵ These messages also did not increase quit considerations among adults who use e-cigarettes (though they did increase brain development risk perceptions among adults who smoke cigarettes, a group for whom these beliefs were much lower in the control and other message conditions). All told, the current study suggests the need for future work to consider the relative impact of warnings regarding nicotine and youth brain development in the context of broader exposures to pro-vaping messages, including ads for vaping products.

Warning messages focused on chemical constituents took three different forms – standing alone, paired with health harm statements, and paired with health harm statements equating e-cigarettes to cigarettes and cigarette smoke. We found little evidence, relative to the current FDA warning, that chemical constituent messages without health harm statements shifted risk beliefs or intentions to use tobacco products among youth or adults who smoke or vape. These messages were not associated with risk beliefs themselves (among youth or adults who smoke and/or vape), reduced willingness to use e-cigarettes in the future among youth, considering quitting e-cigarettes among adults who vape, or using e-cigarettes to quit smoking among adults who smoke). Adding harm and/or cigarette equivalence messaging to these chemical constituent messages did not change their impact on youth beliefs or willingness to vape in the future, either.

The lone source of evidence of benefit from alternative warning label messages in these analyses concerns chemical constituent messages that included an explicit statement of health harm associated with these chemicals and no explicit comparison to cigarettes. Exposure to these messages increased the likelihood of considering cessation among adults who vape (79.8%) compared to FDA warning condition (64.9%). These warnings were not associated with reductions in beliefs about the benefits of switching to e-cigarettes or likelihood to use these products in quit attempts among adults who currently smoke, suggesting they may not have unintended negative consequences. Subsequent exploratory analyses suggest that their impact was apparent only for adults who both vape and smoke, with limited preliminary evidence suggesting that the chemicals + harm warnings increased quit considerations for both e-cigarettes and combustible cigarettes. These findings are consistent with other recent studies showing that messages about exposure to chemical and metal constituents were among the most resonant warning message themes among adults who vape,³⁵ and that their impact may be heightened when combined with messages (including either text and/or images conveying that harm) that emphasizes the harms of these exposures.^30-35 Combined, prior work and the results presented here continue to suggest the potential utility of chemical constituent + harm messaging, without equivalence comparisons to cigarettes, in warning label messages on e-cigarette ads and beyond.

Study Limitations

While we used a national probability sample for adult tobacco product users, the youth sample included probability samples (with a low response rate) and non-probability samples, preventing broad claims about national representativeness. While we weighted responses to adjust for discrepancies between sample demographics and US Census demographic estimates, this strategy does not address sources of difference and bias between responders and nonresponders. Furthermore, while not directly comparable to national prevalence estimates due to differences in sample composition (here youth ages 14-17; prevalence data based on school grade 9-12), observed rates of tobacco use in our sample were somewhat larger than estimated high school tobacco use prevalence from early 2022 (where 15.4% reported past 30-day e-cigarette use and 1.8% reported past 30-day cigarette use).⁴⁶ It is possible that larger-than-expected proportions of current tobacco users may have changed the relative impact of warning messages and/or ads for vaping products on which those warnings were embedded.

Respondents saw warning messages only in a single session, averaging 75 seconds of exposure across three different messages, and responded to outcome measures immediately thereafter. We cannot speak to the impact of repeated exposures or the longer-term impact of one-time exposures. Future work should assess the impact of repeated exposures to vaping risk messages over time, either in the context of evaluations of large-scale media campaigns⁴⁵ or in randomized, controlled trials (as have been used to test the impact of warning labels on cigarette packaging)^47-48 that ensure multiple opportunities for exposure and engagement. The study was also not powered to detect small differences by condition (differences of 6 percentage points or less for dichotomous outcomes) that may be meaningful at the population level. At the same time, we used a (pre-registered) p-value for pre-registered hypothesis tests that was not adjusted for multiple comparisons which, while in our view statistically appropriate,^49-50 remains a source of some contemporary debate about the risks of type I error (falsely positive inferences). Finally, the messages were each longer than the current FDA warning, which we deemed necessary to accurately characterize the state of scientific knowledge about the potential constituents and harms. However, we cannot rule out the possibility that the length and complexity of the messages may have reduced their potential impact on some audiences.

Conclusion

Relative to the current FDA warning, alternative warning label messages were largely ineffective at reducing youth willingness to vape in the future or at changing risk beliefs about vaping products relative to cigarettes among adults who smoke or vape. Messages that describe the chemical constituents of vaping products and articulate the potential health harms of those chemicals may have potential to promote interest in quitting tobacco products among adults who smoke and vape, though future work will need to confirm the replicabilty of these findings.