Identifying message content to reduce vaping: Results from online message testing trials in young adult tobacco users

Abstract

INTRODUCTION

Young adults’ e-cigarette use is associated with perceptions that e-cigarettes are less harmful or addictive than cigarettes, socially acceptable, and appealing. This study developed and tested vaping educational messages addressing these factors: 1) Harm Perceptions, 2) Addictiveness, 3) Social Use, and 4) Flavors.

METHODS

Two message trials were conducted in U.S. Amazon Mechanical Turk workers aged 18-24 using a 2 (content: addiction, harm) x 3 (theme: alone, + flavors, + social) design with multiple messages in each of the six categories. Participants were assigned to view a random subset of messages and report on likeability and perceived message effectiveness (PME). Phase 1 (n = 200) tested 33 messages and 32 images. Phase 2 (n = 769) tested combinations of Phase 1’s 24 most effective messages with 6 images rated most likeable or effective. Linear mixed effects models assessed the effect of content, theme, image, and their interactions on message response.

RESULTS

In both trials, most participants were past 30-day tobacco users. Harm content messages produced higher PME ratings than addiction content messages, and flavor theme messages were correlated with higher likeability scores than “content alone” theme messages. In Phase 2, flavor and social message themes decreased the PME of harm messages. There was no effect of images on either outcome controlling for the independent or interaction effects of content, theme, and image.

CONCLUSIONS

Messages conveying the harms of vaping may be best for reducing vaping in young adult tobacco users; flavor and social themes may diminish their effectiveness.

1. INTRODUCTION

Young adults aged 18-24 report a high prevalence of experimentation with e-cigarettes (i.e., vaping) (Dai & Leventhal, 2019; Delnevo et al., 2016; Kasza et al., 2017; King, Patel, Nguyen, & Dube, 2015; McMillen, Gottlieb, Shaefer, Winickoff, & Klein, 2015), including JUUL (Hammond, Wackowski, Reid, & O'Connor, 2018; Ickes et al., 2019; Leavens et al., 2019; McKelvey, Baiocchi, & Halpern-Felsher, 2018; Vallone, Bennett, Xiao, Pitzer, & Hair, 2018). Studies in youth (Parker et al., 2018; Strong et al., 2019), young adults, and adults demonstrate that lower relative harm perceptions of e-cigarettes predict e-cigarette trial (Brose, Brown, Hitchman, & McNeill, 2015; Choi & Forster, 2014; Cooper, Loukas, Case, Marti, & Perry, 2018; Elton-Marshall et al., 2020). Results from convenience samples of young adults and adults show that the top reasons for trying JUUL are social (e.g., friends are using it) and curiosity (Ickes et al., 2019; Leavens et al., 2019). In adolescents, young adults, and adults, flavored pod use (i.e., menthol/mint, fruit) is also prevalent (Leavens et al., 2019; McKelvey et al., 2018) and first use of a flavored e-cigarette is associated with continued e-cigarette use in young adults (Villanti et al., 2019). Broadly, young adults’ e-cigarette use may be driven by perceptions that e-cigarettes are less harmful or addictive than cigarettes or other products (Berg et al., 2015; Wackowski & Delnevo, 2016), are socially acceptable (Berg et al., 2015; Cavallo et al., 2019), are easy to use (Ickes et al., 2019), and come in appealing flavors (Cavallo et al., 2019; Villanti et al., 2017).

The U.S. Food and Drug Administration’s (FDA) efforts to reduce youth vaping include public education messaging (U.S. Department of Health and Human Services), following the success of its Real Cost smoking prevention campaign (Duke et al., 2018; Duke et al., 2019; Farrelly et al., 2017). Message testing is an essential part of the formative work done to hone campaign messages, with higher perceived message effectiveness used as an indicator of the likely impact of cigarette-related messaging on campaign-targeted beliefs, attitudes, and smoking behaviors (Davis, Nonnemaker, Duke, & Farrelly, 2013; Duke, Nonnemaker, Davis, Watson, & Farrelly, 2014; Zhao et al., 2016). Evaluation of FDA’s Real Cost e-cigarette video messages show similar impacts on perceived message effectiveness in youth (Noar et al., 2020) and young adults (Rohde, Noar, Prentice-Dunn, Kresovich, & Hall, 2020), with longer term evaluation forthcoming. While existing studies detail the likely impact of individual messages on tobacco-related beliefs, attitudes, and behaviors, there is little research to guide the development of effective messages on the risks of e-cigarette use for young adults more broadly. Recent studies highlight addiction as a key theme in e-cigarette prevention media campaigns (Sangalang et al., 2019), though earlier work suggests that young people are uncertain regarding the definition of addiction (M. Roditis, Lee, & Halpern-Felsher, 2016) and that messaging should be relatable and depict short-term consequences of use(M. L. Roditis, Jones, Dineva, & Alexander, 2019). In adult studies of the effectiveness of e-cigarette health harm messages, addiction messages were perceived as less effective in young adults (Wackowski et al., 2019) and less effective in discouraging adult tobacco users from wanting to vape (Rohde, Noar, Mendel, et al., 2020) than other harms. The tone of the messages may also affect effectiveness, with fear and humor appeals both showing potential to affect tobacco-related attitudes and risk perceptions (Zhao, Roditis, & Alexander, 2019).

Given JUUL’s marketing success via social media (Huang et al., 2019), particularly to young people (Allem, Dharmapuri, Unger, & Cruz, 2018; Chu et al., 2018), and the widespread use of image-based social media apps (Instagram, Snapchat) among young adults (Perrin & Anderson, 2019), the goal of the current study was to develop and test a series of vaping prevention messages that could be deployed in a social media context such as Instagram to address factors associated with e-cigarette use in young adults: 1) Harm Perceptions, 2) Addictiveness, 3) Social Use, and 4) Flavors. In earlier work (Phan et al. 2020),we identified that messages on the social aspects of use or flavors were difficult to separate from content on addictiveness or harm. As a result, the messages followed a 2 x 3 design, with two health content areas (harm, addiction) and three contextual themes (health content alone, health content + flavors, health content + social). We used a two-phase approach to test the effect of exposure to messages with these factors, alone and in combination, and associated images on message receptivity (likeability) and perceived effectiveness. Based on previous evidence (Phan et al. 2020; Rohde, Noar, Mendel, et al., 2020), we hypothesized that harm messages would have higher perceived effectiveness than addiction messages and that social and flavor themes would produce greater likeability than health content alone.

2. MATERIAL AND METHODS

2.1. Message development

Study messages were developed using a process similar to message development for hookah prevention (Phan et al. 2020). A subset of the authors (ACV, SEP, JCW) developed 31 candidate messages for review by the study team. Study messages were developed to be brief enough to appear on a single image, similar to an Instagram post. Messages were reviewed and edited during a study team meeting, with n=33 messages confirmed for an initial round of testing in Phase 1 (Supplemental Table 1).

The study team also reviewed a series of candidate images designed to evoke a range of affective responses. These included images of hospital rooms, test tubes, vapor clouds, young people socializing and playing, colorful candies, and older people vaping (Supplemental Table 2). Some of these images were selected to be comparable to images tested in e-cigarette graphic health warning labels (Brewer et al., 2019). Thirty-two images were tested in Phase 1.

2.2. Study samples

Phase 1 participants were Amazon Mechanical Turk (AMT) workers who responded to a human intelligence task (HIT) titled “Feedback on E-Cigarette Education Messages” and met the following eligibility criteria: 1) are aged 18-24 years; 2) live in the United States, 3) have access to a smartphone with internet access, and 4) use one or more social media site(s) at least weekly. Of 957 respondents to the HIT, 200 were eligible and were paid $1.00 for completing the survey in March 2020.

Phase 2 participants were also recruited from AMT with the same eligibility criteria; 2,571 AMT workers responded to the human intelligence task (HIT) titled “Feedback on E-Cigarette Education Messages.” Of these, 769 were eligible and were paid $2.00 for completing the survey in May 2020.

2.3. Experimental procedures

Within the Phase 1 survey, participants were randomized to view a message from each of the six content by theme categories. Following exposure to six study messages in Phase 1, participants were exposed to six randomly-selected images. In Phase 2, a subset of messages were overlaid on selected images and participants were randomized to view three message/image combinations from the six content by theme categories.

2.4. Measures

2.4.1. Preferred terminology

To ensure that study messages reflected common terminology related to vaping nicotine, Phase 1 participants responded to three items assessing preferred terminology for vape products and vaping. The first was “What term do you usually use to refer to electronic vapor products containing nicotine? (Select the one best option).” Response options were “Vape,” “E-Cigarette,” “JUUL,” “Puff bar,” “Vape pod,” “Mod,” and “Other (Please specify).” The second item was “What term do you usually use to refer to the act of using electronic vapor products containing nicotine? (Select the one best option).” Response options were: “Vaping,” “JUULing,” “Puffing,” “Hitting,” and “Other (Please specify).” The final question was designed to assess the specificity of the term “vaping” to nicotine: “What substance do you associate with the word "vaping"? (Select the one best option).” Response options were “Nicotine,” “Marijuana/THC,” and “Other (Please specify).” These items were not included in Phase 2.

2.4.2. Response to study messages

Following each message, several items were asked to assess message receptivity and potential impact. “Likeability” of the message, which has strong predictive power for advertising success (Haley & Baldinger, 2000; Vakratsas & Ambler, 1999), was assessed by asking participants to describe their feelings about the message, with responses on a 5-point scale ranging from “I liked it very much” to “I disliked it very much.” Cognitive reactions were assessed as perceived message effectiveness (PME), using a validated three-item scale of effects perceptions (Baig et al., 2019). These items were: “This message discourages me from wanting to vape” (discouragement), “This message makes me concerned about the health effects of vaping” (concern), and “This message makes vaping seem unpleasant to me” (unpleasantness). Response options were on a five-point Likert scale, from strongly disagree (1) to strongly agree (5) and the mean response was calculated per participant for each message (Baig et al., 2019).

Each message was also followed by a manipulation check to determine whether the message resonated with its intended content and theme targets. The items were: 1) The message focused on the health risks of using vape devices; 2) The message focused on the addictiveness of using vape devices; 3) The message focused on the flavors in vape devices; and 4) The message focused on the social aspects of using vape devices, such as sharing with friends. Responses were assessed on a seven-point Likert scale with higher scores reflecting greater agreement with each statement (1 Strongly disagree to 7 Strongly agree).

2.4.3. Response to study images

Following each image, participants were asked to assess likeability and perceived effectiveness of the study images using the same items for messages, with wording changes (“image” vs. “message”).

2.4.4. Tobacco use

Ever use of cigarettes and electronic vapor products (EVPs) was assessed among all participants, as well as past 30-day use of cigarettes, EVPs, smokeless (chewing tobacco, snuff, dip, snus, or dissolvable tobacco products), cigars (cigars, cigarillos, or little cigars), and hookah (hookah or waterpipe). Past 30-day use was recoded as a categorical variable with the following five categories: no use, other tobacco products only, EVPs (any), cigarettes (any), and cigarettes & EVPs. Past 30-day EVP users were also asked to identify the substance last vaped based on an item from the Monitoring the Future Study (i.e., “The LAST TIME you vaped what was in the mist you inhaled?”) with response options for “Nicotine,” “Marijuana or hash oil,” “Just flavoring,” “Other (please specify),” and “I don't know.”

2.4.5. Sociodemographic characteristics

All participants were asked to provide information on age, sex assigned at birth, sexual identity, race, Hispanic ethnicity, highest level of education completed, and subjective financial status, a validated measure of socioeconomic status for young adults(Williams et al., 2017).

2.4. Statistical analysis

Analyses conducted in 2020 used Stata MP, Version 16.1, estimated the sociodemographic makeup and past 30-day tobacco use of each sample and the mean likability and PME of each message and image, or message/image combination, in both phases. In Phase 1, messages and images were ranked according to mean responses to likeability and PME. The six images with the highest PME ratings were paired with the six content by theme categories; the same process was used with the six images with the highest likeability ratings. In Phase 2, a subset of Phase 1 messages was paired with each image in the six categories to create two versions of the message: one on the most “effective” image and one on the most “likeable” image.

Since participants were randomly assigned to view a selection of messages and images, linear mixed effects models with restricted maximum likelihood (REML) were used to approximate results from a repeated measures ANOVA. These models were developed for likeability and PME outcomes in each phase, as well as the mean ratings of the four manipulation checks per message (harm, addiction, social, flavors). Mixed effects models accounted for independent effects of content and theme, as well as the interactions between content and theme; in Phase 2, models also accounted for independent effects of image and interactions between content and image, theme and image, and all three (content, theme, and image).

3. RESULTS

The Phase 1 and Phase 2 samples had similar sociodemographic characteristics, with most participants being aged 21-24 (64.0% Phase 1, 67.4% Phase 2), male (69.5% Phase 1, 69.7% Phase 2), heterosexual (75.5% Phase 1, 75.9% Phase 2), and white (60.0% Phase 1, 62.4% Phase 2; Table 1). Half reported a Bachelor’s or Advanced degree (50.5% Phase 1, 50.6% Phase 2) and most reported stable subjective financial status (meeting needs with a little left or living comfortably; 73.5% Phase 1, 78.2% Phase 2). Nearly all of the respondents in both samples reported past 30-day tobacco use, with the majority reporting both cigarette and EVP use (54.5% Phase 1, 63.1% Phase 2), followed by past 30-day cigarette use (23.5% Phase 1, 36.0% Phase 2), and past 30-day EVP use (9.5% Phase 1, 0.7% Phase 2). More than half of ever EVP users reported last vaping nicotine (52.9% Phase 1, 51.5% Phase 2), with similar proportions reporting vaping marijuana or hash oil (25.8% Phase 1, 20.7% Phase 2) or just flavoring (21.3% Phase 1, 23.5% Phase 2).

Table 1.

Characteristics of participants in two Amazon Mechanical Turk message testing samples of young adults, Spring 2020

	Phase 1 (n = 200)		Phase 2a (n = 769)
	n	%	n	%
Age group
18-20	72	36.0	251	32.6
21-24	128	64.0	518	67.4
Sex
Male	139	69.5	536	69.7
Female	61	30.5	233	30.3
Sexual identity
Heterosexual	151	75.5	584	75.9
LGBQ	46	23.0	176	22.9
Questioning/other	3	1.5	9	1.2
Race
White	120	60.0	480	62.4
Asian	46	23.0	122	15.9
Black or African American	23	11.5	124	16.1
Other race/multiple races	11	5.5	43	5.6
Hispanic ethnicity
No	141	70.5	557	72.4
Yes	59	29.5	212	27.6
Highest level of education completed
Less than high school	21	10.5	39	5.1
High school/GED	28	14.0	118	15.3
Some college/Associate's degree	50	25.0	223	29.0
Bachelor's/Advanced degree	101	50.5	389	50.6
Subjective financial status
Live comfortably	75	37.5	286	37.2
Meet needs with a little left	72	36.0	315	41.0
Just meet basic expenses	49	24.5	160	20.8
Don't meet basic expenses	4	2.0	8	1.0
Use of tobacco products, past 30 days
None	24	12.0	1	0.1
Other tobacco products only	1	0.5	1	0.1
EVPs	19	9.5	5	0.7
Cigarettes	47	23.5	277	36.0
Cigarettes & EVPs	109	54.5	485	63.1
Last substance vaped (among ever vapers)
Nicotine	82	52.9	313	51.5
Marijuana or hash oil	40	25.8	126	20.7
Just flavoring	33	21.3	143	23.5
Other	0	0.0	2	0.3
Don't Know	0	0.0	24	3.9

^an = 59 respondents participated in both Phase 1 and Phase 2

3.1. Phase 1

Prior to message exposure, Phase 1 participants responded to items reflecting preferred terminology to be used in vaping prevention messages. The top three terms used to refer to EVPs containing nicotine were “vape” (40.5%), “e-cigarette” (33.0%), and “JUUL” (20%) and the majority of respondents used the term “vaping” to refer to the act of using an EVP containing nicotine (70.0%), followed by “JUULing” (19.5%). Most participants (80.5%) associated nicotine with the word “vaping,” though a sizeable proportion associated “vaping” with marijuana/THC (16.5%). These findings provided support for the use of “vape” and “vaping” as the main terms used in Phase 2 study messages.

Manipulation checks identified that the harm content messages had higher ratings of being related to health risks and lower ratings of being related to addiction, messages with the flavor theme had higher ratings related to flavor, and messages with the social theme had higher ratings related to social effects of vaping compared to the addiction content alone message (Table 2).

Table 2.

Manipulation checks of vaping prevention message content in two Amazon Mechanical Turk samples of young adults, Spring 2020

Phase 1 (n = 200)	Harm check	Addiction check	Flavor check	Social check
	Coef	Coef	Coef	Coef
Addiction + Alone	Ref.	Ref.	Ref.	Ref.
Addiction + Flavors	0.35 *	−0.28	1.27 **	−0.06
Addiction + Social	−0.37 *	−0.71 **	−0.03	0.97 **
Harm + Alone	0.93 **	−1.94 **	0.06	−0.17
Harm + Flavors	1.10 **	−1.77 **	1.27 **	−0.21
Harm + Social	0.95 **	−1.68 **	−0.18	0.84 **
Constant	4.51 **	5.69 **	3.31 **	3.51 **
Phase 2 (n = 769)a	Harm check	Addiction check	Flavor check	Social check
	Coef	Coef	Coef	Coef
Addiction + Alone + Likeable image	Ref.	Ref.	Ref.	Ref.
Addiction + Alone + Effective image	−0.33 *	0.46 *	−0.11	−0.36 *
Addiction + Flavors + Likeable image	0.26	0.03	1.29 **	−0.17
Addiction + Flavors + Effective image	0.21	−0.09	1.28 **	−0.28
Addiction + Social + Likeable image	−0.39 *	0.00	0.09	1.33 **
Addiction + Social + Effective image	−0.33 *	−0.10	0.07	1.28 **
Harm + Alone + Likeable image	0.95	−1.33	0.14	−0.27
Harm + Alone + Effective image	1.05 **	−1.36 **	0.04	−0.37 *
Harm + Flavors + Likeable image	0.75 **	−1.26 **	1.66 **	−0.15
Harm + Flavors + Effective image	0.85 **	−1.49 **	1.53 **	−0.25
Harm + Social + Likeable image	0.70 **	−1.28 **	−0.03	0.87 **
Harm + Social + Effective image	0.73 **	−1.10 **	0.03	0.87 **
Constant	4.81 **	5.28 **	3.32 **	3.72 **

^*p<0.05

^**p<0.001

^an = 59 respondents participated in both Phase 1 and Phase 2

Mean PME ranged from 2.82 to 4.01 across the 33 messages tested in Phase 1, with a median PME of 3.40 (Supplemental Table 1). Mean likeability for these messages ranged from 2.62 to 3.38, with a median likability of 3.13 (data not shown). In general, harm content messages produced higher PME ratings (fixed effect: 0.28, 95% CI: 0. 15, 0.41; Table 3) than addiction content messages and flavor theme messages generated higher likeability scores (fixed effect: 0.23, 95% CI: 0.07, 0.39) than health content alone theme messages, controlling for content, theme, and the content by theme interaction. There were no significant interaction effects of content by theme.

Table 3.

Response to content, theme, and image variations within vaping prevention messages in two Amazon Mechanical Turk samples of young adults, Spring 2020

	Phase 1 (n = 200)				Phase 2 (n = 769)a
	Perceived message effectiveness		Likeability		Perceived message effectiveness		Likeability
	Coef	(95% CI)	Coef	(95% CI)	Coef	(95% CI)	Coef	(95% CI)
Content
Addiction	Ref.		Ref.		Ref.		Ref.
Harm	0.28	(0.15, 0.41)	0.01	(−0.14, 0.17)	0.40	(0.27, 0.54)	−0.02	(−0.17, 0.13)
Theme
Alone	Ref.		Ref.		Ref.		Ref.
+ Flavors	0.10	(−0.03, 0.23)	0.23	(0.07, 0.39)	0.32	(0.17, 0.48)	0.19	(0.02, 0.36)
+ Social	−0.09	(−0.22, 0.04)	0.13	(−0.03, 0.29)	0.12	(−0.03, 0.27)	0.21	(0.04, 0.38)
Content x Theme
Harm + Flavors	−0.11	(−0.30, 0.07)	−0.16	(−0.39, 0.06)	−0.44	(−0.65, −0.23)	−0.04	(−0.28, 0.20)
Harm + Social	−0.03	(−0.19, 0.18)	−0.12	(−0.34, 0.10)	−0.24	(−0.43, −0.04)	−0.08	(−0.29, 0.14)
Image
Likeable image					Ref.		Ref.
Effective image					−0.003	(−0.14, 0.14)	−0.08	(−0.24, 0.08)
Content x Image
Harm + Effective image					0.12	(−0.07, 0.31)	0.20	(−0.01, 0.41)
Theme x Image
Flavors + Effective image					−0.04	(−0.26, 0.18)	0.05	(−0.19, 0.29)
Social + Effective image					−0.03	(−0.37, 0.18)	0.02	(−0.22, 0.26)
Content x Theme x Image
Harm + Flavors + Effective image					−0.02	(−0.32, 0.28)	−0.23	(−0.57, 0.10)
Harm + Social + Effective image					−0.10	(−0.37, 0.18)	−0.20	(−0.50, 0.10)
Constant	3.31	(3.15, 3.46)	3.04	(2.87, 3.21)	3.18	(3.06, 3.29)	3.19	(3.07, 3.32)

Bold text indicates 95% confidence intervals that do not overlap the null.

^an = 59 respondents participated in both Phase 1 and Phase 2

Mean PME of the 32 images tested in Phase 1 ranged from 1.94 to 3.40, with a median rating of 2.76; mean likeability of the images ranged from 2.39 to 3.97, with a median rating of 3.08 (Supplemental Table 2). Figure 1 presents the six images with the highest PME ratings (effective) and the six images with the highest likeability ratings (likeable) which were chosen for testing in Phase 2. Each image was assigned to a content by theme category. The 21 messages with the highest PME ratings were also selected for inclusion in Phase 2, along with three messages with high PME among non-current e-cig users (hs2, a4) and e-cigarette users (a2), for a total of 24 messages (Supplemental Table 1). Each of the 24 messages was paired with the likeable and effective messages for that content by theme category, yielding a total of 48 message/image combinations to be tested in Phase 2 (Supplemental Table 3).

Figure 1.

Images with the Phase 1 highest perceived message effectiveness (PME) and highest likeability scores, by assigned content by theme category

3.2. Phase 2

Similar to Phase 1 findings, manipulation checks identified that the harm content messages had higher ratings of being related to health risks and lower ratings of being related to addiction, messages with the flavor theme had higher ratings related to flavor, and messages with the social theme had higher ratings related to social effects of vaping compared to the addiction alone message on the likeable image (Table 2). The addiction alone message + effective image improved response to the addiction manipulation check compared to the same message on the likeable image; this was the only instance in which inclusion of the image altered response to the manipulation checks. All other messages performed similarly on the likeable and effective images.

Mean PME ranged from 2.82 to 3.80 across the 24 messages tested in Phase 2, with a median PME of 3.45 (Supplemental Table 1). Mean likeability for these messages ranged from 2.94 to 3.69, with a median likability of 3.00 (data not shown). Harm content messages (fixed effect: 0.40, 95% CI: 0.27, 0.54; Table 3) and flavor theme messages (fixed effect: 0.32, 95% CI: 0.17, 0.48) received higher PME ratings than addiction content messages, though interactions between content and theme demonstrate lower PME ratings for harm + flavors (fixed effect: −0.44, 95% CI −0.65, −0.23) and harm + social (fixed effect: −0.24, 95% CI: −0.43, −0.04), controlling for all other content, theme, and image main effects and interactions. Flavor theme (fixed effect: 0.19, 95% CI: 0.02, 0.36) and social theme (fixed effect: 0.21, 95% CI: 0.04, 0.38) messages produced higher likeability scores than “alone” theme messages. There were no significant fixed effects of image or any interaction with image.

4. DISCUSSION

Using data from two sequential trials in online samples of U.S. young adults with high past 30-day tobacco use prevalence, findings from this study suggest that educational messages focused on the harms of vaping alone produce the greatest perceived message effectiveness, while messages addressing the themes of flavors and social aspects of vaping produce greater likeability compared to the messages with harm and addiction content alone. Flavor and social message themes, however, decrease the perceived effectiveness of harm content messages, suggesting that there is a potential cost to including these themes in vaping messaging.

Findings from these vaping messaging trials are consistent with similar research on hookah prevention message testing, which found that harm messages produced greater receptivity, positive attitudes toward the messages, emotional response, and harm risk appraisals in young adults than those conveying addiction risks (Phan et al. 2020). In the hookah messaging trial, the flavor theme did not alter receptivity to addiction or harm-related messages, though the social theme produced lower receptivity to the study messages compared to the harm or addiction content alone. This is consistent with our findings that the flavor and social theme may improve likeability of the addiction and harm vaping education messages, though reduce the effectiveness of those messages.

In line with earlier research in adolescents (M. L. Roditis et al., 2019), addiction messages in our trials of young adults addressed loss of independence, loss of choice, effects on mood, getting “hooked” and not being able to “put down your vape.” Regardless of content, addiction messages had similar likeability ratings to harm messages, but were outperformed by harm messages when it came to perceived message effectiveness in our samples. These results are consistent with earlier work on e-cigarette health harm messages, showing addiction messages as least discouraging to adult tobacco users (Rohde, Noar, Mendel, et al., 2020; Wackowski et al., 2019). With these earlier studies (Phan et al. 2020; Rohde, Noar, Mendel, et al., 2020), our findings suggest that addiction messages alone may not produce the greatest impacts on vaping reduction, but including addiction messages in a campaign that also addresses the harms of vaping may be appropriate.

A novel contribution of our trial is our systematic examination of the effect of imagery on message likeability and perceived effectiveness. Generally, the images rated as most likeable were colorful or projected positive social imagery and the most effective messages were darker in color or monochromatic and included vapor or vaping. We expected that messages on the “effective” image would produce greater perceived message effectiveness than those on the “likeable” image. In fact, there was no effect of the image on perceived message effectiveness or likeability after controlling for message content and theme.

Strengths of the current study include the systematic, iterative approach to testing vaping prevention messages in the target sample and the explicit inclusion of “effective” and “likeable” imagery in these trials. Our main outcome of message effectiveness (PME) has also been shown to be superior to message perceptions in estimating the likely impact of a vaping prevention message (Noar et al., 2020). Previous studies in tobacco control have identified effective messages as those scoring above the midpoint on message perceptions scales (Davis et al., 2013; Duke et al., 2014; Zhao et al., 2016) and the mean response to our harm content messages met that threshold. While existing studies on vaping prevention messages have evaluated individual messages (Noar et al., 2020; Rohde, Noar, Prentice-Dunn, et al., 2020), our study aggregates messages by theme and content, allowing for greater synthesis on message development to inform future campaigns. The study is limited by the nature of the online convenience samples and the lack of data collected on campaign target outcomes, including vaping-related knowledge, attitudes, beliefs, and behaviors. Samples recruited via Amazon Mechanical Turk may be more likely to hold a college degree, and less likely to smoke (Walters, Christakis, & Wright, 2018), but potentially more likely to report non-cigarette tobacco use (Kraemer, Strasser, Lindblom, Niaura, & Mays, 2017). While e-cigarette use was not part of our eligibility criteria, the use of the term “e-cigarette” in the title of the HITs may have resulted in the small numbers of non-tobacco users in the two samples. This may limit the generalizability of our findings to primary prevention of vaping, though it may be applicable to secondary prevention efforts. Additionally, the messages differed across conditions which, while appropriate for identifying candidate messages, limits our ability to determine the specific health harms (e.g., lung injury, toxicity of constituents) likely to produce the greatest effectiveness or likeability. Results from this study, limited to single message exposures, do not account for the variety of messages needed in a media campaign to sustain awareness and drive behavior change.

5. CONCLUSIONS

The concurrent rise of e-cigarette use in youth from 2017 to 2019 and 2019 epidemic of e-cigarette and vaping-related lung injury (EVALI) have drawn national attention to the importance of vaping prevention in young people (Adkins et al., 2020; King, Jones, Baldwin, & Briss, 2020; Werner et al., 2020). Our findings suggest that the most effective vaping messages for young adult tobacco users may be the most direct: messages conveying the harms associated with vaping had the highest perceived message effectiveness. Though prior research has identified flavors and social factors as drivers of e-cigarette use in young people and these themes as likeable to young adults, adding these themes to harm-related messages may diminish their effectiveness. Future studies are needed to examine the prospective impact of these messages on vaping-related knowledge, attitudes, beliefs and behaviors in e-cigarette users and non-users.

HIGHLIGHTS.

• Messages addressed health content (addiction, harm) and themes (flavors, social).

• Harm messages produced higher perceived message effectiveness than addiction messages.

• Flavor and social message themes decreased the PME of harm messages.