Reactions to graphic and text health warnings for cigarettes, sugar-sweetened beverages, and alcohol: An online randomized experiment of US adults

Abstract

We aimed to examine reactions to graphic versus text-only warnings for cigarettes, SSBs, and alcohol. A convenience sample of US adults completed an online survey in 2018 (n=1,352 in the analytic sample). We randomly assigned participants to view a: 1) text-only warning without efficacy information (i.e., message intended to increase consumers’ confidence in their ability to stop using the product), 2) text-only warning with efficacy information, 3) graphic warning without efficacy information, or 4) graphic warning with efficacy information. Participants viewed their assigned warning on cigarettes, SSBs, and alcohol, in a random order. Across product types, graphic warnings were perceived as more effective than text-only warnings (p<.001) and led to lower believability, greater reactance (i.e., resistance), more thinking about harms, and lower product appeal (all p<.05); policy support did not differ. Compared to SSB and alcohol warnings, cigarette warnings led to higher perceived message effectiveness, believability, fear, thinking about harms, policy support, and greater reductions in product appeal (all p<.05). The efficacy information did not influence any outcomes. Graphic warnings out-performed text-only warnings on key predictors of behavior despite causing more reactance.

Requiring health warnings on unhealthy products – including cigarettes, sugar-sweetened beverages (SSBs), and alcohol – is a promising policy strategy for improving public health (1–8).

One key question facing policymakers is whether warnings for these products should use images depicting health consequences (i.e., graphic warnings). Numerous experiments have found that graphic cigarette warnings are more effective than text-only warnings at increasing quit attempts, reducing product appeal, eliciting more negative emotions, and more thinking about the harms of smoking (1, 9, 10). These constructs are key mechanisms predicting sustained behavior change as outlined in the Tobacco Warning Model, an empirically-driven model describing the most important mechanisms underlying warnings’ impact on behavior change (11, 12). However, few studies have examined whether graphic SSB and alcohol warnings are similarly effective.

There are key differences between cigarettes, SSBs, and alcohol that might influence consumers’ reactions to graphic warnings. For example, there is no safe level of cigarette smoking, whereas there are arguably safe levels of alcohol and SSB consumption. Moreover, it is illegal to sell tobacco and alcohol to youth but there are no bans on food or beverage sales (13). In 2020, the US Food and Drug Administration issued a final ruling requiring large graphic warnings to appear on cigarette packs. In contrast, alcohol products in the US are required only to have a single, small text-only warning, and SSB containers are not required to bear warnings (although lawmakers in seven US states have proposed legislation to require text warnings appear on SSBs). Understanding differences in consumers’ reactions across these product types can elucidate whether findings from the large body of research on graphic cigarette warnings are likely to apply in the context of SSBs and alcohol, or if more research will be needed to understand product-specific effects.

Finally, the role of efficacy information (i.e., messages designed to increase people’s perceived ability to change their behavior) in moderating graphic warnings’ impact requires further examination. The Extended Parallel Process Model, a threat-management communication theory often applied to research on warnings,(14) suggests that warnings will be ineffective or even backfire if not accompanied by efficacy information, but studies have disproven that assertion for cigarette warnings (1, 15, 16). However, efficacy information remains understudied in the context of SSB and alcohol warnings.

To address these gaps, we conducted an experiment with US adults. First, we examined the impact of graphic versus text-only warnings on consumer reactions that are predictive of warnings’ effects on behavior. We predicted that across product type, graphic warnings would elicit higher ratings on perceived message effectiveness, reactance (i.e., opposition or resistance to a message), fear, thinking about harms, and policy support, and lower ratings on believability and product appeal. Second, we examined whether reactions to warnings differed by product type, predicting that participants would perceive warnings as more effective when they appear on cigarettes than on alcohol or SSBs. Third, we examined whether the inclusion of efficacy information affected reactions to graphic warnings. Finally, we explored whether demographic characteristics moderated the impact of graphic warnings (versus text-only), product type, and efficacy information on perceived message effectiveness.

Methods

Participants

In April 2018, we recruited a convenience sample of 1,413 adults to participate in an online experiment. Inclusion criteria were currently residing in the U.S. and being at least 18 years old. Recruitment occurred through Amazon Mechanical Turk (MTurk). MTurk is an online crowdsourcing platform that provides access to participants who volunteer to complete online tasks (17). MTurk has increasingly been used for social science research, including research on the effect of warning messages (18–20). Experiments conducted on MTurk largely replicate findings from experiments conducted via probability-based samples (21). Participants were able to take the survey on any device with an internet connection, including computers, tablets, and smartphones.

Procedures

Participants provided informed consent and took a 10–15 minute online survey. After completing an experiment about SSB warnings (7), participants completed a 2×2×3 between-within subjects experiment, representing 12 different conditions (Figure 1). The built-in randomizer tool in Qualtrics automatically randomized participants to one of four between-subjects conditions (see Supplementary Figure 1 for CONSORT flow diagram): 1) text-only warning without efficacy information, 2) text-only warning with efficacy information, 3) graphic warning without efficacy information, or 4) graphic warning with efficacy information. These four conditions represent the combination of two between-subjects factors, each with two levels: 1) warning type (text vs. graphic), and 2) whether warnings included efficacy information (yes vs. no). Participants viewed their randomly assigned warnings on three different products (cigarettes, SSBs, and alcohol), presented in a random order within subjects (this randomization also occurred via the built-in randomizer tool in Qualtrics).

Figure 1.

Study stimuli.

We selected commonly known consequences of each product (22–24) that had a strong epidemiological evidence base (25–27). The cigarette warning read: “WARNING: Smoking cigarettes causes lung cancer,” the SSB warning read: “WARNING: Drinking beverages with added sugar causes tooth decay,” and the alcohol warning read: “WARNING: Drinking alcohol causes liver disease.” To create graphic warnings, we selected copyright-free images depicting diseased body parts for each health consequence, as US case law suggests that images should be congruent with the text (28). We adapted efficacy information from required cigarette pack messages in the U.K.; the text read: “Cut back: www.cutback.gov” for SSBs and alcohol and “Quit now: www.quitnow.gov” for cigarettes, since there is no safe level of cigarette consumption. To control for established brand preferences (29), warnings were displayed on images of a mock brand of each product, created by a professional designer.

After viewing the warnings and responding to survey items, participants completed two additional experiments (30, 31) and answered demographic items. Participants received $2.20 for completing the survey. The University of North Carolina Institutional Review Board approved the study. We pre-registered the study before data collection: https://aspredicted.org/w9zi3.pdf.

Measures

Participants rated each warning using measures adapted from previous studies. Perceived message effectiveness was the primary outcome because this measure is sensitive to change in online studies and is predictive of actual behavior change (32). Secondary outcomes included believability (33), reactance (34), fear (35), thinking about harms (11), product appeal (36), policy support (37), and self efficacy (38). Exact item wording and response scales for all measures appear in Supplementary Table 1.

Data analysis

We excluded survey responses from participants who had recently completed a pilot test of our survey instrument, those with duplicate IP addresses or MTurk usernames, and those with missing data on the variables in the main models, yielding an analytic sample of 1,352 (Supplementary Figure 1). Analyses used Stata/SE version 14.1 with two-tailed tests, a critical alpha of 0.05, and listwise deletion for missing data. We first examined whether randomization created equivalent groups using chi-squared tests for categorical variables and t-tests for continuous variables, examining all variables in Table 1 and random assignment to the first experiment in the survey. We used a t-test to examine whether efficacy information changed self efficacy, recoding participants who selected “Not applicable. I do not [consume product]” as “strongly agree”. To examine the unadjusted impact of the experimental factors on our outcomes, we calculated standardized effect sizes (Cohen’s d) with 95% confidence intervals.

Table 1.

Participant characteristics (n=1,352).

Characteristic	n	%
Demographics
Age
18–29 years	361	27%
30–39 years	547	40%
40–54 years	295	22%
55+ years	149	11%
Mean in years (SD)	37	(12)
Gender
Male	704	52%
Female	639	47%
Transgender or other	9	1%
Gay, lesbian, or bisexual	141	10%
Hispanic	122	9%
Race
White	1106	82%
Black or African American	127	9%
Asian	63	5%
Other/multiracial	47	3%
American Indian or Alaskan Native	8	1%
Native Hawaiian or Pacific Islander	1	0%
Education
High school degree or less	170	13%
Some college	313	23%
College graduate or associates degree	699	52%
Graduate degree	170	13%
Household income, annual
$0–$24,999	234	17%
$25,000–$49,999	425	31%
$50,000–$74,999	322	24%
$75,000+	370	27%
Low income (≤ 150% of FPL)	224	17%
Health behaviors
Current smoker	297	22%
Frequency of sugar-sweetened beverage consumption
<1 time per day	866	64%
1 to <3 times per day	312	23%
3 or more times per day	174	13%
Frequency of alcohol consumption
<1 time per month	510	38%
1 to 3 times per month	265	20%
1 to 2 times per week	309	23%
3 to 7 times per week	268	20%

Note. Characteristics did not differ by experimental arms except for Hispanic ethnicity (p<.001). Missing demographic data ranged from 0.6% to 0.9%.

We used multilevel linear models to estimate message-level (Level 1) and person-level (Level 2) predictors of the outcomes, controlling for Hispanic ethnicity, the only characteristic found to be imbalanced across conditions. These models used random intercepts but not random slopes. The first set of models included indicators for the three experimental factors studied (i.e., graphic image, efficacy information, and product type) and all interactions between these three factors, with a Bonferroni corrected p-value of .001 due to the large number of exploratory hypotheses (.05 divided by 49 comparisons (7 models * 7 moderation coefficients) = .001) (39). None of the interactions were statistically significant so for ease of interpretation we present results from models without interactions.

In pre-specified exploratory moderation analyses, we examined whether product consumption, low-income status, and race/ethnicity moderated the impact of the experimental conditions on perceived message effectiveness. For significant interactions, we visually plotted the predicted probabilities for each level of the moderators. These analyses used a Bonferroni corrected p-value of 0.002 (.05 divided by 24 comparisons (6 models * 4 moderation coefficients) = .002).

Results

Participants’ mean age was 37 years and 10% identified as gay, lesbian, or bisexual (Table 1). About half (49%) had an annual household income of less than $50,000. In terms of health behaviors, 22% were current smokers (defined as having smoked at least 100 cigarettes and now smoking some days or every day), about a third (36%) consumed SSBs at least once a day, and most (63%) consumed alcohol at least once a month. The sample was younger, more likely to identify as gay, lesbian, or bisexual, less likely to be Hispanic, and more likely to smoke compared to nationally representative samples (Supplementary Table 2).

Warning type

In unadjusted analyses (Figure 2), graphic warnings were perceived as more effective than text-only warnings (d=.17, 95% CI=.11 to .23). They also led to greater reactance (d=.56, 95% CI=.50 to .62), fear (d=.54, 95% CI=.47 to .60), thinking about harms (d=.29, 95% CI=.22 to .35), and reduced product appeal compared to text-only warnings (d=−.68, 95% CI=−.62 to −.74). Graphic warnings led to lower believability (d=−.11, 95% CI=−.05 to −.18) and policy support (d=−.09, 95% CI=−.02 to −.15). Adjusted analyses (Table 2) revealed the same pattern of findings, except the impact of warning type on policy support was no longer statistically significant. These findings were all in the predicted direction, with the exception of policy support. The impact of graphic versus text warnings on the outcomes did not differ by product type. Similarly, the impact of graphic versus text warnings on perceived message effectiveness did not differ by participants’ product consumption, low-income status, or race/ethnicity.

Figure 2.

Standardized effects (Cohen’s d) of warning characteristics on outcomes, n=1,352, i=4,056. i, total number of observations. Error bars show standard errors.

Table 2.

Effects of warning characteristics on consumer reactions and policy support, n=1,352, i=4,056.

	Perceived message effectiveness	Believability	Reactance	Fear	Thinking about harms	Product appeal	Policy support
Warning Characteristic	B	B	B	B	B	B	B
Level 2
Graphic (vs. text)	0.34**	−0.22*	0.76**	0.69**	0.57**	−0.51**	−.25
Efficacy statement (vs. none)	0.09	0.04	0.09	−0.03	0.06	0.01	.12
Level 1
SSBs (vs. cigarettes)	−0.36**	−0.39**	0.13**	−0.51**	−0.49**	0.11**	−.47**
Alcohol (vs. cigarettes)	−0.59**	−0.63**	0.14**	−0.29**	−0.46**	0.10**	−.50**
ICC	0.14	0.21	0.80	0.61	0.18	0.54	0.96

Note. Boldface indicates statistical significance (*p<0.05, *** p<0.001); i, total number of observations; B, unstandardized regression coefficient from mixed effects linear regression; SSBs, sugar-sweetened beverages; ICC, intraclass correlation. Analyses adjusted for Hispanic ethnicity. None of the interactions between the Level 2 factors were statistically significant, so these models do not include the interaction terms.

Product type

Overall, warnings (both text and graphic) for cigarettes were perceived as more effective than SSB and alcohol warnings in unadjusted analyses (SSB vs. cigarettes: d=−.34, 95% CI=−.26 to −.42; alcohol vs. cigarettes: d=−.26, 95% CI=−.18 to −.34; Figure 2). Compared to SSB and alcohol warnings, cigarette warnings also led to greater believability (SSBs vs. cigarettes: d=−.35, 95% CI=−.28 to −.43; alcohol vs. cigarettes: d=−.28, 95% CI=−.20 to −.35), greater fear (SSBs vs. cigarettes: d=−.38, 95% CI=−.30 to −.46; alcohol vs. cigarettes: d=−.22, 95% CI=−.14 to −.29), more thinking about harms (SSBs vs. cigarettes: d=−.43, 95% CI=−.36 to −.51; alcohol vs. cigarettes: d=−.20, 95% CI=−.13 to −.28), more policy support (SSBs vs. cigarettes: d=−.16, 95% CI=−.09 to −.24; alcohol vs. cigarettes: d=−.17, 95% CI=−.10 to −.25), and less reactance (SSBs vs. cigarettes: d=.09, 95% CI=.02 to .17; alcohol vs. cigarettes: d=.10, 95% CI=.02 to .17). Warnings for cigarettes led to lower product appeal compared to warnings for SSBs and alcohol (SSBs vs. cigarettes: d=.15, 95% CI=.07 to .22; alcohol vs. cigarettes: d=.13, 95% CI=.05 to .20). The pattern of findings in adjusted analyses was identical in direction and statistical significance (Table 2).

Interactions of product type with demographic characteristics

Exploratory moderation analyses revealed that non-smokers perceived cigarette warnings to be more effective than both SSB warnings and alcohol warnings, whereas smokers perceived all three product warnings as equally effective (Supplementary Figure 2, Panel A). The finding that cigarette warnings were perceived as more effective than alcohol warnings was more pronounced among participants who drank alcohol in the past month than those who had not (Supplementary Figure 2, Panel B). SSB consumption, low-income status, and race/ethnicity did not moderate the impact of product type on perceived message effectiveness.

Efficacy information

The efficacy information did not change self efficacy (mean self efficacy: efficacy group=4.65, SD=.54 vs. no efficacy group=4.63, SD=.53, p=.59). Efficacy information did not change any of the study outcomes in unadjusted or adjusted analyses, nor did it moderate the impact of warning type or product type on the outcomes (all p > .14). The impact of efficacy information on perceived message effectiveness did not differ by participants’ product consumption, low-income status, or race/ethnicity.

Discussion

In our experiment with US adults, we found that graphic warnings for cigarettes, SSBs, and alcohol were perceived as more effective than text-only warnings, although the magnitude of the effect was small. This finding suggests that graphic warnings may be more effective at changing actual behavior based on numerous studies linking perceived message effectiveness with actual effectiveness (11, 32, 40–42). Across product type, graphic warnings also led to greater fear, more thinking about the harms of consuming the product, and lower product appeal. These three constructs are particularly important because they have been shown to mediate the impact of warnings on changes in intentions (43–45) and behavior (11, 46).

Graphic warnings also led to greater reactance, as has been found in other studies (34, 47). However, prior research consistently shows that reactance is not enough to undermine the warnings’ beneficial effects (11, 34, 43). Graphic warnings were also rated as less believable than text-only warnings. While photographic images function as visual evidence (48, 49), it is possible that seeing severe (and thus less common) health consequences reduced participants’ beliefs the health effect could happen to them. Graphic warnings also garnered lower policy support than text-only warnings, although the effect was null in adjusted analyses. We note that prior research has found that labeling generally garners higher support than policies such as taxation and restricting product availability (51–53).

Finally, we found that the effects of graphic warnings on the outcomes did not differ by product type; for instance, graphic warnings were perceived as more effective than text-only warnings regardless of whether the warnings were on cigarettes, SSBs, or alcohol. The lack of interaction by product type suggests that the existing evidence base about graphic cigarette warnings may extrapolate to other contexts including SSBs and alcohol, and vice versa.

Across warning type, compared to SSB and alcohol warnings, cigarette warnings generated higher perceived message effectiveness, believability, fear, thinking about the harms, and policy support. Reactance ratings were lower for cigarette warnings than for SSB and alcohol warnings. Taken together, these findings suggest that cigarette warnings out-performed SSB and alcohol warnings in terms of both effectiveness and acceptability. There are several possible explanations for this set of findings. People may be more aware of cigarettes’ harms due to prominent anti-smoking campaigns and existing warning policies. Moreover, smoking is a less common behavior than SSB consumption or alcohol consumption, and may be perceived as less acceptable (54), which could have contributed to greater receptivity to cigarette pack warnings. Moreover, differences in the perceived or actual harm of products, or the current regulatory environment around the products, could have contributed to these differences in perceived message effectiveness. Our sample included lower numbers of smokers than SSB and alcohol consumers, so the sample of largely non-smokers may have been less resistant to cigarette warnings than SSB and alcohol warnings. Finally, although we attempted to create equivalent warnings across product type based on consumer familiarity and scientific accuracy, we did not assess whether the warning topics were matched on perceived severity and it is possible that disease type could have been confounded with product type.

Our study found that the inclusion of a quit or cut-back website in the warnings did not change people’s perceived self efficacy to limit smoking, SSB consumption, or alcohol consumption. Since our efficacy statement did not change perceived efficacy, we do not know whether the inclusion of information that does increase efficacy would have improved the effectiveness of the graphic warnings. We also note that the mean self efficacy in our sample was high, which may have led to ceiling effects. Although the Extended Parallel Process Model (14) posits that fear-inducing messages like graphic warnings could backfire if they do not increase efficacy, we did not find this to be true in our study. Several meta-analyses have found that fear appeals change behavior even in the absence of efficacy information (55–57). Thus, warnings may not need to include efficacy information to be effective. This finding should be reassuring in the US regulatory context given that newly proposed graphic cigarette pack warnings do not include efficacy information. Future studies could test whether more detailed efficacy information (for example, efficacy-related cigarette pack inserts used in Canada (58)) could enhance the effectiveness of graphic warnings.

In our study, smokers did not perceive cigarette pack warnings as more effective than SSB and alcohol warnings, whereas non-smokers did perceive cigarette warnings as more effective than SSB and alcohol warnings. This finding suggests possible defensive responses to cigarette pack warnings among smokers, in line with prior research (59). We observed similar defensive responses among people who drank alcohol in the past month. The finding that cigarette warnings were rated as more effective than alcohol warnings was more pronounced among those who consumed alcohol in the past month than those who had not. The effects of the other experimental factors (efficacy information, warning type) did not differ by low-income status or race/ethnicity, building on prior studies finding equivalent warning effects on diverse populations (1, 6, 60, 61).

Strengths and limitations

Strengths of this study include the factorial randomized design, the use of professionally designed stimuli, and the comparison of graphic and text warnings matched on size, color, and text placement. Limitations include that we only displayed one product for each product type, which may be problematic for SSBs given that reactions to warnings may differ by beverage category (62). Similarly, we only tested one warning for each product; future studies should examine other types of health effects and other types of warnings (for example, nutrient-based warnings like “high in sugar,” which are popular globally). Other limitations include the use of single-item measures due to survey space constraints and the lack of behavioral outcomes. We did not assess whether the warning topics were equivalent on perceived severity so it is possible that disease type could have been confounded with product type. The study tested subtle efficacy-related language that may be more akin to action information than a traditional efficacy statement (e.g., “Cutting back is easy”); this may have led to null effects of efficacy information. Finally, the generalizability of study findings remains unknown given the use of a convenience sample, especially since the sample differed from nationally representative samples on some demographic characteristics (e.g., age, smoking status). However, warnings tend to elicit similar reactions across population groups (1, 8, 61) and experimental studies conducted with convenience samples largely replicate those conducted with probability samples (21).

Conclusions

This study demonstrates the promise of graphic warnings for cigarettes, SSBs, and alcohol, as graphic warnings performed better than text-only warnings on several predictors of behavior change, including perceived message effectiveness, fear, thinking about harms, and product appeal. Cigarette warnings generally out-performed SSBs and alcohol warnings, in terms of both effectiveness and acceptability. Graphic warnings were equally effective for all three products. Future studies should examine whether these findings replicate with behavioral outcomes, in different samples, and when using a wider variety of warnings.