How should sugar-sweetened beverage health warnings be designed? A randomized experiment

Abstract

Health warnings are a promising strategy for reducing consumption of sugar-sweetened beverages (SSBs), but uncertainty remains about how to design warnings to maximize their impact. Warnings already implemented in Latin America use nutrient disclosures, while proposed U.S. warnings would describe the health effects of consuming SSBs. We sought to determine whether warning characteristics influence consumers’ reactions to SSB health warnings. A national convenience sample of U.S. adults (n=1,360) completed an online survey in 2018. In a factorial design, we randomly assigned participants to view SSB health warnings that differed in: 1) inclusion of health effects (“Drinking beverages with added sugar contributes to obesity, diabetes, and tooth decay”); 2) inclusion of a nutrient disclosure (“High in added sugar”); 3) inclusion of the marker word “WARNING;” and 4) shape (octagon vs. rectangle). The primary outcome was perceived message effectiveness (PME, range 1–5). PME was higher for warnings that included health effects (average differential effect [ADE]=0.63, p<0.001) or nutrient disclosures (ADE=0.32, p<0.001) compared to warnings without this information. However, adding a nutrient disclosure to a warning that already included health effects did not lead to higher PME compared to warnings with health effects alone. The marker “WARNING” (ADE=0.21) and the octagon shape (ADE=0.08) also led to higher PME compared to warnings without these characteristics (ps<0.001). The same pattern of results held for the secondary outcomes, fear and thinking about harms. SSB health warnings may have more impact if they describe health effects, use the marker “WARNING,” and are octagon-shaped.

Introduction

Excess consumption of sugar-sweetened beverages (SSBs) remains a pressing public health issue in the United States. Half of adults consume SSBs on any given day,¹ and average caloric intake from SSBs remains well above national dietary guidelines.^2,3 Evidence indicates that SSB consumption increases risk of developing obesity,^4,5 diabetes,^6,7 and heart disease.⁸ To reduce consumption of SSBs, five states have proposed requiring front-of-package (FOP) health warnings on SSB containers.^9–13

Even as interest in SSB health warning policies has grown, questions remain about how to design warnings to maximize their effectiveness. For example, warnings proposed in the U.S. describe the health effects of consuming SSBs.^9–13 In contrast, nutrition warning systems adopted in countries such as Chile do not describe health effects, but instead display a nutrient disclosure that signals when a product exceeds recommended levels of sugar, sodium, saturated fat, or calories. For example, SSBs in Chile display FOP warnings that read “Alto en azúcares” (“high in sugars”).^14,15 Another difference is warning label shape: in Chile, warnings are displayed on octagonal labels, while SSB warnings in the U.S. would likely be displayed on rectangular labels. Additionally, the proposed SSB health warnings in the U.S.^9–13 begin with a marker word (“WARNING” or “HEALTH WARNING”) that signals that the subsequent text is a warning message, while labels in other countries often do not use marker words.^14,16,17

These four warning characteristics – health effects, nutrient disclosures, label shape, and marker words – could influence how effectively SSB health warnings discourage SSB consumption. For example, cigarette warnings that describe health effects elicit higher perceived effectiveness,¹⁸ and warnings with health effects statements or nutrient disclosures have been found to reduce consumers’ intentions to purchase SSBs.^19–21 Others have found that consumers associate the octagon shape with unhealthfulness.²² Including marker words such as “CAUTION,” or “WARNING” (or similar marker symbols²³) may draw attention to warnings,^24–26 but makes messages longer, potentially reducing readability.

Limited research has examined these warning characteristics side-by-side or in combination with one another. The objective of this study was to examine the influence of health effects, nutrient disclosures, marker words, and label shape on perceptions of messages’ effectiveness at discouraging SSB consumption. Based on previous research, we predicted that warnings that included health effects^18,19 or nutrient disclosures^21,27 would elicit higher perceived message effectiveness than warnings without these characteristics, and that octagon-shaped labels would elicit higher perceived message effectiveness than rectangular labels.^22,28 We did not make an a priori prediction regarding marker words because they might increase attention but reduce readability. We also examined whether these four warning characteristics elicit more fear or thinking about the harms of SSB consumption. We focused on perceived message effectiveness,^29–33 fear,^33,34 and thinking about harms^33,35,36 because these outcomes have been found to predict warnings’ actual effectiveness. We also assessed whether warning characteristics affect consumers’ knowledge of the health harms of SSB consumption and identified the warning color combinations perceived to be most effective.²²

Methods

Participants

In April 2018, we recruited a convenience sample of 1,413 U.S. adults ≥18 years using Amazon Mechanical Turk (MTurk), an online platform commonly used by social and behavioral science researchers.^37–39 Research indicates that experiments conducted on MTurk replicate findings from studies conducted both in the lab⁴⁰ and via random-digit dial phone surveys.⁴¹ Participants earned $2.20 for completing the 10–15 minute survey.

Impact of Warning Characteristics on Consumer Reactions

Procedures.

The main experiment varied characteristics of SSB health warnings using a mixed between/within factorial design. First, we randomly assigned participants to one of four between-subjects conditions: 1) control (“Always read the Nutrition Facts Panel”), 2) health effects only (“Drinking beverages with added sugar contributes to obesity, diabetes, and tooth decay,” adapted from California’s proposed warnings⁹), 3) nutrient disclosure only (“High in added sugar,” adapted from Chile’s warnings¹⁴), and 4) health effects and nutrient disclosure. These four conditions represented the combination of two between-subjects factors, each with two levels: 1) whether the warning included health effects and 2) whether the warning included a nutrient disclosure.

Participants viewed their randomly assigned warning message four times, on four labels that differed on two within-subjects factors, each with two levels: whether the message began with the marker word “WARNING” and the shape of the warning label (rectangle vs. octagon). Thus, the experiment had four within-subjects conditions, each representing a different warning label design: 1) no marker and rectangle shape, 2) no marker and octagon shape, 3) “WARNING” marker and rectangle shape, and 4) “WARNING” marker and octagon shape. Participants viewed these four labels in a random order.

In total, we created 16 different warnings: one for each of the four between-subjects conditions, displayed on warnings that varied along each of the four within-subjects conditions (Figure 1). Participants viewed warnings presented mocked up on an unbranded bottle of soda (Figure 2). Presenting warnings on an unbranded soda bottle allowed us to focus participants’ attention on the warning characteristics of interest while also presenting a realistic image of what SSB warnings might look like if implemented. To mimic Chilean labels, we displayed warnings in white text on a black background.

Figure 1.

Experimental conditions

Figure 2.

Example experimental stimulus showing a generic beverage with a sugar-sweetened beverage health warning label and call-out of the enlarged label. Dimensions of image on a desktop computer screen were ~5” × 6.4”.

Measures.

Participants viewed warnings one at a time. After viewing each warning, participants rated the warning on effectiveness at discouraging SSB consumption (primary outcome) and on thinking about the harms of SSB consumption and fear (secondary outcomes). The survey assessed perceived message effectiveness (PME) with an adapted version of the UNC Perceived Message Effectiveness Scale.⁴² PME is commonly used in message development studies⁴³ and was found in a recent meta-analysis to predict messages’ actual behavioral efficacy.⁴⁴ Our three PME items read: “This label makes me concerned about the health effects of drinking beverages with added sugar;” “This label makes drinking beverages with added sugar seem unpleasant to me;” and “This label discourages me from wanting to drink beverages with added sugar.” The 5-point response scale ranged from “strongly disagree” (coded as 1) to “strongly agree” (coded as 5). We averaged responses to these three items to create a composite score (Cronbach’s alpha=0.93, range across conditions: 2.52 to 3.80).

The survey assessed thinking about the harms of SSB consumption using a single item, adapted from studies of cigarette warnings.^45–47 “How much does this label make you think about the health problems caused by drinking beverages with added sugar?” Finally, the survey assessed fear using one item also adapted from previous studies of cigarette warnings,^45,48 “How much does this label make you feel scared?” Response options for these items ranged from “not at all” (coded as 1) to “very much” (coded as 5).

Knowledge of Consequences of SSB Consumption

As a secondary outcome, we also assessed the effect of the between-subjects factors, health effects and nutrient disclosure, on knowledge of the health harms of SSB consumption. After rating all four warnings and completing the two items about color described below, participants indicated whether SSB consumption contributes to: obesity, diabetes, tooth decay, and heart disease. Because SSB consumption may increase risk of these outcomes,^4,8,49,50 we coded responses as correct if participants reported awareness of each health consequence and incorrect otherwise.

Most Discouraging Color Combinations

In a separate task, we also examined the warning label color combination participants perceived as most discouraging. After rating all four warnings, participants viewed a set of six rectangular warnings with the same text (“WARNING: High in added sugar”) but different combinations of background, border, and text color (Supplemental Table 1) displayed in a random arrangement. Participants selected the color combination that “would discourage you most from wanting to drink beverages with added sugar.” Participants then completed an identical item for octagon-shaped warnings.

Attention Check and Demographics

Participants completed an attention check in which they were asked to intentionally not answer an item. Participants also provided information on their demographic characteristics and health behaviors.

The University of North Carolina, Chapel Hill Institutional Review Board approved this study. Prior to data collection, we pre-registered the study’s sample size, primary hypotheses, design, and analysis plan on AsPredicted.org (https://aspredicted.org/7iz2y.pdf).

Analysis

We identified duplicate IP addresses and MTurk usernames and retained the record with the most complete data, or, when the amount of missing data was equivalent, the first record. This resulted in dropping 40 records. We also excluded 13 records for people who previously participated in pilot testing of the experiment, yielding a final analytic sample of n=1,360. These 1,360 participants each rated at least one warning and were included in analyses of the primary outcome see CONSORT flow diagram in (Supplemental Figure 1). We used intent-to-treat analyses, analyzing all participants in their assigned conditions including those who did not pass the attention check.⁵¹ We conducted analyses in Stata/SE version 15.1 (StataCorp LLC, College Station, TX).

We used mixed effects (i.e., multi-level) linear models to assess how the four manipulated warning characteristics (health effects, nutrient disclosure, marker word, and label shape) affected the primary outcome of perceived message effectiveness while accounting for the repeated measures design. We entered the within-subjects factors (marker word, label shape) as Level 1 variables and the between-subjects factors (health effects, nutrient disclosure) as Level 2 variables, treating the intercept as a random effect. Sample characteristics did not differ by experimental arm, so we conducted unadjusted analyses. The initial model included indicators for the four manipulated warning characteristics and all interactions between these four factors. The final model retained only significant interactions from the initial model. We used the same approach to examine the effects of warning characteristics on our secondary outcomes, thinking about harms and fear. We report raw means and average differential effects of each experimental factor on the outcomes as generated by the mixed models. We probed interactions by calculating means and average differential effects at different levels of the moderating factors.

In pre-specified analyses, we examined whether participant characteristics moderated the relationship between warning characteristics and PME. We examined six moderators: overweight/obese status (BMI ≥25 vs. <25 kg/m²), obese status (BMI ≥30 vs. <30 kg/m²), SSB consumption (≥ 4.5 vs. < 4.5 servings/week [sample median]), educational attainment (college degree or more vs. some college or less), income (>150% of the Federal Poverty Level [FPL] vs. ≤ 150% FPL), and race (white vs. non-white).

We assessed the impact of the two between-subjects factors (health effects and nutrient disclosure) on knowledge of SSB health consequences using general (i.e., not mixed) logistic regression, reflecting that participants responded to knowledge items only once, after seeing all of their assigned warnings. The initial models included both factors and their interaction; the interactions were not significant in any model so were removed from final models. To identify the color combinations perceived as most effective, we calculated the proportion of participants who selected each color combination as the “most discouraging” for each label shape (rectangular and octagonal).

Results

Sample

Participants’ average age was 37.4 years (Table 1). About 17% of participants had a household income of 150% FPL or less. The sample was younger, more likely to identify as gay, lesbian, or bisexual, less likely to identify as Hispanic, more likely to smoke, and less likely to have a BMI in the obese category compared to nationally representative samples (Supplemental Table 2). Nearly all participants (98%) passed the attention check. Sample characteristics did not differ by experimental condition.

Table 1.

Participant characteristics, n=1,360 U.S. adults (April 2018)

Characteristic	n	%
Age
18–29 years	361	27%
30–39 years	547	40%
40–54 years	295	22%
55+ years	149	11%
Mean (SD)	37.4	11.5
Gender
Male	704	52%
Female	639	47%
Transgender or other	9	1%
Gay, lesbian, or bisexual	141	10%
Hispanic	122	9%
Race
White	1,106	82%
Black or African American	127	9%
Asian	63	5%
Other/multiracial	47	3%
American Indian or Alaskan Native	8	1%
Native Hawaiian or Pacific Islander	1	0.1%
Education
High school diploma or less	170	13%
Some college	313	23%
College graduate or associates degree	699	52%
Graduate degree	170	13%
Household income, annual
$0-$24,999	234	17%
$25,000-$49,999	425	31%
$50,000-$74,999	322	24%
$75,000+	370	27%
Low income (≤ 150% of the Federal Poverty Level)	224	17%
Current smoker	298	22%
Sugar-sweetened beverage consumption
<1 time per day	866	64%
1 to <3 times per day	312	23%
3 or more times per day	175	13%
Body mass index (BMI, kg/m2)
Underweight	49	4%
Healthy weight	519	38%
Overweight	409	30%
Obese	301	22%
Not reported	82	6%
Mean (SD)	26.6	6.8
Passed attention check	1,338	98%

Note. Characteristics and outcomes did not differ by experimental arms. Missing demographic data ranged from 0.5% to 0.9%, except for BMI (6.0% missing) (see Supplemental Table 3).

Perceived Message Effectiveness

Main effects of experimental factors.

Warnings that included health effects were perceived as more effective than warnings without health effects (average differential effect [ADE]=0.63, p<0.001) (Figure 3). Warnings with nutrient disclosures also led to higher PME compared to warnings without nutrient disclosures (ADE=0.32, p<0.001). Likewise, PME was higher for warnings that included the marker word “WARNING” (ADE=0.21, p<0.001) than warnings without a marker word and for warnings displayed on octagon-shaped labels compared to rectangular labels (ADE=0.08, p<0.001).

Figure 3.

Impact of the four warning characteristics manipulated in the experiment on perceived message effectiveness (5,431 ratings), thinking about harms (5,430 ratings), and fear (5,431 ratings) from 1,360 U.S. adults (April 2018).

Interactions between experimental factors.

Nutrient disclosure interacted with health effects (p for interaction <0.001, Supplemental Table 4). Adding a nutrient disclosure led to higher PME when the warning did not include health effects (Mean [M]=2.75 vs. M=3.41; ADE=0.66, p<0.001) (Figure 4). However, the addition of a nutrient disclosure had no benefit when a health effects statement was also included (M=3.71 vs. M=3.70; ADE=−0.01, p=0.90).

Figure 4.

Interaction between health effects and nutrient disclosure on mean (A) perceived message effectiveness (5,431 ratings), (B) thinking about harms (5,430 ratings), and (C) fear (5,431 ratings) among 1,360 U.S. adults (April 2018). Error bars show 95% confidence intervals.

Marker word interacted with health effects (p for interaction<0.001, Supplemental Table 4). For warnings that did not include health effects, adding a marker word led to higher PME compared to not having a marker word (M=2.91 vs. M=3.24; ADE = 0.32, p < 0.001, Supplemental Fig. 2). For warnings that included health effects, adding a marker word still increased PME, but the impact was smaller (M=3.66 vs. M=3.75; ADE=0.09, p<0.001).

Marker word also interacted with nutrient disclosure (p for interaction<0.001, Supplemental Table 2). For warnings that did not include a nutrient disclosure, adding the marker word led to higher PME compared to warnings without a marker word (M=3.10 vs. M=3.35; ADE=0.25, p<0.001) (Supplemental Figure 3). For warnings with a nutrient disclosure, adding the marker word again led to higher PME (M=3.47 vs. M=3.64; ADE=0.16, p<0.001), though the effect was smaller.

Interactions between experimental factors and participant characteristics.

Only two of the twenty-four interactions between participant characteristics (income, education, race, overweight, obesity, or SSB consumption) and the experimental factors on PME were statistically significant, potentially indicating type I error. Nutrient disclosure had a smaller impact on PME for high SSB consumers compared to low-consumers (p for interaction=0.012). Octagon-shaped labels had a larger impact on PME for participants with an overweight/obese BMI than those with BMI in the normal range (p for interaction=0.038).

Fear and Thinking about Harms

Main effects of experimental factors.

A similar pattern of results emerged for fear and thinking about harms, the secondary study outcomes. Of the warning characteristics, health effects had the largest impact on both thinking about harms (ADE=0.66, p<0.001) and fear (ADE=0.42, p<0.001) (Figure 3). Including a nutrient disclosure also increased thinking about harms (ADE=0.23, p<0.001) and fear (ADE=0.15, p=0.013). The marker word “WARNING” increased thinking about harms and fear (ADE=0.22 and 0.23, respectively, both p’s<0.001). Finally, compared to rectangular labels, octagon-shaped labels elicited more thinking about harms (ADE=0.08, p<0.001) and fear (ADE=0.09, p<0.001).

Interactions between experimental factors.

Nutrient disclosure again interacted with health effects, a finding replicated for both thinking about harms (p for interaction <0.001) and fear (p for interactions <0.05). Including both health effects and a nutrient disclosure again did not perform better than including health effects alone (Figure 4). Marker word again interacted with health effects, showing a similar pattern as for PME (ps for interactions <0.001) (Supplemental Table 4, Supplemental Figure 2). However, unlike for PME, marker word did not interact with nutrient disclosure for either secondary outcome (ps for interactions >0.30).

Knowledge of Consequences of SSB Consumption

Knowledge that SSB consumption contributes to tooth decay was 2.1 percentage points higher among participants exposed to warnings that included health effects (p=0.048) (Supplemental Table 5). Exposure to health effects messages did not affect knowledge that SSBs contribute to obesity or diabetes (ps > 0.25), but led to lower knowledge that SSBs contribute to heart disease, information not included in the warnings, by 9.4 percentage points (60.8% vs. 51.4% answered correctly, p < 0.001).

Color Combinations Selected as Most Discouraging

For octagon-shaped labels, the majority of participants (75%) said that a warning with red background and white text would most discourage them from consuming beverages with added sugar (Supplemental Table 1). Likewise, for rectangle-shaped labels, most (66%) participants indicated this color combination would most discourage them. The between-subjects factors (health effects and nutrient disclosure) did not impact color combination selections (ps>0.19).

Discussion

SSB health warnings are a promising policy strategy for reducing SSB consumption. Yet little is known about how to best design such warnings to maximize their impact. In this experimental study of U.S. adults, we found that warning characteristics influence reactions to SSB health warnings. Specifically, warnings that described health effects, included a nutrient disclosure, began with the marker word “WARNING,” and were displayed on octagon-shaped labels were perceived to be more effective than warnings without these characteristics. These characteristics also increased thinking about the harms of SSB consumption and feelings of fear. Participants selected the red background with white text as the most discouraging color combination for both octagonal and rectangular warnings. Because past research has shown that these reactions (perceived message effectiveness,^29–33,44 thinking about harms,^33,35,36 and fear^33,34) predict warnings’ actual effectiveness, our findings suggest design choices that could increase the impact of SSB health warnings.

SSB health warnings proposed in the U.S. have all included health effects.^9–13 This is a wise choice, given that health effects had the largest impact of the warning characteristics we studied. This finding is consistent with cigarette warning research, which has found that health effects messages are generally more potent than “found in” statements identifying toxic products that contain cigarette smoke chemicals.⁵² Others have suggested health effects increase perceived message effectiveness by providing contextualizing information that increases motivation to think about the warning message and helps consumers understand the harms of a particular product.^18,52 In contrast to the U.S., warning systems implemented in Latin American countries do not describe health effects, instead using nutrient disclosures.^14–16,53 These nutrient disclosures accompany all foods and beverages that exceed thresholds for certain nutrients, not just SSBs. Future research should compare health effects warnings to nutrient disclosures on a larger variety of products in U.S. and non-U.S. samples.

Adding more text to warnings in our experiment had diminishing returns. Across outcomes, the textual warning characteristics we manipulated (health effects, nutrient disclosure, and marker word) interacted with one another, such that the additional impact of a textual characteristic (e.g., a marker word) was generally lower when a message already included another textual warning characteristic (e.g., health effect) than when it did not. The interaction between health effects and nutrient disclosures was particularly large: adding a nutrient disclosure to a warning that did not include health effects increased perceived message effectiveness, thinking about harms, and fear, but adding a nutrient disclosure to a warning that already included a health effects statement had no additional influence on these outcomes. These results suggest that SSB health warnings may perform best when they include only a nutrient disclosure or only health effects, but not both. These findings are consistent with other studies suggesting that “less is more” when showing consumers comparative quality information.⁵⁴ Our findings also replicate studies from the tobacco warnings literature.^18,52 For example, cigarette warnings studies have shown the same pattern of “less is more” interaction such that combining the two forms of risk information (health effects and “found in” statements) did little or no better than presenting either one alone.⁵²

Consistent with previous research on SSB and tobacco warnings,^19,20,55 warning characteristics had similar impact regardless of participants’ income, education level, and race/ethnicity. One exception was that nutrient disclosures had a slightly smaller influence on perceived message effectiveness for high SSB consumers compared to low consumers. This finding could be explained by the defensive processing literature, which suggests that resistance to messages is strongest among people engaging in the behavior targeted by the message.^56,57 The other exception was that the octagon shape had a larger influence on perceived message effectiveness for participants with an overweight/obese BMI.

Strengths of our experiment include the large sample from across the U.S. and that we randomly assigned participants to conditions using a fully factorial design. Limitations include using a convenience sample, which may limit the generalizability of the findings. However, recent research has found that experiments conducted on MTurk generally replicate findings of experiments conducted using probability-based samples.^39,58,59 Previous research has found that the impact of SSB health warnings on consumer perceptions varies by SSB type (e.g., fruit drinks vs. sodas).⁶⁰ Because we only displayed warnings on sodas, we were unable to examine whether SSB type moderated the impact of the manipulated warning characteristics on our study outcomes. We also displayed warnings on non-branded SSBs on a computer screen, and warnings were likely more noticeable than they would be if implemented on actual SSBs in retail settings. Finally, study outcomes were all based on self-report after brief exposure to the warnings. A recent meta-analysis indicates that self-reported perceived message effectiveness (our primary outcome) predicts actual behavior change for tobacco messages,⁴⁴ but future studies should examine whether warnings with these characteristics affect consumer behavior.

Conclusions

To maximize the impact of SSB health warnings, policymakers should consider adopting warnings that describe health effects, begin with the marker word “WARNING,” and are displayed on an octagon-shaped label, as warnings with these characteristics are perceived to be more effective, and elicit more thinking about harms and fear, than warnings without these characteristics. Warnings that include a nutrient disclosure also increase perceived effectiveness over warnings that do not, but to a lesser extent than warnings with health effects. Further, including both a nutrient disclosure and health effects is unlikely to improve effectiveness over health effects alone. Future work should assess whether these principles apply to other types of warnings (e.g., on alcohol or junk food) and in other countries, and should examine whether warnings with these characteristics influence behavioral outcomes.

Highlights.

• Uncertainty remains about how to design impactful health warnings for sugary drinks

• Describing health effects increases perceived effectiveness of sugary drink warnings

• Including a nutrient disclosure also increases perceived effectiveness

• But, including both of these elements reduces each’s impact on perceived effectiveness

• “Marker” words and octagon-shaped labels also increase perceived effectiveness

Abbreviations:

BMI

body mass index

FOP

front-of-package

MTurk

Mechanical Turk

SSBs

sugar-sweetened beverages