Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jun 1.
Published in final edited form as: J Behav Med. 2017 Jan 24;40(3):520–529. doi: 10.1007/s10865-016-9821-z

A brief measure of reactance to health warnings

Marissa G Hall 1,4,✉, Paschal Sheeran 2,4, Seth M Noar 3,4, Kurt M Ribisl 1,4, Marcella H Boynton 1,4, Noel T Brewer 1,4
PMCID: PMC5458745  NIHMSID: NIHMS847804  PMID: 28120228

Abstract

Reactance to persuasive messages involves perceived threat to freedom, anger, and counterarguing that may undermine the impact of health warnings. To understand reactance’s effects, reliable and valid assessment is critical. We sought to develop and validate a brief Reactance to Health Warnings Scale (RHWS). Two independent samples of US adults completed the brief RHWS in studies that presented warnings on cigarette packs that smokers carried with them for 4 weeks (Study 1; n = 2149) or as digital images of cigarette packs that participants viewed briefly (Study 2; n = 1413). The three-item Brief RHWS had good internal consistency and test–retest reliability. The scale correlated with higher trait reactance and exposure to pictorial warnings, supporting its convergent validity. With respect to predictive validity, the Brief RHWS predicted perceived message effectiveness, quit intentions, avoidance of the warnings, and number of cigarettes smoked per day. The Brief RHWS can serve as an efficient adjunct to the development of persuasive messages.

Keywords: Reactance, Defensive processing, Health warnings, Tobacco control, Pictorial warnings, Health communication

Introduction

Health messages aim to encourage people to engage in healthier behaviors, such as quitting smoking or getting vaccinated. However, these messages sometimes elicit opposition due to feelings that one’s autonomy is being threatened. Termed reactance, theorists suggest that this negative reaction to persuasive messages can reduce message effectiveness (Brehm, 1966; Brehm & Brehm, 1981; Witte, 1992). Drawing on the rich body of empirical and conceptual work on reactance (Dillard and Shen, 2005; LaVail et al., 2010; Quick, 2012; Quick & Stephenson, 2007; Rains, 2013; Witte, 1992, 1994), we define reactance as an emotional and cognitive resistance to a message, characterized by (1) perceived threat to freedom, (2) anger toward the message, and (3) counterarguments against the message, such as denial or derogation. Based on this definition, we developed and evaluated the validity of a 9-factor, 27-item Reactance to Health Warnings Scale (RHWS; Hall et al., 2016) in the context of pictorial cigarette pack warnings. We found that the RHWS had high reliability and good construct validity (Hall et al., 2016). As expected, reactance factors were positively correlated with trait reactance (i.e., a personality characteristic reflecting a predisposition to reactance), being a smoker, and exposure to pictorial cigarette pack warnings (vs. a text-only warning control).

Although the RHWS exhibits strong psychometric properties, its length may pose a challenge in many research contexts. For instance, researchers evaluating multiple candidate messages or images may wish to assess reactance, but administering 27 items for each of multiple messages would be overly burdensome and repetitive for study participants. Moreover, many studies of tobacco product warnings, disclosures, and public media campaigns are currently underway to assess their viability and effectiveness for communicating the risks of tobacco use; administration of a short measure of reactance could help researchers to select effective messages that elicit minimal reactance and thereby have a greater impact on tobacco-related beliefs and behaviors. Given these considerations, the current study describes the development and validation of a short reactance scale that could be used not only in the context of anti-tobacco messaging, but also for assessing reactance in other health messaging domains.

Methods

Study 1

Participants

From September 2014 to August 2015, we recruited a convenience sample of adult smokers in North Carolina and California, US to participate in a trial comparing the impact of pictorial versus text-only warnings (Brewer et al., 2016). Participants were 18 years of age or older, proficient in English, and current smokers, defined as having smoked at least 100 cigarettes during their lifetime and now smoking every day or some days. Exclusion criteria included pregnancy, current enrollment in a smoking cessation trial, smoking only roll-your-own cigarettes, smoking fewer than seven cigarettes per week, and living in the same household as another study participant. Details regarding recruitment, design, and methods can be found in Brewer et al. (2016).

Procedures

Smokers received warnings on their own cigarette packs for four weeks (Brewer et al., 2015). Participants brought in an 8-day supply of cigarettes weekly. They were randomly assigned to have one of four pictorial warnings applied to the top half of the front and back panels of their cigarettes packs (Fig. 1), or one of four text-only warnings applied to the side of their cigarette packs, for the duration of the study. Randomization created groups that did not differ on demographics assessed (all p > .05) (Brewer et al., 2016). Participants completed two computer surveys at the first study visit (i.e., baseline and immediately after seeing their assigned warning, or immediate post-test), and one survey at each visit thereafter. Participants received a cash incentive at the end of each visit, up to $185 in North Carolina and $200 in California. At the end of the final follow-up appointment, participants received information about local smoking cessation programs.

Fig. 1.

Fig. 1

Open in a new tab

Pictorial warnings used in Studies 1 and 2

Measures

The immediate post-test survey at the first study visit assessed demographic characteristics, trait reactance [11 items, α = .87, with response options ranging from strongly disagree (coded as 1) to strongly agree (coded as 5)] (Hong & Page, 1989), and positive smoker prototypes (4 items, α = .84) (Gerrard et al., 2008; McCool et al., 2011). We assessed message reactance using the 27-item RHWS (Hall et al., 2016) at the immediate post-test survey, the week 1 follow-up survey, and the week 4 follow-up survey. The outcome variables used to assess predictive validity were obtained at the week 4 follow-up survey, and comprised perceived effectiveness of the warning (six items, α = .90, “How much did having this warning on your cigarette packs make you want to quit smoking? How much did having this warning on your cigarette packs make you concerned about the health effects of smoking? How much would having this warning on cigarette packs…Make other smokers concerned about the health effects of smoking? Make other smokers want to quit smoking? Make non-smokers concerned about the health effects of smoking? Discourage non-smokers from starting to smoke?”), support for requiring pictorial warnings on cigarette packs (one item: “If the US required that graphic warnings covered the top half of the front and back of cigarette packs, would you… strongly oppose this policy, somewhat oppose this policy, somewhat support this policy, or strongly support this policy?”), quit intentions (three items, α = .94, e.g., “How interested are you in quitting smoking in the next month?”) (Klein et al., 2009), avoidance of the warning (three items, α = .90, e.g., “In the last week, how often have you tried to avoid looking at the warning label on your cigarette packs?”) (Population Assessment of Tobacco and Health Study, 2014), and forgoing a cigarette (one item: “In the last week, how often have you stopped yourself from having a cigarette because you wanted to smoke less?”) (Li et al., 2014). We also assessed the number of cigarettes smoked per day.

Data analysis

Analyses used Stata/SE version 14.1 with two-tailed tests and a critical alpha of 0.05. In Study 1, reactance at immediate post-test did not differ among the four text-only warnings (F < 1) or among the four pictorial warnings (F < 1). For Study 2, we previously reported that comparisons of the five warnings in the pictorial condition revealed few differences in reactance (Hall et al., 2016). Thus, in both studies, we combined the warnings into two groups (text-only vs. pictorial) for all analyses.

To prioritize items for the brief measure, we used item response theory (IRT)-based modeling (De Ayala, 2013; Embretson & Reise, 2013) with immediate post-test data from Study 1. For each of the nine reactance subscales, we selected the item with the highest factor loading from our previous validation study (Hall et al., 2016) that was at or below an eighth-grade reading level (Table 1). Then, we entered these nine items into a graded-response IRT model. After testing IRT assumptions, we eliminated two items that violated the assumption of local dependence and one that violated the assumption of monotonicity (De Ayala, 2013; Embretson & Reise, 2013). Then, we ran a second IRT model with the remaining six items, ultimately selecting the three items for the Brief RHWS that contributed the most information about the underlying latent construct of reactance based on the item information curves (the items are listed in Table 3).

Table 1.

Reactance items selected for item response theory testing, Study 1

Subscale Item Factor loading Readabiliy
Anger This warning makes me feel aggravated 0.95 8.8
Anger This warning irritates me 0.76 9.4
Anger This warning annoys me 0.77 4.4
Exaggeration This warning is misleading 0.59 10.0
Exaggeration This warning exaggerates the health effects of smoking 0.61 10.3
Exaggeration The health effect on this warning is overblown 0.63 7.6
Government The government shouldn’t require warnings like this on packs 0.71 10.6
Government Smoking is legal, so the government should stop interfering with smokers’ freedom 0.90 11.7
Government It’s not the government’s job to warn me about the risks of smoking 0.71 5.5
Manipulation This warning is trying to boss me around 0.34 3.9
Manipulation This warning is trying to manipulate me 0.53 7.6
Manipulation This warning is manipulating smokers 0.45 12.3
Personal attack This warning tells me I’m bad because I smoke 0.53 2.8
Personal attack This warning tells me that I’m stupid 0.45 2.8
Personal attack I am being told that I am a fool by this warning 0.34 2.2
Common knowledge I’ve heard the information in this warning a million times 0.71 6.9
Common knowledge The information in this warning is common knowledge 0.73 9.5
Common knowledge I already knew about the harms in this warning 0.67 6.2
Denial I’ll quit long before I suffer the health effect in this warning 0.55 4.7
Denial The health effect in this warning won’t catch up to me for a long time 0.48 3.9
Denial I would worry more about this warning if I expected to smoke for many years 0.40 7.0
Derogation This warning is stupid 0.62 4.4
Derogation This warning is useless 0.67 8.4
Derogation This warning is pointless 0.58 6.0
Self-relevance This warning is meant for other smokers, not me 0.74 4.2
Self-relevance This warning is only meant for hard-core smokers 0.69 4.7
Self-relevance This warning is not relevant to me 0.71 5.4
Open in a new tab

Italicized rows depict items selected for item reponse theory model. Readability is the reading grade level of the message, calcluated as an average of five readability scores, obtained from readability-score.com. Factor loadings from previously-reported data in Hall et al. (2016)

Table 3.

Brief Reactance to Health Warnings Scale

Study 1
Study 2
Immediate post-test
(n = 2149)
Mean (SD)		 Week 1
(n = 1854)
Mean (SD)		 Week 4
(n = 1901)
Mean (SD)		 (n = 1413)
Mean (SD)
Item 1. This warning is trying to manipulate me 2.16 (1.15) 2.01 (1.07) 2.03 (1.04) 3.02 (1.30)
Item 2. The health effect on this warning is overblown 1.96 (1.01) 1.90 (0.95) 1.90 (0.93) 2.08 (1.05)
Item 3. This warning annoys me 2.14 (1.10) 2.03 (1.04) 2.02 (1.00) 2.36 (1.25)
Brief RHWS 2.08 (0.89) 1.98 (0.85) 1.98 (0.84) 2.49 (0.93)
Cronbach’s alpha, Brief RHWS 0.75 0.77 0.80 0.65
Open in a new tab

Response scale ranged from 1 (strongly disagree) to 5 (strongly agree). Missing data range from 0.0% to 0.7%

We ran a confirmatory factor analysis with the three Brief RHWS items at immediate post-test to evaluate factor loadings, although we could not determine model fit because the model was just-identified (Bollen, 1998). We then calculated mean scores and assessed internal consistency using Cronbach’s alpha. We also assessed test–retest reliability for reactance between immediate post-test, week 1, and week 4. For convergent validity, we derived our predictions from the Theory of Psychological Reactance (Brehm, 1966; Brehm & Brehm, 1981), anticipating that higher Brief RHWS scores would be correlated with the RHWS long form, higher trait reactance, greater positive smoker prototypes, and exposure to pictorial warnings (i.e., random assignment to the pictorial condition). In terms of predictive validity, we drew upon the fear appeals literature (Witte, 1992; Witte & Allen, 2000), expecting that higher Brief RHWS scores at immediate post-test would be correlated with several deleterious consequences including lower perceived effectiveness, less support for pictorial warning policy, lower quit intentions, greater avoidance of warnings, a lower likelihood of forgoing a cigarette at the week 4 follow-up survey, and smoking more cigarettes per day. Finally, we ran these validity analyses using the RHWS long form in order to compare the validity of the long form to the Brief RHWS.

Validity analyses controlled for study arm and used pairwise deletion for missing data, using cases with complete data on the variables of interest for each model. Convergent validity analyses treated reactance as the outcome and thus used linear regression with standardized coefficients. For predictive validity analyses, we present standardized regression coefficients for continuous outcomes and odds ratios and 95% confidence intervals for dichotomous outcomes.

Study 2

Participants

In May 2014, we used Amazon Mechanical Turk (MTurk) to recruit a convenience sample of 1500 US smokers and non-smokers 18 years of age or older. MTurk is a web-based crowd-sourcing platform that is widely used for social science research and shown to generate reliable and valid data (Buhrmester et al., 2011; Paolacci & Chandler, 2014; Peer et al., 2014). We excluded 87 respondents who failed standard procedures for ensuring data quality, resulting in a final sample size of 1413 respondents. While we used this sample to validate the long form of the scale, we use it here to test the psychometric properties of the brief measure. The University of North Carolina Institutional Review Board approved the procedures for both studies.

Procedures

Smokers viewed the warnings on a computer screen. Participants took a survey while viewing an image of an unbranded cigarette pack with a randomly assigned warning on the top half. They viewed one of five warnings with an image depicting the health consequences of smoking and related text (n = 1204, Fig. 1) or one of the same five warnings without the image (n = 209). Randomization to pictorial or text condition yielded equivalent groups on eight of nine demographic variables; participants had lower levels of education in the text than in the pictorial condition (p < .05). Participants received $3 for completing the online survey.

Measures

The survey assessed demographics, the 27-item RHWS, and trait reactance using the same measures as Study 1. The survey also assessed positive smoker prototypes using an expanded 6-item version of the scale (α = .86) (Gerrard et al., 2008; McCool et al., 2011). Among smokers, the survey assessed perceived effectiveness of the warning (four items, α = .85, “How much would having this warning on your cigarette packs… Make you concerned about the health effects of smoking? Make you concerned about the health effects of smoking? Discourage non-smokers from smoking? Make non-smokers concerned about the health effects of smoking?”) and avoidance (ten items, α = .89, e.g., “Imagine that all cigarette packs had this warning. How likely is it that you would try to avoid looking at the warning on your cigarette packs?”) (Population Assessment of Tobacco and Health Study, 2014). A more detailed description of Study 2 measures can be found in Hall et al. (2016). Survey items for both studies are available upon request.

Data analysis

We calculated mean scores of the Brief RHWS and internal consistency using Cronbach’s alpha. We calculated the same correlational coefficients examined in Study 1 to assess convergent validity, expecting that the Brief RHWS would be correlated with the RHWS long form, higher trait reactance, higher positive smoker prototypes, and exposure to pictorial warnings. We also predicted that smokers would exhibit more reactance than non-smokers because warnings may more directly threaten smokers’ perceived freedom to use cigarettes. For concurrent criterion validity, we expected that reactance would be associated with lower perceived effectiveness and greater avoidance of the warnings, as in Study 1. As in Study 1, we ran these validity analyses using the RHWS long form. Validity analyses controlled for study arm and education.

Results

About half (48%) of the 2149 Study 1 participants were male, with a mean age of 40 years (Table 2). Study 1 participants were diverse, including a substantial number of sexual minority, African American, low-education, and low-income smokers. About half (47%) of the 1413 Study 2 participants were male, with a mean age of 34 years. Ten percent were gay, lesbian, or bisexual, and 55% had less than a college education.

Table 2.

Participant characteristics at baseline

Study 1
Study 2
Text-only warnings
(n = 1078)
Pictorial warnings
(n = 1071)
Text-only warnings
(n = 209)
Pictorial warnings
(n = 1204)
n (%) n (%) n (%) n (%)
Smoker
 No 0     (0.0) 0     (0.0) 122     (58.4) 694     (57.6)
 Yes 1078     (100.0) 1071     (100.0) 87     (41.6) 510     (42.4)
Age
 18–24 years 171     (16.1) 152     (14.5) 42     (20.1) 257     (21.3)
 25–39 years 377     (35.5) 398     (37.9) 113     (54.1) 659     (54.7)
 40–54 years 338     (31.8) 304     (29.0) 40     (19.1) 200     (16.6)
 55+ years 176     (16.6) 195     (18.6) 14     (6.7) 88     (7.3)
 Mean years (SD) 39.7 (13.4) 39.8 (13.7) 34.1 10.9 33.5 11.4
Gender
 Male 507     (47.4) 532     (50.0) 102     (48.8) 565     (47.0)
 Female 548     (51.2) 512     (48.2) 107     (51.2) 632     (52.6)
 Transgender 15     (1.4) 19     (1.8) 0     (0.0) 4     (0.3)
Gay, lesbian, or bisexual 173     (16.3) 195     (18.8) 28     (13.4) 120     (10.0)
Hispanic 92     (8.6) 89     (8.5) 16     (7.7) 85     (7.1)
Race
 Black or African American 484     (45.8) 510     (48.9) 17     (8.1) 80     (6.6)
 White 393     (37.2) 358     (34.3) 155     (74.2) 958     (79.6)
 Other/multiracial 134     (12.7) 117     (11.2) 18     (8.6) 78     (6.5)
 Asian 28     (2.7) 42     (4.0) 17     (8.1) 77     (6.4)
 Native Hawaiian/other Pacific Islander 11     (1.0) 6     (0.6) 0     (0.0) 2     (0.2)
 American Indian or Alaska Native 7     (0.6) 11     (1.0) 2     (1.0) 9     (0.7)
Education
 High school degree or less 333     (31.1) 344     (32.5) 35     (16.8) 124     (10.3)
 Some college 519     (48.5) 502     (47.4) 80     (38.5) 534     (44.4)
 College graduate 156     (14.6) 156     (14.7) 77     (37.0) 433     (36.0)
 Graduate degree 63     (5.9) 58     (5.5) 16     (7.7) 112     (9.3)
Low income (≤150% of federal poverty level)
 No 506     (47.0) 477     (44.8) –     –     –     –    
 Yes 570     (53.0) 589     (55.2) –     –     –     –    
Household income, annual
 $0–$24,999 566     (53.3) 589     (55.8) 66     (31.6) 333     (27.7)
 $25,000–$49,999 272     (25.6) 266     (25.2) 64     (30.6) 381     (31.7)
 $50,000–$74,999 110     (10.3) 92     (8.7) 32     (15.3) 236     (19.6)
 $75,000+ 115     (10.8) 109     (10.3) 47     (22.5) 252     (21.0)
Study site
 California 594     (55.1) 592     (55.3) –     –     –     –    
 North Carolina 484     (44.9) 479     (44.7) –     –     –     –    
Cigarettes smoked per day, mean (SD) 9.1   (6.5) 9.0   (7.2) –     –     –     –    
Trait reactance, mean (SD) 2.86 (0.7) 2.89 (0.7) 2.93 (0.7) 2.97 (0.7)
Open in a new tab

– Not assessed. Missing demographic data range from 0.0% to 2.2%. Demographics did not differ by trial arm in Study 1. Demographics did not differ by trial arm in Study 2, except for education (p < .05)

Scale psychometrics

The three items in the Brief RHWS each had a strong association with the underlying latent construct of reactance, as demonstrated by the high factor loadings from confirmatory factor analysis (Fig. 2). The Brief RHWS exhibited acceptable reliability in Study 1 (α = .75 at immediate post-test, 0.77 at Week 1, and 0.80 at Week 4; Table 3) and Study 2 (α = .65). In Study 1, the measure had good test–retest reliability at one week (r = .70), three weeks (r = .68), and four weeks (r = .59).

Fig. 2.

Fig. 2

Open in a new tab

Confirmatory factor analysis for Brief Reactance to Health Warnings Scale, Study 1

Convergent validity

Analyses supported the convergent validity of the Brief RHWS. The brief scale was highly correlated with the long form in Study 1 (β = 0.85, p < .001; Table 4), a finding replicated in Study 2 (β = 0.88, p < .001). In Study 1, the Brief RHWS was correlated with higher trait reactance, greater positive smoker prototypes, and exposure to pictorial warnings cigarette packs (all p < .001; Table 4). Again, Study 2 replicated these findings, and also demonstrated that reactance was higher among smokers than non-smokers (β = 0.23, p < .001, data not shown). The brief form and the long form performed similarly in terms of convergent validity in both studies (Table 5).

Table 4.

Correlates of reactance to health warnings (brief form)

Range Study 1
Study 2—smokers
Study 2—non-smokers
n Mean (SD) β n Mean (SD) β n Mean (SD) β
Convergent validity (correlates at immediate post-test)
 Reactance long form 1–5 2106 2.30 (0.63)   0.85** 594 2.87 (0.69) 0.88** – – –
 Trait reactance 1–5 2043 2.87 (0.70)   0.32** 594 3.05 (0.72) 0.37** 808 2.90 (0.67) 0.27**
 Positive smoker prototypes 1–5 2122 1.76 (0.86)   0.19** 594 2.42 (0.90) 0.15** 808 1.87 (0.76) 0.09*
 Exposure to pictorial warning – 2135 –   0.21** 594 – 0.20** 808 – 0.18**
Other validity Predictive validity Concurrent criterion validity
 Perceived message effectiveness 1–4 1882 2.66 (0.86) −0.15** 592 2.70 (0.85) −0.36** – – –
 Support for pictorial warning policy 1–4 1887 3.30 (0.84) −0.30** – – – – – –
 Quit intentions 1–4 1890 2.63 (1.09) −0.18** – – – – – –
 Avoidance of warning 1–5 1847 2.02 (1.14)   0.09** 594 2.74 (0.93) 0.30** – – –
 Forgoing a cigarette – 1889 – OR = 0.82*a – – – – – –
 Cigarettes per day 1–60 1890 7.34 (6.78)   0.07* – – – – – –
Open in a new tab

For predictive validity analyses, we assessed predictors at immediate post-test and the outcome at week 4

βs are standardized regression coefficients. – not assessed. Study 1 analyses controlled for study condition. Study 2 analyses controlled for study condition and education

*

p < .05;

**

p < .001

a

95% confidence interval [.73, .92]

Table 5.

Correlates of reactance to health warnings (long form)

Range Study 1
Study 2—smokers
n β n β
Convergent validity (correlates at immediate post-test)
 Trait reactance 1–5 2020   0.37** 596   0.46**
 Positive smoker prototypes 1–5 2096   0.23** 596   0.12*
 Exposure to pictorial warning – 2106   0.15** 596   0.09*
Other validity Predictive validity Concurrent criterion validity
 Perceived message effectiveness 1–4 1856 −0.19** 596 −0.43**
 Support for pictorial warning policy 1–4 1860 −0.31** –   –
 Quit intentions 1–4 1863 −0.17** –   –
 Avoidance of warning 1–5 1824   0.09** 596   0.32**
 Forgoing a cigarette – 1889 OR = 0.75*a –   –
 Cigarettes per day 1–60 1863   0.06* –   –
Open in a new tab

For predictive validity analyses, we assessed predictors at immediate post-test and the outcome at week 4

βs are standardized regression coefficients. – not assessed. Study 1 analyses controlled for study condition. Study 2 analyses controlled for study condition and education

*

p < .05,

**

p < .001

a

95% confidence interval [.73, .92]

Predictive validity

The Brief RHWS also exhibited predictive validity. Longitudinal analyses indicated that Brief RHWS scores at immediate post-test predicted lower perceived effectiveness of the warnings at the week 4 follow-up survey in Study 1, controlling for study arm (β = −0.15, p < .001; Table 4). In these same analyses, Brief RHWS scores were associated with less support for requiring pictorial warnings on cigarette packs (β = −0.30, p < .001) and lower intentions to quit smoking at the end of the trial (β = −0.18, p < .001). These results were similar when controlling for baseline levels of support for requiring pictorial warnings and quit intentions. Brief RHWS scores were associated with greater avoidance of the warnings (β = 0.09, p < .001) and lower odds of forgoing a cigarette (OR = 0.82, 95% CI = .73–.92). The Brief RHWS was also associated with smoking more cigarettes per day (β = 0.07, p < .05).

Concurrent criterion validity analyses in Study 2 followed a very similar pattern to Study 1. Among smokers, Brief RHWS scores were associated with lower perceived effectiveness of the warnings (β = −0.36, p < .001) and greater avoidance of the warning (β = 0.30, p < .001), controlling for study arm and education. The Brief RHWS and the long form performed comparably in terms of predictive validity in both studies (Table 5).

Discussion

Our brief scale is an efficient and psychometrically strong measure of reactance to health messages. The three items in the Brief RHWS reflect the conceptualization of reactance as an amalgam of perceived threat to freedom, anger in response to the warning, and counterarguing against the warning. The scale exhibited good test–retest reliability and acceptable internal consistency reliability among US adults exposed to pictorial cigarette pack warnings. The internal consistency reliability was lower in Study 2 than Study 1; future studies should evaluate the reliability of the Brief RHWS to ensure that it is adequate. The brief measure correlated with its long form, higher trait reactance, higher positive smoker prototypes, exposure to pictorial warnings, and being a smoker, supporting the measure’s convergent validity. In support of predictive validity, the brief measure correlated with greater avoidance of warnings, lower perceived effectiveness warnings, less support for requiring pictorial warnings on cigarette packs, lower quit intentions, a lower likelihood of forgoing a cigarette, and smoking more cigarettes per day. The Brief RHWS exhibited comparable validity to the long form, indicating that researchers can use the brief form without sacrificing construct validity.

The brief form builds on the strengths of several distinct reactance measurement approaches. A widely-used measure, developed by Dillard and Shen (2005), assesses anger toward and counterarguments against a message, using four close-ended questions assessing anger and an open-ended thought-listing task to assess counterarguments about the warning (Dillard & Shen, 2005; Gollust & Cappella, 2014; Quick, 2012; Rains, 2013; Rains & Turner, 2007). Some prior studies on pictorial warnings have often focused on the emotional element of reactance (Cho et al., 2016; Erceg-Hurn & Steed, 2011). Other researchers testing Extended Parallel Process Model have measured reactance as a combination of perceived manipulation, message minimization, and message derogation (Witte, 1994). Organ donation researchers have assessed reactance with a four-item scale assessing frustration toward being told how to feel (LaVail et al., 2010; Lindsey, 2005; Reinhart et al., 2007). Our measure builds on this work by incorporating the key affective and cognitive components of reactance, and by including a close-ended measure of counterarguing that may be less burdensome for participants and researchers.

Design and development of health warnings may benefit from taking message reactance into account in order to potentially maximize the beneficial impact of the warnings. In the early stages of developing warnings and campaigns, evaluators commonly use perceived effectiveness as a metric for identifying effective messages (Davis et al., 2013). We propose reactance as a useful adjunct to perceived effectiveness. Measuring reactance could identify outlier messages that elicit particularly high amounts of reactance and therefore may be likely to be ineffective or lead to unintended consequences. Looking at reactance alone is likely not the best way to narrow down a list of candidate messages because effective messages often simultaneously elicit reactance, as is the case for pictorial warnings (Hall et al., 2016). Furthermore, in our own work, we have found little variation in message reactance among pictorial warnings (Hall et al., 2016), perhaps because we looked at a small number of warnings that had already been pre-tested by other researchers (Cameron et al., 2015). Thus, assessing message reactance may offer less useful information at later stages of message development after messages have already been refined.

Pictorial cigarette pack warnings cause greater message reactance than text warnings, as several studies and a recent meta-analysis have demonstrated (Erceg-Hurn & Steed, 2011; Hall et al., 2016; LaVoie et al., 2015; Noar et al., 2016). However, given the large body of research indicating the effectiveness of pictorial warnings, it would be unwise to conclude that pictorial warnings are counterproductive because they produce reactance, as others have asserted (LaVoie et al., 2015). Experimental and observational evidence supports the superiority of pictorial warnings over text warnings on numerous outcomes, including quit intentions (Noar et al., 2016), quit attempts (Brewer et al., 2016), knowledge about smoking risks (Brennan et al., 2011), and foregoing cigarettes (Yong et al., 2013). Moreover, pictorial warning implementation is associated with decreases in the number of cigarettes smoked and may have played a role in the reductions in smoking prevalence that we have seen in many countries (Noar et al., 2016). Despite the role reactance may play in weakening the warnings’ impact, pictorial warnings remain a promising strategy for reducing smoking, far better than text-only warnings at changing attitudes, intentions, and behavior (Brewer et al., 2016; Noar et al., 2016).

Reactance could undermine the political will to implement pictorial warnings on cigarette packs or other policy changes. Public support can influence the ability of policymakers to successfully advocate for, enact, and enforce health policies as well as the effectiveness of the new policies (Diepeveen et al., 2013). The relationship between reactance and support for public health policies remains an important area for future research.

Strengths of our studies include consistent validity findings across both studies, the use of an experimental design in both studies, naturalistic exposure to warnings on smokers’ actual packs (Study 1), the longitudinal follow-up assessment (Study 1), and the inclusion of both smokers and nonsmokers (Study 2). However, our use of convenience samples may limit the generalizability of our findings to other populations. The magnitude of some of the correlations in our validity analyses was modest. Although we followed smokers for four weeks in Study 1, the impact of reactance on attitudes and behavior over a longer period of time remains unknown. Finally, we did not include previously validated reactance scales in our surveys due to space constraints.

Conclusions

The Brief Reactance to Health Warnings Scale captures the construct well while maintaining good reliability and validity among smokers exposed to pictorial warnings. We encourage researchers to measure reactance when developing and evaluating health messages in order to understand whether reactance weakens the effects of those messages. Researchers could use this scale as one of a battery of measures to help develop and select pictorial warnings for implementation. The scale may also hold utility beyond the context of pictorial cigarette pack warnings, for example, in evaluating health warnings and other messages for other tobacco products, alcohol, risky sex, exercise, or food and beverages. The scale could also be used or adapted for different types of health communication message formats, such as mass media campaigns.

Acknowledgments

Research reported in this publication was supported by The National Cancer Institute and FDA Center for Tobacco Products (CTP) under Award Number P30CA016086-38S2. F31CA196037 from the National Cancer Institute of the National Institutes of Health supported MGH’s time writing the paper. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration.

Footnotes

Conflict of interest Marissa G. Hall, Paschal Sheeran, Seth M. Noar, Kurt M. Ribisl, Marcella H. Boynton and Noel T. Brewer declare that they have no conflict of interest.

Human and animal rights and Informed consent Study procedures were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments. Researchers obtained informed consent from study participants.

References

  1. Bollen KA. Structural equation models. New York: Wiley; 1998. [Google Scholar]
  2. Brehm KA. A theory of psychological reactance. New York, NY: Academic Press; 1966. [Google Scholar]
  3. Brehm SS, Brehm JW. Psychological reactance: A theory of freedom and control. New York, NY: Academic Press; 1981. [Google Scholar]
  4. Brennan E, Durkin SJ, Cotter T, Harper T, Wakefield MA. Mass media campaigns designed to support new pictorial health warnings on cigarette packets: Evidence of a complementary relationship. Tobacco Control. 2011;20:412–418. doi: 10.1136/tc.2010.039321. [DOI] [PubMed] [Google Scholar]
  5. Brewer NT, Hall MG, Lee JG, Peebles K, Noar SM, Ribisl KM. Testing warning messages on smokers’ cigarette packages: A standardised protocol. Tobacco Control. 2015 doi: 10.1136/tobaccocontrol-2014-051661. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brewer NT, Hall MG, Noar SM, Parada H, Stein-Seroussi A, Bach LE, et al. Effect of pictorial cigarette pack warnings on changes in smoking behavior: A randomized clinical trial. JAMA Internal Medicine. 2016;176:905–912. doi: 10.1001/jamainternmed.2016.2621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Buhrmester M, Kwang T, Gosling SD. Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science. 2011;6:3–5. doi: 10.1177/1745691610393980. [DOI] [PubMed] [Google Scholar]
  8. Cameron LD, Pepper JK, Brewer NT. Responses of young adults to graphic warning labels for cigarette packages. Tobacco Control. 2015;24:e14–e22. doi: 10.1136/tobaccocontrol-2012-050645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cho YJ, Thrasher JF, Swayampakala K, Yong HH, McKeever R, Hammond D, et al. Does reactance against cigarette warning labels matter? Warning label responses and downstream smoking cessation amongst adult smokers in Australia, Canada, Mexico and the United States. PLoS ONE. 2016;11:e0159245. doi: 10.1371/journal.pone.0159245. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Davis KC, Nonnemaker J, Duke J, Farrelly MC. Perceived effectiveness of cessation advertisements: The importance of audience reactions and practical implications for media campaign planning. Health Communication. 2013;28:461–472. doi: 10.1080/10410236.2012.696535. [DOI] [PubMed] [Google Scholar]
  11. De Ayala RJ. Theory and practice of item response theory. New York, NY: Guilford Publications; 2013. [Google Scholar]
  12. Diepeveen S, Ling T, Suhrcke M, Roland M, Marteau TM. Public acceptability of government intervention to change health-related behaviours: A systematic review and narrative synthesis. BMC Public Health. 2013;13:756. doi: 10.1186/1471-2458-13-756. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dillard JP, Shen L. On the nature of reactance and its role in persuasive health communication. Communication Monographs. 2005;72:144–168. [Google Scholar]
  14. Embretson SE, Reise SP. Item response theory. Mahwah, NJ: Psychology Press; 2013. [Google Scholar]
  15. Erceg-Hurn DM, Steed LG. Does exposure to cigarette health warnings elicit psychological reactance in smokers? Journal of Applied Social Psychology. 2011;41:219–237. doi: 10.1111/j.1559-1816.2010.00710.x. [DOI] [Google Scholar]
  16. Gerrard M, Gibbons FX, Houlihan AE, Stock ML, Pomery EA. A dual-process approach to health risk decision making: The prototype willingness model. Developmental Review. 2008;28:29–61. [Google Scholar]
  17. Gollust SE, Cappella JN. Understanding public resistance to messages about health disparities. Journal of Health Communication. 2014;19:493–510. doi: 10.1080/10810730.2013.821561. [DOI] [PubMed] [Google Scholar]
  18. Hall MG, Sheeran P, Noar SM, Ribisl KM, Bach LE, Brewer NT. Reactance to health warnings scale: Development and validation. Annals of Behavioral Medicine. 2016;50:736–750. doi: 10.1007/s12160-016-9799-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hong SM, Page S. A psychological reactance scale: Development, factor structure and reliability. Psychological Reports. 1989;64:1323–1326. [Google Scholar]
  20. Klein WM, Zajac LE, Monin MM. Worry as a moderator of the association between risk perceptions and quitting intentions in young adult and adult smokers. Annals of Behavioral Medicine. 2009;38:256–261. doi: 10.1007/s12160-009-9143-2. [DOI] [PubMed] [Google Scholar]
  21. LaVail KH, Anker AE, Reinhart A, Feeley TH. The persuasive effects of audiovisual PSAs to promote organ donation: The mediating role of psychological reactance. Communication Studies. 2010;61:46–69. [Google Scholar]
  22. LaVoie NR, Quick BL, Riles JM, Lambert NJ. Are graphic cigarette warning labels an effective message strategy? A test of psychological reactance theory and source appraisal. Communication Research. 2015 doi: 10.1177/0093650215609669. [DOI] [Google Scholar]
  23. Li L, Borland R, Fong GT, Jiang Y, Yang Y, Wang L, et al. Smoking-related thoughts and microbehaviours, and their predictive power for quitting: Findings from the International Tobacco Control (ITC) China Survey. Tobacco Control. 2014 doi: 10.1136/tobaccocontrol-2013-051384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lindsey LLM. Anticipated guilt as behavioral motivation an examination of appeals to help unknown others through bone marrow donation. Human Communication Research. 2005;31:453–481. doi: 10.1093/hcr/31.4.453. [DOI] [Google Scholar]
  25. McCool J, Cameron LD, Robinson E. Do parents have any influence over how young people appraise tobacco images in the media? Journal of Adolescent Health. 2011;48:170–175. doi: 10.1016/j.jadohealth.2010.06.012. [DOI] [PubMed] [Google Scholar]
  26. Noar SM, Francis DB, Bridges C, Sontag JM, Ribisl KM, Brewer NT. The impact of strengthening cigarette pack warnings: Systematic review of longitudinal observational studies. Social Science and Medicine. 2016a;164:118–129. doi: 10.1016/j.socscimed.2016.06.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Noar SM, Hall MG, Francis D, Ribisl KM, Pepper J, Brewer NT. Pictorial cigarette pack warnings: A meta-analysis of experimental studies. Tobacco Control. 2016b;25:341–354. doi: 10.1136/tobaccocontrol-2014-051978. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Paolacci G, Chandler J. Inside the turk understanding mechanical turk as a participant pool. Current Directions in Psychological Science. 2014;23:184–188. [Google Scholar]
  29. Peer E, Vosgerau J, Acquisti A. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods. 2014;46:1023–1031. doi: 10.3758/s13428-013-0434-y. [DOI] [PubMed] [Google Scholar]
  30. Population Assessment of Tobacco and Health Study. PATH: Population assessment of tobacco and health. 2014 Retrieved April 3, 2014, from http://www.pathstudyinfo.nih.gov/UI/HomeMobile.aspx.
  31. Quick BL. What is the best measure of psychological reactance? An empirical test of two measures. Health Communication. 2012;27:1–9. doi: 10.1080/10410236.2011.567446. [DOI] [PubMed] [Google Scholar]
  32. Quick BL, Stephenson MT. Further evidence that psychological reactance can be modeled as a combination of anger and negative cognitions. Communication Research. 2007;34:255–276. [Google Scholar]
  33. Rains SA. The nature of psychological reactance revisited: A meta-analytic review. Human Communication Research. 2013;39:47–73. [Google Scholar]
  34. Rains SA, Turner MM. Psychological reactance and persuasive health communication: A test and extension of the intertwined model. Human Communication Research. 2007;33:241–269. [Google Scholar]
  35. Reinhart AM, Marshall HM, Feeley TH, Tutzauer F. The persuasive effects of message framing in organ donation: The mediating role of psychological reactance. Communication Monographs. 2007;74:229–255. [Google Scholar]
  36. Witte K. Putting the fear back into fear appeals: The extended parallel process model. Communications Monographs. 1992;59:329–349. [Google Scholar]
  37. Witte K. Fear control and danger control: A test of the extended parallel process model (EPPM) Communications Monographs. 1994;61:113–134. [Google Scholar]
  38. Witte K, Allen M. A meta-analysis of fear appeals: Implications for effective public health campaigns. Health Education and Behavior. 2000;27:591–615. doi: 10.1177/109019810002700506. [DOI] [PubMed] [Google Scholar]
  39. Yong HH, Fong GT, Driezen P, Borland R, Quah AC, Sirirassamee B, et al. Adult smokers’ reactions to pictorial health warning labels on cigarette packs in Thailand and moderating effects of type of cigarette smoked: Findings from the international tobacco control southeast Asia survey. Nicotine and Tobacco Research. 2013;15:1339–1347. doi: 10.1093/ntr/nts241. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES