The Effects of Framing and Fear on Ratings and Impact of Antimarijuana PSAs

Abstract

A laboratory experiment, funded by the U.S. National Institutes of Health, involved 243 U.S. undergradu-ate students and employed a 2 (gain-framed vs. loss-framed) × 2 (high vs. low threat) plus control group pretest-posttest experimental design to assess the com-bined effects of frame (gain vs. loss) and level of threat of public service announcements (PSAs) about mari-juana on attitudes, beliefs, and intentions related to marijuana, as well as the relationship of message con-dition to ratings of PSAs. Results suggest that loss-framed messages may lead to greater perceived threat, as well as reactance, and gain-framed messages may lead to a greater reduction in positive attitudes toward marijuana than loss-framed messages. Finally, frame and threat may interact in a complex way. Further re-search is suggested to replicate these findings. A sub-stantial body of carefully crafted and systematic re-search studies examining both content and features of messages increasingly informs mass media preven-tion efforts, including the development of public ser-vice announcements (PSAs). Although the significance of messages on commercial broadcast stations may be diminishing with the increasing role and impact of new media, many of the basic questions addressed by this research are likely to apply across media channels. Nonetheless, important questions about what makes a message effective in changing an individual’s attitudes or behavior remain to be answered. In this paper, the authors focus on two theoretically derived strategies that offer possibilities for developing persuasive mes-sages: framing and threat.

INTRODUCTION

Framing

A key element in any prevention message is how the content is framed, emphasizing either the benefits of adopting a particular behavior (a gain-framed message) or the costs of failing to adopt a behavior (a loss-framed message). Research has suggested that framing can alter a message’s persuasive impact (Rothman, Kelly, Hertel, & Salovey, 2003; Rothman & Salovey, 1997; Rothman, Bartels, Wlaschin, & Salovey, 2006; Rothman, Stark, & Salovey, 2006). The premise that shifting how informa-tion is framed affects people’s behavioral decisions was motivated by Prospect Theory (Tversky & Kahneman, 1981) which purports that people in general are less likely to take risks in order to receive potential gains and more likely to take risks to avoid potential losses. Based on this premise, Rothman and colleagues (Rothman & Salovey, 1997; Rothman et al., 2006) proposed that when people are deciding whether to perform a behavior perceived to involve the risk of an unpleasant outcome (e.g., it may detect a health problem), loss-framed appeals will be more persuasive. However, when people are deciding whether to perform a behavior perceived to involve a relatively low risk of an unpleasant outcome (e.g., it prevents the onset of a health problem), gain-framed appeals will be more persuasive. In health contexts, the desirous behavioral outcome serves as a moderator, with gain-framed messages more effective for disease prevention and loss-framed messages more effective in encouraging detection behaviors (Rothman et al., 2006).

Empirical findings across a range of health domains have proved consistent with this framework (e.g., Banks et al., 1995; Cox & Cox, 2001; Detweiler, Bedell, Salovey, Pronin, & Rothman, 1999; Jones, Sinclair, & Courneya, 2003; Myers et al., 1991; Meyerowitz & Chaiken, 1987; Millar & Millar, 2000; Rothman, Salovey, Antone, Keough, & Martin, 1993; Rothman, Martino, Be-dell, Detweiler, & Salovey, 1999; Schneider et al., 2001; See Rothman, Stark et al., 2006; Rothman, Bartels et al., 2006 for recent reviews). Moreover, more recent empirical work has demonstrated that the variability of how people construe a given behavior (e.g., the likelihood it will af-ford an adverse outcome) regulates the impact of loss- and gain-framed appeals (Apanovitch, McCarthy, & Salovey, 2003; Bartels, Kelly, & Rothman, 2006).

Notably, conclusions of a recent meta-analysis on the effect of message framing are contradictory to many of the previously cited studies. O’Keefe & Jensen (2007) reviewed the literature on the relative persuasiveness of gain- and loss-framed messages for “encouraging disease prevention behaviors” (k = 93 studies; n = 21,656). The overall result of their meta-analysis suggests that loss framed appeals, which “emphasize the disadvantages of noncompliance,” are more persuasive than gain-framed appeals, which emphasize “the advantages of compli-ance.” The size of the difference, however, although statistically significant, is quite small, (corresponding to an r = 0.03), and if results specific to one health-promoting behavior were dropped, the difference between the two framing types becomes nonsignificant. While the potential negative outcomes of marijuana use are not directly related to a particular disease, this review has implications for marijuana prevention messages because focusing on the negative outcomes of marijuana use may persuade au-dience members to avoid this behavior.

Threat

The second dimension of message content considered in this study is threat, and to what extent, if at all, mes-sages that contain a specific threat engage and/or persuade the target audience. Some researchers contend that this type of prevention message, often referred to as a fear appeal, is ineffective (Boster & Mongeau, 1984; Sutton, 1982), while others suggest the research is inconclusive (de Hoog, Stroebe, & de Witt, 2007). Nonetheless, it is hard to deny that such a strategy is intuitively appealing when designing persuasive messages; in formative work, when focus group participants are asked what type of message would cause them to avoid risky behavior they often recommend a fear-based message. Much of the work over the last 20 years looking at the role of threat in per-suasive messages has been conducted using Witte and colleagues’ theoretical framework, the Extended Parallel Process Model (EPPM). The goal of the EPPM is to provide guidance on how to manage fear generated from a variety of threats like HIV infection, substance abuse, or a natural disaster (Witte, 1994; Witte, 1992). The EPPM suggests that when people feel at risk for a significant threat (high level of perceived susceptibility and severity), they become scared and are motivated to act. Perceptions of self-efficacy (i.e., perceived ability to perform a recommended response) and response efficacy (i.e., one’s be-liefs about whether or not a recommended response works to avert a threat) determine whether people are motivated to control the danger (changing one’s behavior) or con-trol the fear (changing one’s perception of the threat). For example, a study looking at message design for alcohol abuse prevention campaigns indicated that fear appeals are successful in improving adolescents’ intentions not to drink (Stainback & Rogers, 1983). Witte and colleagues found, in a study to prevent the spread of genital warts, that fear appeals can be persuasive if the level of fear and the perceptions of response efficacy are high (1998). Finally, in a study using a randomized experimental design that examined the role of threat and coping, results demon-strated that high coping increased anticipated regret which served as a mediator of the effect of fear on intentions to use condoms (Smerecnik & Ruiter, 2010).

The Importance of Looking at Framing and Threat Simultaneously

The purpose of the study was to better understand the rela-tive contributions of each dimension in persuading the au-dience to adopt safer attitudes, intentions, and behaviors related to marijuana use. In designing PSAs, developers must tell a coherent and compelling story that provides a reason to avoid the risk behavior and enumerates out-comes related to the choices the viewer ultimately makes. Looking at framing and threat simultaneously allows us to examine the impact, individually and together, of di-mensions that reflect the storylines of many current PSAs. Because the literature on fear-based messages is more de-veloped, a hypothesis was warranted; however, because there is less of a consensus in the framing literature, we developed research questions we sought to answer.

HYPOTHESIS AND RESEARCH QUESTIONS

H1. For individuals who have high levels of efficacy beliefs, messages that are relatively high in threat will produce more positive outcomes than messages that are low in threat.

Consistent with the Extended Parallel Processing Model (EPPM), high-threat messages will be more effective than low-threat messages, but only for individuals who have relatively high levels of efficacy (self-efficacy plus re-sponse efficacy.) According to the theory, these individu-als will cope with the threat by “danger control” (attempt-ing to change their behavior to reduce the threat). High threat for individuals with low levels of efficacy only re-sults in “fear control” (using coping techniques such as denial and dealing with the feelings but lacking a clear strategy about how to reduce the threat).

RQ1. Will gain-framed messages produce greater positive changes in marijuana-related variables than loss-framed messages?

To date, most antimarijuana ads have focused either on someone experiencing a loss (job, freedom, or success) because of marijuana use or someone gaining some-thing (partner satisfaction, new friends, or job/school achievements) due to either a lack of or cessation of mari-juana use. Previous work by Rothman, Stark and Salovey (2006) identified gain-framed messages as the most effective at reducing risk behaviors like marijuana use; however, a relatively recent meta-analysis by O’Keefe and Jensen (2007) casts doubt on this finding. Nevertheless, messages must have a storyline. Our work in developing PSAs for high sensation-seeking (a characteristic often predictive of marijuana use) adolescents and young adults, indicates messages that are loss-framed and focus on neg-ative consequences are often unpopular because they are perceived as “preachy,” which supports the Rothman et al. (2006) findings. Work by O’Keefe & Jensen (2007) and others, such as van ’t Riet and colleagues in examining the interaction between affect and framing (2010), suggest, however, that the definitive approach has yet to be established.

RQ2. Will level of threat and message framing interact to have effects on marijuana-related variables?

Again, as mentioned earlier, drug use prevention PSAs must provide a storyline that convinces viewers to either avoid (or adopt a less harmful approach to) drug use in a way that connects these decisions to positive or negative outcomes of the behavior. The level of threat in-herent in the prevention messages may interact with the posited outcomes posited in ways that support or detract from the overall impact of the message. Recent research on framing effects suggest that exploring the role of a va-riety of moderating variables may be important in under-standing the effects of framing. Van ‘t Riet, Ruiter, Werrif, Candel, & de Vries (2010) proposed that efficacy might be important to consider when thinking about the impact of message framing on individual behavior as “a mediat-ing effect of negative affect might be most likely when recipients perceive a high efficacy” (p. 1271). Others, including Gallagher, Updegraff, Rothman, and Sims (2011), have looked at a single dimension of the EPPM (perceived susceptibility) as a moderator of message framing in a study on breast cancer screening. Consistent with this ex-ploratory work, we examined whether threat is a potential moderator of the effects of message framing.

METHODOLOGY

Study Design

The current study employed a 2 (gain-framed vs. loss-framed PSAs) × 2 (high- vs. low-threat PSAs) plus con-trol group pretest-posttest experimental design. Participants were initially screened for age (had to be at least 18 years of age) and then randomly assigned to one of five conditions: (1) control, (2) high threat/loss frame, (3) high threat/gain frame, (4) low threat/loss frame, or (5) low threat/gain frame. Analyses assessed the impact of mes-sage condition on attitudes and beliefs about marijuana, intentions to use marijuana, peer norms regarding mari-juana use, and efficacy beliefs, as well as the relationship of message condition on ratings of PSAs.

Sample

Participants in this study were recruited from a list of 2,889 freshman and sophomore undergraduate students at a midwestern university. The list was provided by the Of-fice of Institutional Research, and was comprised of 27% African-Americans (purposively over-sampled by a factor of about four times their proportion in the student popula-tion), 70% Caucasians, and 3% other racial/ethnic groups. The recruitment list included 60% males (oversampled by about 33% to compensate for expected lower participation rates).

The initial sample for this study was comprised of the freshman and sophomore undergraduate students (re-cruited from the list described above) who also completed the following sequence of events: completed a pretest (n = 525), created an ID number (n = 406), registered for a session in the media lab (n = 307), and attended a me-dia lab session (final n = 243). We compared individu-als who completed the pretest but did not ultimately sign up or show up for the media lab part of the study on six variables: age, year in school, gender, race/ethnicity, and prevalence of ever using marijuana and marijuana use in the past 30 days. Only one of the variables was significantly different between the two groups: race/ethnicity. While 42.8% of those who completed the pretest were African-American, only 31.4% of those who attended the media lab and were thus in the final database were African-American, χ 2 (1) = 8.64, p = .003.

Those who attended the media lab session were mostly Caucasian (64.5% compared to 35.5% African American), female (54.7%), and first year college students (51.4%).

Ages ranged from 18–21 years old, with the majority of participants being either 18 or 19 years of age (40% and 49.8%, respectively).

Procedure

Recruitment to the Study and Pretest

Undergraduate students were invited to participate in the study via email. When telephone numbers were available, calls were also made to encourage students to check their e-mail and learn about the study. The e-mail message described the study and asked students to click on a link that would lead them to the study website. Students were asked to read through the consent form to learn more about the purpose of the study and their potential involvement. If students decided to participate, they completed an online consent form that involved two steps: checking a box to indicate they read, understood and agreed with study protocol; and creating a 9-digit ID to be used throughout the study. All procedures were approved by the university’s Institu-tional Review Board.

Once participants created their unique ID number, they were instructed to log into the sites to complete the pretest, which consisted of 139 questions and took approximately 10–15 min. In addition to measures of demographics, the pretest asked participants questions about their attitudes toward marijuana, intentions to use marijuana, peer norms supportive of marijuana, refusal self-efficacy and response efficacy to abstaining from marijuana, severity of and susceptibility to consequences of marijuana use, and past marijuana use. To prevent sensitization to the study’s purpose, complete sets of items that mirrored the questions about marijuana use were developed reflecting the content of the control PSAs (HIV testing, seatbelt use, and military recruitment). After completing the pretest, stu-dents were asked to sign up for one of the PSA viewing sessions in the Media Lab.

The Study

The Media Center has theater seating for 20, soundproofing, and the capacity for showing a variety of media on a large screen. The theater has the capability for collecting Turning Point data, which is a data collection system that collects participant responses using a handheld device (re-sponse card). Participants entered numerical responses to PSA rating questions that were read aloud and displayed on a screen after each PSA was shown.

Media lab sessions were limited to 12 people in order to allow for about 3 ft of space between participants to help ensure the participants felt comfortable and that their answers were private. As participants arrived, they were greeted by a member of the research team and asked for the IDs they created at baseline. These IDs were used to:

(1) verify that the participant had completed a pretest, (2) link pretest and posttest data, and (3) link Turning Point data to pretest and posttest data. Students were assigned to seats based on the order in which they arrived and were provided with headphones and Turning Point response cards.

Participants in the intervention conditions viewed a total of five intervention PSAs about marijuana use and nine control PSAs related to seatbelt safety, HIV testing, or military recruitment. PSAs were viewed in seven sets of two, where five sets included one control and one inter-vention PSA and two sets included two control PSAs to help disguise the purpose of the study. After each set, participants were asked to use their Turning Point remote control to answer a series of 14 questions about the second PSA in the set, including perceived effectiveness of the message, relevance, and perceived threat. Participants in the control condition viewed 14 control PSAs shown in seven sets and also completed a series of 14 questions about the second PSA for each set.

In order to control for order effects, a random number chart was used to determine pairs (one control PSA and one intervention PSA in each pair) as well as order of presentation of pairs. Within each pair, the intervention PSA came second so that we could obtain immediate responses from the audience on that PSA using the Turn-ing Point remote control. For the pairs with two control PSAs, a coin was flipped to determine order. After all of the PSAs were viewed and responses were recorded, participants were escorted to an adjoining computer lab for a posttest survey. Participants entered their individual IDs and completed the survey, which included the same measures as in the pretest regarding attitudes, perceptions, and intentions either toward marijuana use or the control PSA topics. After completion of the survey, respondents were thanked for their time (generally 45 to 60 min) and pro-vided with a $25 cash incentive.

Selection and Production of Stimuli

To obtain marijuana prevention PSAs, a wide search was conducted that yielded PSAs from the following sources: a database maintained by the Communication Depart-ment at the study university containing materials from prior research; the Office of National Drug Control Pol-icy’s (ONDCP) website; Music Television’s (MTV) web-site; Black Entertainment Television’s (BET) website; and YouTube’s website. While the research team found 110 marijuana prevention PSAs, 15 that best reflected the dimensions of the study design were selected for the purpose of this study. PSAs for the study met the following criteria: (1) 30 seconds in length; (2) directed at youth or young adults; (3) message content targeted marijuana use; (4) message featured either clear negative consequences of marijuana use or benefits of not using marijuana; (5) the consequence or benefit was relevant to the target age group; (6) the PSA had high sensation-value; and (7) the PSA was available for our use and compatible with me-dia room capabilities. The 15 experimental PSAs were coded based on the two key theoretical variables for this study: framing (gain or loss) and level of threat (high or low). This resulted in five PSAs in three of the four exper-imental cells; no PSAs that met the above criteria were found in the high-threat/gain-frame condition. Working with a media producer that we have collaborated with over the past 20 years in developing successful prevention campaigns related to substance use and sexual behavior, we conducted significant formative work with the target population and produced five new high-threat/gain-frame PSAs, now yielding five PSAs in each of the four threat × framing conditions. Control PSAs about were obtained from public sources such as YouTube and the National Highway Traffic Safety Administration (NHTSA) website. Participants in the intervention conditions viewed five experimental PSAs consistent with their condition (i.e., high-threat and gain-framed, low-threat and loss-framed, etc.) about marijuana use and nine control PSAs related to seatbelt safety, HIV testing, or the military.

Using the Turning Point technology, participants were asked questions about the perceived effectiveness of the message, as well as its relevance and believability. They were also asked about whether they experienced feelings of fear or manipulation. Responses were recorded on the computer and converted to Microsoft Excel for analyses.

Measures

The authors chose to develop scales to measure suscep-tibility and severity of substance use as well as response efficacy for strategies for avoiding substances because val-idated items reflecting these dimensions of the EPPM in the context of substance use were not available in the liter-ature. In addition, it was important that survey items measuring consequences and benefits as well as beliefs and attitudes were reflective of the content of the PSAs. De-veloping items directly related to the stimuli is not an un-common strategy in studies such as this one.

Demographics

Demographic questions were asked only at pretest and included gender, ethnicity, age, year in college, and current grade point average.

Marijuana Use

Marijuana use behavior was measured at pretest only, as the PSA messages were only expected to affect attitudes, beliefs, intentions, and self-efficacy. Participants first re-sponded to the question, “Have you ever smoked marijuana?” Individuals who responded “No,” skipped the remaining behavior questions. Participants who selected “Yes,” completed four items regarding age of initiation and frequency of use (past 30 days, past 3 months, past year). These are standard substance abuse items used on SAMHSA and Monitoring the Future substance use sur-veys (Bachman, Johnston, & O’Malley, 2011; Substance Abuse and Mental Health Services Administration, 2011).

Marijuana-Related Outcome Variables

Attitudes towards marijuana use were measured using eight items developed by Harrington, Lane, Donohew, Zimmerman, Norling et al., (2003), four focusing on neg-ative attitudes (i.e., “Smoking marijuana makes a person lose their friends”), and four focusing on positive attitudes (“How much does smoking marijuana help a person have fun with friends?”). For both sets of attitude items, partici-pants used a Likert Scale with responses from “Not at all” to “Extremely”; coefficient alpha for the 4-item negative attitude scale was 0.90 at pretest and 0.89 at posttest and for the 4-item positive attitude scale, it was 0.85 at pretest and 0.84 at posttest.

Peer norms about marijuana use were measured using a 5-item scale developed for this study. Participants re-sponded using a Likert Scale to items such as, “Most of my friends think smoking marijuana is cool.” The coef-ficient alpha was 0.80 at pretest and 0.81 at posttest. Re-fusal self-efficacy for marijuana use was measured using a 5-item scale previously developed by Harrington et al. (2003). Items included, “How confident are you that you could refuse to use marijuana when you are at a party?” Responses were based on a 5-point Likert-type scale from “Not at all confident” to “Extremely confident.” The co-efficient alpha at pretest was found to be 0.94 and 0.87 at posttest. Response efficacy for marijuana use was measured using a 6-item scale developed for this study. Items included, “When people stop using marijuana, they be-come more focused.” Responses were based on a 5-point Likert Scale. Coefficient alpha was 0.81 at pretest and 0.85 at posttest. Perceived susceptibility to consequences of marijuana use was measured by asking six items of mari-juana users (e.g., “Because I smoke marijuana, I am likely to get lung cancer”) and nonusers (e.g., “If I start smoking marijuana on a regular basis, I am likely to get lung can-cer.”). These items had a coefficient alpha of 0.80 at pretest and 0.81 at posttest. Since substance use and its consequences are not generally measured in the context of the Extended Parallel Processing Model, these items were de-veloped for this study, based on focus group information available to the researchers.

Perceived severity of marijuana use was measured by asking five items (e.g., “Peo-ple who smoke marijuana cause damage to their lungs.”) using a 5-point Likert Scale ranging from “strongly dis-agree” to “strongly agree.” Coefficient alpha was 0.91 for both pre- and posttest.

Behavioral intentions for marijuana use were measured using four items developed for this study. The first item, “How likely is it that you will smoke marijuana in the next month (30 days)?” had a 5-point Likert-type response format ranging from “I definitely will” to “I definitely will not.” Three additional items measured intended mar-ijuana use during the next month, next 3 months, and next year. A Likert-type response format was used with op-tions ranging from “1–2 times” to “40 or more times.” The four items were standardized and summed, with an alpha of 0.97 at pretest, and 0.94 at posttest. Intentions are almost always measured by a single item, though it is well known that multiple item scales are almost always more reliable than single items, and behavior- related outcomes are rarely well measured (Epstein, 1980).

PSA Ratings

Three measures of ratings of the PSAs were assessed: fear response, reactance, and perceived message effectiveness. The first two were assessed using Witte’s measures (Witte et al., 1992; Witte, Berkowitz, Cameron, & McKeon, 1998). Fear response was measured with five items based on Witte’s measures, with an alpha of 0.80. Reactance was measured with Witte’s reactance measure, comprised of two items indicating whether participant felt manipulated by the PSA and/or by the level of believability of the claims of the message; the correlation between these two items was 0.49.

Perceived message effectiveness was assessed with four items (e.g., “How effective do you think this message would be in convincing other college students like you?), with an alpha of 0.80, based on the recent work of Dillard (Dillard & Ye, 2008; Dillard, Weber, & Vail, 2007).

METHODS OF ANALYSIS

Two repeated measures analyses of covariance were conducted to assess the impact of intervention (vs. control) and the joint impact of threat and framing on mediating variables (attitudes, norms, refusal self-efficacy, re-sponse efficacy, perceived susceptibility, and behavioral intentions). Covariates included age, gender, race, and recent frequency of marijuana use. Analysis of covariance was also conducted to assess the impact of threat and framing on reactions to PSAs (fear response, reactance, and perceived message effectiveness). It should be noted that for all analyses presented here participants who reported past 30-day marijuana use were excluded as the antimarijuana PSAs selected featured primary marijuana prevention messages, thus recent marijuana users were deemed to be an inappropriate target group.

RESULTS

Results will be presented first for a comparison between all intervention conditions and the control group, and then for the effects of threat, framing, and the interaction of the two. Finally, we will summarize the results in terms of the hypothesis and research questions.

Impact of Intervention versus Control Messages on Marijuana-Related Variables

As shown in Table 1, results indicated that when com-pared to the control PSAs, the intervention PSAs had a significant impact on positive attitudes about mari-juana and severity of consequences of marijuana use and a nearly significant impact on marijuana refusal self-efficacy. Specifically, positive attitudes about marijuana use in the intervention group significantly decreased from a mean of 2.35 at pretest to 2.21 at posttest, compared to a slight increase from 2.21 to 2.27 for the control group (p = .045). For perceptions of severity of consequences, the intervention group mean response increased slightly from 3.78 at pretest to 3.93 at posttest, compared to a decrease from 3.74 to 3.70 for the control group (p = .03). Responses for refusal self-efficacy in the intervention group increased slightly from a mean of 4.54 at pretest to 4.58 at posttest, compared to a decrease from 4.64 to 4.55 for the control group (p = .07). As seen in effect sizes of effects (d), most were quite small. As expected, the two resulting in statistically significant effects (positive attitudes and severity) were between “small” and “moderate” in Cohen’s typology, around 0.30.

TABLE 1.

Means (SD) for Marijuana-related variables by intervention versus control

Control					Intervention
Dependent
Variable	Pre	Post	Pre	Post	F	df	p	d
Pos. att.	2.21	2.27	2.35	2.21	2.85	1, 207	.045	–0.26
	(1.11)	(0.99)	(1.03)	(0.93)
Neg. att.	3.50	3.66	3.55	3.76	0.58	1, 213	.22	0.09
	(1.13)	(1.08)	(1.04)	(1.00)
Resp. eff.	3.69	3.57	3.78	3.88	1.29	1, 206	.13	0.20
	(1.40)	(1.61)	(1.33)	(1.61)
Norms	2.36	2.24	2.43	2.36	0.33	1, 207	.28	0.09
	(1.10)	(1.04)	(1.04)	(1.03)
RSE	4.64	4.55	4.54	4.58	2.22	1, 207	.07	0.07
	(0.96)	(0.96)	(0.99)	(0.93)
Intentions	1.22	1.26	1.19	1.18	0.46	1, 200	.25	–0.09
	(1.76)	(1.74)	(1.69)	(1.77)
Severity	3.74	3.70	3.78	3.93	3.81	1, 206	.03	0.31
	(1.03)	(0.98)	(0.92)	(0.99)

Impact of Threat and Framing on PSA Ratings

Threat

A summary of results assessing the impact of threat and framing on PSA ratings is shown in Table 2. Higher threat messages were significantly associated with greater fear response to PSAs (p ≤ .05), such that participants felt greater fear when messages were higher threat (M = 2.01) compared to low threat (M = 1.76). Perceived threat of the message did not affect responses of reactance or PME. Because the mean for fear response was a bit less than 2 out of 7 for the higher threat messages, we call them “mod-erate” threat messages rather than “high” threat messages from here and throughout the rest of the manuscript. Effect sizes (d) were generally small to moderate.

TABLE 2.

Means (SD) for PSA ratings by condition

Depend. Variable	Threat		Statistics	Frame		Statistics	Low Threat		Mod. Threat		Statistics
	Low	Mod		Loss	Gain		Loss	Gain	Loss	Gain
PME	4.18	4.28		4.24	4.27		3.86	4.52	4.82	4.00
	(1.33)	(1.41)		(1.40)	(1.34)		(1.23)	(1.35)	(1.29)	(1.28)
F			0.09			0.02					12.00
df			1,56			1,132					1,132
P			0.76			0.90					0.001
d			−0.07			−0.02					−1.15
Fear	1.58	1 . 78		2.20	1.51		1.93	1.55	2.60	1 . 46
	(0.78)	(1.12)		(1.08)	(0.77)		(0.82)	(0.73)	(1.20)	(0.83)
F			0.60			19.35					5.82
df			1,56			1,132					1,132
P			0.44			<.001					0.017
d			−0.21			0.76					−0.83
React.	2.69	3.11		3.22	2.78		3.30	2.56	3.10	3.01
F	(1.14)	(1.19)	2.11	(1.16)	(1.15)	5.74	(1.01)	(1.18)	(1.31)	(1.08)	2.94
df			1,56			1,132					1,132
P			0.15			0.018					0.089
rl			n ^			n					n

Framing

Participants experienced significantly greater feelings of fear (p ≤ .001) for loss-framed messages (M = 2.20) com-pared to gain-framed (M = 1.51). However, participants also experienced significantly more reactance (p = .018) when messages were loss-framed (M = 3.22) compared to gain-framed (M = 2.78). Message framing did not have an impact on PME. Effect sizes were moderate to large.

Interaction

The interaction of threat and framing on PME ratings was significant (p ≤ .001), as depicted in Figure 1. Partici-pants perceived messages as most effective when mes-sages were moderate threat/loss frame (M = 5.04). They perceived messages as least effective when they were low threat/loss frame (M = 3.57) or moderate threat gain frame (M = 3.54). Also, the interaction of threat and framing on fear response was significant (p = .017). Participants reported the greatest feelings of fear for the moderate-threat/loss-frame condition (M = 3.02), and the lowest fear response for the moderate threat/gain-frame and low threat/gain-frame conditions (M = 1.73 and 1.76, respec-tively). The interaction of threat and framing on responses of reactance was evidenced by a nonsignificant trend (p = .089). Participants reported the greatest feelings of reac-tance for the low threat/loss condition (M = 3.40) and the least amount of reactance for the low threat/gain condition (M = 2.84). Effect sizes were all moderate to large.

FIGURE 1.

PSA Ratings by Threat X Frame

Impact of Threat and Framing on Marijuana-Related Variables

Out of eight marijuana-related outcome variables, one variable, positive attitudes, evidenced a significant effect of frame, positive attitudes, with a small/moderate effect size (d = –0.26). Gain-framed messages led to less positive attitudes about marijuana while those in the loss frame condition stayed about the same. When selecting only individuals above the median on efficacy (combina-tion of refusal self-efficacy and response efficacy, as pro-posed in Witte’s EPPM), no variables were significant for the effect of threat, though effect sizes were around ab-solute value 0.3 for positive and negative attitudes. In addition, as shown in Figure 2, the interaction of threat and frame on participants’ reports of positive attitudes toward marijuana use neared significance (F (1, 150) = 6.23, p = .02). Moderate threat/loss-framed messages increased positive attitudes about marijuana (M = 2.18 to M = 2.40), while all of the other conditions reduced positive attitudes, such as low threat/loss frame (M = 2.24 to M = 1.98), low threat/gain frame (M = 2.30 to M = 2.19), and mod-erate threat/gain frame (M = 2.48 to M = 2.09). No other interactions neared significance for either threat or message framing. Confidence intervals for mean differences for change from pretest to posttest for the four conditions are presented in Table 4.

FIGURE 2.

Positive Attitudes about Marijuana by Threat X Frame.

TABLE 4.

Pre to post changes in means and conlidence intervals for positive attitudes about marijuana dependent variable

	Low threat		Moderate threat
Dependent variable	Loss	Gain	Loss	Gain
Mean change in positive attitudes about marijuana Confidence interval for mean change	−0.28 (−0.70, −0.06)	−0.11 (−0.35, +0.13)	+0.26 (−0.08, +0.60)	−0.29 (−0.53, −0.05)

Summary of Results for Hypothesis and Research Questions

Hypothesis 1 was not supported, with no significant ef-fects of threat for any of the marijuana-related variables, though the effect sizes were on the low end of moderate (around 0.3) for positive and negative attitudes about mar-ijuana, with lack of significant due in part to the small sample size of those scoring above the median on effi-cacy, as required by the EPPM. Research question 1 was answered with a partial “yes,” i.e., gain-framed messages did produce greater positive changes than loss-framed messages, though only for positive attitudes about mar-ijuana, whether assessed with significance testing or ef-fect size (this was the only effect size greater in absolute value than 0.25). Research question 2 was answered with a partial “yes,” with the finding of a significant interac-tion of threat and message framing for positive attitudes, such as low threat/loss frame (M = 2.24 to M = 1.98), low threat/gain frame (M = 2.30 to M = 2.19), and moderate threat/gain frame (M = 2.48 to M = 2.09). No other interactions neared significance for either threat or message framing. Confidence intervals for mean differences for change from pretest to posttest for the four conditions are presented in Table 4.

Summary of Results for Hypothesis and Research Questions

Hypothesis 1 was not supported, with no significant effects of threat for any of the marijuana-related variables, though the effect sizes were on the low end of moderate (around 0.3) for positive and negative attitudes about marijuana, with lack of significant due in part to the small sample size of those scoring above the median on efficacy, as required by the EPPM. Research question 1 was answered with a partial “yes,” i.e., gain-framed messages did produce greater positive changes than loss-framed messages, though only for positive attitudes about marijuana, whether assessed with significance testing or effect size (this was the only effect size greater in absolute value than 0.25). Research question 2 was answered with a partial “yes,” with the finding of a significant interaction of threat and message framing for positive attitudes about marijuana, with moderate threat/loss-framed messages resulting in an increase in positive attitudes about marijuana, and other combinations resulting in the desired decrease.

DISCUSSION

In sum, results indicated that the PSAs were generally successful (as compared to the control messages) at changing several mediating variables, including reduction of positive attitudes toward marijuana use, and increased perceived seriousness of consequences. Thus, though the exposure to the PSAs was quite minimal (a total of 21 2 min), the PSAs successfully changed important mediating variables, at least for a short period of time, suggesting that within an intensive campaign they would have the possibility of reducing marijuana use.

Furthermore, the impact of threat and framing on reactions to the PSAs and to mediating variables was interesting and complex. Higher (i.e., moderate rather than low) threat messages and loss-framed messages were associated with greater fear responses and loss-framed messages resulted in more reactance than gain-framed messages. Messages that offered both moderate threat and loss framing resulted in the greatest fear response; messages that were low-threat and loss-framed resulted in the greatest reactance while those that were low-threat and gain-framed resulted in the least reactance. Messages that were low-threat and loss-framed were perceived as the most effective. Finally, the one significant effect of condition on change in positive attitudes toward marijuana use was that moderate threat/loss-framed messages were the only ones that actually increased positive attitudes toward marijuana, i.e., had a negative effect while all of the other conditions reduced those attitudes, i.e., had a positive effect.

A focus of research for members of the research team, both in previous and current work, has been the moderating influence of individual difference variables (such as sensation seeking and impulsive decision making) on attentiveness to and processing of prevention messages (see, e.g., Donohew et al., 2000; Zimmerman et al., 2007). Accordingly, we elected to examine the impact of these constructs on message effectiveness in this study. Surprisingly, there were no significant interactions of sensation-seeking or impulsive decision-making with message condition (control vs. intervention or level of threat or frame type) on either PSA ratings or marijuana related variables. This finding is surprising given that some research has suggested that high sensation-seekers may require a higher level of threat to attend to and/or be persuaded by PSAs. Perhaps the relatively low threat posed by the “moderate” fear messages in this study did not rise to the level required by high sensation-seekers and we thus saw no differences.

In weighing the importance of the results from the perspective of public health—i.e., the likely effects of the PSAs on reducing marijuana use—it appears that while neither threat nor type of framing alone affected attitudes, the combination of moderate threat and loss frame was counter-productive, but all of the other combinations were useful. This suggests that the levels of threat and framing that produced the highest fear response, in combination, may have overshot that impact to the point that individuals were less likely to respond well to the messages. Low threat loss-framed messages, which produced a moderate level of fear response and the highest level of reactance, were nonetheless perceived as likely to be most effective. This combination of results suggests that antimarijuana PSAs—or possibly other types of messages offered in new media—that are both moderate (or higher) threat and loss-framed may produce too high a level of fear response to be effective. On the other hand, those that are both low threat and loss-framed produce the greatest perceived effectiveness and successful reduction in positive attitudes toward marijuana. Thus this is the combination of message characteristics that might be most effective in an actual campaign. Further research on the interaction of perceived threat with efficacy, to test the EPPM, should also be conducted, and all of these conclusions should be assessed within a field implemented campaign to see if the results shown here would be replicated with more intensive exposure, and over a longer period of time.

Limitations

The authors acknowledge a number of potential limitations of this study. First, level of perceived threat was relatively low (below 2 on a 7-point scale) for both PSAs that were coded as “low” and “moderate” threat, perhaps not allowing a very strong test of the impact of threat on outcomes. Second, reliability was low on the reactance measure, although it has been used in a variety of other studies. Third, a number of measures (perceived threat, severity, refusal self-efficacy, response efficacy, and intentions) were developed for this study and have not yet been independently validated, although their correlations with other measures are consistent with their validity. In addition, consistent with many other studies on substance use, the majority of measures utilized in this study were not normally distributed, potentially threatening the validity of results of the statistical tests used. The common practice in the substance abuse prevention field, however, is consistent with what we have done here, namely the use of such non-normally distributed measures, no transformation of the measures, and acknowledgment of this as a potential limitation. Focus groups consisting of target audience members were not used in selection of PSAs and their categorization by level of threat and message framing. Instead PSAs were coded by project coordinators and research assistants slightly older than the target audience. Results were not corrected for tests for multiple dependent variables, as a variety of different subsamples were used and in many cases, sample sizes were quite small; thus we may have arrived at slightly more significant findings than are appropriate for the set of analyses conducted. Finally, regular and heavy users were excluded from the study, based on previous findings that prevention-focused PSA campaigns are unlikely to affect them, but their inclusion might have added useful information.

CONCLUSION

Results of this study suggest that frame may have some interesting effects on perceived threat of messages, with loss-framed messages leading to greater perceived threat, as well as reactance. Frame also had significant effects on positive attitudes toward marijuana, with gain-framed messages leading to a greater reduction in positive attitudes toward marijuana than loss-framed messages. Finally, frame and threat interacted in a significant way, with moderate threat/loss-framed messages actually leading to an increase in positive attitudes toward marijuana, perhaps through participants’ level of reactance to the message. Given the limitations of this study, an attempt should be made to replicate this interaction effect and, if replicated, to extend our understanding through hypothesis testing related to prospect theory and the Extended Parallel Processing Model. But finally, given the failure of the large national youth antidrug media campaign to impact marijuana use (see Hornik, Jacobsohn, Orwin, Piesse, &Katon, 2008; Magura, 2012), perhaps these suggestions about the level of threat and frame for antimarijuana PSAs should be considered in future campaigns.

TABLE 3.

Means (SD) for marijuana-related variables by condition

		Threat								Frame
	Low		Moderate						Loss		Gain
Dep. Variable	Pre	Post	Pre	Post	F	df	P	d	Pre	Post	Pre	Post	F	df	P	d
Pos. att. (0.14)	2.10 (0.11)	1.81 (0.15)	2.13 (0.12)	2.10	2.34	1.80	0.13	0.35 (0.98)	2.19 (0.91)	2.17 (1.06)	2.41 (0.94)	2.19	3.02	1,163	.04	−0.26
Neg. att. (0 10)	3.98 (0.10)	4.02 (0.11)	3.94 (0.11)	4.17	1.36	1.80	0.24	−0.29 (1.10)	3.58 (1.00)	3.82 (1.00)	3.53 (1.01)	3.73	0.22	1,163	.64	−0.07
Rcsp. cfT	-	-	-	-	~	-	-	-	3.71 (1.33)	3.87 (1.53)	3.80 (1.34)	3.91 (1.67)	0.05	1,163	.83	−O.G4
Norms	2.21 (0.14)	2.08 (0.13)	2.19 (0.16)	2.05 (0.14)	0.01	1.80	0.92	−0.02	2.40 (1.02)	2.33 (1.02)	2.44 (1.05)	2.37 (1.05)	0.01	1,163	.92	0
RSE	~	-	-	-	-	-	-	-	4.53 (1.00)	4.57 (0.90)	4.61 (0.98)	4.54 (0.96)	0.22	1,163	.64	0.06
Intentions	1.05 (0.08)	1.01 (0.02)	1.19 (0.09)	1.07 (0.02)	0.43	1.80	0.51	−0.16	1.18 (1.65)	1.21 (1.68)	1.21 (1.72)	1.17 (1.84)	0.99	1,163	.32	−0.12
Severity	4.19 (0.09)	4.35 (0.10)	4.11 (0.10)	4.18 (0.11)	0.41	1.80	0.52	0.14	3.81 (0.93)	4.01 (0.96)	3.77 (0.91)	3.89 (1.00)	0.78	1,162	.38	−0.12

Note. Analyses for threat were conducted only for those whose combined efficacy score (combination of response efficacy and refusal self- efficacy) was above the sample median, consistent with predictions from Witte’s Extended Parallel Processing Model.

GLOSSARY

Gain-framed message

Message that focuses on potential positive outcomes of engaging in a behavior

Loss-framed message

Message that focuses on potential negative outcomes of not engaging in a behavior

Threat

The extent to which a message leads someone exposed to the message to experience a fear response

PSA

Public Service Announcement, can be aired for free or for an advertising fee