Avoiding the Boomerang: Testing the Relative Effectiveness of Antidrug Public Service Announcements Before a National Campaign

Abstract

Objectives. This study examined the relative perceived effectiveness of 30 antidrug public service announcements (PSAs) and assessed the extent to which judgments of effectiveness are related to judgments of realism, amount learned, and positive and negative emotional responses.

Methods. Data were obtained from 3608 students in grades 5 through 12 in 10 schools. The ethnically diverse sample was 50.8% male. Students in 5 experimental conditions viewed sets of 6 antidrug PSAs and filled out a brief evaluation questionnaire following each PSA. Those in the control condition viewed a non–drug-related television program.

Results. The relative perceived effectiveness of the 30 PSAs varied considerably. Sixteen were rated as significantly more effective, and 6 as significantly less effective, than the control program. Relative rated effectiveness was highly related to realism (r = .87), amount learned (r = .88), negative emotion (r = .87), and positive emotion (r = –.35).

Conclusions. Evaluative research is necessary to prevent broadcast of PSAs that could have a negative impact. PSAs should point out the negative consequences of drug use behavior rather than telling adolescents to “just say no.”

On July 15, 1998, New York Times columnist Frank Rich ridiculed an ad aired as part of a $200 million federal effort to minimize adolescents' use of illegal drugs. “In this elegantly shot display of high-concept Madison Avenue creativity,” wrote Rich, “a young woman armed with a skillet angrily smashes an egg and then an entire kitchen to dramatize the destructiveness of heroin. . . . The woman looks like Winona Ryder; she's wearing a tight tank top; there are no visible track marks on her junkie-thin arms; and the kitchen representing her drug-induced hell is echt Pottery Barn, if not Willams-Sonoma. Far from discouraging teenagers from drug use, our anti-heroin heroine—sexy when she gets mad—may inspire some of them to seek out a vixen like her for a date.”1

The problem with critiques such as Rich's is that middle-aged, upper-class, White columnists were not the intended audience for the ad. Nonetheless, later in the column, he makes an important assumption, and one we share. Such ads should be tested rigorously before being aired. With the passage of HR 4328 (Pub L No. 105-277) in 1997, the US government embarked on its first funded media campaign to reduce “risky” behaviors and to promote “healthy” ones. The centerpiece of the campaign, which focuses on minimizing illegal drug use (particularly marijuana and inhalants) among young adolescents who have not yet become “regular” users of an illegal substance, is televised antidrug public service announcements (PSAs). A critical question is whether such a campaign has the potential for producing behavior change in this target population.

Although there is some evidence that mass media campaigns can be successful,2–5 most studies evaluating mass media campaigns have found little or no effect.4,6–8 To a large extent, this failure has been attributed to the fact that most health promotion campaigns have been underfunded, limiting the reach and frequency of the messages. In addition, they have relied on the goodwill of broadcasters to place PSAs in time periods watched by the target audience. While exposure is clearly a critical element in the success of any media campaign, one must also pay careful attention to the content of the message. Recent advances in behavioral science theory and research have pointed out the importance of message content and the need to design theoretically and empirically based behavioral change interventions.9–13 Although these studies have typically not relied on mass media to deliver the message, they do provide evidence that theoretically based messages addressing the beliefs and values of a specific population can significantly change behavior.

More specifically, although knowledge about a disease and about how it is spread appears to have little relation to healthprotective behaviors,14–17 messages targeting outcome expectancies (i.e., beliefs about the consequences of performing the behavior), normative beliefs (i.e., beliefs about the behaviors and normative proscriptions of relevant others), and self-efficacy beliefs (i.e., beliefs that one can perform the behavior, even under a number of difficult conditions) have produced behavior change.10–13,18–21

The effectiveness of these types of messages, however, depends on both the desired behavior and the target population. For example, for some behaviors, attitudinal considerations (i.e., beliefs about the consequences of performing the behavior) may be the primary determinants, while for other behaviors, attitudinal considerations may be relatively unimportant and either perceived norms or self-efficacy may be the critical variable. Similarly, although a given behavior may be attitudinally driven in one population, it may be normatively controlled in another.22,23

In addition, it should be recognized that beliefs that may be important determinants of attitudes, perceived norms, or self-efficacy in one population may be unimportant in another. Moreover, it is unfortunately true that messages often target beliefs that are already strongly held by the population in question (e.g., “smoking is harmful to health”). Thus, for any behavioral change intervention to be effective, it is first necessary to understand the factors underlying the behavior in the population in question. In other words, one must first determine the extent to which the behavior is influenced by attitudes, norms, or self-efficacy and then identify the critical beliefs underlying those important psychosocial variables.

Given that this type of preparatory research is rarely done, it should come as no surprise that most interventions and mass media campaigns fail. Equally important, aside from some qualitative research, typically to determine whether 1 or more target audiences “understand” and “like” or “dislike” the PSA, there have been few attempts to empirically evaluate the potential effectiveness of televised PSAs designed to change healthrelated behaviors. For example, to the best of our knowledge, none of the PSAs investigated in this study were subjected to experimental evaluation before being broadcast. Given the fact that few, if any, were developed with the kind of quantitative research we have described, it is quite likely that there will be wide variation in the PSAs' potential for producing behavioral change. Perhaps more important, although many of these PSAs may in fact be helpful in preventing the initiation of drug use, it would not be surprising to find that at least some are having no effect, while still others may be having negative effects (i.e., may be increasing rather than decreasing the likelihood that adolescents will try or use illicit drugs).

The purpose of this study was to evaluate the potential effectiveness of 30 antidrug PSAs produced by the Partnership for a Drug Free America (PDFA). More specifically, we examine the extent to which adolescent boys and girls view these 30 PSAs as being effective. In addition, we examine the extent to which judgments of effectiveness are related to judgments of realism and learning (i.e., did the adolescents feel that they learned something new from the PSA), as well as to the adolescents' positive and negative emotional responses to the PSA. It is important to recognize that judgments of effectiveness may or may not be related to the extent to which a given message actually prevents illicit drug use behaviors. However, it seems reasonable to assume that judgments of effectiveness may be a necessary (although not a sufficient) condition for producing actual change in the psychosocial determinants of drug use behaviors (i.e., underlying beliefs, attitudes, perceived norms, self-efficacy, and intention). Thus, the present study can be looked at as a first step in evaluating the potential effectiveness of antidrug PSAs.

Generally speaking, we expected to find wide variation in the perceived effectiveness of the 30 PSAs. Moreover, considering the PSA as the unit of analysis, we expected that the more one learned from a PSA and the more “realistic” the PSA, the more it would be seen as effective. Finally, it seemed reasonable to assume that PSAs directed toward outcome beliefs, norms, or self-efficacy would be perceived to be more effective than those that did not address these underlying psychosocial variables.

METHODS

Sample

Study participants were 3608 middle and high school students from 10 separate schools, chosen on the basis of subjects' accessibility and representation of various ethnic, sexual, and racial mixes. More specifically, the students were drawn from the populations of 3 middle schools and 7 high schools from 4 states. Three of the schools were in a rural setting and 7 were in urban environments. While 2 of the urban schools drew only from the areas around them, the other 5 attracted students from more than 15 miles away. For example, 1 of the urban schools drew 25% of its students from the city and the rest from the suburbs.

The schools served a heterogeneous population. Seven of the schools were private and 3 were public. Six were coeducational, 1 was all female, and the remaining 3 were all male. Between 80% and 100% of the student population from each school was included in the study. In each school setting, permission was granted by the school authorities to assess students' reactions to antidrug PSAs. Letters from the school principals describing the study were sent to students' homes. The letter described the increasing use of drugs among adolescents and the attempt to prevent further increases through antidrug PSAs. It also made clear that students were simply being asked to respond to the PSAs and that they would not be asked any questions about either their own drug use or that of their families. Confidentiality was promised. Parents were advised to call the school if they had concerns about the study or their child's participation in it. Only 1 set of parents actually called to request that their children be excused. Some subjects in 1 of the schools refused to participate because they felt that, as was the case with other studies conducted in the area, they should be paid to participate.

The final sample of 3608 students who participated in the study was almost equally divided between males (50.8%) and females (49.2%). The students were in grades 5 through 12, with grade 9 as the mode and grade 10 as the median; the ages ranged from 11 to 20 years, with a median of 15. Whites made up 49.1% of the sample, with African American subjects making up 31.5%, Hispanics 6.4%, Native Americans 4.4%, and Asian/Pacific Islander 3.1%; 5.5% were “other” or of mixed heritage or did not report their ethnicity.

Experimental Conditions

The study involved 5 experimental conditions and 1 control condition. Each experimental condition included 6 antidrug PSAs produced by the PDFA. Condition 1 consisted of ads that focused primarily on negative consequences of drug use. Four of the ads can best be described as dramatic representations, one was a testimonial, and one was a comic representation in which a dog's-eye view was used to mock marijuana smokers. Conditions 2 and 3 used PSAs designed, according to the PDFA, to bolster selfefficacy in refusing to use drugs. Both conditions focused primarily on ads showing avoidance behavior; condition 2 contained ads that appeared to address issues of efficacy and esteem, while condition 3 mainly contained ads suggesting that the viewer should “Just say no.” Two additional conditions (4 and 5) also focused largely on the negative outcomes of drug use. In contrast to condition 1, which focused mainly on marijuana and inhalant use, conditions 4 and 5 focused primarily on methamphetamine and heroin. Generally speaking, on the basis of our analysis of PSA content and the PDFA's descriptions, an attempt was made to develop clusters of ads that would maximize the impact of an underlying theme. Table 1 ▶ outlines the 5 experimental conditions and provides the title or theme of each PSA.

TABLE 1.

—Characteristics of 30 Antidrug Public Service Announcements Viewed by High School Students in 5 Experimental Conditions

Condition	No.	Ordera	Theme or Title	Type	Subtype	Drug
1	1	1	Alex–Straight A	PT-O		MJ
	2	2	Pound Head	DR-O		MJ
	3	3	Drowning	DR-O		Inhalants
	4	4	Free Ride	DR-DC		(Drug dealers)
	5	5	911 (Boyfriend/Girlfriend)	DR-O		Meth
	6	6	Dog's View	CR-O		MJ
2	7	1	In-Line Skaters	M-AHB		Drugs
	8	2	Long Way Home	DR-AB	SEF	Drugs
	9	3	Dealer Shot	PT-AB	SES	Crack, drugs
	10	4	No, No, No	M	Say no	MJ
	11	5	Rite of Passage	DR-AB	SEF	Drugs
	12	6	You Are the Universe	M	SES	(Not drugs)b
3	13	1	Chuck D, Loser	CW		Drugs
	14	2	Friend Offers Pot	DR-AB	Say no	MJ
	15	3	My Heart's Here	PT-AB	SES	MJ
	16	4	Saying No OK	DR-AB	Say no	MJ
	17	5	No Way	DR-AB	Say no	MJ
	18	6	Chris Hill	DR-AB	Say no	MJ
4	19	1	Pot Head	CR-O		MJ
	20	2	Gateway Drug	PT-Info		MJ, drugs
	21	3	Frying Pan	DR-O		Heroin
	22	4	Troy Dead	CT-O		Heroin
	23	5	Lauryn Hill	CW-O	SES	Drugs
	24	6	45% Inner City	Info		Drugs
5	25	1	Noses	CR-O		Inhalants
	26	2	911 (Parent/Child)	DR-O		Meth
	27	3	What Would Make	M-O		Meth
	28	4	Roaches	DR-O		Meth
	29	5	Teeth	DR-O		Heroin
	30	6	Johnny Street	PT-O		Heroin

Note. For types, DR = dramatic representation; CR = comic representation; CT = celebrity testimonial; PT = personal testimonial; CW = celebrity warning; M = montage; O = negative outcomes; AB = avoidance behavior; AHB = alternative health behavior; DC = drug culture. For subtypes, SEF = self-efficacy; SES = self-esteem; Say no = “Just say no.” For drugs, MJ = marijuana; Meth = methamphetamine.

^aOrder of viewing.

^bThis public service announcement was designed to increase self-esteem and did not refer to drug use of any kind.

The control condition was composed of a single 24-minute public television program about techniques of video and news production called In the Mix. The program included 24 seconds, spread over various random street scenes, in which vague references to drugs could be detected. These references included mention, in passing, of perceptions of drug use among adolescents and the word “Crack” spray-painted on a wall in the background of one of the scenes. Table 2 ▶ provides a brief overview of the experiment and shows the number of students and schools participating in each condition.

TABLE 2.

—Overview of Experimental Design and Conditions for Evaluations of 30 Antidrug Public Service Announcements (PSAs)

Condition	na	PSA Approach	Schools
Control	825	(Not applicable)	1–10
1	587	DR-O, MJ, inhalants	1–6, 9
2	639	AB, SEF, SES	1–5, 9, 10
3	546	DR-AB, say no	1–6, 9
4	614	O, heroin, drugs	6–10
5	397	DR-O, heroin, meth	6–9

Note. DR = dramatic representation; O = negative outcomes; AB = avoidance behavior; SEF = self-efficacy; SES = self-esteem; say no = “Just say no”; MJ = marijuana; meth = methamphetamine.

^aNumber of participating students.

Administration

To mitigate any possibility of systematic bias, assignment to condition was done randomly by class. Only nontracked or nonstratified classes were used. Roughly equal numbers of subjects were assigned to each condition within each school. Some classrooms were dropped from the study owing to administration difficulties or time constraints.

The actual testing was done in a classroom setting and administered by a teacher. Materials and stimuli were standardized across conditions, with instructions presented on videotape. Confidentiality and anonymity were emphasized in the instructions, both in written and audio-video form. Teachers were instructed to hand out the materials and then play the videotape, stopping it at indicated points so that students could answer the appropriate questions. The evaluation instrument consisted of a questionnaire and the videotaped stimulus. Answers were recorded on an optical scan sheet.

Outcome Measures

The instrument for the study consisted of a questionnaire with 3 parts. The first was a pretest that asked demographic questions. The second consisted of a series of questions asked after each of the 6 PSAs or after the control program. Generally speaking, these questions focused on subjects' perceptions of the “realism” of the message, their emotional reaction to it, and their beliefs that viewing the message would help them and their friends avoid drugs. The third part of the instrument assessed the respondents' perceptions of the danger and harmfulness of engaging in a number of risky behaviors. It also asked for their perceptions concerning the number of people their own age who engage in each of these behaviors.

On the basis of part 2 of the questionnaire, the following 5 dependent variables were assessed for each PSA as well as for the control program: perceived effectiveness, perceived realism, negative emotional response, positive emotional response, and amount learned.

Perceived effectiveness was based on 4 items from part 2 of the questionnaire: (1) Was the message convincing? (2) Would it be helpful in keeping your friends from using the drug? (3) Would people your age who have never used drugs be more or less likely to want to try the drug after seeing the ad? (4) How confident did the ad make you feel about how best to deal with illegal drugs in the real world? Each question was answered on a 4-point scale (e.g., definitely yes [4], yes [3], no [2], definitely no [1]). The mean of these 4 items was used to indicate the perceived antidrug effectiveness of each PSA (including the program shown in the control condition; α = .79).

Perceived realism was also based on 4 items from part 2 of the questionnaire: (1) Was the message believable? (2) Was the message honest? (3) Would real people act the way the person(s) in the ad did to deal with illegal drugs? (4) If someone did the things shown in the ad, would the things you were shown really happen to that person? (In some of the schools, students were asked, “Were some of the actors in the ad high on drugs?” instead of “Was the message honest?” This did not affect the internal consistency [i.e., α] of the scale, and, as will be described below, because within-school difference scores were used, it did not affect the analyses.) Again, for each of the 6 PSAs, each of the 4 items was answered on a 4-point scale and the mean of these 4 items was used to indicate the mean perceived realism of each PSA (α = .86).

For negative emotional response, respondents indicated the degree to which, while viewing a PSA, they felt sad, angry, afraid, and disgusted. Responses ranged from 1 (not at all) to 4 (very much), and the mean of the 4 items served as a measure of negative emotional response to a given PSA. Higher scores indicate stronger negative emotional reactions to each PSA (α = .95).

For positive emotional response, respondents were asked to indicate the degree to which they felt happy and excited while viewing the PSA. (In some schools, respondents were given both positive emotion items, while in others they were asked to respond only to the “happy” or only to the “excited” item. The 2 items were correlated at r = .65, and, as will be described below, because within-school difference scores were used, variations in the number of items did not affect the analyses.) The positive emotion score also ranged from 1 (not at all) to 4 (very much).

For amount learned, respondents were asked, “How much, if anything, did you see in the ad that you didn’t already know?” They answered on a 3-point scale ranging from 1 (“nothing at all”) to 3 (“a great deal”).

In addition to obtaining each respondent's rating of each PSA, we averaged the respondents' scores over the 6 PSAs to which they were exposed. Thus, for each respondent, we had measures of the mean perceived effectiveness, realism, and amount learned from the set of PSAs they viewed. In addition, we had their mean positive and negative emotional responses to the set of 6 PSAs that they saw.

Three other measures were obtained from part 3 of the questionnaire: perceived harmfulness of engaging in risky behaviors, perceived danger of engaging in risky behaviors “even once,” and perceived behavioral norms (i.e., perceptions of the number of people engaging in these risky behaviors). More specifically, with respect to a set of 8 risky behaviors (sniffing glue, using speed, riding in a car without using a seat belt, drinking beer, drinking hard liquor, smoking cigarettes, chewing smokeless tobacco, and smoking marijuana), the adolescents were asked the following.

For perceived danger, adolescents were asked, “Imagine that your best friend asks you if you think there are some things that are so dangerous he or she should never try them.” The adolescents were asked to answer “No, it is not too dangerous” or “Yes, it is too dangerous” for each of the risky behaviors.

For perceived harmfulness, they were asked, “On a scale from 1 to 5, where 1 means ‘not harmful at all’ and 5 means ‘extremely harmful,’ how harmful do you think it is for someone your age to [engage in each of the risky behaviors]?”

For perceived norms, they were asked, “Of students your own age, how many do you think do, or have done, the following things?” For each behavior, the adolescents responded according to the following 5-item scale: “none” (1), “a few” (2), “some” (3), “more than half” (4), “most” (5).

Internal consistency analyses indicated that, for each of these 3 measures, the risky behaviors we considered (with the exception of the behavior of not wearing a seat belt, which was excluded from the final scales) formed an internally consistent scale (α = .82 to .90). Thus, for each subject, a perceived danger, a perceived harmfulness, and a perceived norm score were computed. The higher the harmfulness and danger scores, the more adolescents felt that engaging in risky behaviors was harmful or too dangerous to try even once; the higher the perceived norm score, the more they believed that most people like them engaged in these risky behaviors. (In addition to checking internal consistency [or reliability—i.e., α] for the sample as a whole, additional internal consistency tests indicated that all the scales constructed from parts 2 and 3 of the questionnaire were equally reliable across grades, sex, and ethnicity.)

To summarize, for each respondent, there are a total of 38 scores: 5 dependent variables (perceived effectiveness, perceived realism, learning, and positive and negative emotional responses) for each of 6 PSAs; 5 mean scores, 1 for each dependent variable (averaged over the 6 PSAs to which the respondent was exposed); and the 3 scores from part 3 (danger, harmfulness, and perceived norms).

To control for possible differences associated with the school the respondent attended, we subtracted the mean scores for the control condition within a given school on each of these ratings from the scores of each individual in that school. For example, within a given school, the mean rating of the effectiveness of the control program was computed. This score was then subtracted from each of the 6 effectiveness ratings provided by a given individual, as well as from the individual's mean effectiveness rating. Similarly, within a given school, the control group's mean rating of harmfulness was computed, and this score was subtracted from the harmfulness ratings of each student in the school. Thus, the actual dependent variables were difference scores, where positive values indicated that the student rated the PSA above the mean of the control group in his or her school and negative values indicated that the student rated the PSA below the mean of the control group in his or her school.

Finally, to partial out possible systematic biases other than school, the difference scores were subjected to covariance analyses with sex, ethnicity, and grade as covariates. Thus, for example, with respect to perceived effectiveness, we conducted a multivariate 5 (message condition) × 6 (PSAs) covariance analysis, with the 6 effectiveness difference scores serving as dependent variables and sex, grade, and ethnicity serving as covariates. This analysis provides estimates of the adjusted mean relative perceived effectiveness of each of the 30 PSAs. Although a complete discussion of this analysis is beyond the scope of this report, it is worth noting that both sex and grade were significant covariates of relative effectiveness, providing additional evidence for the necessity of targeting messages to specific populations. Similar analyses were conducted with respect to each of the other dependent variables. (A critical question concerns the extent to which differences in grade, sex, and ethnicity influence perceptions of PSAs [e.g., influence judgments of effectiveness or realism]. We are currently preparing a separate paper to address this issue.)

Coding the Content of the PSAs

In addition to the ratings of the adolescents, each of the 30 PSAs was coded along several dimensions. More specifically, each PSA was classified in terms of the following characteristics.

Dramatic representation. All PSAs that used dramatic representation to deliver the message (coded as DR in Table 1 ▶) were given a score of 1; all others received a score of 0.

Negative outcome. All PSAs that provided information about the negative consequences of drug use (coded as O in Table 1 ▶) were given a score of 1; all others received a score of 0.

“Just say no.” All PSAs whose main message was to “just say no” to drugs (coded as Say no in Table 1 ▶) were given a score of 1; all others received a score of 0.

Marijuana. PSAs that specifically targeted marijuana use were coded 1; all others were coded 0.

Heroin or methamphetamine. PSAs that specifically targeted the use of heroin or methamphetamine were given a score of 1; all others were coded 0.

Unspecified drug. PSAs that talked about drugs in general or that did not specify any drug were given a score of 1; all others were coded 0.

RESULTS

Table 3 ▶ presents the mean relative perceived effectiveness of each PSA, adjusted for sex, ethnicity, and grade in school. It can be seen that, as expected, there was considerable variation in the relative effectiveness of the 30 PSAs (difference scores range from +0.938 to –0.286). Perhaps even more important, while 16 of the PSAs were rated as significantly more effective in reducing drug use than was the control program, 6 PSAs were seen as significantly less effective than the control program; that is, adolescents viewing these 6 PSAs reported that they and their friends would be more likely to try or to use drugs, and would feel less confident about how to deal with situations involving drugs, than adolescents viewing a public television program about techniques of video and news production. The remaining 8 PSAs did not differ from the control program.

TABLE 3.

—Mean Relative Effectiveness Scores of Antidrug Public Service Announcements (PSAs), Adjusted for Sex, Ethnicity, and Grade in School

PSA	AMRE	Theme or Title	Type	Subtype	Drug
28	0.938**	Roaches	DR-O		Meth
29	0.854**	Teeth	DR-O		Heroin
21	0.678**	Frying Pan	DR-O		Heroin
27	0.638**	What Would Make	M-O		Meth
22	0.597**	Troy Dead	CT-O		Heroin
26	0.559**	911 (Parent/Child)	DR-O		Meth
30	0.538**	Johnny Street	PT-O		Heroin
5	0.531**	911 (Boyfriend/Girlfriend)	DR-O		Meth
3	0.521**	Drowning	DR-O		Inhalants
8	0.494**	Long Way Home	DR-AB	SEF	Drugs
20	0.387**	Gateway Drug	PT-Info		MJ, drugs
11	0.359**	Rite of Passage	DR-AB	SEF	Drugs
9	0.259**	Dealer Shot	PT-AB	SES	Crack, drugs
4	0.243**	Free Ride	DR-DC		(Drug dealers)
23	0.212**	Lauryn Hill	CW-O	SES	Drugs
16	0.077*	Saying No OK	DR-AB	Say no	MJ
2	0.056	Pound Head	DR-O		MJ
1	0.053	Alex	PT-O		MJ
13	–0.001	Chuck D, Loser	CW		Drugs
15	–0.022	My Heart's Here	PT-AB	SES	MJ
24	–0.022	45% Inner City	Info		Drugs
19	–0.029	Pot Head	CR-O		MJ
14	–0.044	Friend Offers MJ	DR-AB	Say no	MJ
18	–0.060	Chris Hill	DR-AB	Say no	MJ
25	–0.069*	Noses	CR-O		Inhalants
10	–0.089*	No, No, No	M	Say no	MJ
17	–0.094*	No Way	DR-AB	Say no	MJ
12	–0.225**	You Are the Universe	M	SES	(Not drugs)a
7	–0.237**	In-Line Skaters	M-AHB		Drugs
6	–0.286**	Dog's View	CR-O		MJ

Note. AMRE = Adjusted mean relative effectiveness. For types, DR = dramatic representation; CR = comic representation; CT = celebrity testimonial; PT = personal testimonial; CW = celebrity warning; M = montage; O = negative outcomes; AB = avoidance behavior; AHB = alternative health behavior; DC = drug culture. For subtypes, SEF = self-efficacy; SES = self-esteem; Say no = “Just say no.” For drugs, MJ = marijuana; Meth = methamphetamine.

^aThis public service announcement was designed to increase self-esteem and did not refer to drug use of any kind.

*P<.05; **P<.01.

To explore factors that might explain why some PSAs were perceived to be more effective than others, we correlated the mean adjusted relative effectiveness scores with the mean adjusted difference scores for realism, learning, positive emotion, and negative emotion. Using the PSA as the unit of analysis, Table 4 ▶ shows that realism (r = 0.87), learning (r = 0.88), and negative emotional responses (r = 0.87) were all highly correlated with perceived effectiveness. In contrast, the more the adolescents had a positive emotional response to the PSA, the less they tended to view it as effective (r = –0.35, P = .06).

TABLE 4.

—Factors Related to the Perceived Relative Effectiveness of Individual Public Service Announcements (PSAs) and Sets of PSAs: Correlations Using the PSA and the Respondent as the Unit of Analysis

	PSA Effectiveness (n=30)	Effectiveness of Set (n=3608)
Realism	0.87	0.77
Learning	0.88	0.43
Negative emotion	0.87	0.29
Positive emotion	−0.35 (P=.06)	0.08
Heroin/methamphetamine	0.77	NA
Marijuana	−0.53	NA
Drug not specified	−0.23 (NS)	NA
Dramatic representation	0.38	NA
Negative outcomes	0.48	NA
“Just say no”	−0.29 (NS)	NA
Perceived danger	NA	0.12
Perceived harmfulness	NA	0.25
Perceived norms	NA	−0.13

Note. NA = not applicable; NS = not significant. Unless otherwise marked, all correlations are significant (P<.01).

We also considered how the content of the PSA related to its perceived effectiveness. Even a cursory glance at Table 3 ▶ shows that the PSAs perceived as most effective were those directed at methamphetamine or heroin, whereas the least effective ones were those directed at marijuana or those that talked about drugs in general. Similarly, it can be seen in Table 3 ▶ that the most effective PSAs provided information about the negative consequences of drug use, whereas the least effective tended to focus on avoidance behaviors and on “just saying no.” These qualitative judgments are supported by the correlations between the 6 content variables and perceived relative effectiveness (Table 4 ▶). PSAs targeting heroin and methamphetamine were judged to be most effective (r = 0.77), whereas those targeting marijuana were judged as least effective (r = –0.52). Those not specifying a drug, or talking about drugs in general, also tended to be judged as ineffective, although this relationship was not significant (r = –0.23). PSAs describing the negative outcomes of drug use (r = 0.48) and those using dramatic representation (r = 0.38) were viewed as more effective than those not using these approaches, while those stressing the “just say no” message tended to be judged as less effective (r = –0.29).

In addition to using the PSA as the unit of analysis, we also looked at the mean relative perceived effectiveness of the set of PSAs using the individual as the unit of analysis. More specifically, we correlated the mean relative effectiveness of the set of PSAs constituting a given condition, with mean realism, learning, and positive and negative emotion scores associated with the set of PSAs making up that condition. In addition, these mean relative effectiveness scores were also correlated with the adolescents' ratings of the harmfulness, danger, and perceived norms associated with engaging in a number of risky behaviors. These correlations were computed within each condition as well as across all conditions. Since the correlations by condition did not differ and were similar to the correlations using the entire sample, only the latter correlations are reported. Because of the very large sample size (n = 3608), all correlations are highly significant. However, as can be seen in Table 4 ▶, the best predictor of the relative perceived effectiveness of the set of PSAs to which a person was exposed is the mean relative perceived realism of the set (r = 0.77). In addition, the more, on average, one learned from the set (r = 0.43) and the stronger one's negative emotional response to the set (r = 0.29), the greater the mean perceived effectiveness of the set. In contrast to the findings using the PSA as the unit of analysis, there was also a small positive correlation between the relative perceived effectiveness of the set of PSAs and the mean positive emotional response to the set (r = 0.08).

Finally, Table 4 ▶ also shows that the more one viewed a set of PSAs as effective, the more one believed that engaging in the set of risky behaviors was harmful (r = 0.25) and too dangerous to try even once (r = 0.12). In addition, the more one evaluated a set as effective, the less likely one was to perceive that others of the same age were engaging in risky behaviors (r = –0.13).

DISCUSSION

Perhaps the most important finding is the wide variation in the perceived effectiveness of the antidrug PSAs produced by the PDFA. While most of the 30 PSAs we considered did, in fact, make adolescents think that they and their friends would be less likely to use drugs or that they would be more confident about how to handle situations in which drugs were offered or available, there were several that had little or no effect and 6 that had negative effects; that is, after viewing these 6 PSAs, the adolescents felt that they and their friends would be more likely to try drugs or that they would be less confident about handling drug-related situations than adolescents exposed to a public television program about techniques of video and news production. These findings clearly point out the necessity of conducting careful evaluations of the potential impact of antidrug PSAs before broadcasting them.

Through a careful consideration of the content of those PSAs that were judged to be most and least effective, some insight into the factors that lead to successful antidrug PSAs was obtained. The most effective PSAs were directed at heroin and methamphetamine, while the least effective were directed at either marijuana or drugs in general. There are at least 2 possible explanations for this finding. First, it may be much more difficult to change young people's beliefs, attitudes, and intentions regarding use of marijuana than use of “harder drugs.” Alternatively, it is possible that the reduced perceived effectiveness of PSAs targeting marijuana reflects the use of different types of PSAs. More specifically, while most of the ads directed at methamphetamine and heroin use were dramatic representations showing negative consequences associated with using these drugs, most of the ads directed at marijuana used a variety of approaches that either modeled avoidance behaviors or essentially told the viewer to “just say no.” Notably, none of the 3 humorous PSAs were judged to be effective and 2 actually were rated as having significant negative effects. Interestingly, 2 were directed at marijuana and 1 at inhalants; humor was not used to convey information about the harder drugs.

In addition, and perhaps not surprisingly, the perceived effectiveness of the PSAs is significantly related to perceptions of harm, danger, and social norms; that is, the greater the perceived effectiveness of the PSA, the more the 7 substance-related risky behaviors (e.g., smoking cigarettes, drinking hard liquor) are seen as harmful and dangerous and the less one believes that people one's own age engage in these behaviors. Since these are correlational data, there are at least 2 explanations for this finding. First, it could be argued that the more a PSA was perceived as effective, the more it increased perceptions of harm and danger and the more it changed perceptions of social norms (i.e., reduced perceptions of the number of people engaging in risky behaviors). Alternatively, one could argue that the PSAs appear to have the biggest impact on those who seem to need them least; or, to put this somewhat differently, those who most need to be influenced by these PSAs (i.e., those who do not view these risky behaviors as harmful or dangerous) are least likely to view the PSAs as effective. Such a view is clearly consistent with the oftenexpressed notion that those at highest risk are those who will be most difficult to influence.

These findings should make it clear that one cannot simply assume that an antidrug PSA will be effective. Not only is it necessary to first conduct empirical research to determine the critical variables influencing a given behavior but, equally important, it is necessary to critically evaluate the potential effectiveness of a given PSA. It is important to recognize, however, that perceived effectiveness is not a surrogate for “true” effectiveness. Indeed, even though adolescents may believe that a given PSA will help them and their friends avoid illicit drug use, this does not guarantee that the PSA will influence their drug use behavior or the psychosocial determinants of that behavior. Nevertheless, it seems reasonable to assume that messages perceived as ineffective (or as having a negative impact) are unlikely to prevent, and may actually facilitate, risky behavior. Thus, on the basis of our findings, it seems safe to recommend the following.

1. Only those PSAs that can be empirically shown to be significantly more effective than a control program should be selected for use in the government's national antidrug campaign. Those PSAs that appear to have a negative impact or that do not differ significantly from a control program should not be aired.

2. PSAs that focus primarily on “saying no” to marijuana should be aired with caution. New PSAs that focus on the negative consequences of smoking marijuana and, perhaps more important, on the negative consequences of trying marijuana, need to be developed and pretested. To be effective, these ads will have to provide new information and be perceived as realistic.

3. It is necessary to determine the causal direction of the relationships linking perceived harm, perceived danger, and perceived social norms to perceived PSA effectiveness. Clearly, very different strategies will be necessary depending on whether the antidrug PSAs do, in fact, increase perceptions of harm and danger or whether those who view drug-related behaviors as “relatively safe” (i.e., those at “high risk”) rate these PSAs as unrealistic and ineffective.