Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Jul 8.
Published in final edited form as: Prev Sci. 2011 Sep;12(3):278–288. doi: 10.1007/s11121-011-0212-y

Affective Antecedents of the Perceived Effectiveness of Antidrug Advertisements: An Analysis of Adolescents’ Momentary and Retrospective Evaluations

Marco C Yzer 1,✉, Kathleen D Vohs 1, Monica Luciana 1, Bruce N Cuthbert 1, Angus W MacDonald III 1
PMCID: PMC3703940  NIHMSID: NIHMS462055  PMID: 21499729

Abstract

Perceived message effectiveness is often used as a diagnostic tool to determine whether a health message is likely to be successful or needs modification before use in an intervention. Yet, published research on the antecedents of perceived effectiveness is scarce and, consequently, little is known about why a message is perceived to be effective or ineffective. The present study’s aim was to identify and test the affective antecedents of perceived effectiveness of antidrug television messages in a sample of 190 adolescents in the 15–19 year age range. Factor-analytical tests of retrospective message evaluation items suggested two dimensions of perceived effectiveness, one that contained items such as convincingness whereas the other contained pleasantness items. Using retrospective data as well as real time valence and arousal ratings, we found that arousal underlies perceived convincingness and valence underlies perceived pleasantness. The results indicated activation of appetitive and defensive motivational systems, which suggests a clear motivational component to the concept of perceived message effectiveness.

Keywords: Perceived message effectiveness, Drug use prevention, Arousal, Valence

By the time they are 18 years old, 47% of all U.S. adolescents will have used illicit substances, such as marijuana, cocaine, or non-prescribed amphetamines at some point in their life (Johnston et al. 2008). Such findings have spurred major antidrug and substance abuse prevention campaigns targeting adolescents and young adults. One prominent example is the National Youth Anti-Drug Media Campaign, which since its initial mandate in 1998 by the U.S. Congress has already received more than $1 billion and an additional $600 million has been earmarked for 2009–2011 (Office of the Law Revision Counsel 2009). Such huge investments are risky, particularly because evaluation research has shown that previous campaign efforts are often ineffective or, counterproductively, can even bring about unfavorable effects on drug use cognitions (Derzon and Lipsey 2002; Hornik et al. 2008).

One approach to try to avoid these potentially costly mistakes is to pretest antidrug messages before launching a campaign. Pretesting involves having people who are in the target audience watch the message, such as a 30-second television anti-drug advertisement, and subsequently asking them how effective they believe the message will be in dissuading drug use. These ratings of perceived effectiveness might reveal that a target audience does not believe that the message will be effective, which allows the interventionist to modify the message in order to avoid null or even counterproductive message effects (Fishbein et al. 2002).

Perceived message effectiveness has been associated with effects of the message on variables such as attitudes and behavioral intentions (e.g., Dillard et al. 2007a, b; Hullett and Boster 2001) which have been recognized as determinants of behavior (e.g., Fishbein et al. 2001). Indeed, perceived message effectiveness has been shown to affect actual behavior through effects on attitudes and other behavioral determinants (e.g., Lavine and Snyder 1996). These findings support the predictive validity of perceived message effectiveness insofar that there is evidence that a message that is perceived as effective can change determinants of drug use and indirectly, through changes in those determinant variables, usage itself.

Note, however, that previous research has assessed perceived message effectiveness with quite disparate measures. Some measures are semantically close to the concept of effectiveness, such as agreement with the statements, “The ad was effective” (Biener 2000) and “Overall, I think this ad is effective for kids my age” (Pechmann et al. 2003). Others have included an efficacy component, such as, “How confident did the ad make you feel about how best to deal with illegal drugs in the real world?” (Fishbein et al. 2002). Still other measures have included a relevance component, such as, “This ad is definitely for people like me” (Stead et al. 2005). An even more common approach is to frame effectiveness in terms of persuasiveness, such as when participants rate the believability or convincingness of a message (Cesario et al. 2004; Dillard and Peck 2000; Fishbein et al. 2002; Mitchell 2000). Collectively these measures have been said to reflect perceived effectiveness but it is clear that they take vastly different approaches. To this point, the dimensionality of perceived message effectiveness measures has not been examined and therefore conclusions about interchangeability of different measures remain unsubstantiated.

Perhaps even more importantly, whereas much research examined effects of perceived message effectiveness on other variables (for review see Dillard et al. 2007a, b), published research that systematically examined antecedents of perceived message effectiveness is scarce. As a result, we do not have a good understanding of the basis of effectiveness judgments of antidrug messages. To inform theorizing in this regard, it is useful to note that antidrug messages often suggest that drug use harms people’s health and well-being. The immediate response to such threat messages is likely to be affective and negative (Beck and Frankel 1981; Fishbein et al. 2002; Lang et al. 2005a, b). For example, Dillard and Peck (2000) showed that watching an antidrug television message induced anger, fear, and surprise, but not happiness or contentment. In addition, measures of negative affect such as anger and fear were positively associated with how persuasive and convincing the antidrug message was perceived to be. These findings indicate emotion as a source of information that directs judgments of stimuli (Bless et al. 1992; Clore et al. 2001). To make an evaluative judgment, people often consider how they feel when exposed to a stimulus and then use those feelings as an indication of their judgment (Schwartz and Clore 1983). Similarly, affective responses to health messages may well be the basis for evaluative judgments of that message (viz., perceived message effectiveness).

But what do these affective responses represent? Research on the dimensionality of emotion supports an interpretation of affective responses as associated with motivational systems. That work describes emotion in terms of two bipolar dimensions involving the pleasantness or unpleasantness of the emotion (valence) and its intensity (arousal: Bradley 2009; Russell 1980, 2003; Russell and Barrett 1999; but see Larsen and Diener 1992). Thus, emotional manifestations (e.g., a feeling, emotion, mood; Russell 2003) can be visualized as a point in a two-dimensional space, where the extent to which the emotional experience is pleasant versus unpleasant (valence dimension) and arousing versus non-arousing (arousal dimension) are the coordinates. For example, the emotion ‘afraid’ is labeled as unpleasant and arousing whereas ‘excited’ is pleasant and arousing; ‘lethargic’ is labeled as unpleasant and non-arousing, and ‘placid’ as pleasant and non-arousing (Russell 1980).

The two-dimensional emotion model suggests the activation of motivational systems, in effect an appetitive and a defensive system (Bradley 2009; Bradley et al. 2001). Generally speaking, an increase of positive stimuli leads to increased activation of the appetitive system and an increase in negative stimuli leads to increased activation of the defensive system. Thus, the valence of an evaluative judgment indicates which motivational system is active, such that a message perceived as pleasant triggers the appetitive system, a message perceived as unpleasant triggers the defensive system, and a message perceived as both pleasant and unpleasant triggers both systems simultaneously (Lang 2006). Arousal indicates the likelihood that the salient motivational system will in fact induce a response. That is, only messages that are perceived as arousing strongly engage the salient motivational system.

Bradley and colleagues (2001) found that the appetitive and defensive systems can be uniquely activated by highly arousing, strongly valenced positive or negative stimuli. They plotted valence and arousal ratings of evocative stimuli (e.g., pictures, words, and sounds) in a two-dimensional arousal-valence affective space, and found a ‘<‘ or boomerang-shaped pattern such that stimuli clustered either on a vector characterized by a positive valence–arousal association (when valence ratings move from neutral to pleasant, arousal ratings move from non-arousing to arousing) or as a vector characterized by a negative valence–arousal association (when valence ratings move from neutral to unpleasant, arousal ratings move from non-arousing to arousing). Importantly, the vector with an upward slope indicates increased positivity ratings and increased arousal and thus increased activation of the appetitive system, whereas the vector with a downward slope indicates increased negativity ratings and increased arousal and thus increased activation of the defensive system (Bradley et al. 2001; Lang 2006). These valence-arousal responses to evocative stimuli covary reliably with biological reflexes that indicate activation of the appetitive or the defensive system (Bradley et al. 2001; Watson 2000). The implication is that emotionally evocative stimuli can be categorized by the motivational system they activate; that is, an appetitive or defense response.

These observations suggest that a sufficient level of induced arousal is a prerequisite for health messages to generate a response. In the current research, we therefore tested an arousal–perceived message effectiveness correlation. Moreover, because a distinctive feature of antidrug messages is that they convey information that is potentially threatening to the message recipient (Dillard and Peck 2000; Lang et al. 2005a, b), we hypothesize that valence–arousal ratings of antidrug messages reflect a defensive motivation vector more than an appetitive motivation vector. If supported, this would be an important finding because whereas the appetitive system induces information seeking (Lang 2006), activation of the defensive system at highly arousing levels elicits responses such as evasion, withdrawal, and attack to attempt threat reduction (Bradley et al. 2001). Such strong responses might lead to less rather than more attention to the information in the antidrug message or even rejection of the message (Lang 2006).

Tests of these ideas should not solely rely on conventional retrospective evaluative measures. Retrospective message evaluations are global judgments informed by recall of different aspects of the message unfolding over time. It is unclear whether these global judgments adequately capture the complexity of health messages and the possibility that different moments of the same message elicit different responses. For example, one part of a hypothetical antidrug advertisement (hereafter ‘ad’) where party-goers are having fun may induce pleasant responses, but a later part of the same ad that presents gruesome pictures of violence apparently triggered by drug use may induce highly arousing, unpleasant responses. A comprehensive analysis of perceived ad effectiveness therefore should consider both real time responses as well as retrospective evaluations. Useful here is work by Kahneman (2000; Fredrickson and Kahneman 1993) that showed that global evaluations of affective experiences can be traced to two pivotal moments. Those moments reflect the most intense sensation felt during the entire experience (the peak) and the sensation felt during the last moments of the experience (the end). Notably, duration of the experience does not affect global evaluations (Fredrickson and Kahneman 1993). These findings led to a rule of how momentary affective experiences integrate into later global evaluations of the entire experience that holds that the peak and the end best predict retrospective global evaluations (e.g., Redelmeier and Kahneman 1996). Relevant for our interest in message evaluation, advertising research has shown that the peak-end rule applies to momentary affect in response to persuasive messages (Baumgartner et al. 1997).

Introduction

The Present Research

To identify the antecedents of perceived effectiveness ratings of antidrug messages we first tested whether measures that have previously been used as perceived message effectiveness measures reflect a single factor or multiple factors. Next, we plotted real time valence and arousal ratings as a function of perceived message effectiveness ratings to test whether antidrug ads that are perceived to be effective are represented by different valence–arousal patterns than ads that are perceived to be ineffective. Last, peak and end values derived from real time valence and arousal ratings were correlated with retrospective perceived message effectiveness ratings. Together, these analyses addressed whether and how affective evaluations contribute to perceived message effectiveness.

In addition, it is conceivable that positive experiences with drug use may have informed a positive stance toward drugs, resulting in a predisposition to negatively evaluate antidrug messages regardless of their content. This would reduce the meaningfulness of affective responses to anti-drug ads as antecedents of perceived message effectiveness. Whereas positive associations between drug use and beliefs about drug use have been well-documented (e.g., Yzer et al. 2004), effects of drug use on evaluations of antidrug ads have not been examined. To be better able to interpret our analyses of an affective basis of perceived message effectiveness, we therefore tested whether past marijuana use is associated with beliefs about marijuana use, and whether pre-existing beliefs determine perceived effectiveness of antidrug ads.

Method

Participants

Participants were recruited from a database maintained by the University of Minnesota’s Institute of Child Development (ICD). The ICD uses public birth records to contact parents in the Twin Cities metro area shortly after each child’s birth. They are asked to return a postcard if they are interested in participating in child development research. Parents are informed that they will learn about the specific research topic when they are contacted by an investigator associated with the ICD, and that they can freely accept or decline any research invitation. Respondents are entered into a database enabling researchers to access lists of participants within specified age ranges. The ICD employs no inclusion or exclusion criteria to build the database.

We used this database to recruit participants aged 15–19 years old by requesting a sampling of potential participants in that age band. Then, using a phone script, we telephoned parents. After making contact, we first spoke with the potential participant’s parents or guardians. We explained that the focus of the research was on how adolescents and young adults process health messages about marijuana and other drugs. With the parent or guardian’s consent, we then explained the research objectives, procedures and reimbursement to the participant. If the participant agreed to participate, we scheduled the research session. Approximately 70% of individuals who were contacted agreed to participate. We have used this recruitment strategy in previous studies of adolescent development (e.g., Luciana et al. 2009).

We recruited 190 participants distributed across the 15–19 year old age range (ns=41, 33, 33, 36 and 47 at each age: M=17 years old, SD=1.49). Except for age, there were no inclusion criteria for this research. A comparison with Minnesota population estimates from 2009 census records (U.S. Census Bureau 2010) showed that our sample is representative in terms of gender (sample vs. census records: %female=58% vs. 50%), and ethnic (%Hispanic=3% vs. 4%) and racial diversity: (%White=88% vs. 88%, %American Indian= 4% vs. 1%, %Asian=2% vs. 4%, and %other=6% vs. 7%). Because parent or child experience with drugs was not an inclusion criterion, and confirmed by data from a 2008 national survey on marijuana use (SAMHSA 2009), the sample is not particularly low or high in risk for drug use. Specifically, our sample’s reports of marijuana use are consistent with national data on this age group (sample vs. national data: %lifetime marijuana use =36% vs. 34%, and %marijuana use in last 30 days=17% vs. 14%).

Procedures and Study Design

Participants came to a laboratory and provided anonymous, non-signed consent or assent and parental consent for participants who were less than 18 years old. Participants first answered questions about demographics, marijuana use attitudes, normative perceptions, self-efficacy, and intention to use. After practicing a moment-by-moment rating tool while watching a 30-second ad for flat screen televisions, participants viewed and rated a set of antidrug ads presented in random order. Last, participants responded to individual differences scales including self-esteem, sensation seeking and externality. These scales were not used for the present analyses. Upon completion of these measures, participants were debriefed and received $25.

The antidrug ads that were used for this study were collected from health, government, and advertising sources. Ads were included in the current experiment if they targeted adolescents or young adults and were of good technical quality. Whereas our measures focused on marijuana use and attitudes, we included ads that targeted marijuana (n=34), non-specified drugs (n=19), methamphetamine (n=16), heroin (n=8), cocaine (n=1), and prescription drugs (n=1), but not alcohol or nicotine. This resulted in a final group of 79 antidrug ads.

To keep the rating task manageable, each of the 79 ads was randomly assigned to one of eight conditions so that each participant viewed and rated about ten ads and did not see any single ad twice. Because it is not feasible to simultaneously rate each moment of an ad on two dimensions, half of the sample rated each antidrug ad on a valence dimension (n=95) and the other half on an arousal dimension (n=95). The study thus used an 8 (Antidrug Ad Set) × 2 (Moment-by-Moment Rating Task: Valence dimension, arousal dimension) between-subjects design. This design was based on pragmatic considerations of avoiding participant fatigue, which meant that we expected no systematic Antidrug Ad Set and Rating Task effects on evaluation ratings. Random assignment to the Antidrug Ad Set and Rating Task conditions was successful; there were no differences among Antidrug Ad Set and Rating Task conditions in terms of gender, age, or lifetime marijuana use.

Measures

Momentary Affective Responses

Participants used a computer mouse to rate each ad on a moment-by-moment basis. A horizontal line at the bottom of the screen gave participants visual feedback of the cursor’s position. The computer logged the average of 10 measurements per second to indicate the cursor position at each second of the particular ad. The positions reflected a 7-point valence or arousal scale, ranging from 0 to 6.

We explained the valence and arousal dimensions using affective states that exemplify the two dimensions (e.g., Russell 1980, 2003). Participants in the valence condition were told about feeling happy versus unhappy to illustrate the task: “You will be asked to describe your feelings along the dimension: happy vs. unhappy. At the right end of the scale you are happy, pleased, satisfied, contented, hopeful. At the left end of the scale is the opposite feeling.” The momentary rating task for participants in the arousal condition described the arousal dimension as: “You will be asked to describe your feelings along the dimension: ‘stirred up’ vs. bored. At the right end of the scale you are stirred up, stimulated, excited, frenzied, jittery, wide-awake, aroused. At the left end of the scale is the opposite feeling.”

Retrospective Evaluations

Participants used nine 7-point items to evaluate the ads they had just viewed. Seven of these tapped global evaluations of the ads. For these items, the phrase stem ‘To me, this ad was…:’ was followed by a scale with the anchors extremely unconvincing—extremely convincing, extremely unbelievable—extremely believable, extremely forgettable—extremely memorable, extremely bad—extremely good, extremely unpleasant—extremely pleasant, extremely negative—extremely positive, and extremely not for someone like me—extremely for someone like me. Two additional items were retrospective versions of the valence and arousal items that participants used to record their momentary ratings. For these two items, the phrase stem ‘This ad made me feel…’ was followed by a scale with anchors bored—stirred up and unhappy—happy.

To assess marijuana use participants were asked whether they had ever, even once, used marijuana. Those who answered ‘yes’ were labeled users and those who answered ‘no’ were labeled non-users. Psychosocial determinants of marijuana use were operationalized in a manner consistent with measurement recommendations (Fishbein et al. 2001) and previous marijuana use research (Yzer et al. 2004). An intention to use marijuana measure was computed by averaging responses to a question about the likelihood of participants using marijuana even once or twice in the next 12 months, and a question about participants’ intention to use marijuana even once or twice in the next 12 months (1 = very unlikely, 7 = very likely; Cronbach’s α=.97). Attitude toward marijuana use was measured with the statement “My using marijuana, even once or twice, in the next 12 months would be,” followed by nine semantic differentials, including extremely bad–extremely good, extremely harmful- extremely beneficial, and extremely pleasant-extremely unpleasant. Scores on the nine items were averaged to yield an indicator of attitude toward marijuana use (α=.95). The injunctive norm concerning marijuana use was assessed by asking participants to indicate on two 7-point scales the extent to which they thought (1) people who are important to them and (2) their close friends would approve or disapprove of their using marijuana, even once or twice, in the next 12 months (1 = strongly disapprove, 7 = strongly approve). These two items held together in an injunctive norm scale, α=.78. To assess descriptive norms we asked; “How many of your friends do you think have used marijuana, even once or twice, in the past 12 months?” and “How many people your age do you think have used marijuana, even once or twice, in the past 12 months?” (1 = almost none, 7 = almost all). These two items held together in a descriptive norm scale, α=.71. A single item assessed self-efficacy: “There can be a variety of obstacles to your using marijuana in the next 12 months. Even in the face of such obstacles, how sure are you that if you really wanted to, you could use marijuana in the next 12 months?” (1 = completely sure I cannot, 7 = completely sure I can).

Results

Scale Development and Characteristics of Retrospective Evaluative Items

Analyses Across Antidrug Ads

The retrospective valence and arousal ratings were treated as individual items to test real time ratings as predictors of retrospective global ratings. We submitted the remaining seven retrospective evaluation items to a maximum likelihood factor analysis with oblique rotation. This analysis indicated two factors. The ‘unpleasant-pleasant’ and ‘negative-positive’ items loaded onto one factor that accounted for 60% of the variance in the original variables: eigenvalue = 3.59, loadings after rotation = .99 (‘unpleasant-pleasant’) and .69 (‘negative-positive’). We labeled this factor Perceived Message Effectiveness—(un)pleasant or PME(un)pleasant. Items that asked about how convincing, believable, memorable, and how good was the ad loaded onto a second factor that accounted for an additional 22% of the variance: eigenvalue = 1.33, loadings after rotation = .97 (‘unconvincing-convincing’), .88 (‘unbelievable-believable’), .78 (‘forgettable-memorable’) and .81 (‘bad-good’). We labeled this second factor Perceived Message Effectiveness—(un) convincing, or PME(un)convincing. Although personal relevance has been used previously as a measure of perceived message effectiveness (Stead et al. 2005), our factor analysis suggested that the item (To me, this ad was… ‘extremely [not] for someone like me’) did not contribute to either of the factors. Consequently, we separated this item from the two overarching factors.

To create the two factors as suggested by the factor analysis, we averaged item scores into the two-item PME(un)pleasant scale (α=.81) and the four-item PME(un)convincing scale (α=.92) across the ads each participant rated. Note that PME(un)pleasant and PME(un)convincing scores reflected ratings of different ads for participants in the eight Antidrug Ad Set conditions. As expected, Antidrug Ad Set condition did not systematically alter PME(un)pleasant and PME(un)convincing, nor did Moment-by-Moment Rating Task condition, nor their interaction (all multivariate and univariate Fs<1.60, ns). These results supported our use of perceived message effectiveness scales across different sets of antidrug ads and rating tasks.

The correlations between the retrospective evaluative measures (Table 1) suggested that judgments about the persuasiveness of an antidrug ad (i.e., ratings of, e.g., the convincingness and memorability captured in PME(un)convincing) were associated with the extent to which the ad induced arousal. It is notable that these items are the most widely use indicators of perceived message effectiveness (e.g., Cesario et al. 2004; Dillard and Peck 2000; Fishbein et al. 2002; Mitchell 2000). The results also pointed to a second type of perceived message effectiveness that clustered items pertaining to an ad’s unpleasant or pleasant nature and its negativity or positivity (PME(un)pleasant). As these items are semantically close to definitions of valence it is perhaps unsurprising but meaningful that these judgments are associated with an ad’s perceived valence. The personal relevance item correlated only moderately with the two types of perceived message effectiveness indicators (i.e., PME(un)pleasant/valence and PME(un)convincing/arousal), suggesting that whereas perceptions of personal relevance are positively associated with perceived message effectiveness, they are not part of effectiveness judgment factors.

Table 1.

Retrospective evaluative measures across antidrug ads: correlations, means and standard deviations

PME(un)convincing Valence Arousal Relevance M SD
PME(un)pleasant .36 .66 .23 .31 3.68 .66
PME(un)convincing .28 .74 .37 4.70 .70
Valence .26 .28 3.54 .60
Arousal .31 4.36 .70
Personal relevance 3.74 1.06
Open in a new tab

All measures reflect 7-pt scales. All correlations p<.001

Analyses for Separate Antidrug Ads

Inspection of the five retrospective evaluative measures for each antidrug ad showed that PME(un)convincing ratings ranged from 3.09 to 6.17, and PME(un)pleasant ratings ranged from 1.91 to 5.52, which indicates considerable variation in judgments of different ads. Moreover, the range of valence ratings and mean valence rating (M=3.54 on a 7-point scale) suggested that the antidrug ads in our sample were more likely to be rated as unpleasant than pleasant. Frequency results confirmed this in that 63% of the 79 ads were rated at or below the valence scale’s midpoint. Only three ads were rated more than one scale point above the midpoint but in contrast 26 ads were rated more than one scale point below the midpoint. It therefore appears that, at least as far as the ads in our pool were concerned, antidrug ads are more likely to be rated as unpleasant than as pleasant.

In further support of the perceived message effectiveness arousal-valence typology, PME(un)convincing ratings for each ad were found to be similar to arousal ratings of the same ad, and PME(un)pleasant ratings were similar to valence ratings. Furthermore, antidrug ads rated as most effective as indicated by PME(un)convincing scores were characterized by high arousal scores but low, that is negative PME(un)pleasant and valence scores. For example, ads rated above the median PME(un)convincing score of 4.60 exhibited strong and negative correlations between valence and arousal, r=−.78, p<.001, and between PME(un)pleasant and PME(un)convincing, r=−.74, p<.001. Antidrug ads rated as less effective, however, did not show such a systematic pattern. For example, ads rated below the median PME(un)convincing score exhibited only moderate (and nonsignificant) valence and arousal correlations, r=.25, ns, whereas PME(un)pleasant and PME(un)convincing were uncorrelated, r=.00. Last, perceived personal relevance of antidrug ads rated as more or less effective were quite similar and at or below the scale midpoint, M=3.96 and M= 3.50 on a 7-pt scale. Although this difference was statistically significant (t(77)=3.94, p<.001), it was not substantial (η2,=.17; see also Table 1) which allows the suggestion that if an ad is perceived as effective, it is not necessarily perceived as effective for oneself.

Figure 1 has a different presentation of valence and arousal results. Mean valence and arousal scores for each ad were used to plot ads in a valence–arousal space. Ads that were rated on or above the valence scale’s midpoint were labeled as pleasant ads and ads that were rated below the midpoint were rated as unpleasant. The plot indicated that for pleasant ads valence was positively associated with arousal, but for unpleasant ads valence was negatively associated with arousal. Using the correlations between valence and arousal in the pleasant and unpleasant antidrug ads to form regression lines (cf. Bradley et al. 2001), we confirmed a positive slope for pleasant antidrug ads, b=.33, and a negative slope for unpleasant ads, b=−.76. These slopes suggest that pleasant and unpleasant ads were associated with distinct processes.

Fig. 1.

Fig. 1

Open in a new tab

Pleasant (open circles) and unpleasant (solid circles) antidrug ads plotted in a valence–arousal space. 1 very negative valence, 4 neutral and 7 very positive valence; 1 low arousal, 7 high arousal

Momentary Affective Responses

We next plotted the momentary valence and arousal responses for each of the 79 ads. The 20 ads that were rated as most effective in terms of PME(un)convincing showed remarkably similar momentary valence and arousal trends: As the ad proceeds in duration, arousal increases and valence becomes more negative. Ads rated as weaker were not associated with systematic arousal and valence trends. Additionally, weaker ads were associated with larger variances of arousal and valence ratings than stronger ads. To illustrate these results, Fig. 2 displays arousal and valence trends averaged across the four strongest and the four weakest antidrug ads.

Fig. 2.

Fig. 2

Open in a new tab

Four strongest antidrug ads (two leftmost panels) and four weakest antidrug ads (two rightmost panels): Average arousal and valence scores and standard error bars per second of the ads

Momentary Affective Responses and Perceived Message Effectiveness

We used momentary ratings to compute five predictors of retrospective message evaluation. Specifically, for each of the valence and arousal momentary data series, we computed the peak (mean of the most extreme responses while viewing each ad), end (mean of the final responses while viewing each ad), the mean (average of all responses to each ad), peak/end (average of the peak and end responses) and linear trend (coefficient of the regression of momentary responses on time; Baumgartner et al. 1997; Kahneman 2000) (Table 2).

Table 2.

Correlations between momentary affective response measures and retrospective evaluative measures

Criterion predictor Retrospective valence Retrospective arousal PME(un)pleasant PME(un)convincing Ma
Linear trend: valence .75 −.45 .77 −.45 −.12
Mean: valence .84 −.54 .81 −.58 3.67
Peak: valence .76 −.37 .76 −.40 4.40
End: valence .90 −.63 .91 −.61 3.57
Peak/end: valence .89 −.57 .89 −.56 3.98
Linear trend: arousal −.38 .67 −.42 .63 .18
Mean: arousal −.43 .81 −.43 .68 4.48
Peak: arousal −.51 .85 −.52 .77 5.02
End: arousal −.42 .81 −.45 .76 4.70
Peak/end: arousal −.47 .84 −.49 .78 4.86
Open in a new tab
a

Means for linear trends are average regression coefficients (momentary responses regressed on time, indicated by ad duration in terms of seconds). For linear trends a negative mean indicates a downward slope and a positive mean indicates an upward slope. All other means reflect a 1–7 scale. All correlations p<.001

The means indicate that arousal levels are above the scale midpoint and valence levels below the scale midpoint. The linear trends similarly show a positive trend for arousal scores and a negative trend for valence scores, indicating that on average when an ad advanced, participants rated what they viewed as increasingly arousing but also as increasingly unpleasant.

Overall it appeared that global, retrospective ratings were a good representation of momentary ratings: Retrospective valence ratings were primarily a function of the end value and peak/end average of the momentary valence ratings, whereas retrospective arousal ratings were primarily a function of the peak value and peak/end average of the momentary arousal ratings. Consistent with a two-dimensional typology of perceived message effectiveness, PME(un)pleasant correlated strongly with momentary valence ratings, or more specifically, with end and peak/end average valence values. PME(un)convincing correlated most strongly with momentary arousal ratings captured as peak, end, and peak/end average values.

Ancillary Analyses: The Role of Pre-existing Beliefs

We compared ratings from adolescents who currently use or have used marijuana (users) with those who have never used (non-users) to test the assumption that users hold more positive beliefs towards marijuana use than non-users. We then tested whether users evaluated antidrug ads more negatively than non-users. Last, we tested the association of marijuana beliefs assessed before message exposure with perceived message effectiveness.

Attitudes, descriptive and injunctive norms, self-efficacy and intention were used as indicators of how participants think about marijuana. Because these variables correlated with each other (.45<r<.86), we submitted them to a multivariate analysis of variance, using usage status (users, non-users) as the independent variable. The multivariate effect (F(5, 184)=72.99, η2=.67) and all univariate effects were large and significant at p<.001. Compared with non-users, users reported a more positive attitude towards marijuana (M=4.53 vs. M=2.17; F(1, 188)=253.18, η2 = .57), more pro-marijuana descriptive norms (M=5.21 vs. M=3.67; F(1, 188)=61.83, η2=.25), more pro-marijuana injunctive norms (M=3.91 vs. M=1.94; F(1, 188)=135.40, η2=.42), higher self-efficacy over ability to use (M=6.35 vs. M=4.45; F(1, 188)=37.80, η2=.17), and a stronger intention to use marijuana (M=5.20 vs. M=1.42; F(1, 188)= 332.12, η2=.64). Users clearly were more positive towards marijuana use than non-users.

To test whether users also rated antidrug ads more negatively than non-users, we used a multivariate analysis of variance with the five retrospective evaluative measures as dependent variables.1 The multivariate effect was not large but significant, F(4, 185)=2.55, p<.05, η2=.07. The univariate effects showed usage status effects on PME(un)pleasant, F(1, 188)=3.85, p<.05, η2=.02, and PME(un)convincing, F(1, 188)=10.68, p<.001, η2=.05, but not on valence, F(1, 188)=1.00, ns, η2=.01, arousal, F(1, 188)= 2.56, ns, η2=.01, or personal relevance, F(1, 188)=1.43, ns, η2=.01. Non-users’ PME(un)pleasant and PME(un)convincing ratings of antidrug ads were more positive (M=3.75 and M=4.82) than users’ ratings (M=3.56 and M=4.49). Hence it appeared that experience with drug use affected judgments of antidrug ads; this effect was most prominent for PME(un)convincing and did not pertain to all evaluative judgments.

Last, we tested whether pre-existing marijuana beliefs predicted subsequent evaluations of antidrug ads. We regressed each of the five retrospective evaluative measures on attitude, descriptive and injunctive norms, self-efficacy and intention to use marijuana. As a set, the psychosocial variables did not explain retrospective message evaluation. The entire set of predictor variables explained only 6% of the variance in PME(un)convincing, 3% of the variance in arousal, 3% of the variance in PME(un)pleasant, 1% of the variance in valence, and none of the variance in personal relevance. The mean of the regression coefficients was 0.04. These results suggested that perceived message effectiveness judgments of antidrug ads were not simply proxies of pre-existing stances towards marijuana but rather reflected actual responses to the ad.

Discussion

Prevention scholars have argued for the usefulness of the concept of perceived message effectiveness (e.g., Dillard et al. 2007a, b; Fishbein et al. 2002). Effectiveness assessments are used to indicate whether a message will be successful or needs modification to avoid adverse effects. Because of a paucity of systematic inquiry into the antecedents of perceived effectiveness ratings, however, not much is known about why a message is rated as effective or ineffective, and thus the literature has little to offer the health message designer who seeks to modify a message rated as ineffective. It is therefore imperative for the field to advance understanding of the concept of perceived message effectiveness. The current study contributes to that understanding. We tested affective antecedents of the perceived effectiveness of antidrug messages and found that both valence and arousal responses to antidrug messages underlie ratings of perceived effectiveness. These two vectors emerged perhaps because valence and arousal responses indicate activation of appetitive and defensive motivational systems, which lends motivational components to the concept of perceived message effectiveness.

Previous research has used a variety of assessments, but perceived effectiveness has been treated as a unidimensional concept nonetheless. In contrast, the present results indicated two types of perceived message effectiveness. A first type included retrospective evaluation measures that reflect perceptions of a message’s pleasantness or unpleasantness. This form of perceived message effectiveness correlated strongly with a retrospective affective valence measure and was predicted by moment-by-moment valence ratings obtained while participants viewed antidrug messages. This first type of perceived message effectiveness therefore reflects in part affective valence, or how unpleasant or pleasant the message is. A second type included retrospective evaluation measures that tap a message’s persuasiveness, and is best interpreted in terms of arousal, or the intensity of the perceived valence. This form correlated strongly with a retrospective arousal measure and was predicted by moment-by-moment arousal ratings.

Our finding of two types of perceived message effectiveness, each with its own affective antecedents, is important. Meta-analytical research has shown that most perceived effectiveness work has relied on measures that gauge the persuasiveness of a message, for example, by asking how convincing or believable the message is (Dillard et al. 2007a, b). In the current research, such items clustered together on a perceived effectiveness factor that was predicted by arousal ratings. Should we then conclude that for antidrug messages to be effective, they need to induce strong arousal? A consideration of the second perceived effectiveness type that we found, which was predicted by affective valence, shows that this would be a premature conclusion. The antidrug ads that were rated as most effective in terms of convincingness and believability were indeed associated with high arousal, as indicated by strong, positive moment-by-moment arousal trends. They were also associated, however, with negative valence, as indicated by strong, negative moment-by-moment valence trends. In other words, antidrug ads that were rated as most convincing were those that aroused viewers and induced negative affect in them. This is important, because increased negative affect increases activation of the defensive system (Lang 2006), particularly when arousal also increases (Bradley 2009). Antidrug ads in our research that were rated as convincing therefore can be argued to activate the defensive system.

Activation of the defensive system has implications. Antidrug messages that warn about bad things that can happen from using drugs indicate a threat to the viewer’s well-being, particularly when the message is not very clear on how to avoid bad outcomes (Lang et al. 2005a, b; Witte 1992). Perception of a threat activates the system responsive to threats, that is, the defensive system (Lang 2006). We propose that in response to an antidrug message the defensive system monitors the degree of threat elicited by the message (see also Witte 1992). Moreover, when in addition the message is highly arousing, memory for the message is hampered, because monitoring threat levels and processing an otherwise arousing message consumes more information processing resources than are typically available (Lang et al. 2005a, b; Langleben et al. 2009). The implication is that an antidrug message’s ability to evoke an affective response may be remembered, but why and how the viewer should avoid drugs—the information that the ad was meant to convey—may be lost (cf. Lang 2000).

Because antidrug ads in our pool were more likely to be rated as unpleasant than as pleasant, our data cannot conclusively address whether ads that are arousing but pleasant produce more favorable effects than ads that are arousing and unpleasant. Research on other media formats is informative, however. Poels and Dewitte (2008), for example, examined print ads for consumer products such as jeans and perfume, and found a positive association between arousal and valence that was associated with activation of the appetitive system. Bolls et al. (2001) examined positively and negatively valenced radio ads for, among others, consumer products, TV shows, and health organizations. Bolls and colleagues found that memory for arousing, positive ads was better than for negative ads. These findings suggest that ads that are both pleasant and arousing produce more favorable effects than ads that are unpleasant and arousing. They also support the applicability of our findings to other media formats, although to our knowledge message arousal and valence have not been examined for media other than pictures (Bradley et al. 2001), print ads (Poels and Dewitte 2008), radio ads (Bolls et al. 2001) and video ads (the present research).

The implications of the present findings for the development of antidrug messages are threefold. First, structural message features such as pacing can be used to induce arousal necessary for attention. In addition, the overall tone of the message should be positive to ensure that an appetitive system is activated, and not (just) a defensive system (Lang 2006). This can be achieved, for example, by modeling effective ways to avoid drugs, and by emphasizing the positive consequences of not using drugs. Second, when pretesting antidrug messages before implementation, perceived effectiveness instruments should include both “convincingness” measures and “pleasantness” items. Reliance on “convincingness” items only carries the risk that the interventionist chooses messages that are seen as convincing because they are arousing, but activate avoidance responses because they are unpleasant. Importantly, note that these two recommendations pertain to perceptions of how effective an antidrug message is. Thirdly, then, to corroborate these contentions, research should also test whether arousal and valence associated with antidrug ads, through effects on perceived message effectiveness, produce changes in known determinants of drug use, such as knowledge, attitude and intention.

Limitations

The present findings add to our understanding of valence and arousal as affective antecedents of two types of antidrug message effectiveness perceptions. We found that antidrug ads that are arousing yet unpleasant are perceived as most convincing. We drew on the emotion and information processing literatures to theorize that these ads are also likely to activate a defensive motivational system, which may lead to weaker antidrug effects than intended. We note, however, that these claims are restricted to the 79 antidrug ads that were available to us. Similarly, the representativeness of our sample of the Minnesota 15–19 year old population is a mixed blessing. Although desirable from a sampling perspective, the fact that both our sample and the population we sampled from are mainly white means that we cannot address the possibility that different ethnic and racial groups respond differently to antidrug ads.

Note also that our ideas are based on interpretations of affect defined by two bipolar valence and arousal dimensions. However, it is still debated whether affective valence is better understood as bipolar (a single dimension ranging from unpleasant to pleasant) or two unipolar dimensions (a dimension ranging from no unpleasant affect to strong unpleasant affect, and a dimension ranging from no positive affect to strong positive affect) (see Clore and Schnall 2005; Schimmack and Crites 2005). The advantage of the latter interpretation is that it allows for mixed feelings, which is to say finding the same situation as both unpleasant and pleasant. Lastly, the appetitive and defensive systems could be activated simultaneously (Lang 2006; Schimmack and Crites 2005). Although the primary findings of our research should not be meaningfully affected by this issue, future research should measure valence using separate dimensions, since it is theoretically possible for a stimulus to be rated as both positive and negative.

In Closing

Antidrug messages are a tool that, if effective, could turn adolescents away from illicit substances. To be effective, though, scientists must know what makes an ad effective or ineffective. The current research offers ideas to that end. We found that valence (positivity or negativity) and arousal (intensity) independently and in combination determine retrospective perceived effectiveness ratings. In our sample, ads that elicited high arousal and negativity were the most convincing to our adolescent viewers. These data could inform antidrug message development, with the ultimate aim to promote healthy choices among adolescents.

Acknowledgments

Preparation of this article was supported by Grant 1R21DA024430-01 from the National Institute on Drug Abuse. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse or the National Institutes of Health.

Footnotes

1

Prior experiences with marijuana, an illicit substance, conceivably affect judgments not only of ads that target marijuana but ads that target other illicit substances as well. A multivariate analysis of variance confirmed that at least in our data prior marijuana use (users, non-users) and the ad’s target drug (marijuana, other) did not interact to explain ad evaluations (η2 ≤ .003). However, the generalizability of this result is limited to our sample and pool of ads, and the possibility remains that experiences unique to marijuana use can produce judgments of anti-marijuana ads that do not generalize to ads that target other substances.

References

  1. Baumgartner H, Sujan M, Padgett D. Patterns of affective reactions to advertisements. The integration of moment-to-moment responses into overall judgments. Journal of Marketing Research. 1997;34:219–232. [Google Scholar]
  2. Beck KH, Frankel A. A conceptualization of threat communications and preventive health behavior. Social Psychology Quarterly. 1981;44:204–217. [PubMed] [Google Scholar]
  3. Biener L. Adult and youth responses to the Massachusetts anti-tobacco television campaign. Journal of Public Health Management and Practice. 2000;6:40–44. doi: 10.1097/00124784-200006030-00007. [DOI] [PubMed] [Google Scholar]
  4. Bless H, Mackie DM, Schwarz N. Mood effects on encoding and judgmental processes in persuasion. Journal of Personality and Social Psychology. 1992;63:585–595. doi: 10.1037//0022-3514.63.4.585. [DOI] [PubMed] [Google Scholar]
  5. Bolls PD, Lang A, Potter RF. The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements. Communication Research. 2001;28:627–651. [Google Scholar]
  6. Bradley MM. Natural selective attention: Orienting and emotion. Psychophysiology. 2009;46:1–11. doi: 10.1111/j.1469-8986.2008.00702.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bradley MM, Codispoti M, Cuthbert BN, Lang PJ. Emotion and motivation I: Defensive and appetitive reactions in picture processing. Emotion. 2001;1:276–298. [PubMed] [Google Scholar]
  8. Cesario J, Grant H, Higgins ET. Regulatory fit and persuasion: Transfer from “feeling right. Journal of Personality and Social Psychology. 2004;86:388–404. doi: 10.1037/0022-3514.86.3.388. [DOI] [PubMed] [Google Scholar]
  9. Clore GL, Schnall S. The influence of affect on attitude. In: Albarracin D, Johnson BT, Zanna MP, editors. The handbook of attitudes. Mahwah, NJ: Lawrence Erlbaum; 2005. pp. 437–489. [Google Scholar]
  10. Clore GL, Gasper K, Garvin E. Affect as information. In: Forgas JP, editor. Handbook of affect and social cognition. Mahwah, NJ: Lawrence Erlbaum; 2001. pp. 121–144. [Google Scholar]
  11. Derzon JH, Lipsey MW. A meta-analysis of the effectiveness of mass-communication for changing substance-use knowledge, attitudes and behavior. In: Crano WD, Burgoon M, editors. Mass media and drug prevention: Classic and contemporary theories and research. Mahwah, NJ: Lawrence Erlbaum; 2002. pp. 231–258. [Google Scholar]
  12. Dillard JP, Peck E. Affect and persuasion. Emotional responses to public service announcements. Communication Research. 2000;27:461–495. [Google Scholar]
  13. Dillard JP, Shen L, Vail RG. Does perceived message effectiveness cause persuasion or vice versa? 17 consistent answers. Human Communication Research. 2007a;33:467–488. [Google Scholar]
  14. Dillard JP, Weber K, Vail RG. The relationship between the perceived and actual effectiveness of persuasive messages: A meta-analysis with implications for formative campaign research. The Journal of Communication. 2007b;57:613–631. [Google Scholar]
  15. Fishbein M, Triandis HC, Kanfer FH, Becker MH, Middlestadt SE, Eichler A. Factors influencing behavior and behavior change. In: Baum A, Revenson TR, Singer JE, editors. Handbook of health psychology. Mahwah, NJ: Lawrence Erlbaum; 2001. pp. 3–17. [Google Scholar]
  16. Fishbein M, Hall-Jamieson K, Zimmer E, von Haeften I, Nabi R. Avoiding the boomerang: Testing the relative effectiveness of antidrug public service announcements before a national campaign. American Journal of Public Health. 2002;92:238–245. doi: 10.2105/ajph.92.2.238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Fredrickson BL, Kahneman D. Duration neglect in retrospective evaluations of affective episodes. Journal of Personality and Social Psychology. 1993;65:45–55. doi: 10.1037//0022-3514.65.1.45. [DOI] [PubMed] [Google Scholar]
  18. Hornik R, Jacobsohn L, Orwin R, Piesse A, Kalton G. Effects of the national youth anti-drug media campaign on youths. American Journal of Public Health. 2008;98:2229–2236. doi: 10.2105/AJPH.2007.125849. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hullett CR, Boster FJ. Matching messages to the values underlying value-expressive and social-adjustive attitudes: Reconciling an old theory with a contemporary measurement approach. Communication Monographs. 2001;68:133–153. [Google Scholar]
  20. Johnston LD, O’Malley PM, Bachman JG, Schulenberg JE. NIH Publication No 08–6418. Bethesda, MD: National Institute on Drug Abuse; 2008. Monitoring the Future national results on adolescent drug use: Overview of key Findings, 2007. [Google Scholar]
  21. Kahneman D. Evaluation by moments: Past and future. In: Kahneman D, Tversky A, editors. Choices, values, and frames. New York: Cambridge University Press; 2000. pp. 693–708. [Google Scholar]
  22. Lang A. The limited capacity model of mediated message processing. The Journal of Communication. 2000;50:46–70. [Google Scholar]
  23. Lang A. Using the limited capacity model of motivated mediated message processing to design effective cancer communication messages. The Journal of Communication. 2006;56:S57–S80. [Google Scholar]
  24. Lang A, Chung Y, Lee S, Schwartz N, Shin M. It’s an arousing, fast-paced kind of world: The effects of age and sensation seeking on the information processing of substance-abuse PSAs. Media Psychology. 2005a;7:421–454. [Google Scholar]
  25. Lang A, Chung Y, Lee S, Zhao X. It’s the product: Do risky products compel attention and elicit arousal in media users? Health Communication. 2005b;17:283–300. doi: 10.1207/s15327027hc1703_5. [DOI] [PubMed] [Google Scholar]
  26. Langleben DD, Loughead JW, Ruparel K, Hakun JG, Busch-Winokur S, Holloway MB, et al. Reduced prefrontal and temporal processing and recall of high “sensation value” ads. Neuroimage. 2009;46:219–225. doi: 10.1016/j.neuroimage.2008.12.062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Larsen RJ, Diener E. Promises and problems with the circumplex model of emotion. In: Clark MS, editor. Review of personality and social psychology: Emotion. Vol. 13. Newbury Park, CA: Sage; 1992. pp. 25–59. [Google Scholar]
  28. Lavine H, Snyder M. Cognitive processing and the functional matching effect in persuasion: The mediating role of subjective perceptions of message quality. Journal of Experimental Social Psychology. 1996;32:580–604. doi: 10.1006/jesp.1996.0026. [DOI] [PubMed] [Google Scholar]
  29. Luciana M, Collins PF, Olson EA, Schissel AM. Tower of London performance in healthy adolescents: The development of planning skills and associations with self-reported inattention and impulsivity. Developmental Neuropsychology. 2009;34:461–475. doi: 10.1080/87565640902964540. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Mitchell MM. Able but not motivated? The relative effects of happy and sad moods on persuasive message processing. Communication Monographs. 2000;67:215–226. [Google Scholar]
  31. Office of the Law Revision Counsel. 21 USC Chapter 22—National Drug Control Policy. 2009 Retrieved October 12, 2009, from http://uscode.house.gov/download/pls/21C22.txt.
  32. Pechmann C, Zhao G, Goldberg ME, Reibling ET. What to convey in antismoking advertisements for adolescents: The use of protection motivation theory to identify effective message themes. Journal of Marketing. 2003;67:1–18. [Google Scholar]
  33. Poels K, Dewitte S. Getting a line on print ads. Journal of Advertising. 2008;37:63–74. [Google Scholar]
  34. Redelmeier DA, Kahneman D. Patients’ memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain. 1996;116:3–8. doi: 10.1016/0304-3959(96)02994-6. [DOI] [PubMed] [Google Scholar]
  35. Russell JA. A circumplex model of affect. Journal of Personality and Social Psychology. 1980;39:1161–1178. doi: 10.1037//0022-3514.79.2.286. [DOI] [PubMed] [Google Scholar]
  36. Russell JA. Core affect and the psychological construction of emotion. Psychological Review. 2003;110:145–172. doi: 10.1037/0033-295x.110.1.145. [DOI] [PubMed] [Google Scholar]
  37. Russell JA, Barrett LF. Core affect, prototypical emotional epdisodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology. 1999;76:805–819. doi: 10.1037//0022-3514.76.5.805. [DOI] [PubMed] [Google Scholar]
  38. Schimmack U, Crites SL. The structure of affect. In: Albarracin D, Johnson BT, Zanna MP, editors. The handbook of attitudes. Mahwah, NJ: Lawrence Erlbaum; 2005. pp. 397–435. [Google Scholar]
  39. Schwartz N, Clore GL. Mood, misattribution, and judgments of well-being: information and directive functions of affective states. Journal of Personality and Social Psychology. 1983;45:513–523. [Google Scholar]
  40. Stead M, Tagg S, MacKintosh AM, Eadie DR. Development and evaluation of a mass media theory of planned behaviour intervention to reduce speeding. Health Education Research. 2005;20:36–50. doi: 10.1093/her/cyg093. [DOI] [PubMed] [Google Scholar]
  41. Substance Abuse and Mental Health Services Administration. National survey on drug use and health, 2008. 2009 [Data file]. Retrieved from www.icpsr.umich.edu/icpsrweb/SAMHDA/studies/26701.
  42. U.S. Census Bureau. American Community Survey, 2009 Summary Tables. 2010 Dec 12;2010 Generated by Marco Yzer; using American FactFinder; < http://factfinder.census.gov>. [Google Scholar]
  43. Watson D. Mood and temperament. New York: Guilford; 2000. [Google Scholar]
  44. Witte K. Putting the fear back into fear appeals: The extended parallel process model. Communication Monographs. 1992;59:329–349. [Google Scholar]
  45. Yzer MC, Cappella JN, Fishbein M, Hornik R, Sayeed S, Ahern RK. The role of distal variables in behavior change: Effects of adolescents’ risk for marijuana use on intention to use marijuana. Journal of Applied Social Psychology. 2004;34:1229–1250. doi: 10.1111/j.1559-1816.2004.tb02005.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES