Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jun 1.
Published in final edited form as: J Commun. 2016 May 13;66(3):433–453. doi: 10.1111/jcom.12227

Predictive Validity of an Empirical Approach for Selecting Promising Message Topics: A Randomized-Controlled Study

Stella Juhyun Lee 1, Emily Brennan 2, Laura Anne Gibson 1, Andy S L Tan 3, Ani Kybert-Momjian 1, Jiaying Liu 1, Robert Hornik 1
PMCID: PMC5113814  NIHMSID: NIHMS797430  PMID: 27867218

Abstract

Several message topic selection approaches propose that messages based on beliefs pretested and found to be more strongly associated with intentions will be more effective in changing population intentions and behaviors when used in a campaign. This study aimed to validate the underlying causal assumption of these approaches which rely on cross-sectional belief–intention associations. We experimentally tested whether messages addressing promising themes as identified by the above criterion were more persuasive than messages addressing less promising themes. Contrary to expectations, all messages increased intentions. Interestingly, mediation analyses showed that while messages deemed promising affected intentions through changes in targeted promising beliefs, messages deemed less promising also achieved persuasion by influencing nontargeted promising beliefs. Implications for message topic selection are discussed.

Keywords: Message Effects, Message Topics, Campaign Design, Experimental Methods, Predictive Validity

Crafting a persuasive message is central to any attempt to change attitudes and intentions through communication. Messages can be decomposed into topic (i.e., what the message is about), structural features (i.e., strategies and physical message dimensions), and content (i.e., message appeals) (Cappella, 2006). Although communication research has extensively theorized and tested the effectiveness of different message structures and content features such as narratives, emotional appeals, and frames (Cappella, 2006), less attention has been paid to the strategies for choosing persuasive message topics.

Nonetheless, some previous studies have proposed approaches for selecting persuasive message topics via assessing the association between beliefs and intentions (or behavior) (Atkin & Freimuth, 2001; Fishbein & Cappella, 2006; Fishbein & Yzer, 2003; von Haeften, Fishbein, Kasprzyk, & Montano, 2001). The rationale is that a message addressing a belief that is strongly associated with intentions will cause more intention (behavior) change than a message addressing a belief weakly associated with intentions. This logic is found primarily in formative research for health campaigns (e.g., Boudewyns & Paquin, 2011; Dillard, 2011; Hornik & Woolf, 1999; Parvanta et al., 2013), but also in the political communication domain where it has been applied to identify potential message topics for eliciting support for public policies (e.g., Haider-Markel & Joslyn, 2008; Krosnick, Holbrook, Lowe, & Visser, 2006; Niederdeppe, Porticella, & Shapiro, 2012). However, a consistent caveat regarding these approaches is that they rely on associations calculated from cross-sectional surveys to identify promising (i.e., persuasive) message topics. These assessments are open to challenge, as the question remains as to whether these approaches actually do predict the promise of message topics when realized in actual messages. This study aims to fill this gap in the literature by experimentally testing this core assumption. Specifically, we use the Hornik and Woolf (1999) method (hereafter, H&W) as a guide to test the common core logic of these message topic selection approaches.

Choosing promising topics for persuasive messages

Cappella (2006) argues that a message is composed of a topic, structural features, and content. A message topic refers to what the message is about. Message structural features include strategies for presenting persuasive appeals (e.g., narratives, tailoring) as well as the physical dimensions of the messages (e.g., sensation value, loudness), while content refers to the specific persuasive appeals for delivering a topic (e.g., emotional appeals). As noted above, research on message topic selection is much less common than research focused on structural features or content. For example, a 10-year review of health mass media campaign research found that many studies had been devoted to implementing and testing structural and content features such as drama, fast pace, humor, sensation, and storylines. However, the studies included did not deal with what an optimal message should be about, or how one should go about choosing persuasive message topics (Noar, 2006). Similarly, in a meta-analysis of message tactics utilized in health communication, most of the studies included in the analysis examined tactics regarding message structure (e.g., gain/loss frames, source credibility, communicator gender) (Keller & Lehmann, 2008).

Despite the lesser focus on message topic selection, some strategies to choose promising message topics have been proposed. These strategies can be broadly categorized into qualitative approaches such as focus group and qualitative interviews, and quantitative approaches that utilize survey data. Focus group interviews gather participants to ask them directly about their thoughts on the behavior the persuasive message is aiming to target, and garner potential message topics from the insights shared by participants (Botta, Dunker, Fenson-Hood, Maltarich, & McDonald, 2008; Friedman & Shepeard, 2007). Quantitative methods can elicit potential message topics via surveys, for example, by asking respondents to indicate what they view as the benefits and costs of a behavior, and then selecting as message topics the benefits or costs cited most frequently (e.g., Bai, Middlestadt, Joanne Peng, & Fly, 2009; Middlestadt, 2012). Some quantitative methods go a step further. These methods—including the H&W approach—move beyond describing belief frequencies to also calculate the association between cognitions about a behavior (e.g., attitudinal, normative, self-efficacy beliefs) and behavioral intentions (or actual behavior), and then use the magnitude of associations as the means to distinguish promising and unpromising beliefs to target as message topics (e.g., Boudewyns & Paquin, 2011; Brennan, Gibson, Liu, & Hornik, 2013a; Dillard, 2011; Hornik & Woolf, 1999; Kenski, Appleyard, Von Haeften, Kasprzyk, & Fishbein, 2001; Maddock, Silbanuz, & Reger-Nash, 2008; Parvanta et al., 2013). For example, if a certain behavioral belief regarding fruit and vegetable consumption (e.g., “My eating fruits and vegetables will help me avoid colon cancer”) has a strong association with intentions to consume fruits and vegetables, while another belief (e.g., “My eating fruits and vegetables will help me lose weight”) is minimally associated with intentions, the first belief would be considered more promising as a potential message topic. Effectively, this analysis makes the inference that people who hold an intention to engage in the target behavior (or, actually engage in the behavior in cases where behavioral status is used as the outcome) do so (in part) because they hold this behavioral belief. Also, it assumes that people who are persuaded to endorse this belief as the result of a campaign message would then increase their intentions to consume fruits and vegetables.1

The demand for an experimental test

Despite the practical benefit of quantitative methods such as the H&W approach, one concern with these methods is their dependence on associations drawn from cross-sectional survey data. They assume that effective messages targeting promising beliefs will cause more intention change (or behavior change) than messages targeting less promising beliefs. However, cross-sectional surveys that measure beliefs (i.e., potential message topics) and intention (or behavior) at the same time do not provide evidence that belief change precedes intention change. In addition, there may be other potential confounding variables whose influence on both beliefs and intention/behavior account for their association. Or even if a casual inference can be made, there may be reverse causation, whereby intentions or behaviors shape beliefs, or mutual causation, whereby there is a bidirectional or reciprocal relationship between beliefs, intentions, and behaviors (Huebner, Neilands, Rebchook, & Kegeles, 2011; Weinstein, 2007).

In applied settings, message producers or campaign planners may have no choice but to rely on approaches that are informed by cross-sectional associations between beliefs and intentions; more elaborate approaches, whether involving longitudinal data collection or experimental tests of each potential message, are unrealistic due to limited time and resources. However, if the logic underpinning these approaches can be experimentally validated in a specific case, confidence in applying these approaches more generally can be increased. Therefore, we test this logic with an experimental design that reduces the major threats to causal claims. We focus on the H&W method in particular, as it shares the core logic with other topic selection approaches, but also provides two additional criteria for choosing promising beliefs beyond the strength of belief–intention associations.

The H&W approach

The H&W approach (1999) is a quantitative approach that utilizes cross-sectional survey data to identify promising beliefs to target as message topics. It begins by specifying a target behavior and then defines a range of candidate beliefs grounded either in the available literature about determinants of that behavior or in behavior change theory, often the reasoned action approach of Fishbein and Ajzen (2010).

A belief is then considered to be a promising message topic if it (a) has a highly positive association with behavioral intentions,2 (b) is not already endorsed by most of the target population, and (c) is assessed to be reasonably open to influence by a message. The first criterion is the primary criterion, which is shared with other similar approaches. The latter two criteria provide additional information regarding the relative promise of different beliefs. Assuming that belief–intention associations are similar across beliefs, messages addressing beliefs not endorsed by the majority of the target population (criterion #2) will produce more intention change in the population than will messages addressing beliefs already endorsed by the majority, simply because in the former situation there are more people available to be affected by the campaign message. Similarly, if belief–intention associations and endorsement levels are similar across beliefs, then a belief that has the potential of being changed with a message will elicit more intention change than an unmalleable belief (criterion #3). While this judgment about potential malleability may be based on evidence of persuasion in prior campaigns, it may also reflect a subjective judgment by experienced message designers and program leaders about whether it is possible to construct a persuasive message targeting the specific belief. For example, one potential limiting factor on belief malleability is real-world experience: If a message is contradictory to the real-world experiences of the target audience, then it may be particularly dificult to create a message persuasive enough to achieve belief change.

The H&W approach has been used a number of times to inform the selection of topics (i.e., beliefs, or sets of beliefs) to target in persuasive communications. For example, the method was most recently used to inform the U.S. Food and Drug Administration (FDA) of potentially promising themes (sets of beliefs) for its new antismoking mass media campaign, The Real Cost (Brennan et al., 2013a). It has also been used in the development of campaigns targeting behaviors such as use of help when trying to quit smoking (Parvanta et al., 2013), support for obesity-related social policies (Niederdeppe et al., 2012), and healthy sleep behaviors (Robbins & Niederdeppe, 2015).

Hypotheses

According to the H&W criteria and other similar approaches, a candidate belief is promising for selection as a message topic if it has positive belief–intention and/or belief–behavior associations, because beliefs are assumed to causally precede intentions and behaviors. In order to strengthen our confidence in the causal claim underpinning these approaches, we need to demonstrate that message topics deemed as promising based on formative research applying the above criterion produce intention or behavior change, while topics deemed as less promising do not (or at least, do so to a lesser extent).

This claim is tested in the specific context of constructing smoking prevention messages for a young adult nonsmoker population. We first hypothesize that exposure to smoking prevention messages addressing topics deemed as promising under the first H&W criterion (criterion #1) will increase intentions not to smoke to a greater extent than will exposure to messages addressing less promising topics (H1a). Assuming support for the first hypothesis, we then further hypothesize that the relationship between exposure to messages addressing promising topics and intentions will be mediated by increased endorsement of the targeted beliefs (H1b).

Method

It is important to note that this study used sets of beliefs rather than individual beliefs, and that these sets of beliefs represent potential campaign themes.3 Traditionally, the H&W approach has been used to compare the relative promise of individual beliefs (i.e., “If I quit smoking, I would set a good example for others such as children” compared with “If I quit smoking, I would have more energy to do the activities that I enjoy” [Parvanta et al., 2013]). However, it is also possible to use the method to compare the relative promise of sets of beliefs that represent a common topic domain, and which more closely align with the notion of an overall campaign theme (Brennan et al., 2013a). For example, one of the sets of beliefs (i.e., themes) used in our analyses was the positive mood effects theme, which consisted of five individual beliefs about the positive effects of smoking on mood such as “If I smoke, I will feel more comfortable in social situations” and “If I smoke, I will feel relaxed.” Rather than comparing the relative promise of the five individual beliefs, analyses compared the promise of the set of beliefs to other sets of beliefs. This methodological choice was made for several reasons. Measuring belief endorsement using multiple belief items (i.e., those belonging to the same set of beliefs) should yield more reliable estimates for effects. In addition, using sets of beliefs allowed exposing respondents to multiple messages that each addressed a common theme with each message based on a different belief item. This step enabled maximizing a sense of repeated exposure to a theme without merely showing the same message multiple times. This is often hard to achieve in one-time, short-term experiments.

The logic of the experimental approach was to expose respondents to messages that address topics (i.e., themes) that are deemed as being more or less promising based on formative research, then measure the resulting intentions and beliefs. Implementing this experimental design required two preliminary steps: (1) secondary analysis of previously collected data to identify more and less promising themes (sets of beliefs); and (2) pilot testing of candidate messages to make sure they influenced target beliefs regardless of their level of promise to affect intentions. These steps also ensured that in the main study, the other H&W criteria (criteria #2 and #3) were effectively held constant. Specifically, the secondary analysis was used to select more and less promising themes that were all about equally endorsed by the population (H&W criterion #2), such that they all had a similarly sized population able to be influenced by the campaign message. Pilot tests were needed to ensure that the messages created for the study were effective in influencing beliefs, and to an approximately equal extent across all themes (H&W criterion #3). That way, any observed differences in the influence of messages addressing themes on intentions could be attributed to the targeted theme’s association (strong vs. weak) with intention (H&W criterion #1), and not to the message’s ability (or not) to change beliefs.

Preliminary step 1: secondary analysis of survey data to choose more and less promising themes

Five promising and five less promising themes, from among 21 candidate themes, were identified through secondary analyses of survey data from 3,033 young adults (18- to 25-year-olds) collected in April 2013 (Brennan, Gibson, Liu, & Hornik, 2013b). The sample was drawn from the opt-in online panel maintained by Survey Sampling International (SSI) and consisted of 37% never smokers, 29% not current smokers, 17% not daily smokers, and 17% daily smokers. The samples used in preliminary steps 1 and 2 and in the main study were all drawn through parallel procedures, and are comparable demographically. The survey questionnaire measured smoking behavior and intentions, and a large number of beliefs about the potential benefits and costs of smoking that fell within the 21 candidate themes. These beliefs were derived from a comprehensive literature review on factors that have been known to be associated with young adults’ smoking behavior (Brennan et al., 2012).

The first H&W criterion was used to distinguish promising themes and less promising themes: When ranked by average odds ratios,4 promising themes were in the top five (of 21) for the strength of association between the belief theme and both intentions and behaviors, and less promising themes were in the bottom five for both analyses.5 With regard to the second H&W criterion, the selected promising and less promising themes had similar scale means (using the average level of endorsement of individual beliefs within a theme to assess endorsement of the theme as a whole) and standard deviations, which ensured that all differences in theme promise would be due to variation in the belief–outcome associations.

Preliminary step 2: pilot testing to establish that messages influence target beliefs within themes

Once the five promising and five less promising themes were chosen, a set of messages were developed to influence the beliefs represented within those themes. All themes were comprised of five individual beliefs. Therefore, 50 messages, one for each of the five beliefs within each of the 10 themes, were created. Pilot testing examined whether the messages developed (a) had persuasive effects on the targeted beliefs, and (b) were equally persuasive regardless of whether or not the theme was considered promising.

Eighteen- to 25-year-old nonsmokers (N = 1,415: never smokers [n = 786] and former smokers [n = 629]) were recruited from SSI’s panel to participate in the pilot testing study undertaken in February 2014. Pilot testing procedures were parallel to those used in the main study, as described in more detail below. Briefly, participants were randomized to 11 conditions where 10 conditions were the treatment groups (corresponding to 10 themes) and one was the control group. Participants in the treatment groups were exposed to the five messages addressing their assigned theme, and subsequently answered questions that measured their endorsement of beliefs. Participants in the control condition answered questions measuring their endorsement of beliefs first, and then were exposed to the messages. For a certain theme, if the level of belief endorsement of the treatment group was significantly higher than that of the control group, the message set corresponding to that theme was considered to be effective in persuasion of beliefs. Effect sizes (Cohen’s d) were also calculated to assess the similarity of persuasion effects across the themes.

Four promising themes and two less promising themes were successful in changing endorsement of the targeted beliefs to a similar extent; effect sizes (Cohen’s d) for the four promising themes ranged from 0.31 to 0.44 with a mean of 0.37, while effect sizes for the two less promising themes were 0.33 and 0.46. The messages for the remaining four themes did not increase belief endorsement and were thus eliminated. The four promising themes—positive mood effects, negative mood effects, self-image concerns, and social concerns—and two less promising themes—social and self-image concerns related to not smoking,6 and harmful chemicals—that increased belief endorsement were used as stimuli in the main study. The five individual beliefs within each theme are listed in Table 1. Additional details and full results from the pilot testing study are available from the authors upon request.

Table 1.

Five Underlying Belief Items for Each of Four Promising Themes and Two Less Promising Themes

Belief Items Alpha Mean SD
Promising themes
 Positive mood effects If I smoke, I will feel more comfortable in social situations 0.90 2.96 0.96
If I smoke, I will feel relaxed
If I smoke, I will feel content
If I smoke, I will enjoy life more
If I smoke, I will feel good
 Negative mood effects If I smoke, I will be able to forget about my problems 0.89 2.91 0.97
If I smoke, I will be able to control my anger
If I smoke, I will feel better when I am sad
If I smoke, I will feel less cranky
If I smoke, I will feel less bored
 Self-image concerns If I smoke, I will look mature 0.75 2.43 0.88
If I smoke, I will look uncool
If I smoke, I will look stupid
If I smoke, I will look immature
If I smoke, I will not look confident
 Social concerns If I smoke, I will get respect from my brother(s) and/or sister(s) 0.58 2.87 0.72
If I smoke, I will be unable to go to places that don’t allow smoking
If I smoke, I will lose respect from my brother(s) and/or sister(s)
If I smoke, I will gain friends
If I smoke, I will be popular
Less promising themes
 Social and self-image concerns related to not smoking If I do not smoke at all, I will look cool 0.90 2.31 0.95
If I do not smoke at all, I will look mature
If I do not smoke at all, I will gain friends
If I do not smoke at all, I will be popular
If I do not smoke at all, I will fit in with others my age
 Harmful chemicals If I smoke, I will inhale acetone, which is also found in nail polish remover 0.95 2.63 1.16
If I smoke, I will inhale formaldehyde, which is also found in glues and adhesives
If I smoke, I will inhale ammonia, which is also found in many household cleaners
If I smoke, I will inhale benzene, which is also found in some types of detergents
If I smoke, I will inhale mercury, which is also found in mascara
Open in a new tab

Note: Alphas, means, and standard deviations (SDs) computed using data from the control group (N = 441), who were not exposed to any messages before answering the belief items.

Main study

Participants and design

A new sample of 18- to 25-year-old nonsmokers (including never smokers [n = 1,011] and former smokers [n = 1,057]) recruited from SSI’s opt-in online panel (total N = 2,068) participated in an online experiment in June 2014. Participants were 41.3% male, and the average age was 21.6 years (SD = 2.31). The sample was 64.1% White, 16.1% African American, and 19.9% other race. Fewer than 10% (5.5%) had less than a high school education, 30.1% had completed high school, 45.3% had some college, and 19.1% had a bachelor’s degree or higher education.

Respondents were randomly assigned to one of seven conditions, where six conditions were treatment conditions (four conditions for promising themes and two conditions for less promising themes) and one was the control condition. In the six treatment conditions, respondents were first exposed to one of six sets (one per condition) of smoking prevention messages. Each set of messages was comprised of five individual messages based on the five individual belief items relevant to the theme (see Table 1). Immediately following message exposure, respondents then answered a set of questions measuring their smoking-related intentions, which was the primary outcome measure. Endorsement of the five beliefs that were addressed in the set of messages (targeted beliefs) as well as endorsement of all other 25 nontargeted beliefs (i.e., drawn from all five remaining treatment conditions) was then measured. The study measured nontargeted beliefs to examine whether messages addressing a particular theme could also affect beliefs not specifically addressed by those messages. The order of all belief questions was randomized. In the control condition, intentions and beliefs were assessed without exposing the respondents to the messages first.

This design allowed assessing the effect of message exposure on smoking-related intentions. That is, a comparison of the mean level of intentions among those who had (treatment conditions) and had not been (control condition) exposed to messages prior to intention measurement determined if exposure to these smoking prevention messages increased intentions not to smoke.

Message stimuli

Each respondent was exposed to five text and static visual messages. Each message included: (a) a general statement indicating acceptance of the belief among former smokers (former smokers were spokespeople because their experience of both smoking and quitting would make the smoking prevention messages more persuasive), (b) personal information about the spokesperson, (c) a statement from the spokesperson that linked the belief to their decision to quit smoking, and (d) a picture of the spokesperson. Two example messages from two themes are shown in Figure 1.

Figure 1.

Figure 1

Open in a new tab

Examples of messages used in the main study.

Two aspects of these stimuli should be noted. First, each respondent received five exposures to messages that were on their surface distinct (and from five unique spokespeople), but in fact all addressed the same overarching theme. This de facto repeated exposure to the same underlying message (e.g., smoking does not have positive mood effects) made it more likely that respondents would change their beliefs (e.g., beliefs about the positive mood effects of smoking) (Hornik, 2002). Second, the personal information and pictures were intended to provide vividness and realism to the messages, and to increase engagement (Kim, Bigman, Leader, Lerman, & Cap-pella, 2012), so that exposure to the messages would ultimately lead to belief change.

Measures

Intention to not smoke

Respondents answered three questions measuring intentions to smoke at any time during the next 12 months. The first question asked respondents to rate how likely it was that they would smoke at any time during the next year (1 = very unlikely to 5 = very likely). fie second question asked respondents whether they would use each of four types of tobacco products (smokeless tobacco, such as chewing tobacco, snuff, or dip; cigarettes; cigars, little cigars, flavored cigars, or cigarillos; pipes or waterpipes) at any time over the next year (1 = definitely not to 4 = definitely yes). The third question asked respondents to rate how much they agreed or disagreed with the statement, “I intend to smoke tobacco sometime in the next 12 months (e.g., cigarettes, cigars, little cigars, cigarillos, pipes, waterpipes)” (1 = strongly disagree to 5 = strongly agree). Responses were reverse coded, standardized, and averaged so that higher scores indicated stronger intention to not smoke (α = .83).

Smoking-related beliefs

Using a 5-point scale (1 = very unlikely to 5 = very likely), respondents were asked to rate the likelihood that specific smoking-related consequences would occur either as a result of smoking daily, or not smoking at all (see Table 1).7 Responses to the five belief items within a theme were averaged to create a scale measuring endorsement of the set of beliefs, in a way such that higher scores indicated antismoking beliefs. Table 1 lists all items, and provides means, standard deviations, and Cronbach’s alphas for the six belief scales based only on control group responses.

Analysis

The primary hypothesis asked whether exposure to messages addressing themes deemed a priori as being promising would increase intentions not to smoke to a greater extent than exposure to messages addressing themes deemed as being less promising. For each theme, a linear regression model with a binary independent variable indicating condition (i.e., treatment vs. control) and intention as the dependent variable was conducted to compare intentions between each treatment and the control condition (e.g., for the positive mood effects theme, the sample for these analyses was restricted to respondents who received messages addressing the positive mood effects theme and those in the control condition). All analyses were conducted with Stata V13.0. Regression coefficients, their confidence intervals, and significance levels were examined to determine if the four promising and two less promising themes differed from the control conditions in the way expected. Cohen’s d effect sizes were also used to compare the magnitude of effect across themes. In addition, mediation analyses were conducted to examine the mechanism of the effect of message exposure on intentions. Confidence intervals for the indirect effects of message exposure affecting intentions through endorsement of targeted beliefs were computed via bootstrapping as suggested by Preacher and Hayes (2004) with the bootmm program of Stata V13.0.

Results

Primary analysis

Results from the primary analyses examining effects of message exposure on intentions are summarized in Table 2. The analyses revealed that exposure to message stimuli was successful in increasing intentions not to smoke regardless of whether individuals were exposed to messages addressing promising or less promising themes. That is, respondents in all six treatment conditions showed higher levels of intentions not to smoke compared with those in the control condition. Cohen’s d values (b/pooled SD) for the six themes all pointed to similar moderate effects of the message stimuli. Also, the confidence intervals for all estimates were substantially overlapping. Thus, while the message stimuli were successful in affecting intentions, no support was found for the first hypothesis that exposure to antismoking messages addressing themes deemed to be promising a priori (under the first H&W criterion) would increase intentions not to smoke to a greater extent than exposure to messages addressing themes deemed to be less promising.

Table 2.

Linear Regressions of Intention on Condition (Treatment vs. Control) for Four Promising Themes and Two Less Promising Themes

N b CI d
Promising themes
 Positive mood effects 625 0.35* (0.20, 0.50) 0.40
 Negative mood effects 641 0.28* (0.13, 0.43) 0.32
 Self-image concerns 645 0.29* (0.14, 0.44) 0.33
 Social concerns 632 0.35* (0.20, 0.49) 0.39
Less promising themes
 Social and self-image concerns related to not smoking 869 0.35* (0.23, 0.46) 0.41
 Harmful chemicals 858 0.36* (0.24, 0.47) 0.43
Open in a new tab

Notes: b = unstandardized regression coefficient; CI = 95% confidence interval; d = Cohen’s d; N = sample size for treatment and control group combined.

*

p < .05.

Mediation analysis

Mediation analyses indicated that for the four promising themes, a substantial amount of the effect of exposure to message stimuli on intentions (i.e., the total effect) was mediated by endorsement of the targeted set of beliefs (Table 3). Indirect effect to total effect ratios for the four promising themes were relatively high (ranging from 0.24 to 0.68), while those of the two less promising themes were relatively low (0.14 and 0.19). Thus, analyses indicated that for the two less promising themes, a smaller portion of the total effect was mediated by endorsement of the targeted set of beliefs.

Table 3.

Mediation Effects of Targeted Belief Endorsement on the Relationship Between Message Exposure and Intentions

Total Effect Indirect Effect Residual Effect Indirect to Total Effect Ratio
Promising themes
 Positive mood effects 0.35a 0.17a 0.18a 0.49
 Negative mood effects 0.28a 0.19a 0.10 0.68
 Self-image concerns 0.29a 0.07a 0.21a 0.24
 Social concerns 0.35a 0.10a 0.25a 0.29
Less promising themes
 Social and self-image concerns related to not smoking 0.35a 0.05a 0.29a 0.14
 Harmful chemicals 0.36a 0.07a 0.28a 0.19
Open in a new tab

Notes: Effects are stated in unstandardized regression coefficients. Confidence intervals (95%) for indirect effects through targeted beliefs were computed using bootstrapping and are percentile-based confidence intervals.

a

Confidence intervals (95%) did not include zero.

Follow-up analysis

All themes had comparable effects on intentions, but only the promising theme message effects were substantially mediated through endorsement of the targeted beliefs. A follow-up analysis was conducted to explore alternative mediation routes for the effect of less promising theme messages on intentions, given that the indirect effects through targeted beliefs were relatively small. In particular, we examined whether the effect of these less promising theme messages on intentions may have been mediated through the more promising beliefs that were not specifically targeted.

By using the nontargeted sets of beliefs measured in the study, it was possible to illuminate this route of effects. As a reminder, all participants answered questions measuring endorsement of the beliefs targeted by the messages to which they were exposed, as well as beliefs targeted by messages in all other conditions to which they were not exposed (nontargeted beliefs). Although the nontargeted beliefs measured in the study are only a subset of the plausible set of beliefs through which effects might have been mediated, they do provide an opportunity to explore this path. If such mediation did occur, there are two expectations: First, beliefs addressed by the less promising themes should be correlated with beliefs addressed by the more promising themes; second, less promising theme messages should then affect intentions through increasing endorsement of beliefs addressed by the promising themes.

The first expectation is addressed by examining the correlation matrix for all belief sets, using only the responses provided by the control respondents (given that belief endorsement in this group could not have been influenced by message exposure). This shows that scores on the scales measuring endorsement of the two less promising themes were moderately correlated with scores on scales measuring endorsement of two of the promising themes, self-image concerns and social concerns, but not with the other two promising themes, positive mood effects and negative mood effects (Table 4). The analysis addressing the second expectation builds on this result. Mediation analyses were carried out for the less promising theme conditions, where targeted beliefs, correlated nontargeted promising beliefs, and uncorrelated nontargeted promising beliefs were all entered as mediators of the relationship between message exposure and intentions. Confidence intervals around the sum of indirect effects for correlated nontargeted beliefs (self-image concerns and social concerns), and for uncorrelated nontargeted beliefs (positive mood effects and negative mood effects), were bootstrapped. Entering those promising belief sets that were and were not correlated with the less promising beliefs into the same mediation model allowed testing the hypothesis that only the correlated promising beliefs would serve as a meaningful mediator.

Table 4.

Correlation Among Beliefs Addressed by Four Promising Themes and Two Less Promising Themes

Promising 1 Promising 2 Promising 3 Promising 4 Less Promising 1 Less promising 2
Promising 1 1.00
Promising 2 0.90* 1.00
Promising 3 0.34* 0.31* 1.00
Promising 4 0.60* 0.54* 0.55* 1.00
Less promising 1 0.04 0.01 0.54* 0.35* 1.00
Less promising 2 0.09 0.08 0.45* 0.40* 0.33* 1.00
Open in a new tab

Notes: Pearson correlation computed using data from the control group (N = 441), who were not exposed to any messages before answering the belief items. Promising 1: positive mood effects theme; Promising 2: negative mood effects theme; Promising 3: self-image concerns theme; Promising 4: social concerns theme; Less promising 1: social and self-image concerns related to not smoking theme; Less promising 2: harmful chemicals theme.

*

p < .05.

For the two less promising themes, Table 5 presents indirect effects through the targeted beliefs and two sets of nontargeted promising beliefs: correlated nontargeted beliefs (self-image concerns and social concerns) and uncorrelated nontargeted beliefs (positive mood effects and negative mood effects).

Table 5.

Mediation Effects of Targeted and Nontargeted Belief Endorsement on the Relationship Between Less Promising Theme Message Exposure and Intentions

Theme Total Effect Indirect Effect Through Targeted Beliefs Combined Indirect Effect Through Correlated Nontargeted Beliefs Combined Indirect Effect through Uncorrelated Nontargeted Beliefs Residual Effect
Social and self-image concerns related to not smoking 0.35a 0.02 0.07a 0.02 0.23a
Harmful chemicals 0.36a 0.03 0.09a 0.02 0.22a
Open in a new tab

Notes: Effects are stated in unstandardized regression coefficients. Confidence intervals (95%) for indirect effects through targeted and nontargeted belief themes were computed using bootstrapping and are percentile-based confidence intervals. Combined indirect effect through correlated nontargeted beliefs refers to the summed indirect effect through self-image concerns beliefs and social concerns beliefs. Combined indirect effect through uncorrelated nontargeted beliefs refers to the summed indirect effect through positive mood effects beliefs and negative mood effects beliefs.

a

Confidence intervals (95%) did not include zero.

The indirect effect of message exposure on intentions via targeted beliefs had confidence intervals that included zero for both of the less promising theme conditions; that is, effects of message exposure on intentions were not mediated through the targeted beliefs. However, the indirect effect through correlated nontargeted beliefs (self-image concerns and social concerns) was significant for both themes. By comparison, the indirect effects through uncorrelated nontargeted beliefs (positive mood effects and negative mood effects) were nonsignificant. This indicates that the effect of exposure to messages addressing less promising themes on intentions was partly mediated only through the other nontargeted promising beliefs that were moderately correlated with the less promising beliefs, but not through those promising beliefs that were uncorrelated.8

Discussion

Summary of results

Exposure to the study’s message stimuli had a significant influence on targeted beliefs and on intentions, even after only brief exposure. However, the study did not provide support for the primary prediction that exposure to smoking prevention messages addressing themes identified as promising (under the belief–intention association criterion) would increase intentions not to smoke to a greater extent than would exposure to messages addressing themes deemed as less promising; contrarily, the messages were equally effective. However, the unexpectedness of these results is ameliorated by the findings from the mediation analyses. Exposure to messages addressing promising themes produced changes in intentions that were substantially mediated through changes in the targeted beliefs; in contrast, effects of exposure to messages addressing less promising themes, although they still produced changes in intentions, were not mediated through the beliefs targeted by those messages. Instead, these effects were partially mediated through those more promising beliefs that were correlated with the less promising beliefs.

Implications

Ultimately, the study results show that the primary H&W criterion (belief–intention/ behavior association) did not serve to discriminate among potential message topics in the expected way, as exposure to messages addressing both promising and less promising themes led to significantly stronger intentions not to smoke. However, more importantly, follow-up mediation analyses indicated that the effect of messages addressing both promising and less promising themes exerted their influence on intentions through endorsement of promising beliefs. These results can be interpreted as evidence that the belief–intention association matters, but that the H&W method and other similar methods require refinement; that is, message topic selection approaches may need to be refined to also consider less promising beliefs that are correlated with promising beliefs as appropriate message topics.

To elaborate, there is substantial theory suggesting that beliefs are intertwined with one another, under the concept of spreading activation (e.g., Dinauer & Fink, 2005; Monroe & Read, 2008). Spreading activation refers to how the activation of one concept in memory can increase the probability that another connected concept is also activated (Judd, Drake, Downing, & Krosnick, 1991). Spreading activation may explain why exposure to messages addressing less promising themes changed intentions by increasing endorsement of nontargeted promising beliefs, if the activation of the beliefs targeted by the message (e.g., beliefs about harmful chemicals) triggered the activation of other nontargeted but correlated promising beliefs (e.g., beliefs about self-image concerns). The results of this study suggest that beliefs regarding smoking behavior might not be independent of each other, but are connected, so that when one belief is targeted or changed it is likely that other associated beliefs will also be affected. In this study, the primary H&W criterion was effective in predicting which beliefs would influence intentions; it failed in neglecting to predict that messages targeting one set of beliefs could also affect other beliefs. Therefore, we argue that a refinement to the H&W method and others like it may involve the specification that some sets of beliefs may be defined as promising because they directly satisfy the belief–intention/behavior association criterion, while other themes may be defined as promising if they are substantially correlated with those themes that directly satisfy the criterion.

In contrast, an alternative explanation for the results is that the primary H&W logic for choosing message topics is wrong. It could be that intentions can be influenced similarly by a wide range of messages, and that repeated exposure to messages recommending the same behavior may be what matters. This might be because a range of messages can all activate the same pre-existing thoughts about a behavior, irrespective of the specific content of that message. Or it may be that repeated exposure, with each respondent seeing five messages making the same core argument, influences respondents’ normative perceptions about the recommended behavior, which then affects intentions (Hornik, 2002). Or, it may be that the fact that these message sets shared structural components (all used a narrative structure, individual speakers with portraits, and a similar argument structure) meant that they had similar effects, and that it is the common structural elements which produce parallel effects regardless of topic. Thus, it could be that the specific theme (i.e., topic) that the message addresses matters less than other components. All of these possibilities suggest that message topic selection based on relative belief–intention/behavior associations deserves less attention. Although these possibilities may be plausible, mediation results from this study do provide support for the belief–intention association criterion, but raise the possibility that it requires refinement (i.e., other beliefs correlated with beliefs that directly satisfy this criterion may also be appropriate message topics). However, it is dificult to determine whether this is the case with this study alone; more studies should be carried out in order to judge whether moving away from a focus on belief–intention/behavior associations improves the effectiveness and efficiency of message topic selection approaches.

Limitations

This study is the first, to our knowledge, to test experimentally the core belief–intention association criterion of several message topic selection approaches. However, this study has some limitations. All of the themes tested here were garnered from an initial pool of potential themes that were already known to have a significant association with intentions (hence the labeling of themes as less promising, rather than as unpromising). Thus, it is plausible that the variation in the degree of “promisingness” between promising and less promising themes might not have been wide enough to provide a rigorous test of the method, despite our efforts to choose the most and least promising belief themes from the pool of available themes. That is, the initial pool of 21 themes were derived from a systematic review of factors known to be associated with smoking (Brennan et al., 2012), such that all of the themes were at least somewhat promising to start with. Also, the study focused on only one type of behavior (i.e., smoking). Future tests of the method should be conducted under different domains (e.g., politics, environment as well as health) and for different behaviors (e.g., vaccination and condom use). Finally, the study was an experiment, conducted online, with an opt-in sample using newly developed message stimuli, and with intentions rather than behavior as the outcome. Evidence from other research designs using different samples, message stimuli, and outcomes would support construct and external validity claims.

Conclusion

Despite these limitations, the study’s contributions are noteworthy. First, it used an experimental approach to test the key prediction of a theory of message topic promise. Second, it was able to show meaningful effects on intentions with limited exposure to messages that contained only a brief written statement and a static image. Third, results indicated that the belief–intention association criterion for identifying promising message topics does matter, as intention change was mainly a function of change in promising belief endorsement. Finally, the results open the possibility of refining existing message topic selection approaches by suggesting that less promising themes that are correlated with promising themes may also be appropriate message topics. Practically, this refinement could be useful as it increases the pool of appropriate beliefs/themes to target in a campaign based on the belief–intention association criterion. A larger pool of candidate beliefs/themes is beneficial given that some of these may not satisfy the other criteria for choosing promising beliefs/themes to target as message topics (e.g., H&W criterion #2 and #3). In terms of future research, it would be helpful to demonstrate that changing beliefs defined as less promising by the belief–intention association criteria and which are uncorrelated with promising beliefs do not produce an effect on intention, as would be predicted based on the results of the current work. This future work will further explain the observed results, and also strengthen the claim that these methods should be refined by considering the degree to which different beliefs or sets of beliefs are correlated with one another.

Acknowledgments

The authors are grateful to the editor and anonymous reviewers for their valuable suggestions for improving this paper. Research reported in this publication was supported by the National Cancer Institute (NCI) of the National Institutes of Health (NIH) under Award Number P20CA095856 and by the NCI of the NIH and FDA Center for Tobacco Products (CTP) under Award Number P50CA179546. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration (FDA).

Footnotes

1

A complementary explanation of how campaigns can lead to intention and behavior change is provided by media priming theory, which suggests that rather than causing belief change, exposure to a campaign message targeting a belief may magnify the association between existing beliefs and intention/behavior. If a campaign aims to bring about intention/behavior change through a priming effect, then it is not necessary to select beliefs that have a strong association with intentions/behaviors for targeting by the campaign (Cappella, Fishbein, Hornik, Ahern, & Sayeed, 2001).

2

The H&W approach is applied to belief–intention associations or belief–behavior associations. Therefore, while we refer to belief–intention associations to simplify the explanation, the logic applies to associations with behavior as well.

3

It is important to note the terminology used throughout the manuscript to distinguish between message topics, beliefs, and themes. Message topic is a general conceptual term that refers to the subject a message conveys. Beliefs are the operationalizations of message topic. Beliefs refer to measured beliefs related to a target behavior, while themes are sets of beliefs which capture a broader topical domain.

4

Logistic regression analyses were used to assess the strength of association between each set of beliefs (theme) and the outcome variables, antismoking intentions (intention analysis) and nonsmoker behavior status (behavior analysis). An odds ratio greater than 1.0 indicated that respondents who held the desired beliefs were more likely to have no intention to smoke or to be a not current tobacco user than were those who did not hold the desired beliefs.

5

With the exception of one less promising theme called “harmful ingredients: health effects framing throat/lungs theme,” which ranked 16th in the intention analysis (just outside the bottom five) but 18th in the behavior analysis (inside the bottom five). Nonetheless, we decided that ranking 16th of 21 was close enough to being in the bottom five and proceeded to use this theme as one of the less promising themes (note, however, that this theme was excluded from the main study affer we found that messages addressing this theme failed to increase endorsement of targeted beliefs).

6

It should be noted that one of the less promising themes, “social concerns related to not smoking,” was successful in changing the targeted beliefs, but had a stronger association with intentions than was expected for a less promising theme. So that this theme could be used as a less promising theme in the main study, we modified the original “social concerns related to not smoking” theme to include some belief items from a related theme, “self-image concerns related to not smoking.” The final set of five belief items included in the “social and self-image concerns related to not smoking” less promising theme were selected in order to maximize the probability of achieving an effect on beliefs, but minimize the belief–intention association.

7

In the original study from which the belief items were drawn, the researchers were interested in whether particular themes were more or less promising when framed in terms of the consequences of smoking versus of not smoking (Brennan, Gibson, Momjian, & Hornik, 2013).

8

In order to address the converse possibility that messages addressing promising themes may have exerted their influence on intentions through change in less promising beliefs, a mediation model for each promising theme condition which included targeted beliefs and less promising beliefs as mediators was examined. Specifically, the indirect effect through targeted beliefs and the sum of indirect effects through social and self-image concerns related to not smoking beliefs and harmful chemicals beliefs were bootstrapped and compared. For all four promising themes, indirect effects through targeted beliefs ranged from 0.07 to 0.18 (average 0.13), and were all significant (95% confidence intervals did not include zero). However, the indirect effects through the less promising beliefs ranged from 0.003 to 0.02 (average 0.01) and none were significant (95% confidence intervals included zero). Thus, analyses indicate that even when endorsement of less promising beliefs is accounted for, for promising themes, the effect of messages on intentions is mainly mediated by change in targeted promising beliefs.

References

  1. Atkin CK, Freimuth V. Formative evaluation research in campaign design. In: Rice R, Atkin C, editors. Public communication campaigns. Vol. 3. Thousand Oaks, CA: Sage; 2001. pp. 125–145. [Google Scholar]
  2. Bai YK, Middlestadt SE, Joanne Peng CY, Fly AD. Psychosocial factors underlying the mother’s decision to continue exclusive breastfeeding for 6 months: An elicitation study. Journal of Human Nutrition and Dietetics. 2009;22(2):134–140. doi: 10.1111/j.1365-277X.2009.00950.x. [DOI] [PubMed] [Google Scholar]
  3. Botta RA, Dunker K, Fenson-Hood K, Maltarich S, McDonald L. Using a relevant threat, EPPM and interpersonal communication to change hand-washing behaviours on campus. Journal of Communication in Healthcare. 2008;1(4):373–381. doi: 10.1179/cih.2008.1.4.373. [DOI] [Google Scholar]
  4. Boudewyns V, Paquin RS. Intentions and beliefs about getting tested for STDs: Implications for communication interventions. Health Communication. 2011;26(8):701–711. doi: 10.1080/10410236.2011.563353. [DOI] [PubMed] [Google Scholar]
  5. Brennan E, Gibson L, Liu J, Hornik R. Identifying potential target beliefs for a mass media campaign aimed at preventing progression to daily tobacco use: Final findings and recommendations for 18–25 year old non-daily tobacco users. Philadelphia, PA: Penn’s Center of Excellence in Cancer Communication Research, Annenberg School for Communication, University of Pennsylvania; 2013a. (CECCR Working Paper Series 379) [Google Scholar]
  6. Brennan E, Gibson L, Liu J, Hornik R. Identifying potential target beliefs for a smoking prevention mass media campaign: Final findings and recommendations for 18 – 25 year old never tobacco users. Philadelphia, PA: Penn’s Center of Excellence in Cancer Communication Research, Annenberg School for Communication, University of Pennsylvania; 2013b. (CECCR Working Paper Series 381) [Google Scholar]
  7. Brennan E, Gibson L, Momjian A, Hornik R. Identifying potential target beliefs for a youth-focused smoking prevention mass media campaign: Final findings and recommendations for 13–17 year old non-smokers. Philadelphia, PA: Penn’s Center of Excellence in Cancer Communication Research, Annenberg School for Communication, University of Pennsylvania; 2013. (CECCR Working Paper Series 383) [Google Scholar]
  8. Brennan E, Momjian A, Jeong M, Naugle D, Parvanta S, Hornik R. Mass media campaigns to reduce smoking among youth and young adults: Documenting potential campaign targets and reviewing the evidence from previous campaigns. Philadelphia, PA: Penn’s Center of Excellence in Cancer Communication Research, Annenberg School for Communication, University of Pennsylvania; 2012. (CECCR Working Paper Series 385) [Google Scholar]
  9. Cappella JN. Integrating message effects and behavior change theories: Organizing comments and unanswered questions. Journal of Communication. 2006;56:S265–S279. doi: 10.1111/j.1460-2466.2006.00293.x. [DOI] [Google Scholar]
  10. Cappella JN, Fishbein M, Hornik R, Ahern RK, Sayeed S. Using theory to select messages in antidrug media campaigns. In: Rice RE, Atkin CK, editors. Public communication campaigns. 3. Thousand Oaks, CA: Sage; 2001. pp. 214–230. [Google Scholar]
  11. Dillard JP. An application of the integrative model to women’s intention to be vaccinated against HPV: Implications for message design. Health Communication. 2011;26(5):479–486. doi: 10.1080/10410236.2011.554170. [DOI] [PubMed] [Google Scholar]
  12. Dinauer LD, Fink EL. Interattitude structure and attitude dynamics: A comparison of the hierarchical and Galileo spatial-linkage models. Human Communication Research. 2005;31(1):1–32. doi: 10.1093/hcr/31.1.1. [DOI] [Google Scholar]
  13. Fishbein M, Ajzen I. Predicting and changing behavior: The reasoned action approach. New York, NY: Psychology Press; 2010. [Google Scholar]
  14. Fishbein M, Cappella JN. The role of theory in developing effective health communications. Journal of Communication. 2006;56:S1–S17. doi: 10.1111/j.1460-2466.2006.00280.x. [DOI] [Google Scholar]
  15. Fishbein M, Yzer MC. Using theory to design effective health behavior interventions. Communication Theory. 2003;13(2):164–183. doi: 10.1111/j.1468-2885.2003.tb00287.x. [DOI] [Google Scholar]
  16. Friedman AL, Shepeard H. Exploring the knowledge, attitudes, beliefs, and communication preferences of the general public regarding HPV: Findings from CDC focus group research and implications for practice. Health Education & Behavior. 2007;34(3):471–485. doi: 10.1177/1090198106292022. [DOI] [PubMed] [Google Scholar]
  17. Haider-Markel DP, Joslyn MR. Beliefs about the origins of homosexuality and support for gay rights: An empirical test of attribution theory. Public Opinion Quarterly. 2008;72(2):291–310. doi: 10.1093/poq/nfn015. [DOI] [Google Scholar]
  18. Hornik R. Public health communication: Making sense of contradictory evidence. In: Hornik R, editor. Public health communication: Evidence for behavior change. Mahwah, NJ: Erlbaum; 2002. pp. 1–19. [Google Scholar]
  19. Hornik R, Woolf KD. Using cross-sectional surveys to plan message strategies. Social Marketing Quarterly. 1999;5(2):34–41. doi: 10.1080/15245004.1999.9961044. [DOI] [Google Scholar]
  20. Huebner DM, Neilands TB, Rebchook GM, Kegeles SM. Sorting through chickens and eggs: A longitudinal examination of the associations between attitudes, norms, and sexual risk behavior. Health Psychology. 2011;30(1):110–118. doi: 10.1037/a0021973. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Judd CM, Drake RA, Downing JW, Krosnick JA. Some dynamic properties of attitude structures: Context-induced response facilitation and polarization. Journal of Personality and Social Psychology. 1991;60(2):193–202. doi: 10.1037/0022-3514.60.2.193. [DOI] [Google Scholar]
  22. Keller PA, Lehmann DR. Designing effective health communications: A meta-analysis. Journal of Public Policy and Marketing. 2008;27(2):117–130. doi: 10.1509/jppm.27.2.117. [DOI] [Google Scholar]
  23. Kenski K, Appleyard J, Von Haeften I, Kasprzyk D, Fishbein M. Theoretical determinants of condom use intentions for vaginal sex with a regular partner among male and female injecting drug users. Psychology, Health & Medicine. 2001;6(2):179–190. doi: 10.1080/13548500120035436. [DOI] [Google Scholar]
  24. Kim H, Bigman CA, Leader AE, Lerman C, Cappella JN. Narrative health communication and behavior change: The influence of exemplars in the news on intention to quit smoking. Journal of Communication. 2012;62(3):473–492. doi: 10.1111/j.1460-2466.2012.01644.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Krosnick JA, Holbrook AL, Lowe L, Visser PS. The origins and consequences of democratic citizens’ policy agendas: A study of popular concern about global warming. Climatic Change. 2006;77(1–2):7–43. doi: 10.1007/s10584-006-9068-8. [DOI] [Google Scholar]
  26. Maddock JE, Silbanuz A, Reger-Nash B. Formative research to develop a mass media campaign to increase physical activity and nutrition in a multiethnic state. Journal of Health Communication. 2008;13(3):208–215. doi: 10.1080/10810730701807225. [DOI] [PubMed] [Google Scholar]
  27. Middlestadt SE. Beliefs underlying eating better and moving more: Lessons learned from comparative salient belief elicitations with adults and youths. The ANNALS of the American Academy of Political and Social Science. 2012;640(1):81–100. doi: 10.1177/0002716211425015. [DOI] [Google Scholar]
  28. Monroe BM, Read SJ. A general connectionist model of attitude structure and change: The ACS (attitudes as constraint satisfaction) model. Psychological Review. 2008;115(3):733–759. doi: 10.1037/0033-295x.115.3.733. [DOI] [PubMed] [Google Scholar]
  29. Niederdeppe J, Porticella N, Shapiro MA. Using theory to identify beliefs associated with support for policies to raise the price of high-fat and high-sugar foods. Journal of Health Communication. 2012;17(1):90–104. doi: 10.1080/10810730.2011.585694. [DOI] [PubMed] [Google Scholar]
  30. Noar SM. A 10-year retrospective of research in health mass media campaigns: Where do we go from here? Journal of Health Communication. 2006;11(1):21–42. doi: 10.1080/10810730500461059. [DOI] [PubMed] [Google Scholar]
  31. Parvanta S, Gibson L, Forquer H, Shapiro-Luft D, Dean L, Freres D, … Hornik R. Applying quantitative approaches to the formative evaluation of antismoking campaign messages. Social Marketing Quarterly. 2013;19(4):242–264. doi: 10.1177/1524500413506004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Preacher KJ, Hayes AF. SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers. 2004;36(4):717–731. doi: 10.3758/BF03206553. [DOI] [PubMed] [Google Scholar]
  33. Robbins R, Niederdeppe J. Using the integrative model of behavioral prediction to identify promising message strategies to promote healthy sleep behavior among college students. Health Communication. 2015;30(1):26–38. doi: 10.1080/10410236.2013.835215. [DOI] [PubMed] [Google Scholar]
  34. von Haeften I, Fishbein M, Kasprzyk D, Montano D. Analyzing data to obtain information to design targeted interventions. Psychology, Health & Medicine. 2001;6(2):151–164. doi: 10.1080/13548500120035409. [DOI] [Google Scholar]
  35. Weinstein ND. Misleading tests of health behavior theories. Annals of Behavioral Medicine. 2007;33(1):1–10. doi: 10.1207/s15324796abm3301_1. [DOI] [PubMed] [Google Scholar]

RESOURCES