Abstract
Hornik and Woolf (1999) proposed using cross-sectional survey data to prioritize beliefs to address with communication campaign messages. The empirical component of the approach combines evidence of (1) association of beliefs with intentions and (2) current level of beliefs to calculate a ‘percentage to gain’ as the potential promise of a belief. However, the method relies on cross-sectional data; its conclusions are open to challenge. Here, a panel study assesses whether the calculated promise of a belief actually predicts future behavior change. A nationally representative sample of 3,204 U.S. youth and young adults were interviewed twice, six months apart. Sixteen beliefs about the benefits and costs of smoking cigarettes are compared with regard to their percentage to gain (calculated from cross-sectional data) and their ability to account for subsequent cigarette use. A belief’s cross-sectional percentage to gain is substantially associated with its ability to predict subsequent behavior change (r=.53, p<.05).
Keywords: campaigns, choosing messages, media effects, health communication, communication research methods, evaluation research, observational research methods, surveys
When designing health communication campaigns, planners choose among message themes—that is, which beliefs to target. For example, a tobacco cessation campaign needs to decide whether to focus on health consequences, or on the money to be saved, or on the social approval to be earned by quitting, or on any of dozens of other possible message themes (Brennan, Gibson, Kybert-Momjian, Liu, & Hornik, 2017; Parvanta, Gibson, Forquer, et al., 2013). How might a planner determine which candidate themes are most promising for changing a behavior among a particular audience prior to implementing the campaign?
Hornik and Woolf (1999) [hereafter, H&W] proposed a strategy using survey data to choose themes. The H&W approach requires that planners consider three key points: the proportion of people who do not endorse the belief at baseline (i.e., the number of people who are available for belief change), the strength of association between the potential targeted belief and the desired outcome (i.e., behavior or behavioral intention), and whether the belief could reasonably be affected by a communication campaign. The first and second of these can be determined through the analysis of cross-sectional survey data, while the third requires the subjective judgment of the campaign planner. The H&W approach has been used to guide several large-scale public communication campaigns (Vallone et al, 2017, Brennan et al., 2017; Parvanta, Gibson, Forquer, et al., 2013; Sangalang et al., 2016).
Hypothetically, campaign planners might actually test messages addressing a variety of beliefs, by developing and pilot testing multiple candidate messages for each belief, but few programs would be able to afford this. There are multiple other approaches to choosing among candidate themes. Some are qualitative: focus groups (Botta, Dunker, Fenson-Hood, Malatrich, & McDonald, 2008), elicitation surveys (Ajzen & Fishbein, 1980), or consultation with experts (Nowak & Siska, 1995). Some advocate using survey research to describe beliefs, behaviors, norms, and other information that can be helpful in designing a campaign (see, e.g., Witte, Meyer, & Martell, 2001). However, most of these methods stop short of recommending how to use survey data to select specific beliefs to target.
Others have suggested approaches analogous to the H&W method, for example using regression weights to choose which beliefs are most likely to produce the desired behavioral outcomes (e.g., Kenski, Appleyard, von Haeften, Kasprzyk, & Fishbein, 2001). Fishbein and Cappella (2006) point to the importance of identifying beliefs that differ between those who intend to engage in the desired behavior and those who do not, but they do not provide a specific formula for deciding which among these discriminating beliefs hold the most promise as a potential message strategy. Maddock, Silbanuz, and Reger-Nash (2008) used discriminant function analysis to determine the beliefs that differed most between those engaging in the desired behavior and those who were not, then developed messages using these beliefs.
The H&W method has some advantages over those other approaches since it takes into account not only the association of the belief with intentions or behavior but also the observed availability of people to change their beliefs and the subjective assessment of the likelihood of a campaign’s influence on the belief. Nonetheless, these methods share a potential weakness, the reliance on cross-sectional data and assuming that an association between a belief and intention or behavior is (a) causal and (b) the result of the belief influencing intention/behavior. It is possible to adjust associations for measured potential confounders, but this does not solve the problem of causal order between variables (Weinstein, 2007) nor address possible unmeasured confounders. The H&W analysis observes a cross-sectional association and projects that if a campaign changes the promising belief it will result in behavior change. While H&W provides practical guidance about message themes to emphasize, the uncertainty of the implicit causal assumptions may limit a campaign planner’s confidence in the recommendations. How can one establish that the H&W method produces valid prioritization among candidate beliefs?
In an earlier study, Lee and colleagues (2016) undertook an experimental test of the validity of the H&W approach. In the current study, we present the results of a complementary non-experimental test of the validity of the H&W approach, making use of longitudinal data to provide additional validity evidence. We hypothesize that beliefs that are categorized as more promising under the cross-sectional H&W analysis will also be more promising when examined with longitudinal analyses. Specifically, we expect that more promising (versus less promising) beliefs measured at baseline will better predict subsequent behavior change.
Methods
Study sample and data collection
Survey data came from a rolling cross-sectional study with a panel component. The survey recruited a nationally representative sample of 13- to 25-year-olds over landline and cell phones, with new samples completing an initial interview (T1) each week from June 2014 to September 2016 (AAPOR response rate #3 = 22%) with 35% of T1 respondents completing a follow-up interview 6 months later (T2). Successfully re-contacted respondents were more likely to be male, younger in age, and baseline non-current tobacco cigarette smokers. Demographics of this retained sample who had completed T2 surveys by March 2017 (n=3204) can be found in Table 1. Analyses were conducted separately for each of 16 anti-smoking beliefs (see below); respondents were included in belief-specific analyses if they answered that belief item at T1 and questions about smoking behavior at T1 and T2. There was some variation in which beliefs were asked about over time but the number of responses for each belief item was never less than 2,617.
Table 1.
Characteristics of the Retained Sample at T1 (N=3,204)
| Characteristic | % or M (SD) |
|---|---|
| Sex (male) | 54.8% |
| Race/ethnicity | |
| Hispanic | 19.5% |
| White (non-Hispanic) | 56.6% |
| African American (non-Hispanic) | 12.0% |
| Other/more than one race | 12.0% |
| Current (past 30-day) tobacco cigarette smoker | 7.7% |
| Ever tobacco cigarette smoker (lifetime) | 21.4% |
| Mean age (SD) | 17.2 (3.4) |
Note. Percentages may not sum to 100% due to rounding.
Measures
Beliefs.
Respondents were told that all belief items were about tobacco cigarettes. Belief items had four possible responses of strongly disagree, disagree, agree, or strongly agree (unless indicated). The 16 belief items are listed in Table 2. Belief items were included reflecting their previously established promise as targets for an age-relevant anti-tobacco communication campaign (Brennan et al., 2017).
Table 2.
Cross-sectional (T1) Distribution of Agreement for Each of 16 Belief Items
| Belief item | Strongly disagree | Disagree | Agree | Strongly agree |
|---|---|---|---|---|
| How do you think your close friends feel or would feel about your smoking cigarettes every day? (strongly disapprove)a | 1.2% | 6.6% | 32.3% | 60.0% |
| …I will breathe in thousands of chemicals | 1.8% | 2.1% | 39.6% | 56.5% |
| …I will become addicted to nicotine | 2.2% | 3.6% | 38.2% | 56.1% |
| How much do you think breathing smoke from other people’s cigarettes harms you? (a lot)b | 2.9% | 11.9% | 30.0% | 55.2% |
| …I will enjoy life more (strongly disagree) | 1.5% | 3.7% | 41.4% | 53.5% |
| …I will develop cancer | 1.4% | 4.8% | 49.4% | 44.3% |
| …I will be controlled by smoking | 3.1% | 8.6% | 44.8% | 43.5% |
| …it will be a turn off to other people | 3.1% | 11.9% | 47.0% | 38.1% |
| …I will feel relaxed (strongly disagree) | 1.9% | 17.9% | 43.8% | 36.4% |
| The tobacco industry intentionally designed cigarettes to make them more addictive | 1.7% | 9.5% | 54.3% | 34.5% |
| …I will lose my teeth | 2.0% | 11.8% | 55.7% | 30.5% |
| …I will get wrinkles | 2.2% | 12.7% | 57.6% | 27.4% |
| …I will look uncool | 8.1% | 21.1% | 44.6% | 26.2% |
| …I will get yellow fingers | 4.0% | 25.2% | 52.2% | 18.6% |
| …I will develop headaches | 1.7% | 17.9% | 62.7% | 17.7% |
| …I will develop sexual and/or fertility problems | 4.0% | 24.5% | 54.6% | 16.9% |
Note. All belief items beginning with ellipses were preceded by “If I smoke every day…” Unless otherwise indicated in parentheses in the table, the “strongest antismoking belief” response option was “strongly agree”.
The response options for this item were: strongly disapprove, disapprove, approve, and strongly approve.
cThe response options for this item were: not at all, a little, somewhat, and a lot.
Past 30-day cigarette (non-) smoking status.
Participants were first asked if they had ever tried a cigarette, even just one puff. Positive responders were asked “During the past 30 days, on how many days did you smoke cigarettes?” Those who reported no use in the past 30 days were coded as past 30-day cigarette non-smokers. The great majority reported no past 30-day use of cigarettes at T1 (92.4%) and at T2 (91.7%).
Among the respondents, 5.2% reported any change in cigarette status between T1 and T2, with 2.9% moving from non-use to use and 2.3% from use to non-use. An unknown proportion of those changes may have reflected unreliability of measurement rather than real change in behavior. However, self-reported smoking status has been demonstrated to be consistent with biological indicators among youth and adults (Caraballo, Giovino, & Pechacek, 2004; Soulakova, Hartman, Liu, Willis, & Augustine, 2012).
Statistical analysis
The analyses begin with cross-sectional H&W analyses with the baseline data. Calculation of H&W percentage to gain statistics (described below) is based on the cross-tabulation of dichotomous versions of belief variables (Parvanta, Gibson, Forquer, et al., 2013); consistent with that approach, responses to belief items for the cross-sectional analyses were recoded into strongest anti-tobacco belief versus other responses. Dichotomies are used in the H&W method because they respond to the campaigner’s question: what is the maximum change in the outcome to be achieved if the campaign moved all respondents to the strongest anti-tobacco position? In contrast, the longitudinal analyses, to maximize sensitivity to the effects of any available variation in beliefs on the largely stable behavioral outcome, make use of the raw four category belief variables to take advantage of the complete range of response. The longitudinal analyses use logistic regression to predict non-smoker status at T2, from each four-category belief measured at T1 and non-smoker status at T1. Analyses only include cases that had non-missing belief data at T1 and non-missing behavior data from the T1 and T2 surveys.
H&W analyses typically use intention as the outcome (rather than behavior). See Brennan and colleagues (2017) for a full discussion of the tradeoffs. However, the present validity analysis is about beliefs accounting for future behavior; for that reason, and to make the presentation straightforward, we use behavior for both cross-sectional and longitudinal analyses.
Results
Cross-sectional Analysis
The H&W approach begins with the cross-tabulation of T1 beliefs and behavior. Table 3 presents an illustrative example. At T1, 92.3% of all respondents were non-smokers. However, 89.3% of those who did not strongly endorse and 96.2% of those who did strongly endorse the focus belief were non-smokers, a strong association (relative odds of 3.06). 56.5% did not give the strongest anti-smoking response and represent the ‘percentage (available) to move’.
Table 3.
Example of Hornik & Woolf Cross-sectional (T1) Analysis: The Proportion Who are Not Current (Past 30 Day) Cigarette Smokers by the Belief That “If I smoke every day, I will be controlled by smoking”
| Belief: If I smoke every day, I will be controlled by smoking | |||
|---|---|---|---|
| Current smoking status | All others | Strongest antismoking beliefa | Total |
| Smoker | 10.7% (192) |
3.7% (52) |
7.7% (244) |
| Non-smoker | 89.3% (1609) |
96.2% (1334) |
92.3% (2943) |
| Total | 56.5% (1801) |
43.5% (1386) |
100.0% (3187) |
Strongest antismoking belief= strongly agree; All others= agree, disagree, strongly disagree
H&W then projects the (non-) smoking behavior expected if a perfectly effective campaign moved everyone to strongly endorse this belief. All respondents would look like the ‘strongly agree’ group. Then, the percentage to gain would be 3.9% (equal to 96.2% non-smokers in the ‘strongest antismoking belief’ group minus 92.3% non-smokers overall). This essential H&W statistic captures both the association of belief and behavior, and the population available to move on the belief. The maximum possible percentage to gain would be 7.7%, given that 92.3% are already non-smokers at T1. This maximum percentage to gain would only be reached if everyone became a non-smoker.
Each of the beliefs are examined in Table 4. Beliefs vary in their percentages available to move (ranging from 40.0% - 83.1%). The association between beliefs and behavior are captured in relative odds ratios (RORs); all of these are positive and all but one is significant at p<.05, although varying sharply in magnitude (1.01 – 5.84). Together, percentage to move and ROR values are reflected in the ‘percentage to gain’ estimates (from 0.04%−4.8%).
Table 4.
Cross-sectional (T1) and Longitudinal (T2) Metrics for Each of 16 Belief Items, Ordered by Percentage to Gain
| Belief item | T1 % to move (% not in the strongest antismoking belief category) |
T1 Association of dichotomous belief with non-smoking behavior (ROR, relative odds ratio)a |
T1 % to gain |
T2 aOR (95% CI) |
N |
|---|---|---|---|---|---|
| …I will develop sexual and/or fertility problems | 83.1% | 3.06 | 4.8% | 1.12 (0.89, 1.41) |
2923 |
| …I will lose my teeth | 69.5% | 3.57 | 4.7% | 1.29 (0.88, 1.42) |
3154 |
| How do you think your close friends feel or would feel about your smoking cigarettes every day? (strongly disapprove)b | 40.0% | 5.84 | 4.7% | 1.80*** (1.45, 2.24) |
3178 |
| …I will look uncool | 73.8% | 3.37 | 4.5% | 1.45*** (1.20, 1.76) |
3113 |
| …I will feel relaxed (strongly disagree) | 63.6% | 3.43 | 4.5% | 1.78*** (1.45, 2.18) |
3160 |
| …I will develop headaches | 82.3% | 2.46 | 4.1% | 1.38* (1.08, 1.77) |
3047 |
| …I will be controlled by smoking | 56.5% | 3.06 | 3.9% | 1.47*** (1.20, 1.80) |
3187 |
| …it will be a turn off to other people | 61.9% | 2.63 | 3.7% | 1.22 (0.99, 1.50) |
3139 |
| …I will enjoy life more (strongly disagree) | 46.5% | 2.90 | 3.4% | 1.62*** (1.29, 2.04) |
3181 |
| …I will develop cancer | 55.7% | 2.49 | 3.2% | 1.28* (1.01, 1.62) |
3181 |
| …I will get wrinkles | 72.6% | 1.95 | 3.0% | 1.12 (0.88, 1.42) |
3124 |
| …I will get yellow fingers | 81.4% | 1.78 | 2.9% | 1.26* (1.02, 1.57) |
3028 |
| …I will become addicted to nicotine | 43.9% | 2.10 | 2.3% | 1.17 (0.93, 1.47) |
3187 |
| How much do you think breathing smoke from other people’s cigarettes harms you? (a lot)c | 44.8% | 1.97 | 2.1% | 1.35** (1.12, 1.62) |
3195 |
| …I will breathe in thousands of chemicals | 43.5% | 1.96 | 2.1% | 1.40* (1.08, 1.90) |
2617 |
| The tobacco industry intentionally designed cigarettes to make them more addictive | 65.5% | 1.01 (n.s.) | 0.04% | 0.96 (0.76, 1.22) |
3155 |
Note. All belief items beginning with ellipses were preceded by “If I smoke every day…” Unless otherwise indicated in parentheses in the table, the “strongest antismoking belief” response option was “strongly agree”. aOR = adjusted Odds Ratio predicting T2 past 30-day cigarette non-use from T1 belief (controlling for cigarette use at T1).
All RORs are significant at p<.05 unless indicated.
The response options for this item were: strongly disapprove, disapprove, approve, and strongly approve.
The response options for this item were: not at all, a little, somewhat, and a lot.
p<.05,
p<.01,
p<.001
This cross-sectional H&W analysis ranks the set of beliefs on their promise as campaign foci. Next we ask: do the beliefs that show more promise based on the cross-sectional analysis predict more behavior change six months later? Would we give the same advice to campaign planners if we had longitudinal data as we did when applying the H&W method to cross-sectional data? Is the H&W method predictively valid?
Lagged Analysis
We present the analyses of the lagged data using logistic regression, predicting follow-up cigarette use from T1 belief and T1 cigarette use. Controlling for cigarette use behavior at T1 accounts for all simultaneous effects of fixed confounders. Smoking at T1, unsurprisingly, is a strong predictor of smoking at T2. The relative odds of a T1 non-smoker (versus a smoker) being a T2 non-smoker was 70.4, p<.001, 95% CI [50.1, 99.1]. To illustrate the prediction analysis, endorsing the belief “If I smoke every day, I will be controlled by smoking” at T1 (in the four category version) is a significant predictor of non-smoking at T2 even when T1 behavior is included, OR=1.47, 95% CI [1.20, 1.80]. A respondent who was one category higher in endorsing this belief at T1 was 47% more likely to be a T2 non-smoker adjusting for baseline smoking. Of the 16 beliefs examined in Table 4, 15 showed a positive association and 11 of these showed a significant effect in predicting T2 non-smoking, after adjusting for T1 non-smoking.1
Predictive Validity of Cross-Sectional Analyses
Finally, are the relative promise of beliefs based on the cross-sectional analyses and the lagged effect analyses consistent? We examine the association between the 16 estimates of percentage to gain and the corresponding 16 adjusted odds ratio estimates for T1 beliefs on lagged behavior from Table 4. The correlation is .53 (p<.05).2 This relationship is illustrated in Figure 1. The association between cross-sectional and lagged analyses is substantial and significant although moderate.
Figure 1.
Correlation Between Percentage to Gain Estimates From Cross-Sectional Analyses and Strength of Association (Odds Ratio) Between Beliefs and Behaviors from Lagged Analyses, For Each of 16 Beliefs
Note. The correlation between percentage to gain and odds ratios is .53 (p<.05).
Discussion
Campaign advice based on H&W cross-sectional percentage to gain estimates is subject to concerns about the implicit causal claim; we show here that the prioritized beliefs, using the percentage to gain statistic, are still better able to predict behavior change over time (r=.53, p<.05).
This result is positive and consistent with a claim that the H&W method is an effective approach to prioritizing beliefs. Still, the observed correlation is moderate, and some beliefs performed inconsistently. For instance, the belief “I will develop sexual and/or fertility problems” exhibited a strong percentage to gain in cross-sectional analyses, yet ranked among the bottom in lagged analyses.
There are two ways to consider this anomalous result. One is to recognize that the H&W method, despite overall usefulness, may endorse some beliefs that are not actually promising; other criteria will need to be considered. A second perspective recognizes that the lagged regression coefficients are actually mismatched to the percentage to gain as a criterion for validity. The H&W method looks at associations and percentage to move. In contrast, the validity test coefficients are only analogous to the association component.
In these data, those two components, percentage to move and cross-sectional association, are not positively related (r=−0.20, p=.45). There is still a good substantive reason to merge these two components into the percentage to gain statistic: a campaign that addresses a belief with many people who are available to be convinced that is also strongly related to behavior may be particularly promising. However, because the validity analysis ignores the percentage to move component, it may undermine the ability of the combined percentage to gain statistics to predict the lagged regression coefficients. In fact, the beliefs that are more associated with behavior cross-sectionally do predict greater subsequent change in behavior (columns 2 and 4 in Table 4) (r=.73, p<.01). Ideally a measure of actual campaign success would be better matched to the percentage to gain; absent that we have used the partially matched criterion – the ability of the belief to predict behavior change. This does reduce expectations for the magnitude of the relationship between ‘percentage to gain’ at T1 and the lagged coefficient predicting behavior change.
This study then represents a step forward, both because it reduces previous concerns related to the use of cross-sectional analysis and uses behavior instead of intention as the focal outcome variable. It should leave campaign planners with more confidence in the use of the H&W method as an approach to reducing uncertainty about which campaign themes to use.
Limitations
The longitudinal approach reduces concern that any potential confounder that simultaneously affects belief and behavior threatens the claim of causal influence. However, potential confounders that affect belief and behavior asynchronously over time are not eliminated by adjusting for baseline behavior. Also, analyses which rely on lagged prediction adjusting for behavior at T1 are likely to systematically underestimate effects of beliefs (Weinstein, 2007). The data also come from a single study with a general population sample, focused on tobacco cigarette use, and using only a limited range of 16 beliefs, each measured by a single item. Our confidence will grow with additional longitudinal applications and specific use with segments of the population that might respond differently to campaign messages.
With regard to the application of the method, more generally, this study leaves some issues unresolved. The H&W method largely views message development once message belief themes are chosen as a task for campaign planners and creative staff: do they think they can develop a persuasive message that will convince people of the chosen belief? Campaign planners may believe a less promising belief is easier to address than a more promising one because the creative staff have a more attractive strategy.
Further strengthening of the causal claim might come from field experimental interventions (although see Lee et al. [2016] for the complexities in mounting such experiments), and from additional longitudinal tests with other behavioral outcomes, including those with lower a priori desirable levels or those with less stability.
Conclusion
The current longitudinal evidence in support of the H&W method is promising, and tentatively warrants its continued use by those creating and implementing communication campaigns. To the extent possible, we recommend that preference be given to beliefs identified by the H&W approach as being more rather than less promising; the analyses presented here show that such beliefs, if successfully targeted, have a greater likelihood of contributing to behavior change over time.
Acknowledgements:
Research reported in this publication was supported by the National Cancer Institute (NCI) of the National Institutes of Health (NIH) and FDA Center for Tobacco Products (CTP) under Award Number P50CA179546. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration (FDA).
Footnotes
The analyses presented here make use of the four-category version of the belief measure in the logistic regressions, and present the relative odds ratios associated with change in one category on the belief measure. This approach effectively treats the belief measure as an interval variable and assumes that the average effect across categories is constant. To assess the sensitivity of the analyses to this assumption we also undertook an alternative analysis, estimating the Goodman and Kruskal partial gamma (Davis, 1967) from the prediction of T2 non-smoking status from each T1 four-category belief, adjusting for T1 smoking status. The gamma statistic only assumes that each belief measure is ordinal and its effects on behavior are monotonic. The correlation of the partial gammas with the reported ORs for the 16 beliefs in Table 4 is 0.93. The results then are not sensitive to the violation of the interval assumption.
The odds ratios may not satisfy equal interval assumptions typically expected for Pearson correlation estimates. To test sensitivity to this possible violation of the assumption, we also calculated the Pearson correlation using the corresponding logistic regression coefficients. That correlation was slightly higher: r = .56, p=.03. The Spearman rank-order correlation, ignores all information about the distance between cases, and is 0.34, p = .20, similar to the Pearson correlation when the outlier case is deleted (r=.34, p=.22). As can be seen in Figure 1, the distances between cases capture important information underlying this association.
Contributor Information
Robert C. Hornik, Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, rhornik@asc.upenn.edu, Phone: (215) 898-7057, 3620 Walnut Street, Philadelphia, PA 19104
Allyson C. Volinsky, Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA.
Shane Mannis, Fors Marsh Group, Arlington, VA
Laura Gibson, Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA
Emily Brennan, Centre for Behavioural Research in Cancer, Cancer Council Victoria, Melbourne, Victoria, Australia
Stella J. Lee, Department of Social and Behavioral Sciences, Harvard T.H. Chan School of Public Health, Boston, MA
Andy S. L. Tan, Population Sciences Division, Center for Community Based Research, Dana-Farber Cancer Institute, Boston, MA; Department of Social and Behavioral Sciences, Harvard T.H. Chan School of Public Health, Boston, MA
References
- Ajzen I, & Fishbein M (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice Hall, Inc. [Google Scholar]
- Botta R, Dunker K, Fenson-Hood K, Malatrich S, and McDonald L (2008). Using a relevant threat, EPPM and interpersonal communication to change hand-washing behaviours on campus. Journal of Communication in Healthcare, 1(4), 373–381. doi: 10.1179/cih.2008.1.4.373 [DOI] [Google Scholar]
- Brennan E, Gibson LA, Kybert-Momjian A, Liu J, & Hornik RC (2017). Promising themes for antismoking campaigns targeting youth and young adults. Tobacco Regulatory Science, 3(1), 29–46. doi: 10.18001/TRS.3.1.4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Caraballo RS, Giovino GA, & Pechacek TF (2004). Self-reported cigarette smoking vs. serum cotinine among U.S. adolescents. Nicotine & Tobacco Research, 6(1), 19–25. doi: 10.1080/14622200310001656821 [DOI] [PubMed] [Google Scholar]
- Davis JA (1967). A partial coefficient for Goodman and Kruskal’s gamma. Journal of the American Statistical Association, 62(317), 189–193. doi: 10.1080/01621459.1967.10482900 [DOI] [Google Scholar]
- Fishbein M, & Cappella J (2006). The role of theory in developing effective health communications. Journal of Communication, 56, S1–S17. doi: 10.1111/j.1460-2466.2006.00280.x [DOI] [Google Scholar]
- Fishbein M, Cappella J, Hornik R, Sayeed S, Yzer M, & Ahern RK (2002). The role of theory in developing effective anti-drug public service announcements In Crano WD & Burgoon M (Eds.), Mass media and drug prevention (pp. 89–117). Mahwah, NJ: Lawrence Erlbaum. [Google Scholar]
- Gibson LA, Parvanta SA, Jeong M, & Hornik RC (2014). Evaluation of a mass media campaign promoting using help to quit smoking. American Journal of Preventive Medicine, 46(5), 487–495. doi: 10.1016/j.amepre.2014.01.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hornik R, & Woolf KD (1999). Using cross-sectional surveys to plan message strategies. Social Marketing Quarterly, 5(34), 34–41. doi: 10.1080/15245004.1999.9961044 [DOI] [Google Scholar]
- Kenski K, Appleyard J, von Haeften I, Kasprzyk D, & Fishbein M (2001). Theoretical determinants of condom use intentions for vaginal sex with a regular partner among male and female injecting drug users. Psychology, Health & Medicine, 6(2), 179–190. doi: 10.1080/13548500123751 [DOI] [Google Scholar]
- Lee SJ, Brennan E, Gibson LA, Tan AS, Kybert-Momjian A, Liu J, & Hornik R (2016). Predictive validity of an empirical approach for selecting promising message topics: A randomized-controlled study. Journal of Communication, 66(3), 433–453. doi: 10.1111/jcom.12227 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maddock JE, Silbanuz A, Reger-Nash B (2008). Formative research to develop a mass media campaign to increase physical activity and nutrition in a multiethnic state. Journal of Communication, 13(3), 208–215. doi: 10.1080/10810730701807225 [DOI] [PubMed] [Google Scholar]
- Nowak GJ, & Siska MJ (1995). Using research to inform campaign design: Examples from the “America responds to AIDS” campaign In Maibach E, & Parrott RL (Eds.), Designing health messages: Approaches from communication theory and public health practice (pp. 169–185). Thousand Oaks, CA: Sage Publications Inc. [Google Scholar]
- Parvanta S, Gibson L, Forquer H, Shapiro-Luft D, Dean L, Freres D, … & Hornik R (2013). Applying quantitative approaches to the formative evaluation of antismoking campaign messages. Social Marketing Quarterly, 19(4), 242–264. doi: 10.1177/1524500413506004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parvanta S, Gibson L, Moldovan-Johnson M, Mallya G, & Hornik RC (2013). Race and gender moderation of the relationship between cessation beliefs and intentions: Is race or gender message segmentation necessary in anti-smoking campaigns? Health Education Research, 28(5), 857–868. doi: 10.1093/her/cyt067 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rosenstock IM (1974). Historical origins of the Health Belief Model. Health Education & Behavior, 2(4), 328–335. doi: 10.1177/109019817400200403 [DOI] [Google Scholar]
- Sangalang A, Volinsky AC, Yang Q, Liu J, Lee SJ, Gibson LA, & Hornik RC (2016). Identifying promising campaign themes to prevent youth initiation of electronic cigarette use. Philadelphia, PA: UPenn Tobacco Center for Regulatory Science, Annenberg School for Communication, University of Pennsylvania. [Google Scholar]
- Soulakova JN, Hartman AM, Liu B, Willis GB, & Augustine S (2012). Reliability of adult self-reported smoking history: Data from the Tobacco Use Supplement to the Current Population Survey 2002–2003 cohort. Nicotine & Tobacco Research, 14(8), 952–960. doi: 10.1093/ntr/ntr313 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vallone D, Cantrell J, Bennett M, Smith A, Rath J, Xiao H, Greenberg M & Hair EC (2017). Evidence of the impact of the truth FinishIt Campaign. Nicotine & Tobacco Research, 20(5), 543–551. doi: 10.1093/ntr/ntx119 [DOI] [PubMed] [Google Scholar]
- Weinstein ND (2007). Misleading tests of health behavior theories. Annals of Behavioral Medicine, 33(1), 1–10. doi: 10.1207/s15324796abm3301_1 [DOI] [PubMed] [Google Scholar]
- Witte K, Meyer G, & Martell D (2001). Effective health risk messages: A step-by-step guide. Thousand Oaks, CA: Sage Publications Inc. [Google Scholar]