Skip to main content
PMC home page
Preventive Medicine Reports logoLink to Preventive Medicine Reports
. 2021 Jun 2;23:101434. doi: 10.1016/j.pmedr.2021.101434

Message perceptions and effects perceptions as proxies for behavioral impact in the context of anti-smoking messages

Sabeeh A Baig a,⁎, Seth M Noar b,c, Nisha C Gottfredson a, Allison J Lazard b,c, Kurt M Ribisl a,c, Noel T Brewer a,c
PMCID: PMC8237599  PMID: 34194959

Highlights

  • •

    In the context of anti-smoking messages, effects perceptions (perceived behavioral impact) mediated message impact on quit intentions and six quitting and related behaviors while message perceptions (persuasive potential) did not.

  • •

    The sizes of indirect effects involving effects perceptions ranged from small to medium.

  • •

    These findings provide support for the diagnostic value of effects perceptions in testing messages for behavior change in the context of tobacco use and, possibly, other risky health behaviors.

  • •

    The use of effects perceptions in formative research may increase the efficiency with which behavior change interventions can be developed and optimized under resource constraints.

Keywords: Effects perceptions, Formative research, Perceived message effectiveness, Multiple mediation, Structural equation modeling

Abstract

Researchers commonly use message perceptions (persuasive potential) or effects perceptions (perceived behavioral impact) in formative research to select tobacco risk messages. We sought to identify whether message perceptions or effects perceptions are more useful as proxies for the behavioral impact of tobacco risk messages. In a three-week trial, 703 U.S. adult smokers (ages ≥ 21) were randomly assigned to receive brief messages on their cigarette packs about toxic chemicals in cigarette smoke (chemical messages) or control messages about properly disposing of cigarette litter. The final follow-up survey assessed message perceptions, effects perceptions, quit intentions, and six behavioral outcomes. We conducted multiple mediation analysis in a structural equation modeling framework to test the indirect effects of messages by way of message perceptions and effects perceptions. Message perceptions did not independently mediate the impact of chemical messages on any of the outcomes (7 p-values ≥ 0.01). In contrast, effects perceptions mediated the impact of chemical messages on avoiding the messages, seeking chemical information, intentions to quit smoking, butting out a cigarette, forgoing a cigarette, and making a quit attempt (6 p-values ≤ 0.001). No mediation was present for social interactions about the message (p-value = 0.72). The effect sizes for these mediated effects were small to medium. Thus, effects perceptions, but not message perceptions, were a proxy for risk messages’ impact on quit intentions and six quitting and related behaviors. These findings point to the diagnostic value of effects perceptions in formative research on tobacco risk messages.

Health messages that aim to change behavior are iteratively developed through multiple rounds of testing, beginning with a large of pool of candidate messages. It is usually unfeasible and inefficient to evaluate many candidate messages based on changes in behavioral outcomes, such as vaccination or quitting smoking, that may actually occur over a period of weeks or months. Thus, interventionists have used audience ratings of candidate messages’ perceived effectiveness (perceived message effectiveness) to identify the most promising messages for further testing in a behavioral trial. Perceived message effectiveness (PME) measures have traditionally assessed either message perceptions or effects perceptions, occasionally combining items for both constructs within a single scale (Dillard and Ye, 2008, Noar et al., 2018b).

Message perceptions assess the persuasive potential of candidate messages while effects perceptions assess the potential of candidate messages to change behavioral antecedents and behavior. An example dimension of message perceptions is credibility (e.g., “How believable was the message in this ad?”; Donovan et al., 2006) while an example dimension of effects perceptions is perceived impact on behavioral motivation (e.g., “This ad makes me want to quit smoking”; Niederdeppe et al., 2011). A growing literature has begun to recommend the use of effects perceptions over message perceptions when developing messages that seek to change behavior (Baig et al., 2021, Rohde et al., 2020). Effects perceptions are conceptually closer to behavior, and effects perceptions items generally use behavioral and personal referents to enhance correspondence with behavior (Noar et al., 2018a). Moreover, when examined simultaneously, effects perceptions are more strongly associated with behavior as well as determinants of message impact that are closer to behavior (e.g., quit intentions) while message perceptions are more strongly associated with only some of the earliest determinants (e.g., attention to the message; Baig et al., 2021, Noar et al., 2020). Finally, effects perceptions scales may be less prone to capturing information about secondary constructs than message perceptions scales (i.e., message perceptions scales may have greater measurement error; Baig et al., 2018).

While the aforementioned observations support the use of effects perceptions in formative research, many researchers have expressed caution about the diagnostic value of PME ratings in identifying messages that are actually effective often due to unsatisfactory validity (Dillard et al., 2007, O’Keefe, 2018). The small number of PME validation studies in behavioral contexts have focused mostly on PME correlations or associations with outcomes of interest such as intentions to quit cigarette smoking (Lee et al., 2011) and seeking information about colonoscopy (Dillard & Ha, 2016), occasionally in longitudinal data (Brennan et al., 2013, Davis et al., 2013, Davis et al., 2016, Dillard and Ha, 2016). A disadvantage of focusing on PME correlations or associations is that there is no direct consideration of the sensitivity of PME to candidate messages. When focusing on bivariate associations and correlations, another disadvantage is that such efforts do not inherently compare different PME constructs. A more informative way to evaluate the diagnostic value of PME ratings is to examine whether they mediate message impact on behavioral or other meaningful outcomes. These validation efforts require a study design with control messages that are expected to be weaker than candidate messages and decompose the impact of candidate messages as a function of PME thereby determining the extent to which PME serves as proxy for message impact. These validation efforts also permit the inclusion of multiple PME constructs as simultaneous mediators in a single model, allowing the constructs to compete against each other for which is the better marker of behavioral impact.

The current study sought to compare the diagnostic value of PME ratings using brief messages about the chemicals in cigarette smoke designed to discourage smoking (chemical messages) and control messages designed to discourage littering of cigarette butts (littering messages) as a case study. The main hypothesis guiding our study was that effects perceptions are a better proxy for behavioral impact than message perceptions (behavioral proxy hypothesis). Briefly viewed messages often do not activate central pathways to persuasion (Bodie et al., 2012, Evans, 2008) and instead elicit affective reactions that are informative to the viewer (Cesario et al., 2008, Slovic et al., 2007, So et al., 2015). Some of the stated dimensions of effects perceptions (e.g., perceived message impact on worry) overlap with these affective reactions whereas the stated dimensions of message perceptions (e.g., message credibility) do not. Thus, we expected effects perceptions to mediate message impact and message perceptions to not mediate impact singularly or sequentially through effects perceptions.

In examining message perceptions and effects perceptions as potential proxies for the behavioral impact of chemical messages, our study sheds some light on the relationship between PME and behavior. Doing so is essential to transforming PME from a criterion variable that is useful for message development into a substantive variable potentially with a defined role in message processing. Given limited guidelines on the use of PME in formative research, our study also provides suggestions for the optimal use of PME in message development.

1. Methods

1.1. Participants and procedures

A randomized clinical trial (RCT) testing the ability of chemical messages and littering messages to both inform smokers about the health risks of smoking and motivate them to quit recruited a convenience sample of 719 U.S. adult smokers from the San Francisco Bay Area. Participants were ages 21 or older (given the legal age of buying tobacco products in California) and current smokers (i.e., had smoked at least 100 cigarettes in their lifetime and now smoke every or some days; Nguyen et al., 2015). Smokers who were enrolled in an ongoing smoking cessation trial, smoked less than seven cigarettes weekly or only roll-your-own cigarettes, or were pregnant were ineligible to participate in the RCT.

Participants attended four visits at the trial office and brought an eight-day supply of cigarettes to all but the last visit. The trial randomized smokers to receive chemical messages (intervention) or messages about properly disposing of cigarette butts (littering messages; control) in nearly equal numbers. An example of a chemical message is, “Cigarette smoke contains uranium. This causes lung tumors and kidney damage.” “Please refrain from littering. Cigarette butts are the most littered item,” is an example of a littering message. Littering messages were attention-matched to the chemical messages and had similar word lengths, literacy requirements, and visual features (Fig. 1). Participants in both arms completed surveys at each trial visit and rotated through three unique messages, one per week. After a run-in week, study staff applied a different message at the next three weekly visits on the side of participants’ cigarette packs while the participants completed a survey. Smokers received up to $300 for their trial participation. Data collection began in September 2016 and finished in March 2017. Full details on design and protocol, including recruitment and randomization, are available in the main trial paper (Brewer et al., 2018a). Additional details on the development of the trial protocol and two pilot studies can be found in a separate paper (Brewer et al., 2015). The Institutional Review Board at the University of North Carolina approved trial procedures.

Fig. 1.

Fig. 1

Open in a new tab

Messages placed on smokers’ cigarette packs in the intervention (A) and control (B) arms.

1.2. Measures

The baseline survey assessed participant demographics and smoking behavior. At the last visit, the survey assessed message perceptions using an established six-item scale (α = 0.94; Davis et al., 2013) and effects perceptions using the three-item UNC Perceived Message Effectiveness Scale (α = 0.90; Baig et al., 2018). The message perceptions scale had the following items: “This message is worth remembering”; “This message grabbed my attention”; “This message is powerful”; “This message is informative”; “This message is meaningful”; and “This message is convincing,” (Davis et al., 2013). The effects perceptions scale had the following items: “This message discourages me from wanting to smoke”; “This message makes smoking seem unpleasant to me”; and “This message makes me concerned about the health effects of smoking,” (Baig et al., 2018). Both measures used a five-point response scale ranging from “strongly disagree” (coded as 1) to “strongly agree” (5).

As tobacco warnings can have a range of effects (Noar et al., 2016, Brewer et al., 2018a), the final visit survey also assessed quit intentions and six behavioral outcomes: number of conversations about the messages in the past week, avoidance of cigarette pack messages, seeking information about the chemicals in cigarette smoke, number of times butting out a cigarette in the past week, number of times forgoing a cigarette in the past week, and weekly recall of quit attempts. Forgoing refers to not smoking a cigarette in a particular moment because of wanting to smoke less while “butting out” refers to beginning to smoke a cigarette and then putting it out before finishing it also because of wanting to smoke less. Social interactions about the messages (Morgan et al., 2017), avoidance of the messages (Hall et al., 2018), and seeking information about chemicals (Lambert & Loiselle, 2007) are behaviors that are potentially productive for quitting. Avoidance is largely driven by affective responses that are predictive of quit intentions and quitting behavior as opposed to message reactance and, as such, signifies deeper processing of messages (Hall et al., 2018). Quitting initiation exists on a spectrum (Partos et al., 2014), and forgoing or butting out a cigarette represent micro-quitting behaviors that are less intense than quit attempts. Each of the seven outcomes either appears on the UNC Tobacco Warnings Model (e.g., social interactions), which succinctly describes how tobacco warnings impact behavior (Brewer et al., 2018b), or has some correspondence with a behavioral antecedent in the model (e.g., avoidance with negative affect), further underscoring their importance. The survey assessed avoidance (α = 0.89) and quit intentions (α = 0.96) using separate three-item scales with desirable psychometric properties. The other five outcomes were assessed using single-item measures (Table 1).

Table 1.

Measures for all outcomes in multiple mediation analyses with ordinal reliability coefficients for multi-item scales.

Construct, measure(s) Response Reference
Effects perceptions (α = 0.89) Strongly disagree (1) Baig et al., 2018
 This message discourages me from wanting to smoke. to
 This message makes smoking seem unpleasant to me. Strongly agree (5)
 This message makes me concerned about the health effects of smoking.



Message perceptions (α = 0.94) Strongly disagree (1) Davis et al., 2013
 This message is worth remembering. to
 This message grabbed my attention. Strongly agree (5)
 This message is powerful.
 This message is informative.
 This message is meaningful.
 This message is convincing.



Number of conversations about the message in the past week 0–100 times –
 In the last week, how many times did you talk to other people about the label on your cigarette packs?



Avoidance of cigarette pack messages (α = 0.89) Never (1) Hyland et al., 2016
 How often did you try to avoid thinking about the labels on your cigarette packs? to
 How often did you try to avoid looking at the labels on your cigarette packs? All of the time (5)
 How often did you put your cigarettes away because you didn’t want others to see the labels on your packs?



Seeking information about the chemicals in cigarette smoke 0 times (1) Nelson et al., 2004
 In the last 3 weeks, how many times have you looked for information about the chemicals in cigarettes or cigarette smoke? to
6 or more times (4)



Quit intentions (α = 0.96) Not at all (1) Klein, Zajac, & Monin, 2009
 How interested are you in quitting smoking in the next month? to
 How much do you plan to quit smoking in the next month? Very (4)
 How likely are you to quit smoking in the next month?



Number of times butting out a cigarette in the past week Never (1) Li et al., 2014
 In the last week, how often have you butted out a cigarette before you finished it because you wanted to smoke less? to
10 or more times (5)



Number of times forgoing a cigarette in the past week Never (1) Li et al., 2014
 In the last week, how often have you stopped yourself from having a cigarette because you wanted to smoke less? to
10 or more times (5)



Weekly recall of quit attempts Yes (1) CDC, 2012
 In the last week, did you stop smoking for 1 day or longer because you were trying to quit smoking? No (0)
Open in a new tab

1.3. Statistical analysis

Analyses used R (ver. 3.5.1; Team, 2000) and the add-on packages lavaan (ver. 0.6–2; Rosseel, 2012) for estimating all path models, psych (ver. 1.8.4; Revelle, 2017) for calculating ordinal reliability coefficients, nnet (ver. 7.3–12; Ripley & Venables, 2016) for estimating multinomial logistic regression models for randomization checks, and ggplot2 (ver. 3.0.0; Wickham, 2016) for creating figures.

We conducted multiple mediation analysis in a structural equation modeling (SEM) framework, estimating SEM models using mean- and variance-adjusted weighted least squares (WLSMV). Doing so allowed us to use pairwise deletion to handle missingness while treating all ordinal variables (i.e., those assessed with Likert-type scales) as ordinal in SEM models. Single-item outcomes did not have any missingness, and partial missingness on multi-item outcomes was negligible (<3%). However, we dropped 16 cases with complete missingness on both mediators, message perceptions and effects perceptions, yielding an analytic sample of 703 smokers.

The measurement model specified separate factors for message perceptions and effects perceptions (Fig. 2). The measurement model also accounted for local dependence between four message perceptions items through pairwise correlated errors and likewise on one pair of effects perceptions items. Baig et al. (2018) found the corresponding confirmatory factor analytic model to adequately capture the bidimensionality of message perceptions and effects perceptions. We specified a single-item outcome as a manifest variable and a multi-item outcome as a latent variable.

Fig. 2.

Fig. 2

Open in a new tab

Path diagram for structural equation model with mediated effects of chemical messages on a given outcome through message perceptions alone (a × b), message perceptions and effects perceptions sequentially (a × c × e), and effects perceptions alone (d × e). Measurement model and additional paths included for completeness appear in gray.

For a given outcome (e.g., quit intentions), the structural model simultaneously regressed the outcome onto message perceptions (b) and effects perceptions (e; Fig. 2). The structural model also regressed message perceptions and effects perceptions onto trial condition (a and d) as well as regressing effects perceptions onto message perceptions (c). Following standard procedures (Hayes, 2017), we calculated three indirect effects of chemical messages on the outcome as the products of relevant path coefficients: through message perceptions alone (a × b), through message perceptions and effect perceptions sequentially (a × c × e), and through effects perceptions alone (d × e). We used model-based z-tests to examine the statistical significance of the indirect effects. We report path coefficients and indirect effects that were partially standardized based on the variances of endogenous manifest and latent variables and not the variance of trial condition, the exogenous binary covariate. In addition, we controlled for trial condition; the corresponding main effects are reported in the trial (Brewer et al., 2018a) and other papers (Hall et al., 2018, Hall et al., 2017). Trial condition was not associated with standard demographics confirming that randomization succeeded in creating balanced conditions (26 ps ≥ 0.06).

We confirmed SEM model fit using the Bayesian Information Criterion (BIC), Comparative Fit Index (CFI), and Root Mean Squared Error of Approximation (RMSEA). A negative BIC (Bollen et al., 2012, Jarosz and Wiley, 2014), large CFI (≥0.95), and small RMSEA (≤0.06) indicated adequate global fit. We also calculated the effect size υ for the three indirect effects (Lachowicz et al., 2018). This is a novel effect size measure for mediation that represents the variance in the outcome jointly accounted for by the independent variable and mediator(s) after correcting “for spurious correlation induced by the ordering of the variables,” (Lachowicz et al., 2018). Given its close relationship to R2Med, the usual verbal categories facilitated interpretation: small (0.02), medium (0.15), and large (0.35) (Cohen, 1992). We used this approach to multiple mediation analysis for all seven outcomes. The lack of appropriate temporal ordering between the mediators and outcomes due to their assessment at a single time point meant that mediated effects could be artificially inflated. Thus, we used a critical alpha of 0.01 (or a 99% confidence interval) to provide for conservative tests of mediation.

2. Results

More than a third of participants in the control and intervention arms were white (≥37%; Table 2). Likewise, more than a third of participants in both arms were African-American (≥34%). A minority of participants in both arms had at least a college degree (≥39%). All SEM models had adequate fit as indicated by the negative BICs (range −204 to −118), small RMSEAs (0.05 to.06), and large CFIs (0.99 to 1.00).

Table 2.

Participant characteristics.

Intervention
 Control
n = 353
 n = 350
% %
Age, years
 21–29 22.4 22.9
 30–39 21.2 22.9
 40–49 21.2 16.6
 50–59 25.8 25.4
 60+ 9.4 12.3



Gender
 Male 54.1 47.4
 Female 42.2 47.4
 Transgender (includes other gender identity) 3.7 5.2



Gay, lesbian or bisexual 25.8 24.9



Race
 White 37.1 37.7
 Black or African-American 36.8 34.3
 Asian 8.5 8.3
 American Indian or Alaska Native 4.8 4.9
 Native Hawaiian or other Pacific Islander 3.4 3.1
 Other 9.4 11.7



Hispanic 11.9 15.7



Education
 High school graduate or less 24.6 18.9
 Some college 34.0 41.7
 Bachelor’s degree 34.3 32.0
 Graduate degree 7.1 7.4



Household income, annual
 $0–$24,999 46.7 43.7
 $25,000–$49,999 24.9 23.1
 $50,000–$74,999 9.9 14.9
 $75,000+ 18.4 18.3



Low income, < 200% of federal poverty level 59.8 55.4



Smoking frequency
 Daily 21.8 24.0
 Non-daily 78.2 76.0
Open in a new tab

2.1. Mediation by message perceptions

Chemical messages did not affect any of the outcomes through the potential mediator message perceptions (a × b in Fig. 1). With respect to the predictor-mediator pathway (a), chemical messages did not elicit message perceptions different from littering messages (p = .08). With respect to the mediator-outcome pathways (b), more positive message perceptions were associated with weaker avoidance (β = −0.28, p = .006) and being less likely to engage in a quit attempt (β = −0.38, p = .003). However, message perceptions were not associated with conversations about the messages (p = .36), seeking information about chemicals (p = .55), quit intentions (p = .01), butting out a cigarette (p = .91), or forgoing a cigarette (p = .21). As a result, none of the indirect effects of chemical messages through message perceptions on these seven outcomes were statistically significant (7 ps ≥ 0.11; Fig. 3). Furthermore, chemical messages explained very small amounts of variance on average in the seven outcomes through message perceptions (υ median [range] = 0.002 [0.00, 0.003]).

Fig. 3.

Fig. 3

Open in a new tab

Standardized indirect effects of chemical messages through message perceptions and effects perceptions with 99% confidence intervals.

2.2. Sequential mediation by message perceptions through effects perceptions

Chemical messages did not affect any of the outcomes sequentially through the potential mediators message perceptions and then effects perceptions (a × c × e in Fig. 1). As already reported for the (first) predictor-mediator pathway (a), chemical and littering messages did not differ on message perceptions. With respect to the mediator-mediator pathway (c), more positive message perceptions were associated with more positive effects perceptions (β = 0.81, p ≤ 0.001). With respect to the final mediator-outcome pathways (e), more positive effects perceptions were associated with stronger avoidance (β = 0.79, p ≤ 0.001), more frequent seeking information about chemicals (β = 0.45, p ≤ 0.001), stronger quit intentions (β = 0.91, p ≤ 0.001), more frequent butting out a cigarette (β = 0.36, p ≤ 0.001), more frequent forgoing a cigarette (β = 0.57, p ≤ 0.001), and being more likely to engage in a quit attempts (β = 0.72, p ≤ 0.001). Effects perceptions were not associated with conversations about the messages (p = .72). Looking at the complete mediational pathway, none of the sequential indirect effects of chemical messages through message perceptions and effects perceptions on these seven outcomes were statistically significant (7 ps ≥ 0.09; Fig. 3). Additionally, chemical messages explained very small amounts of variance in the seven outcomes sequentially through message perceptions and effects perceptions (υ median [range] = 0.004 [0.00, 0.010]).

2.3. Mediation by effects perceptions

Chemical messages affected six of the seven outcomes through effects perceptions. With respect to the predictor-mediator pathway (d), chemical messages elicited more positive effects perceptions than littering messages (β = 0.26, p ≤ 0.001). As reported previously for the mediator-outcome pathways (e), effects perceptions were positively associated with all outcomes except conversations about the messages. As a result, effects perceptions mediated the impact of chemical messages on avoidance (β = 0.20, p = .001), seeking information about chemicals (β = 0.11, p = .008), quit intentions (β = 0.23, p = .002), butting out a cigarette (β = 0.09, p = .004), forgoing a cigarette (β = 0.14, p = .002), and quit attempts (β = 0.18, p = .003; Fig. 3). Effects perceptions did not mediate the impact of chemical messages on conversations about the messages (p = .72). Chemical messages explained small to medium amounts of variance in the seven outcomes through effects perceptions (υ median [range] = 0.02 [0.00, 0.053]).

3. Discussion

In a diverse sample of adult smokers, message perceptions did not independently mediate the impact of chemical messages on three behaviors that may play a role in quitting initiation, a motivator of quitting, and three quitting behaviors that varied in intensity. Message perceptions also did not sequentially mediate chemical message impact on these seven outcomes through effects perceptions. On the other hand, excluding conversations about the messages, effects perceptions mediated the impact of chemical messages on the six remaining outcomes. Therefore, effects perceptions exclusively functioned as a proxy for chemical message impact on six outcomes that included various informative and quitting behaviors, supporting our behavioral proxy hypothesis. Formative research should prioritize effects perceptions for identifying promising messages for behavior change.

An open question is whether effects perceptions are a true mediator of message impact. In this regard, effects perceptions may actually be a proxy for an orientation among audience members to sustained engagement with a message in ways that lead to behavior change (as opposed to persuasion in the case of message perceptions). This orientation may come about via affective reactions to messages that inform the viewers’ receptivity to the messages. According to regulatory fit theory, “feeling right” about a message or message receptivity may induce an orientation to sustained message engagement that is productive in a given context (Cesario et al., 2008, Cesario et al., 2004). The interpretation of effects perceptions as a proxy for this orientation finds support in our mediational findings. Additional support comes via findings from our cross-sectional validation study that effects perceptions were a better predictor of behavior (Baig et al., 2021). The same study also found that effects perceptions were more strongly associated with later behavioral antecedents indicative of sustained message engagement (Baig et al., 2021). Future studies should formally examine the relationship between regulatory fit and effects perceptions as a first step to understanding the hypothesized orientation.

Unlike effects perceptions, message perceptions did not mediate the impact of chemical messages on any of the outcomes with the corresponding effect sizes being very small. Thus, message perceptions were minimally diagnostic of a message’s potential for behavior change. A possible explanation for the lack of mediation is that chemical and littering messages did not sufficiently vary on message perceptions. Indeed, multiple rounds of message testing systematically excluded weaker messages yielding highly potent trial messages (Baig et al., 2016, Noar et al., 2018c). However, chemical messages did elicit more positive effects perceptions than littering messages even though both types of messages were attention-matched (Brewer et al., 2018a). As such, an alternative explanation for the lack of mediation by message perceptions is that they are not sensitive to messages with similar design features (e.g., word length). Future studies should attempt to replicate our findings using a diverse pool of candidate messages. Doing so will shed further light on whether there is any value to assessing message perceptions in formative research.

Our study provides support for the diagnostic value of effects perceptions in message testing. Chemical messages (i.e., the intervention) and effects perceptions (i.e., the mediator) jointly accounted for meaningful amounts of variance in avoidance of the messages, chemical information seeking, quit intentions, butting out a cigarette, forgoing a cigarette, and quit attempts. Given that the corresponding effect sizes were small to medium, a more cautious view is that there are considerable amounts of variance in these outcomes that remain unexplained. As such, the extent to which effects perceptions capture a message’s potential to change behavior also remains uncertain. Such a view ignores the stated purpose of using effects perceptions (or PME in general), which is to identify potent messages for further testing in a behavioral trial or similar study. In other words, message testing with effects perceptions is not a substitute for studying behavioral impact, but a means to making the latter more efficient. This efficiency is crucial as the development of brief behavioral interventions often begins with a large pool of candidate messages while being subject to constraints on research resources. The use of effects perceptions would increase the specificity of formative research to identify potent messages for behavior change and facilitate the optimization of a proposed intervention under resource constraints. Our findings generally support the use of effects perceptions in this capacity.

Strengths of our study include a diverse sample of smokers, randomization to trial condition, and structural equation models that uniquely estimated mediated effects. Focusing on a variety of outcomes provided ample opportunities to replicate our basic mediational findings. A limitation of our study is the assessment of mediators and outcomes at the last trial visit, which made it more difficult to confirm the temporal ordering of effects. However, similar mediational patterns across six outcomes weaken the possibility of reverse causation. The assessment of mediators and outcomes at a single time point may have also inflated their covariances contributing to potentially spurious indirect effects. Using a more conservative critical alpha of 0.01 (or 99% confidence intervals) and effect sizes to contextualize indirect effects may have mitigated this possibility. Future studies should replicate our findings using a longitudinal design with explicit temporal ordering of mediators and outcomes. Our study also did not sufficiently capture the heterogeneity in message testing scenarios, whether from candidate messages or PME instruments. Future studies should also replicate our findings in other behavioral contexts such as alcohol consumption and e-cigarette use with different message perceptions and effects perceptions scales to optimize the applicability of PME in formative research across behavioral contexts.

4. Conclusion

Effects perceptions mediated the impact of chemical messages on behavior, pointing to their diagnostic value in formative research on health messages. Effects perceptions may represent an orientation among message recipients to further message engagement that is productive for behavior change. Thus, formative assessments of effects perceptions are practically useful for message development and may contribute to our understanding of message processing. The value of message perceptions above and beyond effects perceptions remains to be established.

CRediT authorship contribution statement

Sabeeh A. Baig: Conceptualization, Methodology, Validation, Formal analysis, Data curation, Writing - original draft, Visualization, Project administration. Seth M. Noar: Conceptualization, Methodology, Validation, Investigation. Nisha C. Gottfredson: Conceptualization, Methodology, Validation. Allison J. Lazard: Conceptualization, Methodology, Validation, Investigation. Kurt M. Ribisl: Conceptualization, Methodology, Validation, Investigation, Funding acquisition. Noel T. Brewer: Conceptualization, Methodology, Validation, Investigation, Resources, Funding acquisition.

Acknowledgments

Acknowledgements

Research reported in this publication was supported by Grant Number P50CA180907 from the National Cancer Institute and FDA Center for Tobacco Products. Research reported in this publication was also partially supported by the National Institute on Drug Abuse through grant funding awarded to Dr. Gottfredson (K01DA0351523). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Food and Drug Administration.

Declaration of Competing Interest

None of the authors have received funding from tobacco product manufacturers. Dr. Brewer has served as a paid consultant in litigation against tobacco companies. Dr. Ribisl has served as an expert consultant in litigation against cigarette manufacturers and Internet tobacco vendors. Dr. Noar has served as a paid expert witness in government litigation against tobacco companies. The views expressed in this paper are his and not those of the FDA. The authors declare no conflict of interest.

Contributor Information

Sabeeh A. Baig, Email: sbaig1@alumni.jh.edu.

Seth M. Noar, Email: noar@unc.edu.

Nisha C. Gottfredson, Email: gottfredson@unc.edu.

Allison J. Lazard, Email: lazard@unc.edu.

Kurt M. Ribisl, Email: kurt_ribisl@live.unc.edu.

Noel T. Brewer, Email: ntb@unc.edu.

References

  1. Baig S.A., Byron M.J., Boynton M.H., Brewer N.T., Ribisl K.M. Communicating about cigarette smoke constituents: An experimental comparison of two messaging strategies. J. Behav. Med. 2016;40(2):352–359. doi: 10.1007/s10865-016-9795-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Baig S.A., Noar S.M., Gottfredson N.C., Boynton M.H., Ribisl K.M., Brewer N.T. UNC perceived message effectiveness: Validation of a brief scale. Ann. Behav. Med. 2018;53(8):732–742. doi: 10.1093/abm/kay080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baig S.A., Noar S.M., Gottfredson N.C., Lazard A.J., Ribisl K.M., Brewer N.T. Incremental criterion validity of message perceptions and effects perceptions in the context of anti-smoking messages. J. Behav. Med. 2021;44(1):74–83. doi: 10.1007/s10865-020-00163-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bodie G.D., Burleson B.R., Jones S.M. Explaining the relationships among supportive message quality, evaluations, and outcomes: A dual-process approach. Commun. Monogr. 2012;79(1):1–22. doi: 10.1080/03637751.2011.646491. [DOI] [Google Scholar]
  5. Bollen K.A., Ray S., Zavisca J., Harden J.J. A comparison of Bayes factor approximation methods including two new methods. Sociol. Methods Res. 2012;41(2):294–324. doi: 10.1177/0049124112452393. [DOI] [Google Scholar]
  6. Brennan E., Durkin S.J., Wakefield M.A., Kashima Y. Assessing the effectiveness of antismoking television advertisements: do audience ratings of perceived effectiveness predict changes in quitting intentions and smoking behaviours? Tobacco Control. 2013;23(5):412–418. doi: 10.1136/tobaccocontrol-2012-050949. [DOI] [PubMed] [Google Scholar]
  7. Brewer, N. T., Jeong, M., Mendel, J. R., Hall, M. G., Zhang, D., Parada Jr, H., Boynton, M. H., Noar, S. M., Baig, S. A., Morgan, J. C., & al., et. (2018). Cigarette pack messages about toxic chemicals: A randomised clinical trial. Tobacco Control, tobaccocontrol-2017-054112. DOI:10.1136/tobaccocontrol-2017-054112. [DOI] [PMC free article] [PubMed]
  8. Brewer N.T., Hall M.G., Lee J.G.L., Peebles K., Noar S.M., Ribisl K.M. Testing warning messages on smokers’ cigarette packages: A standardised protocol. Tobacco Control. 2015;25(2):153–159. doi: 10.1136/tobaccocontrol-2014-051661. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Brewer N.T., Parada H., Hall M.G., Boynton M.H., Noar S.M., Ribisl K.M. Understanding why pictorial cigarette pack warnings increase quit attempts. Ann. Behav. Med. 2018;53(3):232–243. doi: 10.1093/abm/kay032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. CDC, 2012. Current cigarette smoking among adults-united states, 2011. MMWR. Morbidity and Mortality Weekly Report, 61(44), 889. [PubMed]
  11. Cesario J., Grant H., Higgins E.T. Regulatory fit and persuasion: Transfer from “feeling right”. J. Pers. Soc. Psychol. 2004;86(3):388–404. doi: 10.1037/0022-3514.86.3.388. [DOI] [PubMed] [Google Scholar]
  12. Cesario J., Higgins E.T., Scholer A.A. Regulatory fit and persuasion: Basic principles and remaining questions. Soc. Pers. Psychol. Compass. 2008;2(1):444–463. doi: 10.1111/j.1751-9004.2007.00055.x. [DOI] [Google Scholar]
  13. Cohen J. A power primer. Psychol. Bull. 1992;112(1):155–159. doi: 10.1037//0033-2909.112.1.155. [DOI] [PubMed] [Google Scholar]
  14. Davis K.C., Nonnemaker J., Duke J., Farrelly M.C. Perceived effectiveness of cessation advertisements: The importance of audience reactions and practical implications for media campaign planning. Health Commun. 2013;28(5):461–472. doi: 10.1080/10410236.2012.696535. [DOI] [PubMed] [Google Scholar]
  15. Davis K.C., Duke J., Shafer P., Patel D., Rodes R., Beistle D. Perceived effectiveness of antismoking ads and association with quit attempts among smokers: Evidence from the tips from former smokers campaign. Health Commun. 2016;32(8):931–938. doi: 10.1080/10410236.2016.1196413. [DOI] [PubMed] [Google Scholar]
  16. Dillard J.P., Ha Y. Perceived message effectiveness predicts information-seeking behavior. Commun. Res. Rep. 2016;33(4):356–362. doi: 10.1080/08824096.2016.1224173. [DOI] [Google Scholar]
  17. Dillard J.P., Ye S. The perceived effectiveness of persuasive messages: Questions of structure, referent, and bias. J. Health Commun. 2008;13(2):149–168. doi: 10.1080/10810730701854060. [DOI] [PubMed] [Google Scholar]
  18. Dillard J.P., Weber K.M., Vail R.G. The relationship between the perceived and actual effectiveness of persuasive messages: A meta-analysis with implications for formative campaign research. J. Commun. 2007;57(4):613–631. doi: 10.1111/j.1460-2466.2007.00360.x. [DOI] [Google Scholar]
  19. Donovan R.J., Jalleh G., Carter O.B.J. Tobacco industry smoking prevention advertisements’ impact on youth motivation for smoking in the future. Social Mark. Q. 2006;12(2):3–13. [Google Scholar]
  20. Evans J.S.B.T. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 2008;59(1):255–278. doi: 10.1146/annurev.psych.59.103006.093629. [DOI] [PubMed] [Google Scholar]
  21. Hall, M. G., Sheeran, P., Noar, S. M., Boynton, M. H., Ribisl, K. M., Parada Jr., H., Johnson, T. O., & Brewer, N. T. (2017). Negative affect, message reactance and perceived risk: How do pictorial cigarette pack warnings change quit intentions? Tobacco Control, 27(e2), e136–e142. DOI:10.1136/tobaccocontrol-2017-053972. [DOI] [PMC free article] [PubMed]
  22. Hall M.G., Mendel J.R., Noar S.M., Brewer N.T. Why smokers avoid cigarette pack risk messages: Two randomized clinical trials in the united states. Social Sci. Med. 2018;213:165–172. doi: 10.1016/j.socscimed.2018.07.049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hayes A.F. Guilford Publications; 2017. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. [Google Scholar]
  24. Hyland A., Ambrose B.K., Conway K.P., Borek N., Lambert E., Carusi C., Taylor K., Crosse S., Fong G.T., Cummings K.M. Design and methods of the Population Assessment of Tobacco and Health (PATH) study. Tobacco Control. 2016;26(4):371–378. doi: 10.1136/tobaccocontrol-2016-052934. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Jarosz A.F., Wiley J. What are the odds? A practical guide to computing and reporting Bayes factors. J. Probl. Solving. 2014;7(1) doi: 10.7771/1932-6246.1167. [DOI] [Google Scholar]
  26. Klein W.M.P., Zajac L.E., Monin M.M. Worry as a moderator of the association between risk perceptions and quitting intentions in young adult and adult smokers. Ann. Behav. Med. 2009;38(3):256–261. doi: 10.1007/s12160-009-9143-2. [DOI] [PubMed] [Google Scholar]
  27. Lachowicz M.J., Preacher K.J., Kelley K. A novel measure of effect size for mediation analysis. Psychol. Methods. 2018;23(2):244–261. doi: 10.1037/met0000165. [DOI] [PubMed] [Google Scholar]
  28. Lambert S.D., Loiselle C.G. Health information-seeking behavior. Qual. Health Res. 2007;17(8):1006–1019. doi: 10.1177/1049732307305199. [DOI] [PubMed] [Google Scholar]
  29. Lee S., Cappella J.N., Lerman C., Strasser A.A. Smoking cues, argument strength, and perceived effectiveness of antismoking PSAs. Nicotine Tob. Res. 2011;13(4):282–290. doi: 10.1093/ntr/ntq255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Li L., Borland R., Fong G.T., Jiang Y., Yang Y., Wang L., Partos T.R., Thrasher J.F. Smoking-related thoughts and microbehaviours, and their predictive power for quitting. Tobacco Control. 2014;24(4):354–361. doi: 10.1136/tobaccocontrol-2013-051384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Morgan J.C., Southwell B.G., Noar S.M., Ribisl K.M., Golden S.D., Brewer N.T. Frequency and content of conversations about pictorial warnings on cigarette packs. Nicotine Tob. Res. 2017;20(7):882–887. doi: 10.1093/ntr/ntx180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Nelson D., Kreps G., Hesse B., Croyle R., Willis G., Arora N., Rimer B., Viswanath K.V., Weinstein N., Alden S. The Health Information National Trends Survey (HINTS): development, design, and dissemination. J. Health Commun. 2004;9(5):443–460. doi: 10.1080/10810730490504233. [DOI] [PubMed] [Google Scholar]
  33. Nguyen K., Marshall L., Hu S., Neff L. State-specific prevalence of current cigarette smoking and smokeless tobacco use among adults aged≥ 18 years—United States, 2011–2013. MMWR Morb. Mortal. Wkly Rep. 2015;64(19):532. [PMC free article] [PubMed] [Google Scholar]
  34. Niederdeppe J., Farrelly M.C., Nonnemaker J., Davis K.C., Wagner L. Socioeconomic variation in recall and perceived effectiveness of campaign advertisements to promote smoking cessation. Soc. Sci. Med. 2011;72(5):773–780. doi: 10.1016/j.socscimed.2010.12.025. [DOI] [PubMed] [Google Scholar]
  35. Noar S.M., Barker J., Yzer M. Measurement and design heterogeneity in perceived message effectiveness studies: a call for research. J. Commun. 2018;68(5):990–993. doi: 10.1093/joc/jqy047. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Noar S.M., Bell T., Kelley D., Barker J., Yzer M. Perceived message effectiveness measures in tobacco education campaigns: a systematic review. Commun. Methods Meas. 2018;12(4):295–313. doi: 10.1080/19312458.2018.1483017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Noar S.M., Hall M.G., Francis D.B., Ribisl K.M., Pepper J.K., Brewer N.T. Pictorial cigarette pack warnings: A meta-analysis of experimental studies. Tobacco Control. 2016;25(3):341–354. doi: 10.1136/tobaccocontrol-2014-051978. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Noar S.M., Kelley D.E., Boynton M.H., Morgan J.C., Hall M.G., Mendel J.R., Ribisl K.M., Brewer N.T. Identifying principles for effective messages about chemicals in cigarette smoke. Prev. Med. 2018;106:31–37. doi: 10.1016/j.ypmed.2017.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Noar S.M., Rohde J.A., Prentice-Dunn H., Kresovich A., Hall M.G., Brewer N.T. Evaluating the actual and perceived effectiveness of e-cigarette prevention advertisements among adolescents. Addict. Behav. 2020;109:106473. doi: 10.1016/j.addbeh.2020.106473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. O’Keefe D.J. Message pretesting using assessments of expected or perceived persuasiveness: evidence about diagnosticity of relative actual persuasiveness. J. Commun. 2018;68(1):120–142. doi: 10.1093/joc/jqx009. [DOI] [Google Scholar]
  41. Partos T.R., Borland R., Thrasher J.F., Li L., Yong H.-H., O’Connor R.J., Siahpush M. The predictive utility of micro indicators of concern about smoking: Findings from the international tobacco control four country study. Addict. Behav. 2014;39(8):1235–1242. doi: 10.1016/j.addbeh.2014.04.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Revelle, W. R., 2017. Psych: Procedures for personality and psychological research.
  43. Ripley B., Venables W. Nnet: Feed-forward neural networks and multinomial log-linear models. R Package Version. 2016;7 [Google Scholar]
  44. Rohde J.A., Noar S.M., Prentice-Dunn H., Kresovich A., Hall M.G. Comparison of message and effects perceptions for the real cost e-cigarette prevention ads. Health Commun. 2020;1–9 doi: 10.1080/10410236.2020.1749353. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Rosseel Y. Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA) J. Stat. Softw. 2012;48(2):1–36. [Google Scholar]
  46. Slovic P., Finucane M.L., Peters E., MacGregor D.G. The affect heuristic. Eur. J. Oper. Res. 2007;177(3):1333–1352. doi: 10.1016/j.ejor.2005.04.006. [DOI] [Google Scholar]
  47. So J., Achar C., Han D., Agrawal N., Duhachek A., Maheswaran D. The psychology of appraisal: Specific emotions and decision-making. J. Consumer Psychol. 2015;25(3):359–371. doi: 10.1016/j.jcps.2015.04.003. [DOI] [Google Scholar]
  48. Team, R. C., 2000. R language definition. Vienna, Austria: R Foundation for Statistical Computing.
  49. Wickham H. Springer; 2016. ggplot2: Elegant Graphics for Data Analysis. [Google Scholar]

Articles from Preventive Medicine Reports are provided here courtesy of Elsevier

RESOURCES