Can We Resolve the Disconnect Between How Communication Interventions Work and How We Evaluate Them?

This issue of Health Education & Behavior carries an article by Livingood, Allegrante, and Green (2016), “Culture Change From Tobacco Accommodation to Intolerance: Time to Connect the Dots,” which describes the underappreciated role of health communication in the transformation of the social and cultural norms around tobacco smoking, from widespread acceptance and accommodation of smoking to its growing denormalization and stigmatization.

Why has the contribution of health communication to this public health success remained largely unrecognized? The authors argue that the reason lies in the mismatch between how effective health communication campaigns work and how they are typically evaluated. Unlike drugs or treatment interventions that have a direct effect on individuals, health communication interventions work through multiple routes and affect outcomes on several levels—community, group, and individual. In contrast, the standard of intervention evaluation—randomized clinical trial (RCT)—focuses on linear cause-and-effect and individual outcomes.

This criticism of the RCTs’ focus on individual behavior and internal validity at the expense of more complex causal mechanisms and external validity is not new (Mercer, DeVinney, Fine, Green, & Dougherty, 2007; Sanson-Fisher, Bonevski, Green, & D’Este, 2007). Past studies have found that the majority of health promotion interventions focused on and evaluated individual behavior (Golden & Earp, 2012; Pons-Vigués et al., 2014). In the book Public Health Communication: Evidence for Behavior Change, Hornik (2002) cautioned against designing and evaluating media interventions as if they were pills designed to cure a disease. He reviewed the contradictory research findings on the effects of health communication campaigns where highly controlled RCTs showed minimal results but the uncontrolled observational studies revealed large effects in changing behavior and argued that the inability of RCTs to find large effects stems from the design’s inattention to how communication campaigns really work.

Health communication campaigns work through several different routes. The first is the direct exposure of individuals to the campaign messages. The second is the indirect exposure to the campaign messages through interpersonal conversation, also known as social diffusion (Hornik, 2002) or the two-step flow model (Katz & Lazarsfeld, 1955). This interpersonal conversation can occur face-to-face but is becoming increasingly commonplace in the social media. The third is institutional diffusion, which occurs when the media campaign precipitates changes in public policy (Yanovitzky & Stryker, 2001). These different routes of influence usually work together and reinforce each other, but only the first one—the individual effects model—has traditionally been the focus of evaluations.

Studies that looked at the complementary routes of influence of communication interventions have found that alternative routes (such as the two-step flow) sometimes explained a greater proportion of individual behavior than direct exposure to the campaign messages. For example, the Truth® campaign was found to change beliefs about smoking harms and benefits through both the individual exposure to the advertisements and the conversations about those advertisements (Hwang, 2010). A Dutch campaign encouraging smokers to quit had greater effects on smoking cessation behavior and intentions through interpersonal conversation than through direct exposure to the campaign messages (van den Putte, Yzer, Southwell, de Bruijn, & Willemsen, 2011). Institutional and social diffusion routes, but not direct exposure, were shown to be the way news media stories about binge drinking affected youth binge drinking behavior (Yanovitzky & Stryker, 2001).

In addition to the different routes of exposure, communication campaigns vary regarding whether they place emphasis on the individual and individual behavior or on the larger social or institutional forces that are responsible for the health problem (Dorfman & Wallack, 1993). For example, many antismoking campaigns (such as the Centers for Disease Control and Prevention’s Tips From Former Smokers) motivate people to quit smoking by showing real-life negative consequences of tobacco use. In contrast, messages produced by the California Department of Public Health focused on denormalizing the tobacco industry by showing its deceptive and unethical behavior (Bal, 1998; Stevens, 1998). The latter were particularly effective at changing social norms (Dorfman & Wallack, 1993).

Changes in social norms have been the key to denormalization of smoking (Livingood et al., 2016). In order to more fully understand how communication interventions change social norms and how social norms influence behavior, public health researchers should conceptually and operationally define social norms in their studies. Lapinski and Rimal (2005) differentiated collective norms (which exist on the level of a social system) and perceived norms (which are individual interpretations of the collective norms). They further distinguished between descriptive (prevalence of the behavior) and injunctive (punishment for not carrying out the proscribed behavior) social norms. Measuring social norms presents challenges, especially for collective norms. Because perceived norms exist on the level of individual, they are relatively easy to measure by asking individuals what proportion of their peers engage in a specific behavior (descriptive norms) or to what extent the society approves of this behavior (injunctive norms; Rimal & Real, 2003). Measuring collective norms has to be done on a level of a social group, for example, by content analyzing media depictions of behavior (descriptive norm) or analyzing policy (injunctive norm; Lapinski & Rimal, 2005). Perceived descriptive norms have been the target of communication interventions, especially for reducing college binge drinking, although these social norms interventions focused on correcting misperceptions about numbers of peers engaged in heavy drinking have not been very effective in changing the behavior (Wechsler et al., 2003).

Given the disconnect between the way communication interventions change social norms and how these interventions are evaluated (which often ignores two out of three routes of influence), how can we better evaluate communication interventions to capture their real effects?

Livingood et al. (2016) suggest using other approaches to evaluation than RCTs, such as qualitative methods, systems-based science, network analysis, agent-based modeling, and applying research methods from history, anthropology, journalism, and humanities. Hornik provided an overview of alternative nonexperimental and quasiexperimental evaluation designs that could be used when an RCT is inappropriate or not feasible (Hornik, 2002).

A theoretical guide to studying cultural shifts on the macrosystem level could be provided by cultivation theory (Gerbner, 1967), which posits that the broad meanings or the symbolic environment presented in the media affect the perceptions of media users such that heavier users’ views become more aligned with the media representations. Some research on applying cultivation theory to changes in smoking norms (mostly to explain how portrayal of smoking on TV and in movies might make youth more likely to smoke; Gutschoven & Van den Bulck, 2005; Yang, Salmon, Pang, & Cheng, 2015) has been done, but a more complete application of cultivation theory as originally conceptualized (as a macrosystems level theory) is needed (Potter, 2014). Such analyses would go beyond counting the number of impressions but decipher the meaning behind portrayals (positive or negative) in the whole media environment, including social media. Some attempts at such broad analyses of the macrolevel communication environment are currently being undertaken. For example, University of Pennsylvania Tobacco Center of Regulatory Science (TCORS) is analyzing tobacco-related messages in traditional and emerging social media across multiple platforms and studying how exposure to those messages affects tobacco-related beliefs, attitudes, and behavior in a nationally representative sample of youth and young adults (Hornik & Lerman, 2013).

Another route is to continue using RCTs in designing and evaluating communication interventions but to do it in a more rigorous way accounting for the ways communication interventions exert their influence. A recent study in Burkina Faso is an RCT of a media-based intervention (which, if effective, would be a very cost-effective intervention; Head et al., 2015). The authors were evaluating the effects of a radio campaign intervention on reducing child mortality using a cluster-randomized trial design afforded by weak national media, which allows for randomization of exposure across the regions. However, their outcome was reduction in child mortality and they were not measuring media-related conversations or changes in social norms.

Design and evaluation of communication interventions should be guided by theory (Fishbein & Yzer, 2003; Hornik & Yanovitzky, 2003; Rice & Atkin, 2012). Researchers should hypothesize the route of influence and specifically evaluate which components are being targeted (beliefs, attitudes, social norms, affect, or behavior). In designing a media intervention, researchers should consider not only the direct route of effect but also the social diffusion (through conversations, both face-to-face and in social media) and through institutions and policy changes. For example, some studies specifically evaluated the extent to which a campaign stimulated interpersonal discussion (Anderson & Holody, 2014). Messages should be pretested for their effects on conversation (which could be done in focus groups) to ensure the messages generate discussion but that this discussion is not counterproductive (which has been documented with youth discussing anti-marijuana ads; David, Cappella, & Fishbein, 2006). Finally, designers of communication interventions should focus not only on developing the most effective messages (that are pretested for efficacy in individual studies) but ensure sufficient exposure by selecting proper channels and sufficient reach and saturation, including the new social media.

Denormalization of smoking and reduction in smoking rates has been one of the public health success stories, but this process is far from over. The reduction in smoking rates has been most pronounced among the higher educated, higher socioeconomic-class people, while persons living below poverty line, with lower education, or with disability or who are lesbian, gay, or bisexual are still smoking at high rates (Jamal et al., 2015). The explosive growth in the popularity of electronic cigarettes (e-cigarettes) has been argued by some to be a welcome disruption to the status quo and to have the potential to drastically reduce smoking rates if smokers who are unable or unwilling to quit switched to e-cigarettes. However, e-cigarettes are frequently advertised as an alternative to smoking that can be used anytime anywhere (Grana & Ling, 2014), and their use in public place has the potential to renormalize smoking. They could also serve as a gateway to smoking for youth (Leventhal et al., 2015). Another issue is the growing momentum to legalize marijuana. In their efforts aimed to pass legalization initiatives and in the aftermath of the legalization, advocates argue for normalization of marijuana with advertising agencies working to glamorize marijuana consumption (Bennett, 2014).

As Livingood et al. (2016) point out, we often do not measure what is “normal” until we question it or until the norms change. Thus, the fact that Gallup started measuring opinions about smoking in public in the 1990s is in itself evidence that norms were changing. As researchers and public health practitioners, we should learn from the case of the social denormalization of smoking discussed in this article and explore further: What cultural changes are happening now and are we adequately measuring them? Besides smoking, new tobacco products, and marijuana, are there other cultural shifts taking form, such as in the areas of meat consumption, driving under the influence, gun ownership, vaccines, and waste and recycling, and how are they enabled by health communication? What are other behaviors that seem normal now that might become unacceptable a few decades later? How are changes in social norms affected by the new communication technologies and new media?