Effects of Prior Exposure to Conflicting Health Information on Responses to Subsequent Unrelated Health Messages: Results from a Population-Based Longitudinal Experiment

Abstract

Background

Accumulating evidence suggests that exposure to conflicting health information can adversely affect public understanding of and trust in health recommendations. What is not known is whether prior exposure to such information renders people less receptive to subsequent unrelated health messages about behaviors for which the evidence is clear and consistent.

Purpose

This study tests this “carryover” effects hypothesis, positing that prior exposure to conflict will reduce receptivity to subsequent unrelated health messages, and examines potential affective and cognitive pathways through which such effects might occur.

Methods

A three-wave, online, population-based survey experiment (N = 2,716) assessed whether participants who were randomly assigned to view a series of health news stories and social media posts featuring conflict at Times 1 and 2 were ultimately less receptive at Time 3 to ads from existing health campaigns about behaviors for which there is scientific consensus, compared to those who saw the same series of stories and posts that did not feature conflict.

Results

Structural equation modeling revealed evidence of carryover effects of exposure to conflict on two dimensions of message receptivity: greater resistance to the unrelated ads and lower perceptions of the health behaviors featured in the ads. Modeling indicated that carryover effects were a function of generalized backlash toward health recommendations and research elicited by prior exposure to conflicting information.

Conclusions

Findings suggest that the broader public information environment, which is increasingly characterized by messages of conflict and controversy, could undermine the success of large-scale public health messaging strategies.

Prior exposure to conflicting health information can render people less receptive to subsequent health messages about behaviors for which the evidence is clear and consistent.

Introduction

Many public health strategies including media campaigns and other behavioral interventions rely on messages to promote population health improvement. Yet the success of these strategies is not guaranteed, and an important yet largely untested concern is that the broader public information environment might undermine their effectiveness. When people are exposed to campaigns that promote, for example, tobacco cessation or healthy eating, they process these messages within a broader information landscape that is increasingly dominated by conflicting and often controversial information about myriad health topics [1–5]—sometimes about the very topics featured in the campaigns. Conflicting health information can take many forms, including inconsistent findings across research studies, opposing recommendations across professional organizations, and debate and disagreement about such research and recommendations among key stakeholders or sources [6]. Previous research shows that exposure to conflicting information can adversely affect public understanding of and trust in or backlash towards health recommendations [7–13]. What is not known is whether prior exposure to such information renders people less receptive to subsequent unrelated health messages about behaviors for which the evidence is clear and consistent—a phenomenon that has sometimes been described as “carryover” effects [8].

Carryover effects might emerge due to the activation of negative affect and cognitions following exposure to conflicting information, whereby these responses produce an enhanced reaction to a subsequent experience. Such activation would be consistent with observations of excitation transfer in response to message exposure [14, 15]. If people are routinely exposed to conflicting health information, the negative affect and cognitions resulting from each exposure—for example, in the form of confusion or backlash—could build over time [16] and, in turn, influence engagement with recommended health behaviors [7, 17, 18] and even responses to subsequent unrelated health messages about which there is minimal conflict. To date, however, there is little convincing evidence for carryover effects as defined here. Existing research at best hints at this possibility, finding that exposure to conflict about one health topic can influence cognitive and behavioral responses about other topics [8] and, in some cases, cognitions that are not topic-specific (e.g., attitudes toward health research in general) [9, 19]. Ultimately, such topic-specific or more general health cognitions could drive decreased receptivity to subsequent unrelated health messages, but this possibility has not been directly assessed.

The goal of the current study is to test the carryover effects hypothesis, positing that prior exposure to conflicting health information will reduce receptivity to subsequent unrelated messages about uncontested health topics, and examine potential pathways (e.g., confusion, backlash, and negative affective responses) through which such effects might occur. A robust test of carryover effects requires prioritizing causal inference through the use of an experimental design that features not only exposure to conflict but also subsequent exposure to unrelated messages. To accomplish our goal, we fielded a three-wave, online, population-based survey experiment and assessed whether participants who were randomly assigned to view health news stories and social media posts featuring conflict at Times 1 and 2 were ultimately less receptive at Time 3 to ads from existing health campaigns about behaviors for which there is scientific consensus, compared to those participants who saw the same stories and posts but with no information about conflict.

Evidence for Carryover Effects of Prior Exposure to Conflicting Health Information

Nascent empirical evidence signals the potential for exposure to conflicting health information to have carryover effects on responses to subsequent unrelated health messages, but it stops short of demonstrating them. Importantly, existing studies have not included a subsequent exposure as part of their design, and instead have relied on proxy indicators of carryover effects: Cognitive and behavioral intentional responses to a health topic that differs from the one featured in the exposure to conflict. For example, Nagler and colleagues [8] found that women exposed to conflicting information about mammography were not only more ambivalent about mammography screening but also about other types of cancer screening. Although the authors suggest that exposure to conflict could ultimately affect responses to subsequent unrelated health messages or recommendations for behaviors about which there is little conflict (e.g., colorectal cancer screening), they did not directly test this possibility, which would have required embedding a subsequent message exposure in the experimental design.

Even though research to date relies on proxy indicators of carryover effects, the results of this work nonetheless raise cause for concern, as these studies find experimental effects of exposure to conflict about one health topic on cognitive and behavioral intentional responses to other health topics and, in some cases, on more general health cognitions. In addition to the Nagler et al. [8] mammography study, Chang [9] found that exposure to conflicting research about nutrition or physical activity produced less favorable attitudes toward health research in general. Similarly, in a study on shifting antibiotic recommendations, Lyons and colleagues [19] found that exposure to shifting recommendations decreased participants’ intentions to follow future professional guidance on other health issues and reduced their general acceptance of uncertainty in medical guidelines.

Conceptualizing Message Receptivity as the Central Marker of Carryover Effects

If extant research has examined proxy indicators of carryover effects, then a robust indicator of such effects would be actual receptivity to a subsequent unrelated health message. Yet, in considering how to conceptualize and operationalize message receptivity, there is no standard measure in the message effects and persuasion literature that encompasses this concept [20, 21]. Rather, there is a constellation of measures that may serve as indicators of receptivity, likely tapping into distinct dimensions of the construct. The current study focuses on two such dimensions: (1) resistance to the unrelated message, and (2) perceptions of the behavior featured in the unrelated message.

There are a number of widely used measures that reflect message resistance. For instance, resistance can manifest in negative affective responses to a message (e.g., feeling annoyed or frustrated by the ad shown) [22]. Another indicator may be lower perceived message effectiveness. Although perceived message effectiveness has been conceptualized and operationalized in many ways [23], one compelling definition that has been advanced is the message’s perceived argument strength (e.g., how convincing an ad is, whether the ad gives someone a strong reason to engage in a given behavior) [24, 25]; lower perceptions of a message’s argument strength may signal greater resistance. Similarly, message resistance may be reflected in lower perceptions of response efficacy (e.g., being less confident that there is evidence to support the ad’s claims, being less likely to think your health would benefit if you engaged in the behavior specified in the ad) [26]. Still other indicators include greater counterarguing (e.g., being skeptical of the ad’s claims) [27], greater reactance (e.g., freedom threat, or feeling like the ad is trying to manipulate) [28], or other negative cognitive responses to message content (e.g., thinking the ad’s claims are exaggerated or flawed) [29]. Taken together, these six measures reflect resistance, one dimension of message receptivity.

A second dimension of message receptivity—perceptions of the behavior featured in the unrelated message—might best be captured by attitudinal and behavioral intentional responses. These perceptions, which are central constructs in behavior change theories such as the reasoned action approach [30], have been considered by message effects and persuasion researchers to be markers of message effectiveness [31]. Thus more negative attitudinal responses to the behavior specified in the unrelated message, along with lower behavioral intentions to engage in said behavior, would signal reduced message effectiveness and, by extension, lower receptivity.

Pathways Through Which Carryover Effects on Message Receptivity Could Occur

If carryover effects on subsequent message receptivity were to occur, what mechanistically might be happening to produce them? Previous research on the effects of exposure to conflicting health information points to three likely levers: confusion, backlash, and negative affective responses. Informed by decision theory’s concept of ambiguity [32], researchers have argued that exposure to conflict can give rise to confusion (a marker of perceived ambiguity) and, because perceiving confusion is a state that many people find to be uncomfortable, this “ambiguity aversion” can manifest in backlash, or negative responses toward the subject of that ambiguity [17, 33, 34]. For example, in the nutrition context, cross-sectional [17] and longitudinal [7] survey research found that exposure to conflicting nutrition information—about topics such as alcohol, coffee, and fish consumption—was associated with confusion about and backlash toward nutrition recommendations and research, and these cognitive responses, in turn, were associated with lower intentions to engage in widely recommended health behaviors (fruit and vegetable consumption, physical activity). These associations between exposure to conflict and nutrition-related confusion and backlash were substantiated in an experimental study conducted in the context of carbohydrates and dietary fats [11]. Similar effects of exposure to conflict have been observed in the e-cigarettes context [18].

A third potential lever of carryover effects may be negative affective responses to conflict. There is some experimental evidence that exposure to conflict can produce negative emotional responses to the content of this exposure (e.g., annoyance, frustration, and distress in response to a news story featuring conflicting mammography information) [8]. To date, there is no empirical work linking such emotional reactions to potential carryover effects in the context of conflicting health information, but some support for this possibility comes from experimental research published in the advertising and strategic communication domain. Fennis and Bakker [14] identified an “irritation transfer” effect, in which irritation elicited by disliked ads or a larger number of ads carried over to a subsequent neutral ad (particularly among individuals with higher need to evaluate), negatively influencing attitudes toward that unrelated neutral ad and its brand. More recently, Song and Wen [35] found that emotional responses—primed through a writing task that asked participants to reflect on personal life events that made them feel angry or afraid—can carry over and influence responses to subsequent unrelated messages. There is also theoretical reason to expect carryover effects of exposure to conflict to occur via negative affective responses. Drawing on both Fennis and Bakker [14] and Zillmann [15], Nagler [17] speculated that carryover effects of conflicting health information exposure could occur through excitation transfer and priming of negative affect, suggesting that backlash could be a marker of such negative affect. In the current study, we disentangle these constructs and assess backlash and negative affect as separate pathways through which carryover effects might occur.

The Current Study

To provide a convincing demonstration of carryover effects, the current study required two key components. First, a robust experimental design should include not only exposure to conflict, but also a subsequent exposure to at least one unrelated health message (operationalized here as ads from existing health campaigns). Not only should this subsequent message feature a topic that is unrelated to the subject of the conflict exposure, but it should be a topic about which there is broader scientific consensus—and thus for which we would not expect greater resistance or more negative perceptions. Second, the exposure to conflict should mimic people’s routine interactions with the public information environment without compromising causal inference. Previous research on the effects of exposure to conflicting health information, including studies that have hinted at potential carryover effects, have almost always relied on a one-time single exposure, whereby participants are randomly assigned to view a news story or other message that either does or does not include conflicting information about a health topic. This traditional approach, while scientifically strong, does not resemble the way the public actually engages with health information: Not as a single dose, but through routine cumulative interactions over time, and such repeated exposures are more apt to be consequential [36]. The current study therefore randomly assigns participants to receive conflicting or no conflicting information about several health topics via multiple news stories and social media posts at two time points across several weeks (Fig. 1).

Fig. 1.

Study design. Note. Timing between waves was carefully controlled, ensuring that exposures to conflict would remain salient—in experiments, media messages typically decay in 1–2 weeks [37], and message recall and effects decline over time absent subsequent exposures [38]—while also allowing participants adequate time to complete each survey, thus maximizing retention across waves. NORC AmeriSpeak panelists invited to participate were given 2 weeks to complete the T1 survey; to encourage study cooperation, NORC sent 2 email reminders to sampled panelists during this window. One week after completing T1, participants were invited to complete T2 within a 7-day period. One week after completing T2, participants were invited to complete T3, again within 7 days. Participants who completed the surveys on the first day it was available to them could complete the three-wave study in as few as 2 weeks (15 days); those who completed the surveys on the last day available could complete the study in 6 weeks (42 days). *Exposure or no exposure to conflicting news stories and social media posts about 6 health topics, with 3 exposures at both Times 1 and 2. Topics included mammography screening, prostate-specific antigen (PSA) testing, Vitamin D supplementation, carbohydrate consumption, alcohol consumption, and breastfeeding. Stimuli were developed based on real news stories and pretested with a separate national sample. ^☨Topic-specific outcomes assessed at posttest assessment I are not analyzed in the current study, nor are generalized outcomes assessed at posttest assessment II that are not directly germane to carryover effects (e.g., information seeking and sharing about health topics featured in the news stories and social media posts).

Previous research has conceptualized and operationalized conflicting information in various ways [6, 39]. In the current study, the primary dimension of conflict featured in the experimental exposure is conflict about evidence and how to interpret it. In other words, our definition focuses on expert disagreement—sometimes referred to as “consensus uncertainty” [40]—and includes debates over what counts as “good” evidence (i.e., relative contributions of observational studies versus randomized controlled trials), what counts as safe (i.e., acceptable level of potential harm may vary across stakeholders), or what professional guidance ought to be (i.e., organizations may weigh and interpret evidence in different ways, leading to distinct recommendations). Experts can include individual researchers and scientists (e.g., public health or nutrition experts) or professional organizations (e.g., American Cancer Society or U.S. Preventive Services Task Force). An additional dimension featured in our experimental exposures is conflict that is heated, given evidence that controversy often, though not always, co-occurs with conflict [41].

Ultimately, if the carryover effects hypothesis is supported, then we would expect to see that participants exposed to conflict at Times 1 and 2 are less receptive to unrelated ads from existing health campaigns at Time 3, compared with participants who received no-conflict exposures. This receptivity would be expected to manifest in two ways: First, through greater message resistance—whose indicators might include negative affective responses to the unrelated ads, greater counterarguing, greater reactance, other negative cognitive responses to ad claims, lower perceived argument strength, and lower response efficacy—and, second, through lower perceptions of the behaviors featured in the ads (i.e., more negative attitudinal and behavioral intentional responses). We also would expect to see evidence of carryover effects operating through the likely levers identified in past research (confusion, backlash, and negative affective responses), although previous work does not enable clear a priori predictions regarding which of these pathways may prove most consequential.

Method

Study Design and Procedure

We fielded a three-wave, online, survey-based experiment in July–August 2020 using a sample of U.S. adults drawn from NORC’s nationally representative AmeriSpeak panel (Fig. 1). To simulate routine exposure to health news stories and social media posts in the broader information environment, the experimental exposure to conflict occurred at two time points at least seven days apart across a rolling 28-day period. Participants were exposed to news stories and social media posts about six health topics—mammography screening, prostate-specific antigen (PSA) testing, Vitamin D supplementation, carbohydrate consumption, alcohol consumption, and breastfeeding—with each participant seeing stories or posts about three topics at Time 1, and the remaining three at Time 2. Participants were randomized at study entry to one of two experimental conditions that differed only in whether conflict was presented in these stories and posts (conflict, no conflict). They responded to posttest surveys following the Times 1 and 2 experimental exposures; the three proposed mediators of carryover effects (confusion, backlash, and negative affective responses to conflict) were assessed following the Time 2 exposure. At Time 3, after the exposure to conflict period, all participants were exposed to three ads from existing health campaigns about three behaviors for which there is broad scientific consensus: fruit and vegetable consumption, colorectal cancer screening, and physical activity. To assess carryover effects, we measured participants’ receptivity to these messages—captured by both resistance to the unrelated messages and perceptions of the behaviors featured in the unrelated messages—via a third posttest survey.

The study protocol was preregistered at both ClinicalTrials.gov (ID NCT04247529) and Open Science Framework (OSF; https://osf.io/3ygwh/) and was approved by the University of Minnesota Institutional Review Board.

Data Source

Participants were recruited via email from NORC’s AmeriSpeak panel, a probability-based panel of approximately 43,000 households designed to be representative of the U.S. household population. NORC recruits panelists by randomly selecting U.S. households using area probability and address-based sampling; sampled households are contacted via mail, telephone, and face-to-face field interviews [42]. The AmeriSpeak panel provides sample coverage of approximately 97% of the U.S. household population. To be eligible to participate in the study, participants needed to be English-speaking U.S. adults aged 18+ who are AmeriSpeak panelists. All incentives for study participation were managed and delivered by NORC; participants received $3 incentives for each survey wave completed, using a cash-equivalent points system, and earned an additional $5 incentive for completing all three surveys. Participants could complete the surveys via mobile device (phone or tablet; 54.2–57.7% across waves) or desktop computer (42.3–45.8% across waves).

NORC recruited 6,046 eligible panelists at Time 1 to account for expected loss-to-follow-up at each wave, which was anticipated to be higher than normal due to the COVID-19 pandemic, with the goal of at least 1,800 participants completing all three surveys—powered to detect small effects (Cohen’s d = 0.10–0.20), consistent with effect sizes typically observed in communication research [43, 44]. Ultimately, retention was not adversely impacted by the pandemic: A total of 4,898 participants completed the Time 1 survey (81.0% completion rate); of these, 3,920 completed the Time 2 survey (80.0% retention rate), and of these, 2,716 completed the Time 3 survey (69.3% retention rate). There was no significant differential loss to follow-up across experimental conditions (Supplementary Appendix Fig. S1).

Stimuli

Experimental stimuli: Exposure to conflict (Times 1 and 2)

We developed both news stories and social media posts to serve as experimental stimuli in an effort to better represent the current media ecology. To maximize ecological validity, we developed our stimuli by selecting from the universe of real news stories—incorporating additional “conflict about evidence” language from related Google news searches—and we modeled our fictitious social media posts on those one might come across in a typical feed. We drafted stimuli (both stories and posts) about eight health topics for which there has been considerable conflict about evidence in recent years: mammography screening, PSA testing, Vitamin D supplementation, carbohydrate consumption, alcohol consumption, breastfeeding, egg consumption, and e-cigarette use (see Supplementary Appendix Fig. S2 for details on stimuli development).

Stimuli development yielded 16 possible conflict exposures (eight health topics, with a news story and social media post version for each), with 16 corresponding no-conflict exposures. In August 2019, we pretested these candidate stimuli with a sample of 710 participants aged 18+ recruited from a national opt-in Qualtrics panel. Participants were randomly assigned to one of eight conditions, where they viewed four candidate stimuli (two news stories and two social media posts about four different health topics; all conflict or no-conflict). Participants were asked to look at “some recent health information, including short news segments and social media posts from Twitter and Facebook”; to increase realism, we noted that, for the purposes of this research study, “we have removed all paid advertising content, as well as the media source (for news segments) and profile information (for social media posts).” Across health topics, mean patterns suggested no meaningful differences across conditions in perceptions of news story accuracy, credibility, or understandability; this pattern held for social media posts as well. However, for one of the eight topics (egg consumption), results suggested that we did not manipulate conflict as intended—which is important to empirically assess, since people do not always detect key message content and features [45]—as there were no meaningful differences across conditions for both the news story and social media post versions; thus this topic was dropped from consideration. We ultimately jettisoned the e-cigarette stimuli as well, because in Fall 2019 there was considerable controversy surrounding e-cigarettes and lung damage and disease [46]. We were concerned that the differences in perceived conflict we had observed across conditions in the pretest might be attenuated in the main study.

Ultimately, this analysis yielded six health topics for the main study. We selected either a news story or social media post for each topic based on the pretest results, retaining the version that demonstrated greater mean differences in perceived conflict across conditions: mammography screening (Twitter post), PSA testing (Facebook post), Vitamin D supplementation (news story), carbohydrate consumption (news story), alcohol consumption (news story), and breastfeeding (Facebook post; Fig. 2 and Supplementary Appendix Fig. S2). In both the conflict and no-conflict conditions, participants were randomly shown three of the six stimuli at Time 1 and the remaining three at Time 2, though programing ensured that no participants saw only news stories or only social media posts at a given time point. Given multi-mode survey administration, we created both mobile and desktop versions of the stimuli, maximized for each display. Prior to viewing the stimuli, participants viewed similar instructions as in the pretest, here asking them to “look at some information about several health topics that have received attention over the past year.”

Fig. 2.

Examples of experimental stimuli: Alcohol consumption and prostate-specific antigen (PSA) testing, conflict (left) and no-conflict (right) conditions (mobile versions; see Supplementary Appendix Fig. S2 for all stimuli). Note. Participants were told, “We’d like to ask you to look at some information about several health topics that have received attention over the past year, including short news stories and social media posts from Twitter and Facebook,” and that, for the purposes of this research study, we removed all paid advertising content, as well as the media source (for stories) and profile information (for posts). All images are in the public domain; byline and profile names were fictitious (details on stimuli construction in Supplementary Appendix Fig. S2).

Unrelated messages: Ads from existing health campaigns (Time 3)

To identify health topics for which there was perceived scientific consensus, we surveyed a separate sample of 140 Qualtrics panelists in August 2019, asking them to indicate how much conflicting information and controversy they think there is about 25 nutrition and health topics. The topics that showed the lowest levels of perceived conflict and controversy (~10–20%) were eating fruit and vegetables, drinking water, wearing seat belts, being screened for colon or colorectal cancer, engaging in sun protection behaviors (e.g., using sunscreen, wearing sun-protective clothing), and engaging in physical activity. We focused on these six topics when identifying candidate ads from existing health campaigns to serve as the Time 3 unrelated health messages. Potential ads were identified by the investigator team with additional crowdsourcing assistance from students enrolled in a University of Minnesota health communication course. Criteria for ad selection were: (a) having a clear behavioral recommendation, with either explicit or implied evidence in support of it; (b) avoiding formats that could distract from the central behavioral recommendation (e.g., no humor or disgust appeals); and (c) avoiding sources or spokespersons who could differentially resonate across participants (e.g., celebrities, ordinary people). We identified the most promising ad candidates for three of the health topics—fruit and vegetable consumption, colorectal cancer screening, and physical activity—and created a bank of nine possible ads, with three per topic (Supplementary Appendix Fig. S3). In the main study, participants at Time 3 were randomly exposed to one ad for each of the three health topics, such that each participant saw a total of three ads. By randomly drawing these ads from a larger bank, we hoped to avoid case-category confound [21, 47]. Participants were told “we would like to show you three examples of ads from health campaigns from the past few years in the U.S. and other countries,” noting that, for the purposes of this research study, “we have removed information about the sources or sponsors of these health campaigns.” Ads were optimized for both mobile and desktop viewing.

Measures

Message receptivity outcomes: Resistance to unrelated messages and perceptions of behaviors featured in unrelated messages

To assess message resistance, we drew on six measures, reviewed in the introduction: negative affective responses to the ads (four items, including whether the ad made someone feel “frustrated” and “annoyed”; 1 “very slightly or not at all,” 5 “extremely”) [8]; counterarguing (4 items, including “I thought of points that went against what the add was saying” and “While reading the ad, I was skeptical of its claims”; 1 “strongly disagree,” 5 “strongly agree”) [27]; reactance (1 item to assess freedom threat: “I felt like the ad was trying to manipulate me”; 1 “strongly disagree,” 5 “strongly agree”) [28]; other negative cognitive responses to ad claims (four items, including thinking the claims in the ad were “flawed” and “believable” (reverse-coded); 1 “strongly disagree,” 5 “strongly agree”) [29]; lower perceived argument strength (two items: “The ad was convincing” and “The ad gave me a strong reason to [engage in the behavior specified in the ad]” (both reverse-coded); 1 “strongly disagree,” 5 “strongly agree”) [25], and lower response efficacy (two items: “How confident are you that there is evidence to support the ad’s claims?” (reverse-coded; 1 “not at all confident,” 7 “extremely confident”) and “How much do you think your health would benefit if you [engaged in the behavior specified in the ad]” (reverse-coded; 1 “not at all,” 7 “extremely”)) [26]. All items are provided in Supplementary Appendix Table S1; means and standard deviations are reported in Supplementary Appendix Table S2. Participants responded to these questions for each of the three ads they saw at Time 3, and responses were averaged across ads to create a score for each of the six measures.

Per our preregistration plan, we assessed whether these six measures tap into a single underlying construct, expecting strong correlations among measures. In fact, most were strongly correlated with one another (r = 0.50–0.88); the one exception was negative affective responses (r = 0.09–0.32), and a principal components factor analysis substantiated this finding. We therefore chose to analyze negative affective responses as a separate dependent latent variable, restricting the message resistance latent variable to the remaining five measures. Higher scores reflected more negative affective responses and greater resistance to the unrelated messages.

To assess perceptions of the behaviors featured in the unrelated messages, we drew on measures of attitudinal and behavioral intentional responses [30]. To assess attitudes, participants were asked whether their engaging in each behavior would be harmful/beneficial, unpleasant/pleasant, and unnecessary/necessary (on 7-point semantic differential scales), and to assess behavioral intentions, they were asked how likely it is that they will engage in each behavior (1 “very unlikely,” 7 “very likely”; Supplementary Appendix Tables S1 and S2). Participants responded to these questions for each of the three ads they saw, and responses were averaged across ads to create attitudinal and behavioral intentional response scores. The four measures supported one latent variable, as items were moderately correlated (r = 0.37–0.58) and loaded on the same factor. Overall, a higher score reflected more positive perceptions of the behaviors featured in the unrelated messages.

Potential mediators of carryover effects: confusion, backlash, and negative affective responses to conflict

The three potential mediators were measured at Time 2, following the second round of experimental exposure to conflict (see Supplementary Appendix Tables S1 and S2 for all items and descriptive statistics). Because the current study tested the effects of cumulative exposure to conflict across a range of health topics, both confusion and backlash were defined in general health rather than content-specific terms: Confusion was defined as perceived ambiguity about health recommendations and research in general, and backlash was defined as negative beliefs toward health recommendations and research in general. All items were adapted from past research [8, 17, 33]. For example, items used to assess generalized health confusion included “It is not clear to me how best to stay healthy” and “I find health recommendations to be confusing” (six items; 1 “strongly disagree,” 5 “strongly agree”), while items used to assess generalized health backlash included “I am tired of hearing what I should or should not do to stay healthy” and “Health recommendations should be taken with a grain of salt” (seven items; 1 “strongly disagree,” 5 “strongly agree”). The six confusion measures supported one latent variable, as items were moderately correlated (r = 0.26–0.57) and loaded on the same factor. The seven backlash measures also supported one latent variable; items were moderately correlated (r = 0.22–0.67) and loaded on a single factor. For both confusion and backlash, higher scores indicated more negative cognitive responses.

To assess negative affective responses to conflict, participants were presented with 7 positive and negative emotions and asked to indicate how they felt “having read these news and social media posts,” first at Time 1 and again at Time 2 [8]. The four negatively valenced items (“frustrated,” “annoyed,” “distressed,” and “worried”; 1 “very slightly or not at all,” 5 “extremely”) supported one latent variable and were moderately correlated (r = 0.38–0.68). A higher score reflected more negative affective responses.

Analytic Approach

Descriptive statistics, including frequencies and means with standard deviations, were calculated for all study variables (Supplementary Appendix Tables S2–S4). Given our goal of assessing effects of prior exposure to conflict on receptivity to subsequent unrelated health messages, as well as potential pathways through which such effects might operate, we took a structural equation modeling (SEM) approach with maximum likelihood estimation. This allowed us to address all of our research questions at once—namely, whether exposure to conflict over time reduces receptivity and whether such effects operate through key potential mediators—using a latent variable approach, which is particularly valuable given the lack of a standard validated measure of message receptivity. Models tested internal consistency through confirmatory factor analyses of the latent variables and pathways through structural regression analysis. To test the carryover effects hypothesis, we examined the effect of exposure to conflict on each of the three message receptivity outcomes (message resistance, perceptions of behaviors featured in unrelated messages, negative affective responses to unrelated messages). To explore the processes that underlie carryover effects, we examined path coefficients among the experimental exposure, three potential mediating variables (generalized health confusion, generalized health backlash, and negative affective responses to conflict), and three message receptivity outcome variables, as well as the overall indirect effect of exposure to conflict on message receptivity outcomes.

We developed separate models for each of the three outcomes: Model 1 tested effects on message resistance (latent variable, five indicators standardized to M = 0 and SD =1; Fig. 3); Model 2 on perceptions of behaviors featured in unrelated messages (latent variable, four indicators; Fig. 4a); and Model 3 on negative affective responses to unrelated messages (latent variable, four indicators; Fig. 4b). Each model consisted of a single indicator for the randomization group (conflict), and three latent mediating variables: generalized health confusion (six indicators), generalized health backlash (seven indicators), and negative affective responses (four indicators). The factor loading for the first item on each latent variable was constrained to 1.0. We conducted a sensitivity analysis removing indicators with factor loadings <0.6, which did not affect our conclusions.

Fig. 3.

Structural equation model of the effect of experimental exposure to conflict on resistance to unrelated health messages (Model 1). Note. Model illustrates the relationships tested using structural equation modeling. To improve figure clarity, correlations are not included. Solid lines represent statistically significant standardized path coefficients (^**p <0.05); dashed lines represent nonsignificant standardized path coefficients. Latent variables are denoted by circles and indicators by rectangles. “Recs” = recommendations, “R” = reverse coded. Model 1 fit statistics: χ² = 1440.42, df = 209, p < 0.0001; RMSEA = 0.047 (0.045, 0.050); CFI = 0.958; SRMR = 0.041.

Fig. 4.

(a) Structural equation model of the effect of experimental exposure to conflict on perceptions of behaviors featured in unrelated health messages (Model 2). (b) Structural equation model of the effect of experimental exposure to conflict on negative affective responses to unrelated health messages (Model 3). Note. Since Figure 3 (Model 1) specifies indicators for mediating variables, for parsimony these are not presented again here. Figure 4a (Model 2) fit statistics: χ² = 1210.12, df = 187, p < 0.0001; RMSEA = 0.045 (0.043, 0.048); CFI = 0.953; SRMR = 0.054. Figure 4b (Model 3) fit statistics: χ² = 1209.31, df = 178, p < 0.0001; RMSEA = 0.047 (0.044, 0.049); CFI = 0.958; SRMR = 0.045.

No specific hypotheses were set about the relationships between confusion, backlash, and negative affective responses to conflict, or between each of the indicators for a specific latent variable; instead, they were allowed to covary where it improved the model fit. Factor loadings, factor correlations, residual variances, and path coefficients were inspected for sign and magnitude. Model fits were evaluated based on the model chi-square statistic, the root mean square error of approximation (RMSEA) and its 90% confidence interval, the Bentler comparative fit index (CFI), and the standardized root mean square residual (SRMR). For the final models, the relative contribution of direct and indirect effects, in addition to the total effect for each variable, were estimated. Across models, p-values of <.05 were considered statistically significant. All analyses were conducted in SAS version 9.4 (Cary, NC).

Results

Participant characteristics

Half (50.0%) of participants who completed all data collection identified as female, and the average age was 50.1 (SD = 16.2) years. Three-quarters (74.5%) identified as White, non-Hispanic; 11.3% as Black, non-Hispanic; and 6.9% as Hispanic. Over one-third (38.9%) had a Bachelor’s degree or higher, while 18.5% had a high school degree or less. Participant characteristics by experimental condition are reported in Supplementary Appendix Table S3. Although there were some differences in participant characteristics across waves (Supplementary Appendix Table S4), overall there was no significant differential loss to follow-up across conditions (Supplementary Appendix Fig. S1).

Effects of Exposure to Conflict on Resistance to Unrelated Health Messages

Support for the carryover effects hypothesis was found in a positive indirect effect of exposure to conflict at Times 1 and 2 on resistance to unrelated health messages at Time 3 (0.09, p < .001; Fig. 3). Exposure to conflict produced greater confusion (0.21, p < .001), backlash (0.19, p < .001), and negative affective responses to conflict (0.36, p < .001), and backlash, in turn, was associated with stronger message resistance (0.29, p < .001). However, independent of the estimated indirect effect, there was a small but negative direct effect of exposure to conflict on resistance to Time 3 ads (-0.05, p = .011), suggesting that prior exposure to conflict slightly reduced participants’ resistance—a finding that runs counter to the carryover effects hypothesis. Given the positive indirect effect and negative direct effect, the total effect of exposure to conflict on message resistance was small and nonsignificant (0.03, p = .081).

Effects of Exposure to Conflict on Perceptions of Behaviors Featured in Unrelated Messages

Support for the carryover effects hypothesis was again evident in the indirect effect of exposure to conflict at Times 1 and 2 on perceptions of the health behaviors featured in the unrelated messages at Time 3 (−0.07, p < .001; Fig. 4a). Exposure to conflict increased confusion (0.22, p < .001), backlash (0.19, p < .001), and negative affective responses (0.34, p < .001), and backlash was associated with subsequently lower perceptions of the behaviors in the Time 3 ads (i.e., more negative attitudes and behavioral intentions; −0.35, p < .001). As with message resistance, there was also a small but positive residual direct effect of exposure to conflict on behavioral perceptions (0.05, p = .020), indicating that prior exposure slightly increased such perceptions. Again, indirect and direct effects with opposite signs yielded a small and nonsignificant total effect of exposure to conflict on behavioral perceptions (−0.02, p = .312).

Effects of Exposure to Conflict on Negative Affective Responses to Unrelated Messages

Similar to message resistance and behavioral perceptions, the observed indirect effect of exposure to conflict at Times 1 and 2 on negative affective responses to the unrelated messages at Time 3 supported the carryover effects hypothesis (0.17, p < .001; Fig. 4b). Exposure to conflict still increased confusion (0.21, p < .001), backlash (0.20, p < .001), and negative affective responses to conflict (0.35, p < .001), but the relationships of these variables with Time 3 negative affect deviated from the pattern observed above. Here, both confusion (0.21, p < .001) and negative affective responses to conflict (0.46, p < .001) were associated with increased negative affective responses to Time 3 ads, whereas backlash was associated with reduced negative affective responses to these ads (−0.20, p < .001). There was also a negative residual direct effect of exposure to conflict (−0.17, p < .001), suggesting that exposure reduced Time 3 negative affect. Indirect and direct effects, which again were in opposite directions, canceled each other out, yielding no total effect of exposure to conflict on negative affective responses to the T3 ads (0.000, p = .978).

Discussion

Despite growing evidence that exposure to conflicting health information is widespread and can have adverse effects on affective, cognitive, and behavioral responses [6, 39], research has not established whether such exposure can carry over and adversely influence responses to subsequent unrelated health messages, especially those about health behaviors for which the evidence is clear and consistent. To date, nascent research on carryover effects has focused exclusively on proxy indicators, or cognitive and behavioral intentional responses to a health topic that differs from the one featured in the exposure to conflict. The current study builds on this existing work by offering a robust test of the carryover effects hypothesis. Using a longitudinal experimental design, we assessed whether multiple exposures to conflict across several weeks produces less receptivity to subsequent unrelated health messages about behaviors for which there is broad consensus. In addition, informed by prior research on conflicting health information, we examined several pathways through which such carryover effects might occur: generalized health confusion, generalized health backlash, and negative affective responses to conflict. We found evidence of carryover effects: For two dimensions of message receptivity—resistance to the unrelated health messages, and perceptions of the behaviors featured in the unrelated messages—modeling revealed indirect effects of exposure to conflict, whereby carryover effects were a function of generalized backlash toward health recommendations and research elicited by prior exposure to conflicting information. However, after accounting for indirect effects, we also found evidence of small direct effects of exposure to conflict on message receptivity that runs counter to the carryover effects hypothesis. These residual direct effects suggest that there is a process—hereto unmeasured, nor hypothesized—through which exposure to conflict could in fact strengthen message receptivity; we speculate on this possibility below. Ultimately, the presence of both indirect and direct effects with opposite signs yielded nonsignificant total effects across models.

The finding that backlash was the primary lever of carryover effects on both message resistance and behavioral perceptions is consistent with past theorizing. Nagler [17] suggested that carryover effects might operate through excitation transfer and priming of negative affect, arguing that backlash could reflect such negative affect. The current study parses backlash from negative affect and instead assessed these as separate pathways through which carryover effects might occur—whereby excitation could manifest not just in negative affective responses, but also in negative cognitive responses like backlash. That backlash proved to be more consequential in carryover effects transmission is not surprising, as conceptually it is a marker of concern; if someone is questioning and generating negative thoughts about health recommendations and research, it stands to reason that such interrogation could translate into greater questioning, counterarguing, and, ultimately, resistance to subsequent health messages. In contrast, negative affective responses could be too diffuse to prompt a resistance response one to two weeks later, while confusion responses may not be as universally negative in valence (e.g., for some health topics, like mammography screening and PSA testing, perceiving ambiguity in research and recommendations may be legitimate and not necessarily deleterious).

The indirect effects of exposure to conflict on negative affective responses to the unrelated messages do not follow the same pattern as the effects on message resistance and behavioral perceptions. Both confusion and negative affective responses to conflict increased negative affective responses to Time 3 ads, whereas backlash reduced negative affective responses to these ads. Although we can only speculate as to why, perhaps backlash is a marker of disengagement, which, in turn, would likely lead to less of an affective reaction. It is also plausible that negative affective responses may be a conceptually distinct form of message receptivity; the fact that the measure was not strongly correlated with other measures of message resistance might support this possibility. Future research should investigate whether negative affect is an indicator of message receptivity, and, if so, what implications it has for how people behaviorally respond to a message.

Across the three models, we consistently observed evidence of a favorable direct effect of exposure to conflict on responses to the unrelated ads about uncontested health topics. This pattern stands in contrast to the adverse indirect effect of exposure to conflict observed through the affective and cognitive responses to the conflict-laden media messages. Why might exposure to conflict lead people to evaluate the unrelated ads more favorably? One possibility is that media coverage of conflicting claims about carbohydrate consumption and Vitamin D supplementation, for example, led people to appreciate messages about behaviors for which the evidence is clear and consistent. From this vantage point, information about behaviors for which there is conflict serves as a comparison standard against which subsequent messages are evaluated. Although this suggests there may be value in people being able to recognize conflict in the media environment and, in turn, recognize when it is not present, the pattern of indirect effects observed in this study suggests that repeated exposure to conflict may lead people to draw broader negative inferences about the value of health recommendations and the sources that disseminate them—beliefs that over time may prove to be more detrimental. If evidence for this differential effect of exposure to conflict proves to be robust, future research should prioritize delimiting the conditions under which these favorable and adverse responses are observed.

Study results should be considered in light of several observations. First, data were collected in Summer 2020 during the COVID-19 pandemic, which could limit generalizability of study results. Recent research allays some concerns about online survey experiments fielded during the COVID era, finding little evidence that the pandemic has changed how participants respond to experimental treatments, although reduced effect sizes may reflect increased inattentiveness [48]. Thus, if anything, the effects observed here may underestimate carryover effects that might occur during times when participants have less consistent pressure on their capacity for cognitive engagement. Second, and relatedly, the obtained effects were small but meaningful. Their magnitude is comparable to what is often observed in communication research [43, 44], and on a population level such effects can translate into important public health impact [49]. Moreover, given media discourse surrounding COVID, participants in the no-conflict condition were likely to have been exposed to conflicting information about COVID [50]; the fact that we were able to observe carryover effects across a two- to six-week period in response to just six news stories and social media posts suggests that the cumulative downstream effects of exposure to conflict may be considerable when naturally occurring in the public information environment. Third, the message receptivity outcomes captured responses across the Time 3 ads to which participants were exposed; investigating ad- or behavior-specific effects was not the focus of the current study, but would be worth investigating in future research. So, too, is it important for future research to prioritize measurement development and validity of several key constructs in this study. For example, several measures that reflect message resistance, including reactance and perceived message effectiveness [23], have suffered from a “a lack of conceptual and operational precision and agreement” (28, p. 1), and, more broadly, there is no standard measure in the message effects and persuasion literature that encompasses message receptivity [20, 21]. Similarly, although past research has provided some evidence of the unidimensionality of confusion and backlash measures, respectively [17], greater attention to measurement validity is warranted. Last, our conceptualization of conflict did not include personal conflict (e.g., researcher X versus researcher Y), industry versus public health conflict, or political conflict. Although these are legitimate dimensions of conflict worthy of investigation—particularly political conflict, as health issues increasingly become politicized [51]—this was beyond the scope of the current study and should be tested in future work. In addition, whereas this study presented stories and posts that explicitly emphasized conflict, other presentations of conflict are also prominent in the media environment but are not addressed here and may be processed differently by audiences. For instance, there could be two messages that conflict with one another delivered simultaneously or over time; or, a corrective message could lead to audiences inferring conflict (i.e., between a corrective message and that which it aims to correct).

Conflicting and often controversial health information continues to proliferate—perhaps most immediately about COVID-19 [50], but this is just the latest incarnation of a long-standing, cross-topic, and cross-source phenomenon. The immediate impacts of exposure to conflict on affective, cognitive, and behavioral responses are well understood [6]. The contribution of the current study is moving beyond these more proximate responses and documenting their influence on downstream responses to subsequent unrelated health messages about uncontested health topics using a robust longitudinal experimental design. Results sound an alarm: The broader public information environment, with its ubiquitous messages of conflict and controversy, could undermine the success of population-based public health messaging strategies. Determining how to intervene and interrupt the carryover phenomenon, and assessing whether such effects vary across population subgroups, is a critical next step.

Funding

This work was supported by a grant from the National Cancer Institute (5R21CA218054-02; PI: RHN). This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Compliance with Ethical Standards

Conflict of Interest The authors declare that they have no conflicts of interest.

Ethical Approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent Informed consent was obtained from all individual participants included in the study.