Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Oct 1.
Published in final edited form as: Soc Sci Med. 2023 Aug 29;334:116194. doi: 10.1016/j.socscimed.2023.116194

Sustaining Positive Perceptions of Science in the Face of Conflicting Health Information: An Experimental Test of Messages about the Process of Scientific Discovery

Rebekah H Nagler 1, Sarah E Gollust 2, Marco C Yzer 1, Rachel I Vogel 3, Alexander J Rothman 4
PMCID: PMC10552003  NIHMSID: NIHMS1929281  PMID: 37660521

Abstract

Background:

The public is often exposed to conflicting health information, with evidence of concerning consequences, yet little attention has been paid to identifying strategies that can mitigate its effects.

Objective:

The current study tests whether three different approaches to communicating about the process of scientific discovery—a rational appeal using analogical evidence, a rational appeal using testimonial evidence, and a logic-based inoculation approach—could reduce the adverse effects of exposure to conflict by positively framing how people construe the scientific process, increasing their perceived knowledge about the scientific process, and helping them to respond to critiques about the scientific process, which, in turn, might make them less apt to counterargue the science they subsequently encounter in health news stories and other exposures to conflict.

Methods:

We fielded a survey experiment in May 2022 with a national sample of U.S. adults (N = 1,604).

Results:

Providing any of the three messages about science prior to exposure to conflicting health information encouraged both positive construal of science and greater science knowledge perceptions and discouraged counterarguing science, compared to a control condition in which people were only exposed to conflict. Of the three messaging approaches tested, the testimonial evidence message was slightly more effective, but was also considered slightly more accurate, credible, and trustworthy.

Conclusions:

Developing and implementing messages that describe the process of scientific discovery could prove successful, not only in improving public perceptions of science but perhaps ultimately in better equipping people to make sense of conflicting information and its causes. However, additional research on such strategies is needed, particularly as part of larger interventions with multiple messages across multiple exposures, if they are to have implications for health and science communication.

Keywords: science communication, conflicting health information, survey-based experiment, health communication

Introduction

Conflicting health information is increasingly a threat to public health promotion efforts. This information—which includes competing findings across research studies, opposing health guidance across professional organizations, and debate about such research and guidance among health experts and other stakeholders (Nagler & LoRusso, 2018)—pervades news media coverage of a wide range of health topics, such as nutrition (Ihekweazu, 2021), mammography screening (Nagler, Fowler, et al., 2019), e-cigarettes (Wackowski et al., 2020), and COVID-19 (Wang & Zhou, 2022). Such media content often features explicit journalistic reference to conflict that underscores, for example, disagreement among scientists (Annenberg Public Policy Center, 2018) and inconsistency in different organizations’ health recommendations (Nagler, Fowler, et al., 2019). Importantly, conflicting information does not go unnoticed by the public, and there is now considerable evidence documenting its adverse effects on public confusion about and trust in health recommendations (e.g., Chang, 2015; Clark et al., 2019; Iles et al., 2022; Nagler, Yzer, et al., 2019). Exposure to messages about shifting public health guidance and conflicting scientific evidence may be particularly aversive (Iles et al., 2022), “carrying over” to reduce people’s receptivity to unrelated health messages—even about widely recommended behaviors for which there is broad scientific consensus, such as fruit and vegetable consumption and physical activity (Nagler et al., 2022).

Although the prevalence and consequences of exposure to conflicting information are well recognized and concerning, little is known about potential approaches to address and mitigate its effects. One possible strategy for bolstering people’s ability to make sense of conflicting health information is better communication about the process of scientific discovery, since the very nature of the scientific process is one of the reasons that conflicting (or seemingly conflicting) messages arise. For example, different study designs can yield distinct conclusions, which the public often perceives as contradictory; yet the use of multiple methods to answer the same research question, or triangulation, is central to the production of scientific knowledge. This scenario has frequently occurred in the nutrition context, where the results of observational studies, such as those that found that people who ate diets rich in antioxidants (e.g., vitamins A, E, and C) were healthier, did not hold up in rigorous randomized trials (Sinvani et al., 2013). In addition, the COVID-19 pandemic has showcased what happens when the process of scientific discovery is rapidly accelerated. Rather than what typically unfolds—namely, incremental gains in scientific knowledge over time—the introduction of a novel coronavirus prompted a huge redirection of intellectual resources all at once, with scientists from all corners turning their attention to understanding the virus, its ability to spread, and the consequences of infection. This meant a swift evolution in scientific understanding, which, in turn, translated into what has been described as ever-shifting guidance about behaviors including mask wearing and booster vaccinations (Scott, 2022).

We conceptualize communication about the process of scientific discovery as messages that explain how scientific knowledge or understanding is produced and how, by its very nature, doing science means that what we think might be true is likely to change as more evidence accumulates. Or, as Nagler (2010) described in a content analysis of conflicting nutrition information in the news media, these messages “explain how scientific advancement often involves taking two steps forward and one step back—a process that can produce findings that seem to be at odds with one another” (p. 250). To be clear, communicating about the process of scientific discovery is distinct from communicating about scientific uncertainty, which has received considerable research attention (Peters & Dunwoody, 2016; Ratcliff et al., 2022). Much of the work on communicating about uncertainty has focused on communicating what is and is not currently known—referred to as “deficient uncertainty” (describing a “known lack of knowledge”; see, e.g., Gustafson & Rice, 2020) or “hedging” (underscoring the limitations or caveats of scientific research; see, e.g., Jensen, 2008). In contrast, there has been comparatively little empirical attention to communication about the scientific process itself. One recent study tested whether a forewarning message, which described “the scientific process as one of discovery,” would attenuate negative responses to conflicting guidance about COVID-19 (Gretton et al., 2021). There was some evidence that it did, though the study tested a single messaging strategy and focused on conflicting information about COVID-19, specifically.

What might be the effects of communicating about the process of scientific discovery, and through what mechanisms might such a message affect the public’s perceptions of science? Would it lead people to think and/or feel differently about how science works? Would it lead them to feel like they know more about the scientific process? Or by providing them with a better sense of how science happens, might it give them the tools to counter subsequent threats to that understanding? The current study tests three message strategies that target these potential cognitive mechanisms of positive construal of science, science knowledge perceptions, and counterarguing science, respectively.

The first messaging approach is a rational appeal using analogical evidence. The defining feature of rational appeals is that they are grounded in arguments. With this particular approach, analogies are used to support a claim, comparing one idea or situation to another, and they may be most useful when dealing with a novel topic (Shen & Bigsby, 2012). For example, an airplane turbulence analogy could be used to illustrate the process of scientific discovery (Figure 1a). Pilots provide different safety guidance, by turning off or on the seatbelt sign, when circumstances on board change; so, too, do scientists, when they discover new evidence that seems to run counter to what the science suggested previously. This analogical approach may be an effective way to positively frame how people construe the scientific process: The analogy or comparison presented invites people to view the scientific process through a new lens, thus making it more accessible and compelling and, in turn, encouraging a positive construal. This possibility is consistent with work from political science, which suggests not only that analogies function as cognitive shortcuts but also as ways of making less familiar and often complex information easier to explain and, ultimately, more acceptable (Barabas, Carter & Shan, 2020).

Figure 1 :

Figure 1 :

Open in a new tab

Experimental stimuli: Rational appeal/analogical evidence message about science (1a), rational appeal/testimonial evidence message about science (1b), inoculation message about science (1c)

The second approach, a rational appeal using testimonial evidence, uses arguments or claims that are supported by people’s personal experiences, opinions (including expert testimony), and/or eyewitness accounts (Shen & Bigsby, 2012). For example, a quote from a scientist could be used to support the argument that shifts in science and health recommendations reflect how science works: “Science is not the truth. Science is finding the truth. When science changes its opinion, it didn’t lie to you; it learned more” (Figure 1b). This testimonial approach may be an effective way to augment people’s knowledge about the scientific process. Heuristically, testimonials encourage people to pay attention to how information is presented (de Wit, Das & Vet, 2008), and when that information is provided by a scientist or expert, it could be considered worth learning, thus producing a perceived increase in knowledge about the scientific process. Moreover, compared to other evidence-based appeals, testimonial or narrative formats are more likely to discourage reactance or other forms of message resistance, thereby increasing the likelihood of message engagement and acceptance (de Wit, Das & Vet, 2008; Keer, vas den Putte, de Wit & Neijens, 2013).

Logic-based inoculation, the third messaging approach (hereafter referred to as “inoculation”), is based on inoculation theory, which is a resistance to persuasion theory (McGuire, 1964). As with inoculations in the medical context, an inoculation message exposes people to a weakened form of an oppositional message, with the goal of activating attitudinal defenses to arm against future oppositional messages. A logic-based inoculation approach specifically provides information on the reasoning process used (Iles, Gillman, Platter, Ferrer & Klein, 2021)—here, an explanation for why conflicting information can arise, and how understanding the process of scientific discovery could help people to reconcile such conflict (Figure 1c). This information, coupled with forewarning (“Don’t let seemingly conflicting research findings or recommendations distract or prevent you from trusting the scientific process”), may be an effective way to help people respond to critiques about the scientific process—and, in turn, make them less apt to counterargue the science they subsequently encounter in health news stories and other exposures to conflict. All three of these messaging approaches are well described in the message effects and persuasion literature and have evidence to support their use (Compton, 2012; Hornikx, 2005; Reinard, 1998; Shen & Bigsby, 2012); however, they have not been applied to the context of communicating about the scientific process, nor have they been directly compared to one another for their relative effectiveness.

We fielded a survey experiment with a national sample of U.S. adults to assess whether exposing participants to such messages about science prior to exposure to conflicting health information would reduce negative responses on indicators of people’s perceptions of science—responses that can occur in the face of exposure to conflict (Chang, 2015; Nagler et al., 2022). We hypothesized that any of these messaging approaches would be more successful in reducing negative responses, compared to a control condition in which people read conflicting health information but with no pre-exposure message about science (H1). This expectation was based on the logic that each messaging strategy would engage a potential cognitive target that may serve to mitigate the adverse effects of exposure to conflicting health information: An analogical evidence message should encourage positive construal of science, a testimonial evidence message should strengthen science knowledge perceptions, and an inoculation message should discourage counterarguing science. We also asked whether there were differences among these three message strategies in how well they reduced negative responses on these indicators of perceptions of science (RQ1), and we examined whether a specific message strategy moved its expected corresponding cognitive target more than the other message strategies did (e.g., did the rational appeal/analogical evidence message move positive construal of science more than the inoculation and rational appeal/testimonial evidence messages did?) (RQ2). Last, we hypothesized that any of the message strategies would reduce negative affective responses to conflicting health information—responses that have been observed in past research (e.g., Nagler et al., 2022; Nagler, Yzer, et al., 2019)—compared to a control condition with no pre-exposure message about science (H2).

Methods

Study Design and Procedure

We designed a single factor between-subjects survey experiment in which each participant was exposed to conflicting information about two health topics, but first was randomly assigned to one of four pre-exposure conditions: 1) control condition (no message about science; only exposure to conflicting information); 2) positive construal of science target condition (rational appeal/analogical evidence message about science); 3) science knowledge perceptions target condition (rational appeal/testimonial evidence message about science); and 4) counterarguing science target condition (inoculation message about science). We fielded the survey experiment in May 2022 with participants recruited via Prolific Academic, an online research platform. To be eligible to participate in the study, potential participants had to be English-speaking U.S. adults aged 18+ who were enrolled in the Prolific platform. Eligible participants received an email invitation that described the goal of the study as “better understand[ing] how the public responds to health and science information in the media.” Those who agreed to participate were randomized to one of the four conditions at the time of study entry using simple randomization. After message exposure, participants responded to posttest survey questions about their perceptions of science, as well as individual-level factors that might moderate any observed effects (not analyzed in the current study) and demographics.

We recruited 1,604 participants, compensating them $1.40 for participating in the study (rate = $10.50/hour for an ~8-minute study). A sample size of approximately 400 in each condition provided 80% power to detect a small effect (Cohen’s d = 0.20), which is typical for communication research (Snyder et al., 2004; Valkenburg & Peter, 2013). The study protocol and analysis plan was preregistered at Open Science Framework and approved by the University of Minnesota Institutional Review board.

Stimuli

Experimental stimuli: Messages about science.

We developed three mechanism-based messages about science: 1) a rational appeal/analogical evidence message, whose cognitive target is positive construal of science; 2) a rational appeal/testimonial evidence message, whose cognitive target is science knowledge perceptions; and 3) an inoculation message, whose cognitive target is counterarguing science (Figure 1). Both rational appeals were based on content gleaned from real news stories and social media posts (e.g., Twitter posts from medical and public health experts that provided examples of analogical or testimonial evidence germane to the process of scientific discovery). The inoculation message was based on inoculation stimuli used in past research on communication about politicized health controversies (Fowler et al., 2022). The three messages were similar in format, each with a common header (“Tips for making sense of science”), though there was some variation in length (range = 113–198 words). All three included the same lead sentence—“Sometimes it might seem like science and health recommendations are always changing, with research study findings contradicting each other and scientists changing their minds about what you should or should not do to stay healthy”—and each was preceded by the same instructions. These explained that “communicating about science is challenging” and so “science educators have put together the following tips about science.” They also were told that “identifying information about the science educators was removed for the purposes of this research study.”

Exposure to conflicting health information.

All participants, regardless of randomized assignment to condition, were exposed to one health news story (randomly chosen from a possible two stories) and one social media post (randomly chosen from a possible two posts), each of which featured a health topic for which conflicting information exists: Carbohydrate consumption (news story), alcohol consumption (news story), mammography screening (social media post), and prostate-specific antigen (PSA) testing (social media post). These stimuli—which were developed for a previous study on the effects of conflicting health information and are detailed elsewhere (see online supplemental material)—all featured conflict about evidence and how to interpret it. Although this conceptualization focuses on expert disagreement (or conflict between scientists), it also includes debates over what counts as quality evidence (i.e., relative weight of evidence from observational cohort studies versus randomized trials), what counts as safe (i.e., variation in key stakeholders’ tolerance for potential harm), and what professional recommendations ought to be (i.e., organizations consider and interpret evidence differently, which can produce distinct and sometimes shifting guidelines). In the survey instructions that preceded the exposure to conflict, participants were asked to read “a recent health news story and social media post, both of which feature health topics for which there are conflicting research findings and/or health recommendations.” Participants were told that the “news media source and social media profile were removed for the purposes of this research study.”

Measures

Primary outcome: Perceptions of science.

The primary outcome variable, perceptions of science, was captured using three measures, each of which assessed one of three cognitive targets: positive construal of science, science knowledge perceptions, and counterarguing science. Both the positive construal and knowledge perceptions items were developed for this study, because, to our knowledge, there are no standard measures of these constructs available in the literature—at least not in the context of understanding the scientific process. The posttest survey instructions explained that participants would be asked some questions about the news story and social media post they just read, and that the questions would refer to these as the “stories” they read.

To assess positive construal of science, participants indicated their responses to two, randomly ordered, sliding scale items: “Given the stories you just read, how comfortable do you feel with how science works?” (response options: 0 = “not at all comfortable” to 100 = “very comfortable”), and “Given the stories you just read, to what extent do you feel negatively or positively about science?” (response options: 0 = “completely negative” to 100 = “completely positive”). The two items (r = 0.78) were averaged to create a positive construal of science scale, where a higher score indicated a more positive construal of science (M = 73.48, SD = 21.09).

To assess science knowledge perceptions, participants indicated the extent to which they agreed or disagreed with four randomly ordered statements: “I feel like I understand how science works,” “I recognize that change is a part of how science happens,” “The scientific process makes sense to me,” and “The stories I just read make me think differently about how science works” (response options: 1 = “strongly disagree” to 5 = “strongly agree”). The fourth item was negatively associated with the other three items, and averaging the four items generated a knowledge perceptions scale that was less internally consistent (Cronbach’s α = 0.69). From a face validity perspective, the fourth item might be tapping into a distinct construct, or there could be a threat to construct validity, with some participants interpreting “differently” as normatively good and others interpreting it as normatively bad. We therefore dropped the fourth item, yielding a three-item scale that was both stronger on its face and more internally consistent (Cronbach’s α = 0.78; M = 4.15, SD = 0.68); a higher score signaled greater perceived knowledge. A sensitivity analysis, where models were run using the four-item version of the scale, yielded the same pattern of results.

To assess counterarguing science, participants indicated the extent to which they agreed or disagreed with four randomly ordered statements: “I found myself agreeing with the stories [reverse coded],” “I thought of points that went against what the stories were saying,” “I accepted the stories’ claims [reverse coded],” and “While reading the stories, I was skeptical of their claims” (response options: 1 = “strongly disagree” to 5 = “strongly agree”). The items, which were adapted from previous research (Nagler et al., 2022; Niederdeppe et al., 2015), were averaged to create a counterarguing scale (Cronbach’s α = 0.75; M = 3.06, SD = 0.69), where a higher score indicated greater counterarguing of the science presented in the health news stories and social media posts featuring conflict.

Small to moderate correlations among the three measures suggest that they reflect distinct constructs, and patterns of associations were consistent with a priori expectations (e.g., more positive construal of science was associated with less counterarguing, r = −0.32, and greater knowledge perceptions, r = 0.61).

Secondary outcome: Negative affective responses.

To assess negative affective responses to conflicting health information, participants were presented with seven, randomly ordered, positive and negative emotions and asked to indicate how they felt having “read these stories” (Nagler et al., 2022; Nagler, Yzer, et al., 2019). The four negatively valenced items—“frustrated,” “annoyed,” “distressed,” and “worried” (response options: 1 = “very slightly or not at all” to 5 “extremely”)—were averaged to create a negative affective response scale (Cronbach’s α = 0.83; M = 1.79, SD = 0.77). A higher score reflected more negative affective responses.

Perceptions of the messages about science.

After responding to the primary and secondary outcome questions, we showed participants the same experimental message about science that they read earlier and asked them to rate the message on several dimensions—pecifically, whether they believed that the information was inflammatory, credible, trustworthy, accurate, and easy to understand (response options: 1 = “not at all” to 5 = “extremely”). Each of these five items was treated as a separate outcome variable. Participants in the control condition were not asked these questions, as they did not read a message about science prior to their exposure to conflicting information.

Analysis

Descriptive statistics were calculated for all study variables, and bivariate correlations were calculated to estimate associations among outcome variables. Analysis of variance (ANOVA) models were used to assess mean differences in perceptions of the three messages about science (e.g., in terms of credibility, accuracy, understandability); Tukey’s HSD post-hoc pairwise comparisons were examined to compare the three conditions to one another. For the primary analysis, we performed t-tests to assess whether any message about science would reduce negative responses on indicators of people’s perceptions of science (primary outcome)—as measured by positive construal of science, knowledge perceptions, and counterarguing—and negative affective responses to conflict (secondary outcome), relative to the no message about science control condition. For these analyses, the three message about science conditions were combined into a single intervention condition. Then we conducted a secondary analysis using ANOVA models to assess whether there were mean differences across the four conditions on each measure of the primary outcome and the secondary outcome. Tukey’s HSD post-hoc pairwise comparisons were examined to compare all four conditions to one another, as well as to assess message–target correspondence (i.e., whether each message about science moved its respective cognitive target more than the other messages did). P-values less than 0.05 were considered statistically significant. Because there were very minimal missing data (<0.5%), we analyzed the full dataset; there was no need for imputation or other methods to address missing data. All analyses were conducted using Stata 15.

Results

Participant Characteristics

Just over one-third (38.8%) of participants were 18–29 years old, and the same proportion were 30–44 years old; 15.1% were between 45–59 years, and 7.4% were 60+ or older. About half (49.3%) of participants identified as male and 47.4% as female; all others identified as nonbinary (2.4%), another gender not listed (0.6%), or preferred not to answer (0.4%). A majority of participants were White (81.9%), 8.3% were Black or African American, 8.8% were Asian, and 6.2% reported being another race(s) or preferred not to answer; 8.9% were Hispanic or Latino/a. A majority of participants were highly educated, with more than half (52.4%) completing at least four years of college and 33.0% completing some college or technical school; 14.2% had a high school degree or less.

Perceptions of the Messages about Science

There were small but statistically significant differences across the three messages about science in participants’ perceptions of accuracy (F(2,1197) = 6.34, p = 0.002), credibility (F(2,1197) = 4.25, p = 0.014), and trustworthiness (F(2,1197) = 4.19, p = 0.014) (Table 1). Participants in the rational appeal/analogical evidence condition tended to rate their message lower on these dimensions; those in the rational appeal/testimonial evidence message tended to rate their message higher, while those in the inoculation condition had ratings that generally fell in between. However, for each dimension, all of these ratings were above the mid-point of the scale. Across conditions, the messages were perceived to be consistently noninflammatory and easy to understand (F(2,1196) = 0.86, p = 0.424 and F(2,1198) = 2.63, p = 0.072, respectively).

Table 1.

Mean differences in perceptions of the three messages about science

Message about science condition Accurate Credible Trustworthy Inflammatory Easy to understand
Rational appeal/testimonial evidence message 3.89 (0.95)a 3.83 (0.98)a 3.80 (0.97)a 1.54 (0.88)a 4.24 (0.79)a
Rational appeal/analogical evidence message 3.65 (0.99)b 3.63 (1.03)b 3.59 (1.05)b 1.50 (0.82)a 4.20 (0.86)a
Inoculation message 3.80 (0.88)ab 3.73 (0.90)ab 3.70 (0.93)ab 1.57 (0.87)a 4.11 (0.79)a
Open in a new tab

Note. Standard deviations are provided in parentheses. For each condition, n range = 396–403. For all variables, range = 1 (“not at all”) to 5 (“extremely”). Within each column, means with different subscripts differ significantly at p < .05 as indicated by Tukey’s HSD (comparing all three message about science conditions). Participants in the control condition, who did not read a message about science, were not asked these questions.

Effects of Exposure to Messages about Science on Perceptions of Science

Participants who received any message about science prior to exposure to conflicting health information reported significantly more positive construal of science (t(1600) = −2.81, p = 0.005; Cohen’s d = 0.16), less counterarguing (t(1602) = 2.78, p = 0.006; Cohen’s d = 0.16), and greater knowledge perceptions (t(1602) = −4.87, p = 0.000; Cohen’s d = 0.25) than did those who received no message about science prior to their exposure to conflict (Table 2). We therefore find support for H1.

Table 2.

Mean responses to outcome variables (perceptions of science and negative affective responses to conflict) by (a) no message (control condition) vs. any message (three message conditions combined) and (b) all four experimental conditions

Experimental condition Perceptions of science Negative affective responses to conflict
Positive construal of science Counterarguing Knowledge perceptions
(a) Comparing no message vs. any message
No message about science control 70.93 (20.98)a 3.15 (0.69)a 4.02 (0.71)a 1.81 (0.75)a
Any message about science (combined) 74.33 (21.06)b 3.03 (0.69)b 4.19 (0.66)b 1.79 (0.78)a
(b) Comparing all four experimental conditions
No message about science control 70.93 (20.98)a 3.15 (0.69)a 4.02 (0.71)a 1.81 (0.75)a
Rational appeal/testimonial evidence message 75.09 (21.52)b 3.00 (0.71)b 4.23 (0.66)bc 1.71 (0.72)a
Rational appeal/analogical evidence message 73.93 (20.87)ab 3.05 (0.68)ab 4.15 (0.69)bc 1.81 (0.78)a
Inoculation message 73.99 (20.82)ab 3.06 (0.67)ab 4.20 (0.63)bc 1.83 (0.84)a
Open in a new tab

Standard deviations are provided in parentheses. For each condition, n range = 396–403. For positive construal of science, range = 0–100. For all other outcome variables, range = 1–5. Within each column, means with different subscripts differ significantly at p < .05 as indicated by t-tests (comparing no message vs. any message) or Tukey’s HSD (comparing all four experimental conditions).

Our first research question (RQ1) asked whether there were differences across the three messages about science in the mechanisms they engaged to reduce negative perceptions of science. Overall, differences were observed for each outcome—positive construal of science (F(3,1598) = 2.89, p = 0.035), counterarguing (F(3,1600) = 3.22, p = 0.022), and knowledge perceptions (F(3,1600) = 7.33, p = 0.000). The pattern of means in Table 2 suggest that the rational appeal/testimonial evidence message reduced participants’ negative perceptions to a slightly greater degree than the other two messages. Post-hoc pairwise comparisons showed that, for positive construal of science (p = 0.027; Cohen’s d = 0.20) and counterarguing (p = 0.012; Cohen’s d = 0.21), the rational appeal/testimonial evidence message was the only message condition that was significantly different from the control condition; each of the three message conditions compared favorably against the control condition in producing higher perceived knowledge (rational appeal/testimonial evidence message, p = 0.000, Cohen’s d = 0.30; rational appeal/analogical evidence message, p = 0.045, Cohen’s d = 0.18; inoculation message, p = 0.001, Cohen’s d = 0.26).

RQ2 asked whether exposure to a specific message about science would move its respective cognitive target more than the other messages about science. As shown in Table 2, there was little evidence for this message–target correspondence. The rational appeal/testimonial evidence message did move its corresponding target, knowledge perceptions, more than the control, but not more than the rational appeal/analogical evidence message nor the inoculation message.

Effects of Exposure to Messages about Science on Negative Affective Responses to Conflict

We hypothesized (H2) that exposure to a message about science would reduce negative affective responses to conflicting health information, compared to a no message about science control condition. This hypothesis was not supported (t(1602) = 0.46, p = 0.647); across conditions, little negative affect was reported (Table 2).

Discussion

This study offers an initial test of an important unanswered question: Can communicating about the process of scientific discovery affect how the public thinks about and understands science, with the ultimate goal of bolstering their ability to make sense of conflicting health information and mitigate its adverse effects? Overall, we found that any of the three messages about science—the rational appeal/analogical evidence message, the rational appeal/testimonial evidence message, or the inoculation message—reduced negative responses on indicators of people’s perceptions of science, compared to a no message about science control condition. Specifically, we observed more positive construal of science, greater knowledge perceptions, and less counterarguing among those who saw a message about science prior to exposure to conflicting health information, compared with those participants who saw only conflicting information. These were small effects, as communication effects often are (Snyder et al., 2004; Valkenburg & Peter, 2013), but on a population scale they could translate into measurable impact (Cappella & Hornik, 2009). In our study, participants were exposed to messages about science only once and for a very short time. If such a short intervention could move people on these cognitive targets, then a layered intervention, with additional exposures and combined messaging strategies, could prove powerful in its potential to interrupt carryover effects and otherwise reduce the negative influence of exposure to conflict.

Of the three messaging approaches we tested, the testimonial evidence message seemed to perform the best. Although this pattern is consistent with evidence that testimonials can be persuasive (Reinard, 1998; Shen & Bigsby, 2012), it would be premature to recommend this approach over the other two based solely on the data presented here. The perceptions of the messages about science data should prompt us to consider whether the testimonial evidence message may have performed better because it was considered by participants to be slightly more accurate, credible, and trustworthy. We also cannot know whether the testimonial evidence message might have been more effective for reasons beyond having testimonial content; for example, perhaps the mere mention of “scientists” was enough to prime participants’ responses. Given that we did not observe evidence of the predicted message–target correspondence, we do not currently know what it was about the testimonial evidence message that produced the set of effects observed here. Additional research should examine whether the three tested messaging approaches do in fact move these cognitive targets and, if not, explore whether more effective mechanism-based messages can be developed and implemented (for a review of potential messaging strategies, see, e.g., Dillard & Shen, 2012 and O’Keefe, 2015).

We did not find evidence that exposure to a message about science reduced negative affective responses to conflicting health information, compared to a no message about science control condition. This is likely due in part to the fact that exposure to conflict in this study did not generally elicit negative affective responses such as frustration and distress; negative affective responses have been observed in some (e.g., Nagler et al., 2022; Nagler, Yzer, et al., 2019) but not all (e.g., Iles et al., 2022) previous research. It is still an open question whether moving people on cognitive targets such as positive construal of science, knowledge perceptions, and counterarguing might better equip them to deal with the conflicting messages they encounter—for example, by reducing their confusion, backlash, and, in turn, their susceptibility to carryover effects, all of which have been consistently documented in prior work (e.g., Chang, 2015; Clark et al., 2019; Iles et al., 2022; Nagler et al., 2022; Nagler, Yzer, et al., 2019). The current study stops short of testing this complete pathway, but it does underscore the promise of mechanism-based messages about science in forestalling the negative sequelae of conflicting information exposure. Recent research, which found that a forewarning message about science reduced negative responses to conflicting guidance about COVID-19 (Gretton et al., 2021), provides additional evidence for the promise of such messaging strategies.

Limitations

Several study limitations should be acknowledged. First, although this experiment was fielded with a large national sample, these are not population-based data and thus generalizability to the broader U.S. population is constrained (e.g., the sample skewed younger and more highly educated). Second, examining potential subgroup effects was beyond the scope of this study, yet it is conceivable that there could be differential effects of the messaging strategies we tested. For example, November 2020 data from Pew’s American Trends Panel, which were reported in the NSF’s National Science Board’s 2022 Science and Engineering Indicators report, indicate that some members of the public have a better understanding of the scientific process (e.g., 66% believed the scientific method “produced findings that are meant to be continually tested and updated over time,” while 34% believed the process produces “unchanging core principles and truths” or were unsure) (National Science Board, 2022). Those with a better understanding—who have elsewhere been described as being higher in “civic science literacy” (Howell & Brossard, 2021) or “research literacy” (Shi et al., 2022)—may be more influenced by the messaging strategies in this study, as the messages could prime science-related beliefs and knowledge that they already hold. Third, the current study does not enable us to test whether these messaging strategies are equally effective in addressing different types of conflicting health information. Given evidence that certain types of conflict, such as conflicting information coming from the same source (e.g., a shifting health recommendation), may be particularly aversive (Iles et al., 2022), this is an important future research inquiry. Last, given the dearth of research on communicating about the process of scientific discovery, both the message stimuli and several measures were developed for the purposes of this research. Additional work on developing mechanism-based messages, including those that test different types of inoculation strategies in the context of conflicting science (such as explicitly including a weakened version of counterarguments; McGuire, 1964), is necessary. So, too, is research that refines measures of potential cognitive targets. This may be particularly important in the case of counterarguing, given several possible targets—for example, counterarguing the science presented in stimuli versus counterarguing the application of that science into a health recommendation versus counterarguing the scientific process in general—and the need to disentangle which specific components of a message participants have counterargued.

Conclusions

The public information environment remains awash in conflicting health information, and given compelling evidence that exposure to such content has adverse downstream consequences, there is a pressing need for strategies that can interrupt and attenuate such impacts. The results of the current study suggest that developing messages that describe the process of scientific discovery could prove successful—not only in improving perceptions of science but perhaps ultimately in better equipping the public to make sense of conflicting information and its causes. Although we find initial evidence for this possibility here, additional tests of such strategies, particularly as part of larger interventions with multiple messages across multiple exposures, could have important implications for health and science communication. One content analysis found that only 17.2% of news stories featuring messages about contradictory nutrition research and recommendations contained information on the process of scientific discovery (Nagler, 2010). Such content could help readers to make sense of conflicting research findings and health guidance, and yet this data point suggests such messages may be rare in health news stories, especially those featuring conflict. Perhaps embedding such content within health and science news stories—and even integrating it into public health campaigns—could help to buttress audiences against the impacts of seemingly ever-shifting health advice.

Highlights.

  • Strategies are needed to mitigate the effects of conflicting health information.

  • Data come from an online survey experiment fielded with 1,604 U.S. adults.

  • Messages about the process of scientific discovery shape public perceptions of science.

  • Such messages strengthen positive construal of science and knowledge perceptions.

  • Messaging about science may attenuate the impacts of conflicting health information.

Acknowledgements

The authors thank Rachel Dallman for graphic design support. Additional thanks to the Liberal Arts Technologies and Innovation Services (LATIS) team at the University of Minnesota for their assistance with data collection.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interests with respect to the research, authorship, and/or publication of this article.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain

References

  1. Annenberg Public Policy Center (APPC). (2018). How well do major media outlets cover the process of scientific discovery? Annenberg Science Media Monitor -- Report 1. Available from https://www.annenbergpublicpolicycenter.org/feature/science-media-monitor-report-1/
  2. Barabas J, Carter B, & Shan K (2020). Analogical framing: How policy comparisons alter political support for health care reform. American Politics Research, 48(5), 596–611. 10.1177/1532673X20926125 [DOI] [Google Scholar]
  3. Cappella J, & Hornik R (2009). The importance of communication science in addressing core problems in public health. In Buzzanell PM & Carbaugh D (Eds.), Distinctive qualities in communication research (pp. 73–86). Routledge. [Google Scholar]
  4. Chang C (2015). Motivated Processing: How People Perceive News Covering Novel or Contradictory Health Research Findings. Science Communication, 37(5), 602–634. 10.1177/1075547015597914 [DOI] [Google Scholar]
  5. Clark D, Nagler RH, & Niederdeppe J (2019). Confusion and nutritional backlash from news media exposure to contradictory information about carbohydrates and dietary fats. Public Health Nutrition, 22(18), 3336–3348. 10.1017/S1368980019002866 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Compton J (2012). Inoculation Theory. In Dillard J & Shen L (Eds), The SAGE Handbook of Persuasion: Developments in Theory and Practice (2nd ed.) (pp. 220–236). SAGE Publications, Inc. 10.4135/9781452218410 [DOI] [Google Scholar]
  7. de Wit JBF, Das E, & Vet R (2008). What works best: Objective statistics or a personal testimonial? An assessment of the persuasive effects of different types of message evidence on risk perception. Health Psychology, 27(1), 110–115. DOI: 10.1037/0278-6133.27.1.110 [DOI] [PubMed] [Google Scholar]
  8. Dillard J, & Shen L (2012). The SAGE Handbook of Persuasion: Developments in Theory and Practice (2nd ed.). SAGE Publications, Inc. 10.4135/9781452218410 [DOI] [Google Scholar]
  9. Fowler EF, Nagler RH, Banka D, & Gollust SE (2022). Effects of politicized media coverage: Experimental evidence from the HPV vaccine and COVID-19. Progress in Molecular Biology and Translational Science, 188(1), 101–134. 10.1016/bs.pmbts.2021.11.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gretton JD, Meyers EA, Walker AC, Fugelsang JA, & Koehler DJ (2021). A brief forewarning intervention overcomes negative effects of salient changes in COVID-19 guidance. Judgment & Decision Making, 16(6), 1549–1574. 10.1017/S1930297500008548 [DOI] [Google Scholar]
  11. Gustafson A, & Rice RE (2020). A review of the effects of uncertainty in public science communication. Public Understanding of Science, 29(6), 614–633. 10.1177/0963662520942122 [DOI] [PubMed] [Google Scholar]
  12. Hornikx J (2005). A review of experimental research on the relative persuasiveness of anecdotal, statistical, causal, and expert evidence. Studies in Communication Sciences, 5, 205–216. 10.5169/seals-790921 [DOI] [Google Scholar]
  13. Howell EL, & Brossard D (2021). (Mis)informed about what? What it means to be a science-literate citizen in a digital world. Proceedings of the National Academy of Sciences, 118(15), e1912436117. 10.1073/pnas.1912436117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Ihekweazu C (2021). Is Coffee the Cause or the Cure? Conflicting Nutrition Messages in Two Decades of Online New York Times’ Nutrition News Coverage. Health Communication, 1–15. 10.1080/10410236.2021.1950291 [DOI] [PubMed] [Google Scholar]
  15. Iles IA, Gillman AS, O’Connor LE, Ferrer RA, & Klein WMP (2022). Understanding responses to different types of conflicting information about cancer prevention. Social Science & Medicine, 311, 115292. 10.1016/j.socscimed.2022.115292 [DOI] [PubMed] [Google Scholar]
  16. Iles IA, Gillman AS, Platter HN, Ferrer RA, & Klein WMP (2021). Investigating the potential of inoculation messages and self-affirmation in reducing the effects of health misinformation. Science Communication, 43(6), 768–804. 10.1177/10755470211048480 [DOI] [Google Scholar]
  17. Jensen JD (2008). Scientific Uncertainty in News Coverage of Cancer Research: Effects of Hedging on Scientists’ and Journalists’ Credibility. Human Communication Research, 34(3), 347–369. 10.1111/j.1468-2958.2008.00324.x [DOI] [Google Scholar]
  18. Keer M, van den Putte B, de Wit J, & Neijens P (2013). The effects of integrating instrumental and affective arguments in rhetorical and testimonial health messages. Journal of Health Communication, 18(9), 1148–1161. 10.1080/10810730.2013.768730 [DOI] [PubMed] [Google Scholar]
  19. McGuire WJ (1964). Inducing resistance to persuasion: Some contemporary approaches. In Berkowitz L (Ed.), Advances in experimental social psychology (Vol. 1, pp. 191–229). Academic Press. [Google Scholar]
  20. Nagler RH (2010). Steady diet of confusion: Contradictory nutrition messages in the public information environment. Unpublished dissertation, University of Pennsylvania. Philadelphia, PA. [Google Scholar]
  21. Nagler RH, Fowler EF, Marino NM, Mentzer KM, & Gollust SE (2019). The Evolution of Mammography Controversy in the News Media: A Content Analysis of Four Publicized Screening Recommendations, 2009 to 2016. Womens Health Issues, 29(1), 87–95. 10.1016/j.whi.2018.09.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Nagler RH, & LoRusso SM (2018). Conflicting Information and Message Competition in Health and Risk Messaging. In Parrott R (Ed.), Encyclopedia of Health and Risk Message Design and Processing. Oxford University Press. 10.1093/acrefore/9780190228613.013.292 [DOI] [Google Scholar]
  23. Nagler RH, Vogel RI, Gollust SE, Yzer MC, & Rothman AJ (2022). Effects of Prior Exposure to Conflicting Health Information on Responses to Subsequent Unrelated Health Messages: Results from a Population-Based Longitudinal Experiment. Annals of Behavioral Medicine, 56(5), 498–511. 10.1093/abm/kaab069 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Nagler RH, Yzer MC, & Rothman AJ (2019). Effects of Media Exposure to Conflicting Information About Mammography: Results From a Population-based Survey Experiment. Annals of Behavioral Medicine, 53(10), 896–908. 10.1093/abm/kay098 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. National Science Board (NSB), National Science Foundation. (2022). Science and Technology: Public Perceptions, Awareness, and Information Sources. Science and Engineering Indicators 2022. NSB-2022–7. Available from https://ncses.nsf.gov/pubs/nsb20227 [Google Scholar]
  26. Niederdeppe J, Heley K, & Barry CL (2015). Inoculation and Narrative Strategies in Competitive Framing of Three Health Policy Issues. Journal of Communication, 65(5), 838–862. 10.1111/jcom.12162 [DOI] [Google Scholar]
  27. O’Keefe DJ (2015). Persuasion: Theory and research (3rd ed). Sage Publications. [Google Scholar]
  28. Peters HP., & Dunwoody S. (2016). Scientific uncertainty in media content: Introduction to this special issue. Public Understanding of Science, 25(8), 893–908. 10.1177/0963662516670765 [DOI] [PubMed] [Google Scholar]
  29. Ratcliff CL, Wicke R, & Harvill B (2022). Communicating uncertainty to the public during the COVID-19 pandemic: A scoping review of the literature. Annals of the International Communication Association, 1–30. 10.1080/23808985.2022.2085136 [DOI] [Google Scholar]
  30. Reinard JC (1998). The persuasive effects of testimonial assertion evidence. In Allen M & Preiss RW (Eds.), Persuasion: Advances through meta-analysis (pp. 69–86). Hampton Press. [Google Scholar]
  31. Scott D (2022, January 14). The most consistently botched part of the U.S. pandemic response: Why are public health officials so bad at talking to us? Vox. Available from https://www.vox.com/coronavirus-covid19/22870268/cdc-covid-19-guidelines-isolation-boosters-masks
  32. Shen L, & Bigsby E (2012). The effects of message features: Content, structure, and style. In Dillard J & Shen L (Eds), The SAGE Handbook of Persuasion: Developments in Theory and Practice (2nd ed.) (pp. 20–35). SAGE Publications, Inc. 10.4135/9781452218410 [DOI] [Google Scholar]
  33. Shi W, Rothman AJ, Yzer MC, & Nagler RH (2022). Effects of Exposure to Conflicting Information About Mammography on Cancer Information Overload, Perceived Scientists’ Credibility, and Perceived Journalists’ Credibility. Health Communication, 1–10. 10.1080/10410236.2022.2077163 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Sinvani LD, Nouryan CN, Pekmezaris R, & Wolf-Klein GP (2013). Vitamins and dietary supplements: Controversy and confusion. Consultant360, 21(5). Available from https://www.consultant360.com/articles/vitamins-and-dietary-supplements-controversy-and-confusion [Google Scholar]
  35. Snyder LB, Hamilton MA, Mitchell EW, Kiwanuka-Tondo J, Fleming-Milici F, & Proctor D (2004). A Meta-Analysis of the Effect of Mediated Health Communication Campaigns on Behavior Change in the United States. Journal of Health Communication, 9(suppl 1), 71–96. 10.1080/10810730490271548 [DOI] [PubMed] [Google Scholar]
  36. Valkenburg PM, & Peter J (2013). Five Challenges for the Future of Media-Effects Research. International Journal of Communication (1932–8036), 7, 197–215. Available at http://ijoc.org [Google Scholar]
  37. Wackowski OA, Sontag JM, Singh B, King J, Lewis MJ, Steinberg MB, & Delnevo CD (2020). From the Deeming Rule to JUUL—US News Coverage of Electronic Cigarettes, 2015–2018. Nicotine & Tobacco Research, 22(10), 1816–1822. 10.1093/ntr/ntaa025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Wang D, & Zhou Y (2022). The popularity of contradictory information about COVID-19 vaccine on social media in China. Computers in Human Behavior, 134, 107320. 10.1016/j.chb.2022.107320 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES