Effects of Anti- vs. Pro-vaccine Narratives on Responses by Recipients Varying in Numeracy: A Cross-sectional Survey-Based Experiment

Abstract

Purpose

To inform their health decisions, patients may seek narratives describing other patients' evaluations of their treatment experiences. Narratives can provide anti-treatment or pro-treatment evaluative meaning that especially low-numerate patients struggle to derive from statistical information. Here, we examined whether anti-vaccine (vs. pro-vaccine) narratives had relatively stronger effects on the perceived informativeness and judged vaccination probabilities reported among recipients with lower (vs. higher) numeracy.

Methods

Participants (n=1113) from a nationally representative US internet panel were randomly assigned to an anti-vaccine or pro-vaccine narrative, as presented by a patient discussing a personal experience, a physician discussing a patient's experience, or a physician discussing the experience of 50 patients. Anti-vaccine narratives described flu experiences of patients who got the flu after getting vaccinated; pro-vaccine narratives described flu experiences of patients who got the flu after not getting vaccinated. Participants indicated their probability of getting vaccinated and how informative they perceived the narratives to be.

Results

Participants with lower numeracy generally perceived narratives as more informative. By comparison, participants with higher numeracy rated especially anti-vaccine narratives as less informative. Anti-vaccine narratives reduced judged vaccination probabilities compared to pro-vaccine narratives, especially among participants with lower numeracy. Mediation analyses suggested that low-numerate individuals' vaccination probabilities were reduced by anti-vaccine narratives – and to a lesser extent boosted by pro-vaccine narratives – due to their perceiving narratives as more informative. These findings held whether narratives were provided by patients or physicians.

Conclusions

Patients with lower numeracy may rely more on narrative information when making their decisions. Findings have implications for the development of health communications and decision aids.

Introduction

The CDC recommends flu vaccinations for almost everyone 6 months and older.⁽¹⁾ Getting vaccinated reduces the probability of getting the flu, and can make symptoms milder among those who do get sick.⁽²⁾ Vaccinations have been associated with a 74% reduction in children's flu-related admissions to pediatric intensive care, and a 71% reduction in adults' flu-related hospitalizations.^(3,4) A publicly available CDC pamphlet covers such statistical information about vaccine benefits.⁽⁵⁾

Patients often want to receive narratives to learn about other patients' experiences, in addition to statistical evidence.⁽⁶⁾ Health care providers are patients' most influential source for flu vaccination decisions,⁽⁷⁾ and may provide the CDC's statistical information in combination with narratives describing other patients' experiences.^(8,9,10) In addition, patients may search for the narratives of other patients' experiences online, where they are likely to find anti-vaccine narratives describing patients getting sick after being vaccinated. ⁽¹¹⁾ However, a drawback of providing narratives is that they may sway people's decisions, even if they provide information that disagrees with the accompanying statistical information.^(12,13)

The effect of narratives' anti-treatment or pro-treatment evaluative meaning

Narratives can highlight anti-treatment vs. pro-treatment evaluative meaning, by describing whether other patients were dissatisfied or satisfied with specific treatments.⁽¹⁴⁾ Anti-treatment narratives often describe dissatisfied patients who were sick after a treatment, while pro-treatment narratives describe satisfied patients who were healthy after a treatment.^(15,16)

It has been proposed that anti-treatment narratives will be more influential on decision making than pro-treatment narratives, because they can seem relatively more informative.⁽¹⁴⁾ According to the ‘social amplification of risk’ framework, one reason that anti-treatment narratives are treated as more informative than pro-treatment narratives, is that anti-treatment narratives question existing health messages about the benefits of treatments while pro-treatment narratives present more of the same.⁽¹⁷⁾ Among other things, this framework also predicts that a narrative will be treated as more informative when its (anti-vaccine or pro-vaccine) message is repeated more often.⁽¹⁷⁾

Although no studies have directly compared the perceived informativeness of a single anti-treatment vs. pro-vaccine treatment narrative, it has been found that people's responses to a set of narratives tend to be affected by the proportion of anti-treatment narratives or pro-treatment narratives, even if it is not representative of actual statistics.^{(15, 16, 18)} Similarly, individuals who read relatively more anti-vaccine narratives report lower vaccination intentions despite also having received statistical information about vaccine efficacy and adverse effects. ⁽¹⁹⁾

The anti-treatment and pro-treatment narratives that have been used in prior studies that tested effects of disproportionate presentations confounded two features that could potentially affect their perceived informativeness: whether or not they questioned the benefits of treatments, and whether or not their content involved positive or negative health outcomes. ^{(15, 16)} That is, prior anti-treatment narratives described patients who were sick after receiving treatment, thus questioning treatments offered by health care providers, and presenting negative patient experiences. In contrast, prior pro-treatment narratives described the patients who were healthy after receiving treatment, thus confirming the benefits of treatments offered by health care providers, and presenting positive patient experiences.ⁱ

To allow for a systematic comparison of the effect of a single anti-treatment vs. pro-treatment narrative on decisions, anti-treatment and pro-treatment narratives should describe similar patient experiences. Therefore, we created narratives about flu shots that varied in their anti-vaccine or pro-vaccine evaluative meaning, while providing otherwise equivalent patient experiences. That is, anti-vaccine narratives described patients who wished they had not gotten a flu shot because they got sick after getting a flu shot, and pro-vaccine narratives described patients who wished they had gotten a flu shot because they got sick after not getting one.

The effect of narratives' information source

A meta-analysis has suggested that narratives are more likely to influence choices when they are shared by patients in first-person perspective, than when they are shared by health care providers in third-person perspective.⁽¹³⁾ In an experiment that directly tested this idea, participants facing two treatment options were more likely to choose the one that was recommended in a patient's narrative over the one that was recommended in a health care provider's narrative.⁽²⁰⁾ Possibly, this effect occurs because narratives are perceived as more informative when they are provided by patients than by health care providers.

After years of seeing patients with similar conditions, health care providers will be able to build a narrative of their ‘proven experience.’ ⁽²¹⁾ One study found that participants perceived a claim as more informative when it came from an expert describing a pattern among multiple people rather than from a lay person describing one personal experience.⁽²²⁾ However, it has not yet been tested whether the number of patients referred to in a health-care provider's narrative affects patients' perceptions of the informativeness of the presented narrative, and subsequent decision making.

The role of numeracy

How people respond to different narratives may depend on their numeracy. Numeracy refers to people's ability to use numbers, ^(23,24) which varies widely in the general population.^(25,26) Low numeracy is related to generating less valid health risk perceptions,⁽²⁷⁾ misunderstanding risk/benefit information,⁽²⁸⁾ making decision errors,^(29,30) and worse disease management.⁽³¹⁾ More importantly, individuals with lower numeracy are often less motivated to engage with statistical information, ^(32,33) and give more weight to non-numeric information, even if it is irrelevant. ^(34,35)

One study reported that individuals with lower numeracy relied more on narrative intelligence reports than on statistical evidence, in judgments of terror risk.⁽³⁶⁾ Non-numeric information may be especially helpful to low-numerate individuals in deriving evaluative meaning, which they find harder to extract from statistical information.⁽³⁷⁾ To date, we are not aware of studies that have examined the role of numeracy in responses to anti- or pro-vaccine narratives, as provided by patients or health-care providers.

Research questions

Based on our review of the literature, we posed three research questions. Our first research question asked whether anti-vaccine (vs. pro-vaccine) narratives would especially affect participants with lower numeracy, in terms of eliciting relatively (a) higher ratings of perceived informativeness and (b) lower vaccination probabilities, as compared to participants with higher numeracy. Our second research question asked whether (a) more perceived informativeness and (b) higher vaccination probabilities would be observed for narratives provided by another patient rather than by a health care provider discussing the experience of one or fifty patients, especially among recipients with lower numeracy. Our third research question asked whether any statistically significant differences in judged vaccination probabilities (as examined for research questions 1a, 2a) were mediated by differences in level of perceived informativeness (as examined for research questions 1b, 2b).

We presented anti-vaccine or pro-vaccine narratives to participants from a US nationally representative sample. The participants had varying numeracy. Narratives were provided by one patient as a personal experience, by a health care provider as another patient's experience, or by a health care provider as the experience of 50 patients.

Methods

Sample

Participants were members of the nationally representative University of Southern California's Understanding America Study (UAS) panel.⁽³⁸⁾ Since 2014, the UAS has mailed invitations to randomly selected US addresses to recruit adults aged 18 and older. If needed, interested individuals were provided with internet access and a computer or tablet. UAS members get regular invitations to participate in online surveys. They receive $20 for approximately every 30 minutes of survey time. At the time of our survey, the UAS had 2024 members who had completed the entry survey. Of those, 1761 received an invitation for our survey. A total of 1113 answered the questions relevant to our analyses, resulting in a 63% response rate. Sample characteristics are discussed in the Results.

Procedure

The Institutional Review Board of the University of Southern California approved data collection. We collected our data during the influenza pre-season (21 September through 2 November 2015), with flu activity peaking between December and February.⁽³⁹⁾ Although we intended to close the survey on 2 November 2015, it inadvertently remained available through the flu season until February 29^th of 2016. Including the additional 147 surveys completed after our intended closing date did not affect the main findings reported here (α=.05).

Participants were randomly assigned to one of six equivalent narratives (for full text, see Appendix). Flesch-Kincaid readability statistics for all narratives was at U.S. grade level 4.^(40,41) Average adult literacy in the U.S. varies around grade level 7-9, so readability guidelines recommend that patient communications do not exceed that range.^(42,43) The first dimension that we varied across our narratives was whether they were anti-vaccine or pro-vaccine. Narratives communicated anti-vaccine evaluative meaning by describing patients who wished they had not gotten a flu shot, because they had the flu after getting a flu shot. Narratives communicated pro-vaccine evaluative meaning by describing patients who wished they had gotten a flu shot, because they had the flu after not getting a flu shot. The second dimension was whether the narrative involved one patient discussing a personal experience, a doctor discussing one patient's experience, or a doctor discussing the experience of 50 patients (e.g. patient n=1, doctor n=1 and doctor n=50). Following previous work,⁽¹⁹⁾ all participants additionally received the same statistical information from a publicly available CDC pamphlet about vaccine benefits (Supplemental Materials available from the authors upon request).⁽⁵⁾ We counterbalanced the order of the presented narrative and the CDC pamphlet.

After reading the presented narrative, all participants judged their probability of getting a flu shot this year and next year, and indicated their perceptions of the narrative's informativeness. These questions were also asked about the CDC pamphlet, but our research questions for this paper focus on responses to the narratives. All questions appear in the Supplemental Materials that are available from the authors upon request. Our funders had no role in the study.

Measures

Numeracy

As part of their panel entry survey, participants received an 8-item numeracy measure compiled from existing scales (e.g., “In the Acme Publishing Sweepstakes, the chance of winning a car is 1 in 1000. What percentage of tickets for the Acme Publishing Sweepstakes win a car?”).⁽²⁶⁾ Participants' responses were coded for accuracy. Missing responses were coded as incorrect, but 98.7% of our participants had no missing responses. Cronbach's alpha across the eight items was sufficient to warrant the computation of the proportion of correct responses (α=.73).

Judged vaccination probability

Participants gave their probability of getting a flu shot during the current flu season (Fall 2015-Spring 2016) and the next (Fall 2016-Spring 2017). Participants responded on visual probability scales ranging from 0-100%, which are less likely to produce an overuse of the 50% response as compared to fill-in-the-blank probability questions.⁽⁴⁴⁾ Agreement between the two judged vaccination probabilities is seen in Cronbach's alpha (α=.97), as well as the Spearman-Brown coefficient (r_sb=.97), which is a better estimate for reliability when scales have two items.⁽⁴⁵⁾ Hence, internal consistency was sufficient to warrant the computation of the average across the two measures.

Perceived informativeness of narrative

Participants rated the narrative they received on 1-5 rating scales, including (1) how convincing the information was; (2) how easy the information was to understand; (3) how interesting the information was; (4) how worrisome the information was; (5) how useful the information would be if they were deciding whether or not to get a flu shot; (6) how relevant the information was to them personally; (7) how trustworthy the source of the information was; (8) on how much experience the information was based; and (9) on how much evidence the information was based. Cronbach's alpha was sufficient across the nine responses to warrant the computation of an overall mean score (α=.88).

Perceived risk and harm reduction

We also asked participants about two other variables that have been associated with getting vaccinated. ⁽⁴⁶⁾ First, they judged the probability of getting the flu if they did not get a flu shot, and if they did get a flu shot. Visual probability scales ranged from 0-100%. We subtracted the latter from the former, to indicate perceived risk reduction from getting the flu shot. Second, participants assessed how sick they would be if they got the flu, if they did not get a flu shot, and if they did get a flu shot. Rating scales ranged from 1 (=not sick at all) to 5 (=very sick). We computed whether the latter was higher (=1) the same (=0) or lower (-1) as compared to the former, to indicate perceived harm reduction from getting the flu shot.

Experiences

Participants reported on experiences relevant to getting vaccinated.^(47,48,49) First, we asked participants how long ago they had flu shots and the flu, with options including “never” as well as specific previous seasons and “don't know or don't remember.” Second, participants indicated who helped them to make decisions about whether or not to get flu shots, with options including “no one,” as well as health care professionals and family members.

Analysis plan

Analyses were conducted in SPSS 21, with α=.05 significance levels. To examine rates of participation, we compared our participants to the rest of the nationally representative panel on demographics and measures completed in the panel entry survey. These analyses excluded the 30 panel members who skipped the numeracy assessment. We identified potential control variables as those significantly correlated with numeracy.

Next, we conducted separate Analyses of Covariance (ANCOVA) on perceived informativeness and on judged vaccination probabilities, comparing groups of participants who had received narratives varying in evaluative meaning (anti-vaccine vs. pro-vaccine) and information source (patient n=1 vs. doctor n=1 vs. doctor n=50), while including continuous variables for numeracy and its interactions with these two conditions. We controlled for main effects of and interactions with presentation order, as well as perceived harm and risk reduction, and demographic variables correlated with both numeracy and vaccination intentions (as identified in the Results). Varying the included control variables did not affect the main finding of the paper. To test our first research question, we examined the effect of the interaction between anti-vaccine vs. pro-vaccine evaluative meaning and numeracy on (a) perceived informativeness and (b) judged vaccination probabilities. To test our second research question, we examined the effect of the interaction between information source and numeracy on (a) perceived informativeness and (b) judged vaccination probabilities. Auxiliary analyses involved one-sample t-tests to examine whether perceived harm reduction and perceived risk reduction were greater than 0, as reported among participants who received anti-vaccine vs. pro-vaccine narratives, and ANCOVAs to examine group differences in these variables.

We tested our third research question by conducting a moderated mediation analysis that examined whether the significant interaction effect of participants' numeracy and narratives' anti-vaccine vs. pro-vaccine evaluative meaning on judged vaccination probabilities was statistically explained by perceived level of informativeness. We also conducted separate multi-mediation analyses for anti-vaccine and pro-vaccine narratives. All mediation analyses used Hayes' SPSS PROCESS macro, ⁽⁵⁰⁾ and controlled for narrative's information source, presentation order, perceived risk reduction, perceived harm reduction, and variables correlated with numeracy. We did not conduct such a moderated mediation analysis to examine the interaction effect of numeracy and information source, because we found no significant interaction on vaccination probabilities, or on perceived informativeness. Following previous work,⁽³⁶⁾ the numeracy variable was continuous in all analyses, but was divided into three groups of similar sizes (with cut-offs at the 30^th percentile score of .33 and the 66^th percentile score of .63) to simplify graphs (Figures 1-2).

Figure 1.

Effect of numeracy and anti-vaccine versus pro-vaccine narratives on perceived informativeness.

Note: Numeracy was divided into three groups of equivalent sizes for this graph, but entered as a continuous variable in all analyses. Error bars reflect standard errors. There was a significant interaction between numeracy and anti-vaccine vs. pro-vaccine narratives, in addition to a significant main effect of numeracy (p<.05, for each).

Figure 2.

Effect of numeracy and anti-vaccine versus pro-vaccine narratives on judged probability of getting vaccinated.

Note: Numeracy was divided into three groups of equivalent sizes for this graph, but entered as a continuous variable in all analyses. Error bars reflect standard errors. There was a significant interaction between numeracy and anti-vaccine vs. pro-vaccine narratives, in addition to a significant main effect for anti-vaccine vs. pro-vaccine narratives (p<.05, for each).

Results

Sample characteristics

Participants who completed our survey were slightly more numerate than the rest of the panel, on average (M=.48, SD=.25 vs. M=.46, SD=.25), t(2022)=-2.14, p=.03, despite having similar medians (=.50). This median was also observed in previous samples.⁽²⁶⁾ Those who completed our survey were also more likely to have a college degree (44% vs. 38%), χ(1)=7.15, p=.01. They were similarly likely to report being female (52% vs. 53%), χ(1)=.28, p=.59 and non-white (16% vs. 16%), χ(1)=.00, p=.97. They had similar ages (M=49.06, SD=15.17 vs. M=48.12, SD=15.60), t(2031)=-1.38, p=.17.

Higher numeracy was significantly correlated with being male (r=.26, p<.001), white (r=.18, p<.001), college-educated (r=.41, p<.001), perceiving harm reduction from getting a flu shot (r=.07, p=.01), and making flu shot decisions without help from others (r=-.13, p=.01). Our analyses therefore controlled for these variables. Numeracy was not correlated with age (r=-.02, p=.10), perceived risk reduction from getting a flu shot (r=.06, p=.06), ever having had a flu shot (r=.06, p=.06), or ever having had the flu (r=.05, p=.10).

Perceived informativeness of narrative (Research Questions 1a and 2a)

Figure 1 shows the perceived informativeness of narratives, by participants' numeracy and anti-vaccine vs. pro-vaccine narratives. There was a main effect of numeracy, F(1, 1083)=25.41, p<.001, with lower numeracy being related to generally perceiving narratives as more informative (r=-.20, p<.001). There was no main effect of anti-vaccine vs. pro-vaccine narratives, F(1, 1083)=.12, p=.73.

When testing for Research Question 1a, we found a significant interaction between numeracy and the anti-vaccine vs. pro-vaccine narratives, F(1, 1083)=6.48, p=.01. That is, participants with higher numeracy perceived anti-vaccine narratives as less informative than participants with lower numeracy (r=-.28, p<.001). They also perceived pro-vaccine narratives as less informative than participants with lower numeracy, but the pattern was less strong (r=-.13, p<.01). There were no other significant main effects or interactions, including for information source by numeracy (Research Question 2a).

Judged vaccination probability (Research Questions 1b and 2b)

Figure 2 shows judged vaccination probabilities by participants' numeracy and anti-vaccine vs. pro-vaccine narratives. There was a main effect of narratives' evaluative meaning, suggesting that across participants varying in numeracy, anti-vaccine narratives produced significantly lower vaccination probabilities than did pro-vaccine narratives (M=54.59, SD=42.55 vs. M=58.47, SD=41.29), F(1, 1083)=3.97, p<.05. We found no main effect of numeracy, F(1, 1083)=.84, p=.36.

When testing for Research Question 1b, we found a significant interaction between numeracy and narratives' evaluative meaning, F(1, 1083)=4.09, p=.04. That is, individuals with lower numeracy gave significantly lower vaccination probabilities than did higher numeracy individuals after reading anti-vaccine narratives (r=.16, p<.001), but gave similar vaccination probabilities after reading pro-vaccine narratives (r=.05, p=.28). There were no other significant main effects or interactions, including for information source by numeracy (Research Question 2b).

Perceived risk and harm reduction (Auxiliary Analyses)

Flu shots were perceived as relatively effective across all participants, as seen in perceived risk reduction being significantly greater than 0 both for those reading anti-vaccine narratives (M=7.87, SD=29.66, t(570)=6.34, p<.001) and those reading pro-vaccine narratives (M=13.25, SD=30.44, t(541)=10.13, p<.001). Similarly, perceived harm reduction was significantly greater than 0 among participants who received anti-vaccine narratives (M=.32, SD=.63, t(570)=12.26, p<.001), and participants who received pro-vaccine narratives (M=.46, SD=.62, t(541)=17.46, p<.001).

Yet, anti-vaccine narratives did lead to less favorable perceptions of flu shots, as compared to pro-vaccine narratives. That is, we found lower perceived risk reduction due to vaccination after reading anti-vaccine narratives rather than after reading pro-vaccine narratives (M=7.87, SD=29.66 vs. M=13.25, SD=30.44), as seen in a significant main effect of anti- vs. pro-vaccine narratives, F(1, 1085)=7.11, p=.01. Similarly, we found lower perceived harm reduction due to vaccination after anti-vaccine narratives than after pro-vaccine narratives (M=.32, SD=.63 vs. M=.46, SD=.62), as seen in a significant main effect of anti- vs. pro-vaccine narratives, F(1, 1085)=4.62, p=.03. There were no other significant main effect or interactions on either of these two dependent variables.

Mediation analyses (Research Question 3)

As noted in the analysis plan, we examined whether low-numerate individuals' lower judged vaccination probabilities, as reported after reading anti-vaccine vs. pro-vaccine narrative narratives, were statistically explained by their perceiving especially anti-vaccine narratives as being more informative. Figure 3A shows the mediation model for anti-vaccine narratives, and Figure 3B for pro-vaccine narratives. Both models indicated that the relationship between numeracy and judged vaccination probabilities was significantly mediated by narratives' perceived informativeness (95% CI=1.52, 8.84 for anti-vaccine narratives; 95% CI=-13.04, -3.28 for pro-vaccine narratives). Indeed, significant mediation can occur even with non-significant relationships such as those for pro-vaccine narratives (Figure 3B).⁽⁵¹⁾

Figure 3.

Mediation model for (A) anti-vaccine narratives and (B) pro-vaccine narratives.

Note: Both models represent significant mediation (p<.05). Relationship between numeracy and judged vaccination probability is shown after considering perceived informativeness (with effect before considering perceived informativeness between parentheses). A separate moderated mediation analysis showed that the mediation path was significantly different for anti-vaccine vs. pro-vaccine narratives (95% CI=-18.87, -6.03).

A moderated mediation analysis confirmed that the mediation pattern was significantly different for anti-vaccine narratives as compared to pro-vaccine narratives (95% CI=-16.62, -4.39), in two ways. First, narratives' evaluative meaning significantly moderated the relationship between numeracy and perceived informativeness of the narratives (B=.56, p<.01), with decreases in numeracy being associated with perceiving more informativeness for anti-vaccine (Figure 3A) than for pro-vaccine narratives (Figure 3B). Second, narratives' anti- vs. pro-vaccine evaluative meaning significantly moderated the relationship between narratives' perceived informativeness and judged vaccination probabilities (B=20.18, p<.001). That is, perceiving anti-vaccine narratives as more informative was associated with lower vaccination probabilities (Figure 3A), while perceiving pro-vaccine narratives as more informative was associated with higher judgments of vaccination probabilities (Figure 3B).

Overall, these analyses suggested that individuals with lower numeracy reported vaccination probabilities that were reduced by anti-vaccine narratives, and to a lesser extent boosted by pro-vaccine narratives – due to perceiving narratives as more informative than did individuals with higher numeracy.

Discussion

In a national US sample, we examined how recipients varying in numeracy responded to anti-vaccine and pro-vaccine narratives. We found that individuals with lower numeracy generally perceived narratives as more informative. By comparison, participants with higher numeracy found especially anti-vaccine narratives less informative. Anti-vaccine narratives reduced judged vaccination probabilities as compared to pro-vaccine narratives, especially among participants with lower numeracy. A mediation analysis suggested that lower-numerate individuals' judged vaccination probabilities were reduced by anti-vaccine narratives and to a lesser extent boosted by pro-vaccine narratives – due to their perceiving narratives as being relatively more informative than did higher-numerate individuals.

Yet, we found no evidence of anti-vaccine narratives actually being perceived as more informative, as compared to pro-vaccine narratives, as would have been predicted by the social amplification of risk framework.⁽¹⁷⁾ Moreover, our findings were unaffected by whether narratives were provided by a patient describing a personal experience, a physician describing a patient's experience, or a physician describing 50 patients' experience. Narratives' source of information had no effect on reported judged vaccination probabilities or on perceptions of how informative narratives were. Perhaps because our narratives were designed to present equivalent content, they showed similar effectiveness for first person and third person narratives.

One notable limitation of our study is that participants reported their probabilities of getting vaccinated, rather than their actual vaccination behaviors. However, vaccination intentions are correlated to actually getting vaccinated. ⁽⁵²⁾ A second limitation is that the pro-vaccine narrative derived its evaluative meaning from patients wishing they had gotten a flu shot, because they got sick after not getting a flu shot. That is, the pro-vaccine narrative provided no direct evidence that the vaccine would have been effective. Yet, our analyses on both perceived risk reduction and harm reduction confirmed that participants who read the pro-vaccine narratives perceived vaccines as relatively effective (or significantly greater than 0). Although participants who read the anti vaccine narratives also perceived vaccines as relatively effective, perceived effectiveness was greater after reading the pro-vaccine than after reading the anti-vaccine narratives. A third limitation is that we did not measure participants' perceptions of vaccines before entering our study. Hence, we were unable to test how participants' perceptions of vaccines changed as a result of our anti-vaccine vs. pro-vaccine narratives, or whether results held after controlling for participants' different initial perceptions. However, we did use random assignment to anti-vaccine vs. pro-vaccine narratives, allowing us to conclude that these caused the reported effects on perceived informativeness and judged vaccination probabilities. Moreover, the conclusions from the reported mediation analyses held after additionally controlling for past vaccination behavior, which is correlated to perceptions of vaccines (95%CI=1.99, 9.20 for anti-vaccine narratives; 95% CI=-11.95, -2.71 for pro-vaccine narratives; 95% CI=-15.93, -4.60 for moderated mediation).⁽⁴⁶⁾ A fourth limitation is that narratives did not attempt to explain actual proportions of patients getting sick with or without a flu shot. It has been argued that narratives could be included in decision aids to show a statistically representative proportion of positive and negative patient experiences, in combination with stating the actual proportions observed in the literature and health care providers' experience.^(15,16,53)

Despite these limitations, our findings have implications for the design of narratives as part of decision aids and patient communications. Recipients with lower numeracy may perceive narratives as more informative, without distinguishing between anti-treatment and pro-treatment narratives. In other contexts, decision makers have even treated negative information as more informative than positive information, by giving more weight to negative traits than to positive traits when forming impressions of others, ⁽⁵⁴⁾ and more weight to losses than to gains when choosing between gambles.⁽⁵⁵⁾ The relative influence of anti-treatment and pro-treatment narratives may partially be corrected by presenting narratives that are more representative of typical patient experiences, or including a warning that narratives may not be representative of typical patient experiences.^{(15,16,56,57)} Another strategy for counteracting disproportionate influence of narratives on patients' decisions is to provide visual displays.⁽¹⁵⁾ Visual displays can help people with higher and lower numeracy to better process statistical risk information.^(58,59) Thus, different information formats may be needed for effectively informing the decisions of patients varying in numeracy.

Appendix: Narratives

ANTI-VACCINE, PATIENT N=1. A patient at your doctor's office tells you the following about flu shots: Here is what happened to me. i did get a flu shot. Then I got very sick with flu-like symptoms. I had a high fever, severe body aches, and chills. I had to stay in the hospital overnight. After I came home, I spent about two weeks in bed. I was unable to do anything. I had to miss important events, including a big family wedding. I ended up wishing I had not gotten the flu shot that year

PRO-VACCINE, PATIENT N=1. A patient at your doctor's office tells you the following about flu shots: Here is what happened to me. I did not get a flu shot. Then I got very sick with flu-like symptoms. I had a high fever, severe body aches, and chills. I had to stay in the hospital overnight. After I came home, I spent about two weeks in bed. I was unable to do anything. I had to miss important events, including a big family wedding. I ended up wishing I had gotten the flu shot that year.

ANTI-VACCINE, DOCTOR N=1. Your doctor tells you the following about flu shots: “Here is what happened to one of my patients. She did get a flu shot. Then she got very sick with flu-like symptoms. She had a high fever, severe body aches, and chills. She had to stay in the hospital overnight. After she came home, she spent about two weeks in bed. She was unable to do anything. She had to miss important events, including a big family wedding. She ended up wishing she had not gotten the flu shot that year.”

PRO-VACCINE, DOCTOR N=1. Your doctor tells you the following about flu shots: “Here is what happened to one of my patients. She did not get a flu shot. Then she got very sick with flu-like symptoms. She had a high fever, severe body aches, and chills. She had to stay in the hospital overnight. After she came home, she spent about two weeks in bed. She was unable to do anything. She had to miss important events, including a big family wedding. She ended up wishing she had gotten the flu shot that year.”

ANTI-VACCINE, DOCTOR N=50. Your doctor tells you the following about flu shots: “Here is what happened to one of my patients. She did get a flu shot. Then she got very sick with flu-like symptoms. She had a high fever, severe body aches, and chills. She had to stay in the hospital overnight. After she came home, she spent about two weeks in bed. She was unable to do anything. She had to miss important events, including a big family wedding. She ended up wishing she had not gotten the flu shot that year. I have met about 50 patients with similar regrets.”

PRO-VACCINE, DOCTOR N=50. Your doctor tells you the following about flu shots: “Here is what happened to one of my patients. She did not get a flu shot. Then she got very sick with flu-like symptoms. She had a high fever, severe body aches, and chills. She had to stay in the hospital overnight. After she came home, she spent about two weeks in bed. She was unable to do anything. She had to miss important events, including a big family wedding. She ended up wishing she had gotten the flu shot that year. I have met about 50 patients with similar regrets.”