Abstract
Purpose
Beliefs that the risks from a COVID-19 vaccine outweigh the risks from getting COVID-19 and concerns that the vaccine development process was rushed and lacking rigor have been identified as important drivers of hesitancy and refusal to get a COVID-19 vaccine. We tested whether messages designed to address these beliefs and concerns might promote intentions to get a COVID-19 vaccine.
Method
We conducted an online survey fielded between March 8–23, 2021 with US Veteran (n = 688) and non-Veteran (n = 387) respondents. In a between-subjects experiment, respondents were randomly assigned to a control group (with no message) or to read one of two intervention messages: 1. a fact-box styled message comparing the risks of getting COVID-19 compared to the vaccine, and 2. a timeline styled message describing the development process of the COVID-19 mRNA vaccines.
Results
Most respondents (60%) wanted a COVID-19 vaccine. However, 17% expressed hesitancy and 23% did not want to get a COVID-19 vaccine. The fact-box styled message and the timeline message did not significantly improve vaccination intentions, F(2,358) = 0.86, p = .425, = .005, or reduce the time respondents wanted to wait before getting vaccinated, F(2,306) = 0.79, p = .453, = .005, compared to no messages.
Discussion
In this experimental study, we did not find that providing messages about vaccine risks and the development process had an impact on respondents’ vaccine intentions. Further research is needed to identify how to effectively address concerns about the risks associated with COVID-19 vaccines and the development process and to understand additional factors that influence vaccine intentions.
Introduction
All currently available COVID-19 vaccines have demonstrated high efficacy against COVID-19 [1]. Maximizing the public health benefits from these vaccines depends on achieving high levels of vaccine coverage. One of the major barriers to achieving widespread coverage of COVID-19 vaccines is public hesitancy and reluctance to receive them [2]. Prior research has established that attitudes towards vaccines and intentions to receive or refuse them are driven by a multitude of factors [3]. Within this literature, there is consistent evidence that people who are hesitant or reluctant to receive a vaccine often have doubts about the benefits of vaccines and concerns about their general safety [4].
Throughout the pandemic, how people perceive the threat posed by COVID-19 has been shown to vary based on a number of factors such as age [5], political ideology [6, 7], and philosophical beliefs [8]. How people perceive the risk of COVID-19 matters as those who do not identify COVID-19 to be a serious threat are likely to undervalue the benefits of vaccination [4] and are therefore less likely to choose to receive a vaccine [9–12]. Furthermore, people who do not consider COVID-19 to present a serious threat may also be reluctant to receive a vaccine because they are more likely to believe that they are more at risk from potential harms from vaccination than from acquiring COVID-19 [4, 9, 10].
The perceived safety of a vaccine is another factor that can have a strong influence on peoples’ decisions about whether or not to get vaccinated [4, 13]. Current evidence suggests that people who are hesitant about receiving a COVID-19 vaccine are often concerned about the safety of the COVID-19 vaccine development process [14]. Producing a number of highly effective vaccines within a year of an infectious disease outbreak is undoubtedly one of humanity’s greatest medical achievements. It is important to acknowledge that this achievement was only possible because of decades of prior research and clinical trials on vaccine technology (e.g., mRNA and viral vectors) and on the public health response to coronaviruses following outbreaks of SARS and MERS [15, 16]. Unfortunately, the importance and rigor of this prior research has often been overlooked in public discourse, which has placed greater emphasis on the speed of the vaccine development process. As a consequence, concerns that the safety of the vaccines was compromised by rushed development or the use of experimental/untested technology have emerged [14]. High-profile media coverage of very rare side-effects following COVID-19 vaccination (e.g., anaphylaxis and thrombosis with thrombocytopenia syndrome) may have compounded this issue by disproportionately raising public concern about vaccine safety and further fueling public hesitancy [17, 18]. Relatedly, the emergency use authorization given to the COVID-19 vaccines likely also raised concerns about the safety of the vaccines and, in turn, increased public hesitancy [19, 20].
It is critical to find communication strategies to address the public’s concerns about the safety and development process of COVID-19 vaccines. Providing succinct visual communications designed to directly address these frequently cited reasons for COVID-19 vaccine hesitancy may be an effective method for reducing concerns about the vaccines and encouraging uptake. Indeed, prior research has shown that using graphics to visually communicate public health information has numerous potential benefits including improving understanding about health information [21, 22] and motivating engagement in protective health behaviors [23]. In addition, short visual communications are conducive to rapid and widespread dissemination, which can further amplify their impact at low cost [24]. In the hope of achieving these benefits, government and health organizations have been swift in developing short, visual, informative messages about the COVID-19 vaccines and in disseminating them on their websites (e.g., https://www.cdc.gov/coronavirus/2019-ncov/downloads/vaccines/COVID-19-mRNA-infographic_G_508.pdf) and social media accounts. However, as few studies have sought to test the efficacy of short visual communications designed to directly address frequently cited concerns about the COVID-19 vaccines, doing so represents a significant contribution to the literature and future public health interventions. In order to examine the effectiveness of these types of communications, in the present study we tested whether messaging about the comparative risks of getting COVID-19 vs. the risk of COVID-19 vaccines, and also messaging about the timeline of mRNA vaccine development, might promote intentions to get a COVID-19 vaccine.
Materials and method
Study population and recruitment
Respondents were recruited by Qualtrics Online Panels between March 8–23, 2021. This experiment was conducted in the third wave of a three-wave longitudinal study [25]. The study was administered online (in English) and was approved (deemed exempt) by the IRBs at the University of Utah and the Salt Lake City VA (Veterans Experiences During the COVID-19 Pandemic: IRB_00133198). Respondents consented to participate in this voluntary study and were compensated for their participation based on the terms of their panel agreement.
Procedure
In a fully between-subjects experiment, respondents saw either a Schwartz et al. [26] factbox style message about the risks of getting a vaccine compared to getting COVID-19 (factbox message), a message about the development of COVID-19 vaccines (timeline message; both messages are presented in Fig 1), or no message (control group).
Both the factbox and timelines messages were developed by the study team as infographics displayed on a single page. As well as reflecting concerns about the COVID-19 vaccines reported in the existing literature, the messages were also designed to address the concerns reported by respondents in the first survey of the three-wave study this experiment was embedded in. Specifically, in the first wave survey we found that concerns that the risks of getting the vaccine outweighed the risks of getting COVID-19 (46%) and about the vaccine development process (21%) were reported as the most important reasons for not getting a COVID-19 vaccine among these respondents [27].
The factbox style message was chosen to convey information about the potential risks associated with getting COVID-19 compared with getting vaccinated as prior research suggests this format can improve comprehension and recall [28–30].
The timeline message was chosen in order to better understand whether these visualizations have an impact on attitudes towards vaccines, given their widespread use throughout the pandemic in academic outlets [15], in the mainstream media (https://www.nytimes.com/interactive/2020/04/30/opinion/coronavirus-covid-vaccine.html), and on social media (https://twitter.com/moderna_tx/status/1258370971855175680).
Respondent assignment to study conditions (no message [control], factbox message, timeline message) was carried out using the built-in randomizer function in Qualtrics’ survey flow.
Respondents first reported their COVID-19 vaccine intentions (“When a coronavirus vaccine becomes available to you, how interested are you in getting the vaccine?”) using a 5-point scale (1 = “I definitely do NOT want the vaccine”, 2 = “I do NOT want to get the vaccine”, 3 = “Unsure”, 4 = “I WANT to get the vaccine”, 5 = “I definitely WANT the vaccine”) and how long they intended to wait before getting vaccinated (“How soon after the COVID-19 vaccine becomes available to you would you become vaccinated?”) using an 8-point scale (1 = “Immediately”, 2 = “Less than one month”, 3 = “One month to less than 3 months”, 4 = “3 months to less than six months”, 5 = “6 months to less than 1 year”, 6 = “1 year to less than 2 two years”, 7 = “I would wait 2 years or more”, 8 = “I would never get it”).
We also asked respondents about their views regarding COVID-19 vaccine safety (“In your view, how safe is the COVID-19 vaccine?”) using a 5-point scale (1 = “Not at all safe”, 2 = “Slightly safe”, 3 = “Somewhat safe”, 4 = “Moderately safe”, 5 = “Extremely safe”) and the possibility of experiencing side effects (“How worried are you about experiencing side effects from the COVID-19 vaccine?”) using a 5-point scale (1 = “Not at all concerned”, 2 = “Slightly concerned”, 3 = “Somewhat concerned”, 4 = “Moderately concerned”, 5 = “Extremely concerned”).
Pre-registered analyses
We ran two omnibus one-way ANOVA analyses to examine whether the COVID-19 vaccine risk comparison factbox message and vaccine development timeline message influenced 1) respondents’ intentions to get vaccinated and 2) the time they would wait to get vaccinated. Analyses were repeated controlling for age and gender.
We ran two additional omnibus one-way ANOVAs to examine whether 1) the factbox message comparing the risks associated with getting COVID-19 to the risks associated with receiving a COVID-19 vaccine reduced respondents’ concern about experiencing side effects from the COVID-19 vaccine and 2) whether the timeline message regarding the vaccine development process increased respondents’ views about how safe the COVID-19 vaccine is. All analyses were performed using RStudio statistical software Version 1.4.1106 [31].
Exploratory analyses
These analyses were not planned a-priori, but were conducted after the unexpected finding that neither of the messages had a significant impact on respondents’ vaccination intentions or the time they wanted to wait before getting vaccinated.
We conducted exploratory analyses using omnibus one-way ANOVAs to examine the influence of the messages on vaccine intentions and time to getting vaccinated on two subsets of the total sample.
To examine the possibility that any impact of the messages on vaccination intentions could be dependent on whether respondents took the time to read and process the information provided in them, we re-ran our pre-registered analyses on a subset of respondents who spent at least 10 seconds on the page displaying the intervention messages.
Another possibility is that any impact of the messages on vaccination intentions may be limited only to respondents who were initially hesitant about getting a COVID-19 vaccine [32]. That is, an effect of the intervention might be dampened by ceiling effects on vaccination intentions. Thus, for the second subset we selected respondents who had reported that they were hesitant about getting a COVID-19 vaccine in an earlier survey fielded in December 2020 (i.e., reported that they either “do NOT want to get the vaccine” or “definitely do NOT want to get the vaccine”).
In light of the null effect of the information-based messages, we sought to identify which variables were associated with respondents’ vaccine intentions. To do so, we ran two multiple regression models: one to predict respondents’ vaccine intentions and one to predict how long they intended to wait before getting vaccinated. Predictor variables were the same in both models and included respondents’ age (≤65, 65 to 74, ≥75), Veteran status (Veteran or not), total comorbidities [33], health literacy [34], numeracy [35, 36], Race/Ethnicity (Non-Hispanic white or not), worry about getting COVID-19, COVID-19 risk perceptions, Emory Vaccine Confidence Index [37], beliefs about the importance of flu and COVID-19 vaccines, trust in health care [38], disbelief in science [39], beliefs in COVID-19 conspiracy theories [40], political beliefs, and medical maximizing [41].
Results
A total of 1075 respondents completed the third wave of the study, of which 688 (64%) were United States Veterans. Median age of the sample was between 55 and 74 years old; 841 respondents (78%) were male, 819 respondents (76%) were non-Hispanic White, and the median household income was $70,000-$99,999. Sample retention from the first wave was 52% overall, 65% for Veteran and 38% for non-Veterans. Characteristics of respondents who completed only the first wave and those who completed all three are available on the project repository [25]. There were 361 (33%) respondents who reported having not received any vaccine doses, 243 (23%) who reported having received 1 dose, and 471 (44%) who reported having received 2 doses. As we were only interested in the effect of the interventions on respondents who had not yet been vaccinated, any respondents who reported having received at least 1 dose of a COVID-19 vaccine were excluded from the experimental portion of the survey, leaving a final sample of 361 respondents (Table 1).
Table 1. Respondent characteristics overall and according to group assignment.
Overall | Control | Factbox | Timeline | ||
---|---|---|---|---|---|
(N = 361) | (N = 120) | (N = 122) | (N = 119) | ||
Age–No. (%) | |||||
18 to 34 | 46 (13) | 15 (13) | 15 (12) | 16 (13) | |
35 to 54 | 80 (22) | 19 (16) | 35 (29) | 26 (22) | |
55 to 74 | 205 (57) | 75 (63) | 64 (53) | 66 (56) | |
75 or older | 28 (8) | 10 (8) | 8 (7) | 10 (8) | |
Did not respond | 2 (1) | 1 (1) | - | 1 (1) | |
Gender–No. (%) | |||||
Female | 123 (34) | 41 (34) | 43 (35) | 39 (33) | |
Male | 235 (65) | 79 (66) | 77 (63) | 79 (66) | |
Non-binary/Third gender or Transgender man/Transman | 3 (1) | - | 2 (2) | 1 (1) | |
Race/Ethnicity–No. (%) | |||||
Non-Hispanic White | 252 (70) | 86 (72) | 78 (64) | 88 (74) | |
Non-Hispanic Black | 49 (14) | 17 (14) | 15 (12) | 17 (14) | |
Hispanic | 32 (9) | 9 (8) | 14 (11) | 9 (8) | |
Asian/Asian American | 15 (4) | 4 (3) | 8 (7) | 3 (3) | |
American Indian/Alaskan Native | 2 (1) | - | 2 (2) | - | |
Another race | 7 (2) | 3 (3) | 4 (3) | - | |
Multiracial | 4 (1) | 1 (1) | 1 (1) | 2 (2) | |
Income–No. (%) | |||||
$0 - $49,000 | 131 (36) | 51 (43) | 48 (39) | 32 (27) | |
$50,000 to $99,000 | 125 (35) | 41 (34) | 42 (34) | 42 (35) | |
$100,000 or more | 87 (24) | 24 (20) | 27 (22) | 36 (30) | |
Prefer not to say | 18 (5) | 4 (3) | 5 (4) | 9 (8) | |
Residence–No. (%) | |||||
Rural | 77 (21) | 23 (19) | 30 (25) | 24 (20.2) | |
Small (less than 100,000) | 58 (16) | 15 (13) | 17 (14) | 26 (21.8) | |
Suburban near large city | 157 (44) | 54 (45) | 55 (45) | 48 (40.3) | |
Mid-sized city (100,000 to 1million) | 29 (8) | 11 (9) | 8 (7) | 10 (8.4) | |
large city more than 1 million | 40 (11) | 17 (14) | 12 (10) | 11 (9.2) |
We found no statistically significant differences between groups regarding vaccination intentions, F(2,358) = 0.86, p = .425, = .005, or the time respondents would wait to get vaccinated, F(2,306) = 0.79, p = .453, = .005. In fact, as shown in Fig 2 and Table 2, although there were no statistically significant differences between groups, mean scores for both the intervention groups were slightly lower for vaccine intentions (i.e., less willing to get a COVID-19 vaccine) and slightly higher for the time respondents would wait to get vaccinated (i.e., would want to wait longer) as compared to those in the control group who did not see any message.
Table 2. Outcome measures according to group assignment.
Control | Factbox | Timeline | Control | Control | Factbox | |
---|---|---|---|---|---|---|
(N = 120) | (N = 122) | (N = 119) | vs. Factbox | vs. Timeline | vs. Timeline | |
Means±SD | Mean differences (95%CIs) | |||||
Vaccine intentions (Scale range: 1–5) | 3.83±1.40 | 3.61±1.56 | 3.61±1.53 | 0.22 (-0.16 to 0.59) | 0.22 (-0.15 to 0.59) | 0.00 (-0.39 to 0.39) |
Time to vaccine (Scale range: 1–8) | 2.90±2.58 | 3.36±2.77 | 3.20±2.70 | -0.46 (-1.19 to 0.26) | -0.29 (-1.03 to 0.45) | 0.17 (-0.58 to 0.92) |
Vaccine safety (Scale range: 1–5) | 3.57±1.38 | 3.52±1.43 | 3.45±1.33 | 0.04 (-0.31 to 0.40) | 0.11 (-0.23 to 0.46) | 0.07 (-0.28 to 0.42) |
Vaccine side effects (Scale range: 1–5) | 2.66±1.50 | 2.52±1.38 | 2.87±1.42 | 0.14 (-0.23 to 0.51) | -0.22 (-0.59 to 0.16) | -0.36 (-0.71 to 0.00) |
These results did not differ when controlling for respondents’ age, (no statistically significant effect of group on vaccination intentions, F(2,355) = 0.06, p = .945, < .001, and no statistically significant effect of group on time respondents would wait to get vaccinated, F(2,303) = 0.05, p = .950, < .001), or gender, (no statistically significant effect of group on vaccination intentions, F(2,352) = 1.46, p = .233, = .008, and no statistically significant effect of group on time respondents would wait to get vaccinated, F(2,301) = 1.54, p = .217, = .010). Furthermore, we found no statistically significant difference on respondents’ views about how safe COVID-19 vaccines are, F(2,358) = 0.20, p = .815, = .001, and worry about side effects, F(2,357) = 1.89, p = .152, = .010, between the three groups.
Exploratory results
We found that results from the subset of respondents who spent at least 10 seconds on the page displaying the intervention messages (n = 298) did not differ from the overall sample: there were no statistically significant differences in vaccination intentions, F(2,295) = 1.51, p = .224, = .010, and no significant difference in time respondents would wait to get vaccinated between groups, F(2,252) = 1.08, p = .342, = .008.
For the subset of respondents who had previously reported that they were hesitant about getting a COVID-19 vaccine (in an earlier survey fielded in December 2020; n = 86), we also found no statistically significant differences between groups on respondents’ vaccination intentions, F(2,83) = 0.32, p = .724, = .008, or the time they would wait to get vaccinated, F(2,61) = 1.05, p = .356, = .033.
As shown in Fig 3, believing that it is important for adults to get the COVID-19 vaccine predicted greater intentions to get a COVID-19 vaccine (0.71, 95%CI[0.56 to 0.86], p < .001) and to get one sooner, (-1.50, 95%CI[-1.79 to -1.21], p < .001). Conversely, belief in COVID-19 related conspiracy theories predicted lower intentions to get a COVID-19 vaccine (-0.12, 95%CI[-0.23 to 0.00], p = .043) and wanting to wait longer before getting one (0.26, 95%CI[0.03 to 0.49], p = .027). Preferring to taking action in situations where it is not clear whether or not medical action is needed also predicted greater COVID-19 vaccine intentions (0.07, 95%CI[0.01 to 0.14], p = .032).
Discussion
Major advances in vaccine development are at risk of being undermined by increasing levels of vaccine hesitancy [42]. In the immediate term, public hesitancy towards COVID-19 vaccines presents a notable barrier to limiting the spread of COVID-19 [2]. Messaging designed to address prominent concerns about COVID-19 vaccines might encourage people who are otherwise hesitant about getting vaccinated to receive one.
In the present study, we found no evidence that neither a message comparing the potential risks of getting COVID-19 to the potential risks associated with COVID-19 vaccines nor a timeline used to communicate the vaccine development process, had any effect on respondents’ intentions to get a COVID-19 vaccine, the time that they would want to wait before getting vaccinated, perceived safety of COVID-19 vaccines, or worry about vaccine-related side effects. These results were unanticipated and remained consistent when controlling for the age and gender of respondents and following exploratory analyses of respondents who spent at least 10 seconds looking at the messages and among those who had previously expressed that they did not want to get a COVID-19 vaccine.
The messages were designed to address concerns about the risks from COVID-19 vaccines and the vaccine development process as they are reasons frequently cited by people who are hesitant about receiving a COVID-19 vaccine as we found in a previous survey of these same participants [27] as well as in the literature more generally [14, 17]. The absence of an intervention effect is unfortunate as effective low-cost messaging strategies could contribute substantially to combatting growing levels of vaccine hesitancy and help reduce the impact of the COVID-19 pandemic.
One potential reason for these null findings could simply be that the information in the two messages was not sufficiently convincing to influence the vaccine intentions of people who are concerned about the risks of vaccines and the development process. For individuals who did not have concerns about the development or safety of COVID-19 vaccines, it could also have been the case that the information in the messages may have drawn their attention to these issues which may have then served to undermine their prior confidence in vaccine development and safety [43]. It is unlikely that these null effects were due to insufficient statistical power. Observed effect sizes in this experiment were very small (such that even if significant, the meaningfulness of these differences would have been questionable), and if anything, the intervention groups showed slightly more vaccine hesitancy than the control group, not less.
Given the cross-sectional nature of this experiment, it is not possible to examine whether repeated exposure to these messages may have yielded more positive results. Repeated messaging about health issues is often more effective at changing attitudes and behaviors [44], however, given the prevalence of media coverage about COVID-19 and vaccine development it is also possible that efficacy of these messages may be suppressed by respondents having high prior exposure to similar information. In addition, while many studies have shown changes in respondents’ vaccine intentions directly after seeing information about COVID-19 vaccines [32, 45–50] it could be the case that an effect of the provided information on respondents’ intentions may only emerge over a longer period than was covered in the study. Another possibility, given how public attitudes and intentions towards the COVID-19 vaccines have varied over time [51], is that we may have observed different results if the study had been conducted at another time during the pandemic, or with a longer follow up period.
The design of the message–regarding both the style and content, may have also contributed to the null finding. However, given the data available from the present study it is not possible to posit informed explanations for the null findings based on respondents’ perceptions of the messages themselves. This does reveal an important avenue for future research, which might leverage qualitative methods (e.g., focus groups or open responses) in order to better understand how people perceived the messages used in the present study.
The present findings are aligned with research which suggests that providing corrective information based on an information-deficit assumption is not always effective at addressing people’s concerns and doubts [52–54] and highlight the need for testing public health messages to ensure they are as effective as possible as well as to identify potentially counterproductive effects they may have. Future research might explore whether different message styles or sources (e.g., non-partisan experts or popular figures) may be more effective at sufficiently addressing concerns about vaccine development and safety. The results of the exploratory regression analyses suggested that the most influential predictor of respondents’ COVID-19 vaccine intentions was their general beliefs about the importance of COVID-19 vaccines for adults. Our findings suggest that negative views about the importance and need for COVID-19 vaccines are strongly held beliefs which are difficult to address using informative health messages. However, as a number of studies have reported small benefits in vaccine intentions following communications about COVID-19 vaccine side effects and safety development concerns [32, 49], health messages aiming to reassure people about the importance of COVID-19 vaccines and focusing on the benefits remain an important strategy for rapid and widespread communication of this critical health information [46–48].
Identifying methods for addressing complex and deeply held beliefs in COVID-19 related conspiracy theories, which we found predicted lower vaccine intentions, remains a key goal for public health communication research. As respondents’ preferences for taking medical action instead of watching and waiting in cases where it is unclear which option is needed predicted vaccine intentions, communications which focus on the active nature of “getting a vaccine” may resonate with these individuals. Thus, we believe these findings are important in highlighting the need for research into communications that address other influences of vaccine intentions that might be more receptive to this type of health messaging strategy and to identify alternative methods for addressing concerns about safety and development.
Limitations
Despite evidence that self-reports are good predictors of health behaviors [55], it is important that these findings are considered in the context of known limitations of this method (e.g., social desirability in respondents’ answers). The study design (i.e., US-based recruitment, online, and in English) prevents generalization of the present findings outside of the US and to people who may have limited internet access and lower English proficiency. Furthermore, our sample consists of both Veteran and non-Veteran respondents completing the third (and final) survey of a longitudinal study, which began in December, 2020 and thus are not representative of the general population.
Conclusion
In the present study, providing corrective information about the risks of getting COVID-19 compared to receiving a COVID-19 vaccine and about the development of COVID-19 vaccines was not effective at promoting intentions to get a COVID-19 vaccine. Further research is needed to identify how to effectively address concerns about the risks of side effects from COVID-19 vaccines and the vaccine development process.
Acknowledgments
The views expressed in this paper are those of the authors and do not necessarily represent the position or policy of the U.S. Department of Veterans Affairs or the United States Government.
We thank and are grateful to Karina Pritchett BA (Spencer Fox Eccles School of Medicine at University of Utah, Salt Lake City, UT.) for her assistance with the design of the materials. Karina was not compensated for this work beyond her typical employment compensation.
Data Availability
The data and materials for this study are publicly available at: https://osf.io/63gte/.
Funding Statement
Funding for the study was provided by the VA (VA C-19-20-205; for recruitment of Veterans) to Drs. AF and LS and Dr. AF's Jon M. Huntsman Presidential Endowed Chair (for recruitment of non-Veterans). Dr. AT was supported by grant No. 51300302 from the American Heart Association Children’s Strategically Focused Research Network fellowship awarded to Dr. AF. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The views expressed in this paper are those of the authors and do not necessarily represent the position or policy of the U.S. Department of Veterans Affairs or the United States Government.
References
- 1.Olliaro P. What does 95% COVID-19 vaccine efficacy really mean? Lancet Infect Dis. 2021;0. doi: 10.1016/S1473-3099(21)00075-X [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sallam M. COVID-19 vaccine hesitancy worldwide: A concise systematic review of vaccine acceptance rates. Vaccines. 2021;9: 160. doi: 10.3390/vaccines9020160 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Larson HJ, Jarrett C, Eckersberger E, Smith DMD, Paterson P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: A systematic review of published literature, 2007–2012. Vaccine. 2014;32: 2150–2159. doi: 10.1016/j.vaccine.2014.01.081 [DOI] [PubMed] [Google Scholar]
- 4.Karafillakis E, Larson HJ. The benefit of the doubt or doubts over benefits? A systematic literature review of perceived risks of vaccines in European populations. Vaccine. 2017;35: 4840–4850. doi: 10.1016/j.vaccine.2017.07.061 [DOI] [PubMed] [Google Scholar]
- 5.Bruine de Bruin W. Age Differences in COVID-19 Risk Perceptions and Mental Health: Evidence From a National U.S. Survey Conducted in March 2020. J Gerontol B Psychol Sci Soc Sci. 2020. [cited 19 Apr 2021]. doi: 10.1093/geronb/gbaa074 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Calvillo DP, Ross BJ, Garcia RJB, Smelter TJ, Rutchick AM. Political Ideology Predicts Perceptions of the Threat of COVID-19 (and Susceptibility to Fake News About It). Soc Psychol Personal Sci. 2020;11: 1119–1128. doi: 10.1177/1948550620940539 [DOI] [Google Scholar]
- 7.Kerr J, Panagopoulos C, van der Linden S. Political polarization on COVID-19 pandemic response in the United States. Personal Individ Differ. 2021;179: 110892. doi: 10.1016/j.paid.2021.110892 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Byrd N, Białek M. Your health vs. my liberty: Philosophical beliefs dominated reflection and identifiable victim effects when predicting public health recommendation compliance during the COVID-19 pandemic. Cognition. 2021;212: 104649. doi: 10.1016/j.cognition.2021.104649 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Betsch C, Schmid P, Heinemeier D, Korn L, Holtmann C, Böhm R. Beyond confidence: Development of a measure assessing the 5C psychological antecedents of vaccination. PLOS ONE. 2018;13: e0208601. doi: 10.1371/journal.pone.0208601 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Brewer NT, Chapman GB, Rothman AJ, Leask J, Kempe A. Increasing Vaccination: Putting Psychological Science Into Action. Psychol Sci Public Interest. 2017;18: 149–207. doi: 10.1177/1529100618760521 [DOI] [PubMed] [Google Scholar]
- 11.Denford S, Mowbray F, Towler L, Wehling H, Lasseter G, Amlôt R, et al. Exploration of attitudes regarding uptake of COVID-19 vaccines among vaccine hesitant adults in the UK: a qualitative analysis. BMC Infect Dis. 2022;22: 407. doi: 10.1186/s12879-022-07380-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Malik AA, McFadden SM, Elharake J, Omer SB. Determinants of COVID-19 vaccine acceptance in the US. EClinicalMedicine. 2020;26: 100495. doi: 10.1016/j.eclinm.2020.100495 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Karlsson LC, Soveri A, Lewandowsky S, Karlsson L, Karlsson H, Nolvi S, et al. Fearing the disease or the vaccine: The case of COVID-19. Personal Individ Differ. 2021;172: 110590. doi: 10.1016/j.paid.2020.110590 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dror AA, Eisenbach N, Taiber S, Morozov NG, Mizrachi M, Zigron A, et al. Vaccine hesitancy: the next challenge in the fight against COVID-19. Eur J Epidemiol. 2020;35: 775–779. doi: 10.1007/s10654-020-00671-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Ball P. The lightning-fast quest for COVID vaccines—and what it means for other diseases. Nature. 2020;589: 16–18. doi: 10.1038/d41586-020-03626-1 [DOI] [PubMed] [Google Scholar]
- 16.Li Y-D, Chi W-Y, Su J-H, Ferrall L, Hung C-F, Wu T-C. Coronavirus vaccine development: from SARS and MERS to COVID-19. J Biomed Sci. 2020;27: 104. doi: 10.1186/s12929-020-00695-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Dodd RH, Pickles K, Nickel B, Cvejic E, Ayre J, Batcup C, et al. Concerns and motivations about COVID-19 vaccination. Lancet Infect Dis. 2021;21: 161–163. doi: 10.1016/S1473-3099(20)30926-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tran BX, Boggiano VL, Nguyen LH, Latkin CA, Nguyen HLT, Tran TT, et al. Media representation of vaccine side effects and its impact on utilization of vaccination services in Vietnam. Patient Prefer Adherence. 2018;12: 1717–1728. doi: 10.2147/PPA.S171362 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Kreps S, Prasad S, Brownstein JS, Hswen Y, Garibaldi BT, Zhang B, et al. Factors Associated With US Adults’ Likelihood of Accepting COVID-19 Vaccination. JAMA Netw Open. 2020;3: e2025594. doi: 10.1001/jamanetworkopen.2020.25594 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Quinn SC, Kumar S, Freimuth VS, Kidwell K, Musa D. Public Willingness to Take a Vaccine or Drug Under Emergency Use Authorization during the 2009 H1N1 Pandemic. Biosecurity Bioterrorism Biodefense Strategy Pract Sci. 2009;7: 275–290. doi: 10.1089/bsp.2009.0041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Schubbe D, Scalia P, Yen RW, Saunders CH, Cohen S, Elwyn G, et al. Using pictures to convey health information: A systematic review and meta-analysis of the effects on patient and consumer health behaviors and outcomes. Patient Educ Couns. 2020;103: 1935–1960. doi: 10.1016/j.pec.2020.04.010 [DOI] [PubMed] [Google Scholar]
- 22.Thorpe A, Scherer AM, Han PKJ, Burpo N, Shaffer V, Scherer L, et al. Exposure to Common Geographic COVID-19 Prevalence Maps and Public Knowledge, Risk Perceptions, and Behavioral Intentions. JAMA Netw Open. 2021;4: e2033538. doi: 10.1001/jamanetworkopen.2020.33538 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fagerlin A, Valley TS, Scherer AM, Knaus M, Das E, Zikmund-Fisher BJ. Communicating infectious disease prevalence through graphics: Results from an international survey. Vaccine. 2017;35: 4041–4047. doi: 10.1016/j.vaccine.2017.05.048 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Malik A, Khan ML, Quan-Haase A. Public health agencies outreach through Instagram during the COVID-19 pandemic: Crisis and Emergency Risk Communication perspective. Int J Disaster Risk Reduct. 2021;61: 102346. doi: 10.1016/j.ijdrr.2021.102346 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Thorpe A, Fagerlin A, Scherer L, Drews F, Butler J, Stevens V, et al. Veterans Experiences during the COVID-19 pandemic (VISION-19). 2022. [cited 28 Jun 2022]. doi: 10.17605/OSF.IO/63GTE [DOI] [Google Scholar]
- 26.Schwartz LM, Woloshin S, Welch HG. The Drug Facts Box: Providing Consumers with Simple Tabular Data on Drug Benefit and Harm. Med Decis Making. 2007;27: 655–662. doi: 10.1177/0272989X07306786 [DOI] [PubMed] [Google Scholar]
- 27.Thorpe A, Fagerlin A, Drews FA, Shoemaker H, Brecha FS, Scherer LD. Predictors of COVID-19 vaccine uptake: An online longitudinal study of US Veterans and non-Veterans. medRxiv. 2022. p. 2022.04.19. 10.1101/2022.04.19.22273818 [DOI] [Google Scholar]
- 28.Schwartz LM, Woloshin S. The Drug Facts Box: Improving the communication of prescription drug information. Proc Natl Acad Sci. 2013;110: 14069–14074. doi: 10.1073/pnas.1214646110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Brick C, McDowell M, Freeman ALJ. Risk communication in tables versus text: a registered report randomized trial on ‘fact boxes’. R Soc Open Sci. 7: 190876. doi: 10.1098/rsos.190876 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.McDowell M, Rebitschek FG, Gigerenzer G, Wegwarth O. A Simple Tool for Communicating the Benefits and Harms of Health Interventions: A Guide for Creating a Fact Box. MDM Policy Pract. 2016;1: 2381468316665365. doi: 10.1177/2381468316665365 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.RStudio Team. RStudio: Integrated Development Environment for R. RStudio. Boston, MA: PBC, Boston, MA; 2021. Available: http://www.rstudio.com/. [Google Scholar]
- 32.Freeman D, Loe BS, Yu L-M, Freeman J, Chadwick A, Vaccari C, et al. Effects of different types of written vaccination information on COVID-19 vaccine hesitancy in the UK (OCEANS-III): a single-blind, parallel-group, randomised controlled trial. Lancet Public Health. 2021;6: e416–e427. doi: 10.1016/S2468-2667(21)00096-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. J Chronic Dis. 1987;40: 373–383. doi: 10.1016/0021-9681(87)90171-8 [DOI] [PubMed] [Google Scholar]
- 34.Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: Evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract. 2006;7: 21. doi: 10.1186/1471-2296-7-21 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Fagerlin A, Zikmund-Fisher BJ, Ubel PA, Jankovic A, Derry HA, Smith DM. Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Med Decis Making. 2007;27: 672–680. doi: 10.1177/0272989X07304449 [DOI] [PubMed] [Google Scholar]
- 36.McNaughton CD, Cavanaugh KL, Kripalani S, Rothman RL, Wallston KA. Validation of a Short, 3-Item Version of the Subjective Numeracy Scale. Med Decis Mak Int J Soc Med Decis Mak. 2015;35: 932–936. doi: 10.1177/0272989X15581800 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Frew PM, Murden R, Mehta CC, Chamberlain AT, Hinman AR, Nowak G, et al. Development of a US trust measure to assess and monitor parental confidence in the vaccine system. Vaccine. 2019;37: 325–332. doi: 10.1016/j.vaccine.2018.09.043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Shea JA, Micco E, Dean LT, McMurphy S, Schwartz JS, Armstrong K. Development of a Revised Health Care System Distrust Scale. J Gen Intern Med. 2008;23: 727–732. doi: 10.1007/s11606-008-0575-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Hartman RO, Dieckmann NF, Sprenger AM, Stastny BJ, DeMarree KG. Modeling attitudes toward science: development and validation of the credibility of science scale. Basic Appl Soc Psychol. 2017;39: 358–371. doi: 10.1080/01973533.2017.1372284 [DOI] [Google Scholar]
- 40.Piltch-Loeb R, Zikmund-Fisher BJ, Shaffer VA, Scherer LD, Knaus M, Fagerlin A, et al. Cross-Sectional Psychological and Demographic Associations of Zika Knowledge and Conspiracy Beliefs Before and After Local Zika Transmission. Risk Anal. 2019;39: 2683–2693. doi: 10.1111/risa.13369 [DOI] [PubMed] [Google Scholar]
- 41.Scherer LD, Zikmund-Fisher BJ. Eliciting medical maximizing-minimizing preferences with a single question: development and validation of the MM1. Med Decis Making. 2020;40: 545–550. doi: 10.1177/0272989X20927700 [DOI] [PubMed] [Google Scholar]
- 42.WHO. Vaccination: European Commission and World Health Organization join forces to promote the benefits of vaccines. 2019. [cited 22 Mar 2021]. Available: https://www.who.int/news/item/12-09-2019-vaccination-european-commission-and-world-health-organization-join-forces-to-promote-the-benefits-of-vaccines [Google Scholar]
- 43.Betsch C, Ulshöfer C, Renkewitz F, Betsch T. The Influence of Narrative v. Statistical Information on Perceiving Vaccination Risks. Med Decis Making. 2011;31: 742–753. doi: 10.1177/0272989X11400419 [DOI] [PubMed] [Google Scholar]
- 44.Frew PM, Owens LE, Saint-Victor DS, Benedict S, Zhang S, Omer SB. Factors associated with maternal influenza immunization decision-making: Evidence of immunization history and message framing effects. Hum Vaccines Immunother. 2014;10: 2576–2583. doi: 10.4161/hv.32248 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Kaplan RM, Milstein A. Influence of a COVID-19 vaccine’s effectiveness and safety profile on vaccination acceptance. Proc Natl Acad Sci. 2021;118. doi: 10.1073/pnas.2021726118 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Schwarzinger M, Luchini S. Addressing COVID-19 vaccine hesitancy: is official communication the key? Lancet Public Health. 2021;6: e353–e354. doi: 10.1016/S2468-2667(21)00108-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Palm R, Bolsen T, Kingsland JT. The Effect of Frames on COVID-19 Vaccine Hesitancy. medRxiv. 2021; 2021.01.04. doi: 10.1101/2021.01.04.21249241 [DOI] [Google Scholar]
- 48.Motta M, Sylvester S, Callaghan T, Lunz-Trujillo K. Encouraging COVID-19 Vaccine Uptake Through Effective Health Communication. Front Polit Sci. 2021;3. Available: https://www.frontiersin.org/articles/10.3389/fpos.2021.630133 [Google Scholar]
- 49.Thorpe A, Fagerlin A, Drews FA, Butler J, Stevens V, Riddoch MS, et al. Communications to Promote Interest and Confidence in COVID-19 Vaccines. Am J Health Promot. 2022;36: 976–986. doi: 10.1177/08901171221082904 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Freeman D, Loe BS, Chadwick A, Vaccari C, Waite F, Rosebrock L, et al. COVID-19 vaccine hesitancy in the UK: the Oxford coronavirus explanations, attitudes, and narratives survey (Oceans) II. Psychol Med. 2020; 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Jamieson KH, Romer D, Jamieson PE, Winneg KM, Pasek J. The role of non–COVID-specific and COVID-specific factors in predicting a shift in willingness to vaccinate: A panel study. Proc Natl Acad Sci. 2021;118. doi: 10.1073/pnas.2112266118 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Dubé E, Gagnon D, MacDonald NE. Strategies intended to address vaccine hesitancy: Review of published reviews. Vaccine. 2015;33: 4191–4203. doi: 10.1016/j.vaccine.2015.04.041 [DOI] [PubMed] [Google Scholar]
- 53.Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33: 459–464. doi: 10.1016/j.vaccine.2014.11.017 [DOI] [PubMed] [Google Scholar]
- 54.Nyhan B, Reifler J, Richey S, Freed GL. Effective Messages in Vaccine Promotion: A Randomized Trial. Pediatrics. 2014;133: e835–e842. doi: 10.1542/peds.2013-2365 [DOI] [PubMed] [Google Scholar]
- 55.Webb TL, Sheeran P. Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol Bull. 2006;132: 249–268. doi: 10.1037/0033-2909.132.2.249 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data and materials for this study are publicly available at: https://osf.io/63gte/.