Skip to main content
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2022 Nov 14;41(1):136–144. doi: 10.1016/j.vaccine.2022.11.014

Misinformation and COVID-19 vaccine hesitancy

Tara Zimmerman a,, Kristina Shiroma b, Kenneth R Fleischmann b, Bo Xie b, Chenyan Jia c, Nitin Verma b, Min Kyung Lee b
PMCID: PMC9659512  PMID: 36411132

Abstract

Background

COVID-19 vaccine hesitancy has emerged as a major public health challenge. Although medical and scientific misinformation has been known to fuel vaccine hesitancy in the past, misinformation surrounding COVID-19 seems to be rampant, and increasing evidence suggests that it is contributing to COVID-19 vaccine hesitancy today. The relationship between misinformation and COVID-19 vaccine hesitancy is complex, however, and it is relatively understudied.

Methods

In this article, we report qualitative data from two related but distinct studies from a larger project. Study 1 included semi-structured, open-ended interviews conducted in October–November 2020 via phone with 30 participants to investigate the relationship between misinformation and COVID-19 vaccine hesitancy. Study 1′s results then informed the design of open-ended questions for Study 2, an online survey conducted in May–June 2021 to consider the relationship between misinformation and vaccine hesitancy further. The data were examined with thematic analysis.

Results

Study 1 led to the identification of positive and negative themes related to attitudes toward COVID-19 vaccines. In Study 2, responses from vaccine-hesitant participants included six categories of misinformation: medical, scientific, political, media, religious, and technological. Across both Study 1 and Study 2, six vaccine hesitancy themes were identified from the data: concerns about the vaccines’ future effects, doubts about the vaccines’ effectiveness, commercial profiteering, preference for natural immunity, personal freedom, and COVID-19 denial.

Conclusions

The relationship between misinformation and vaccine hesitancy is complicated. Various types of misinformation exist, with each related to a specific type of vaccine hesitancy-related attitude. Personal freedom and COVID-19 denial are vaccine attitudes of particular interest, representing important yet understudied phenomena. Medical and scientific approaches may not be sufficient to combat misinformation based in religion, media, or politics; and public health officials may benefit from partnering with experts from those fields to address harmful misinformation that is driving COVID-19 vaccine hesitancy.

Keywords: Vaccine hesitancy, COVID-19, Misinformation, Disinformation, Interviews, Surveys

1. Introduction

During the COVID-19 pandemic, anti-vaccine beliefs have been at an all-time high, and vaccine hesitancy has become a major threat to public health [1]. The World Health Organization has labeled the increased virulence of misinformation during the COVID-19 pandemic an “infodemic” [2]. Throughout 2020, scientists, healthcare professionals, and politicians speculated about vaccine timelines, distribution, and effectiveness; and media platforms presented widely varying opinions as well as misinformation that negatively influenced people’s attitudes about COVID-19 vaccines [3]. By the end of 2020, the self-reported likelihood of getting vaccinated against COVID-19 had declined sharply from 74 % in April to 56 % in December [4]. While the availability of COVID-19 vaccines improved between the fall of 2020 and late spring 2021, indications of public distrust in vaccines continued. This public distrust contributed to waves of COVID-19 cases and deaths in the US, increasingly among those who were not vaccinated [4].

Both misinformation and disinformation stem from the communication of false information, yet they differ in intent. Misinformation is false or misperceived information communicated without the intent to deceive or control others. Disinformation involves the intentional propagation of false information to deceive or control others, whereas misinformation involves propagating false information unknowingly [5], [6]. For simplicity, particularly because an individual’s motives are not always apparent, henceforth we will use the term misinformation broadly to encompass potential disinformation as well. The most extreme form of misinformation consists of conspiracy-related theories, beliefs that our lives are controlled by a small number of people secretly plotting against us and that big events, such as a pandemic, are evidence of this elite group’s working to control every-one else [7], [8]. Misinformation predates the Internet, but it is fueled and propagated broadly and quickly via social media [9]. In 2016, a BuzzFeed News analysis found that stories based on misinformation and disinformation from hyperpartisan or hoax sites generated more shares, reactions, and comments on social media than did the top stories from major news outlets [10], [11], [12]. Misinformation, particularly conspiracy-related theories, gains acceptance during times of crisis [13].

Although personal belief systems and human values such as benevolence, intellectual curiosity, purity, liberty, and opposition to authority [14] are associated with vaccine hesitancy, so are beliefs in misinformation. Misinformation about COVID-19 has been connected not only to medicine and science, but also to religion [15], [16], [17], media [18], [19], technology [20], [21], and racism [22], [23], [24]. Evidence suggests that belief in misinformation contributes to vaccine hesitancy [3], [4], [24], as does unwillingness to comply with public health agents’ recommendations [25]. Thus, global public health crises are often global information crises [26] characterized by an upturn of misinformation [9], [10], [27].

To study resistance to vaccines, Martin and Petrie’s [28] Vaccine Attitudes Examination (VAX) scale offers a useful framework. The VAX scale measures four key factors for distrust in vaccines: (1) mistrust of vaccine benefit, owing to a perceived lack of safety, effectiveness, and/or protectiveness; (2) worries about unforeseen future effects, including unforeseen problems, for adults and/or children; (3) concerns about commercial profiteering, reflecting the belief that vaccines are promoted by authorities and corporations to advance their financial interests; and (4) preference for natural immunity, based on the belief that natural exposure achieves safer, longer lasting immunity. Intermediate to high levels of both mistrust in vaccine benefit and concerns about future unforeseen side effects are the most important determinants of uncertainty and unwillingness to be vaccinated against COVID-19 [29], [30].

Despite emerging evidence in the literature, complexities in the relationship between misinformation and COVID-19 vaccine hesitancy remain relatively understudied. To understand these complexities in order to improve vaccination acceptance, much research is needed. In this article, we therefore ask the following research question: What is the relationship between misinformation and vaccine hesitancy?

2. Methods

In this article, we report findings from two related but distinct studies that were part of a larger research project funded by the National Science Foundation. First, we conducted semi-structured interviews via telephone with older adults to ascertain their attitudes toward emerging COVID-19 vaccines (Study 1). Participants from an earlier survey study [31] were asked to indicate interest in this follow-up interview study by including an email address as contact information. We chose to focus on older adults in the interviews, given that the COVID-19 pandemic had disproportionately severe outcomes for older adults in terms of lethality and health consequences, social isolation, and challenges due to lack of digital literacy [32]. Inclusion criteria were as follows: being 65 years of age or older, living in the US, completing the prior survey, and agreeing to be interviewed. Of the 243 older adults who submitted valid responses to the prior survey, 123 (51 %) indicated willingness to participate in the follow-up interviews. We emailed a call for participation to the 123 potential participants, and 51 (41 %) responded. From these 51 potential participants, we selected 30 (59 %) to participate on the basis of their availability for an interview, while also striving for equal distribution in race and ethnicity; political leaning; education; gender identity; and electronic health literacy (eHEALS) score [33]. The interview results then informed the open-ended questions for Study 2, an online survey (N = 718) of people who had not pursued vaccination for COVID-19 by May–June 2021. This research project was approved by the authors’ Institutional Review Board.

2.1. Interviews

2.1.1. Interview participants

In October–November 2020, for Study 1, we conducted semi-structured interviews with 30 older adults living in the US. This number of participants was deemed sufficient by the researchers because the goal of the study was richness and depth of data, satisfying the goal of theoretical saturation [34], [35]. The sample size was acceptable, because the adequacy of data in qualitative research is influenced by many different factors [36], [37]. Interview participants were recruited from a previous online survey sample (N = 454) recruited on Prime Panels, a platform that aggregates online research panels of research participants. Prime Panels’ aggregation of potential participants affords a greater diversity of samples across key demographic variables in the US [38], [39]. However, given that such samples tend to skew younger than the general population [38] and because COVID-19 disproportionately affected older adults, we chose to stratify our sample in terms of age, focusing particularly on recruiting adults older than 65. The resulting survey sample comprised 243 older adults and 211 younger adults [39], so we focused only on the subsample of older adults to interview participants for interviews. The final question in the online survey gave participants the option to sign up for a potential follow-up interview by submitting a short web form with their names and contact information. This optional question mentioned that participants selected for interviews would receive a $20 Amazon gift card as compensation upon completion of their interviews. Of the 243 older adults who participated in the survey, 122 signed up for the potential follow-up interview. The final sample of 30 participants who were interviewed ranged in age from 65 to 87 years; 19 self-identified as female and 11 as male; 1 as Black or African-American and 29 as White; 11 as Democrats, 11 as Republicans, and 8 as Independents.

2.1.2. Interview materials

We developed an interview guide to engage participants in recalling and discussing key examples of COVID-19 health information. We used critical incident technique [40], a research method in which participants are asked to focus on a particular event as well as their reaction to it, and asked questions such as the following: “What was the first piece of COVID-19 health information you heard?” (See Appendix A.) The semi-structured nature of the interviews allowed us the flexibility to clarify or follow up on participants’ answers if necessary [41].

2.1.3. Interview procedure

Two researchers conducted each interview, one as the primary interviewer and the other as note taker. Each interview lasted approximately 45 min. Interviews were audio recorded and automatically transcribed using Otter.ai [42].

2.1.4. Interview data analysis

Two authors independently coded the interview transcripts using Dedoose software for qualitative thematic analysis [43]; the research team discussed, revised, and refined themes through several iterations. Participants’ positive and negative intentions toward COVID-19 vaccines provided the initial codes that guided this iterative review. Next, we gathered the codes, assessed them for patterns, and developed overall themes. We reviewed these themes and compared them against the data as a whole to ensure their accuracy. We coded negative attitudes deductively with the VAX scale [28], finding three of the scale’s four attitudes in the data. Because research on the vaccine debate has tended to focus on those who resist vaccines, the factors that promote vaccine acceptance are less well understood, so we identified subthemes for positive attitudes inductively.

2.2. Survey

2.2.1. Survey participants

After Study 1, again using CloudResearch’s Prime Panels [44] online recruitment platform, we conducted Study 2, an online survey of 725 adults in May–June 2021. All Study 2 participants were from the US and were over 18 years old. To ensure that we were surveying people who were vaccine hesitant, we asked participants about their vaccination status, and we excluded participants who reported having already been vaccinated or having scheduled or attempted to schedule vaccination appointments. A statistical power analysis indicated that a minimum of 432 participants would be needed to achieve 80 % power if α = 0.05 and f = 0.15. In May–June 2021, the US population who had not attempted to schedule an appointment for COVID-19 vaccination was fairly small, particularly for adults 65 and older. According to the CDC, about 85 % of these older adults had received at least 1 dose of the vaccine in June 2021. At this point, we were told by Prime Panels that we had reached almost all older adults within their sample who had not attempted to schedule a vaccine, so we stopped recruiting. A total of 718 participants provided responses to the survey’s open-ended questions, and their responses were included for the present analysis (seven participants left all open-ended questions blank). We recruited both older (65 + years, n = 319) and younger (18–64 years, n = 406) adults. The average age of the 718 survey participants was 51.9 years (SD = 19.6, Median = 56). More than half (65.4 %) were female, 34.3 % were male, and 0.3 % chose not to answer. The majority were White (84.8 %), followed by 1.3 % Asian, 0.3 % Pacific Islander, 9.9 % Black/African American, 1.9 % Hispanic/Latino/Latina, 1.0 % American Indian/Alaska Native, and 2.7 % Other; 0.9 % chose not to answer. Approximately 22.4 % of the participants self-identified as Democrats; 33.4 %, as Independent; and 44.2 %, as Republicans.

2.2.2. Survey materials

Our specific open-ended survey questions were designed to elicit information that shaped participants’ interest in the COVID-19 vaccine, how this information increased or decreased their interest in being vaccinated, what their reasons might be for getting vaccinated or not, and barriers to getting vaccinated. These questions were not piloted, but they were informed by the piloted interview questions in Study 1. The questions were broad and open-ended enough so that they were not age-specific. The five open-ended survey questions are listed in Appendix B.

2.2.3. Survey procedure

We developed Study 2 to understand (1) factors that influenced trust and distrust in information related to COVID-19 and (2) the vaccine hesitancy that emerged in the interview data in Study 1. We asked participants to answer five open-ended questions about why they might be more or less likely to be vaccinated for COVID-19. Their responses ranged from a few words to a few sentences in length. The survey was administered via the online survey tool Qualtrics.

2.2.4. Survey data analysis

We used thematic analysis [43] to identify codes and themes inductively. We then used content analysis to code the themes deductively [45], with the VAX scale as a guide [28]. We used each of the 718 participants’ answers as a single unit of analysis. We created a code book defining all codes in detail, and then two authors independently coded all data. The two coders independently coded the same 20 % of the data and calculated intercoder reliability using Krippendorff’s [45] alpha and ReCal2 [46], which achieved α in the range of 0.661–0.921 for each of six categories of misinformation fueling COVID-19 vaccine hesitancy. Each coder then coded an additional 40 % of the data.

3. Results

From the interviews in Study 1, we identified themes related to both positive and negative attitudes toward COVID-19 vaccines. In Study 2, we explored the negative attitudes more broadly, and we identified six subthemes in the responses to the five open-ended questions; three of the subthemes aligned with Study 1 (unforeseen future effects, fear of commercial profiteering, and doubting effectiveness), and three were new (preference for natural immunity, personal freedom, COVID-19 denial). We also identified six categories of misinformation that participants attributed as causes for their COVID-19 vaccine hesitancy (medical, scientific, political, media, religious, technological).

3.1. Themes identified in the Study 1 interviews

We identified two major themes in the Study 1 interviews: positive and negative attitudes toward COVID-19 vaccines—those in which a participant’s sentiments were positive toward the vaccines with no major concern or hesitation mentioned, and those in which a participant’s sentiments were simply negative. Subthemes were also identified under each key theme. In addition, we identified misinformation that people cited as reasons for not being vaccinated.

3.1.1. Positive attitudes toward COVID-19 vaccines

Participants’ positive attitudes toward COVID-19 vaccines can be described with two subthemes: seeing the vaccines as a reason for hope, and expressing confidence in the vaccines’ effectiveness and in health experts’ recommendations. Participants reported that news of a pending vaccine made them “feel a little bit more positive” and “more hopeful.” For example, in response to news about upcoming COVID-19 vaccines, one participant said that “the stuff about the vaccines have sounded pretty encouraging.” As another said with greater enthusiasm, “I just heard last week about Pfizer developing the vaccine and it may be available by Christmas, by December. So, I was just very happy about hearing that. Have some hope.” Another explained the impact of this news within the greater context of the pandemic: “Even though [the vaccine news] is relatively new information for me, it’s given me a really, really good hope and hopeful attitude. I’m pretty positive that this is going to work out, and the vaccine will be ready.”

With respect to confidence in the vaccines’ effectiveness and in health experts’ recommendation, one participant said, “I’m also going to listen to my physician. Whatever the physician says, I will most likely do that.” This person had trust and confidence in advice from a personal doctor, and scientists and medical experts were mentioned generally as reliable sources of vaccine information: “I trust the people who are relying on the scientists who are working with infectious diseases for years and years … to give out the updates on when we are going to get a safe vaccine.” Some participants specifically expressed trust in pharmaceutical companies to deliver a safe vaccine: “I’m hearing this [information about vaccines and trials] on the new … from the drug companies that are doing the vaccines who I would trust would put out valid information.” Pharmaceutical companies’ research and development was noted as a reason to trust vaccine information: “I trust [information about the vaccine] because it came from Pfizer. It’s a pharmaceutical company developing the vaccine, and they have been doing this research and using it on subjects of people who volunteer to take the vaccine. I’m hoping and praying that what they’re saying is true.” Yet another individual felt the same, referring to the manufacturer’s name recognition: “I’m very confident. It’s based in science. It’s based in test groups, and it’s coming from a very reputable company. It’s not some company that let’s say, I never heard of … Pfizer is a respectable medical corporation.”

3.1.2. Negative attitudes toward COVID-19 vaccines

Participants’ negative attitudes about COVID-19 vaccines included three subthemes: concern regarding commercial profiteering, skepticism about overall effectiveness, and fear of unforeseen future side effects.

Negative attitudes toward COVID-19 vaccines revealed fear that bad actors, particularly pharmaceutical companies, were promoting the vaccines for personal and monetary gain. Potential profit motives of drug manufacturers provided one example: “The person that was presenting me information I believe was actually the CEO of Pfizer. And I thought that could skew the information a bit. It didn’t seem to be an impartial piece that I was looking at.” Other participants were more apprehensive about drug manufacturers’ motives: “They always care about making the profits more than anything else so they’re likely to just ignore things that aren’t good.” Another had a similar concern about competition between pharmaceutical companies: “The companies that are making it want to be the first ones to make it because they’ll be the ones to make the money off it.”

Some participants were concerned about profit motives of other stakeholders, reflecting misinformation and conspiratorial thinking prevalent at the time. Fear of profiteering might be directed at specific individuals: “I have lost my faith in that Fauci guy … finding out that his wife has been involved with the vaccines or with something in it, and it will benefit him [financially].” Others feared that the vaccine was being used for benefits by larger, unspecified groups: “Politicians who just want to make the current administration look bad. They don’t want a quick fix. They want the collapse of the economy. And they don’t care how many people die because the collapse of the economy brings chaos, which brings their revolution.”

The question of COVID-19 vaccines’ effectiveness and whether they would actually prevent individuals from contracting the disease were repeatedly mentioned by participants. Some focused on the projected percentage of effectiveness: “I don’t totally trust the 10 % that’s not included in the vaccine [90 % effectiveness]. I don’t know whether to take it or not. I’d feel a lot better if it was 99 % effective.” Some drew comparisons with the flu vaccine: “I’m no scientist but if you think about the flu vaccine it doesn’t necessarily cover every strain, and we don’t know if COVID will mutate in some way.” Another concern was the length of time the vaccine would be effective: “It’s probably gonna be like the flu where you’re gonna have to get a shot every year to keep getting vaccinated for it.”

Other concerns about effectiveness focused on complications in vaccine rollout and delivery: “I think they just need a little more time, more testing, more time to develop this to make sure it’s going to be effective.” Confusion and stress were often related to the vaccine’s rollout: “Take two doses? You know, it is stressful. You don’t know who’s going to get it first. You don’t know how they’re going to get it out.” Delivery concerns were attributed to transportation and handling: “Was it cold when I got the shot? Did it get warm at some point being transported? Specialty refrigerators that are needed to transport that, they aren’t even built yet.” The perceived politicization of the vaccine’s creation, approval, and rollout led some to lose trust in the vaccine: “I guess because of the political influence of the CDC, I no longer have confidence that I had [in the vaccine]”; “I’m one of those people who [was] ready and willing to go out and take the vaccine if it came out. I’m not as willing [to get the vaccine] now because it seems to be politicized again.” One participant simply wanted more evidence of the vaccine’s effectiveness: “Prove it is what I want them to do. Prove it, don’t just tell me something works. Prove it.”

Many participants were hesitant to be among the first to take the vaccine: “I’ve been asked to be a volunteer and be a guinea pig for it, and I’ve said no. Not for COVID. I’ll be the guinea pig for some other things, but not for that … It’s just too risky.” Another spoke similarly: “I’d like them to have a vaccine, but I’m not really anxious to be first in line to try it. I’d rather other people try it before me just to make sure it’s safe and it’s not going to have bad side effects.” Another expressed the same sentiment: “I’m afraid it’s not going to be safe because I don’t believe it’s going to be tested well enough before they start letting every-one use it.” One participant’s fear of unforeseen negative consequences of the vaccine was based on personal experience with previous vaccinations: “I’m nervous about it … I’m not allergic but I still get these weird [vaccine reactions].”

Several participants said that they wanted others to get the vaccine first: “I would kind of wait for somebody else to have that vaccine and see what happens first before I would take it.” One expressed internal conflict about the same fear: “Part of me wants to be first in line, and just sock it to me. And then there’s another part that says, look, if you’re home alone this long period of time, wait a little bit longer and see if people are getting ill from this [vaccine].” More extreme fear was evidenced in comments about deaths related to the vaccine: “I was just watching the news today where in one of the vaccine trials, the person died while she was doing the trial. She passed away. So, they are stopping a lot of vaccines, putting it on hold is all.” These interview results led us to conduct further data collection in May–June 2021 to focus specifically on people who were not vaccinated or were actively pursuing getting vaccinated.

3.2. Reasons identified in responses to Study 2′s open-ended questions

Building on the negative attitudes toward COVID-19 vaccines identified in the Study 1 interviews, we identified seven reasons from the Study 2 survey data that people gave for not getting the COVID-19 vaccine (Table 1 ). One of these, health and scheduling barriers, involved participants’ either facing challenges in getting vaccinated or waiting until additional information was known about how the vaccine might interact with their medical condition, based on advice from their personal physicians. The remaining six reasons for not getting the vaccine were all connected to vaccine hesitancy: unforeseen future effects, doubt of the vaccines’ effectiveness, commercial profiteering, preference for natural immunity, personal freedom, and COVID-19 denial.

Table 1.

Reasons for not getting the COVID-19 vaccine.

Category Description Number of Responses %
Unforeseen future effects* Worries about unforeseen problems for adults and/or children 355 49
Fear of commercial profiteering* Belief that vaccines are promoted by authorities and corporations to advance their financial interests 92 13
Doubting effectiveness* Mistrust of vaccine benefit due to a perceived lack of safety, effectiveness, and/or protectiveness 87 12
Preference for natural immunity* Belief that natural exposure achieves safer and longer lasting immunity 26 4
Health/scheduling barriers Difficulty getting the vaccine logistically or due to specific health problems 96 13
Personal freedom Resistance to governmental mandates, religious beliefs, or conspiracy-related theories 156 22
COVID-19 denial The disease is overblown, non-threatening, or a hoax 41 6

*VAX scale vaccine attitude.

3.2.1. Unforeseen future effects

Concerns regarding future negative effects of the COVID-19 vaccine were mentioned many times by participants. This reason for not getting the vaccine included comments like the following: “It hasn’t been tested over a long period of time”; “blood clots and the chance of altering my DNA.”

3.2.2. Doubt of the Vaccine’s effectiveness

Participants expressed doubt regarding the effectiveness of the COVID-19 vaccines. They were concerned about the vaccines’ safety, effectiveness, and protectiveness: “It doesn’t work as well as they say in the media”; “no one knows how long potential immunity will last.”

3.2.3. Commercial profiteering

Commercial profiteering as an objection to the COVID-19 vaccine was mentioned by participants, with the belief that vaccines are promoted by authorities and corporations to advance their own financial interests: “Don’t trust it…Pushed by big pharma”; “I don’t trust our world leaders to do what is best for anyone but themselves.”

3.2.4. Preference for natural immunity

Just as in previous studies of vaccine hesitancy, some people stated a strong preference for natural immunity as their reason for not getting the COVID-19 vaccine. Participants believed that natural exposure to the disease would result in safer, longer lasting immunity than a vaccine would: “I have natural antibodies”; “I’ve had COVID so I have acquired immunity.”

3.2.5. Personal freedom

Many survey respondents considered the vaccine to be a threat to their personal beliefs and freedoms. We identified three subcategories for personal freedom: resistance to governmental mandates, religious objections, and suspicion of government. Many people simply said they did not want to be vaccinated, emphasizing their resistance to governmental mandates: “I don’t want the government telling [me] what I have to put in my body”; “not interested in being ‘forced’”; “I think for myself”; “I think this has a lot to do with government control.” The statements of those who gave religion as their reason for not being vaccinated ranged from broad comments, such as “My Bible” and “Religion,” to more specific claims. For example, one person said that “God protects me from it, if I get it then he is calling me home,” and another said, “I’m afraid it’s the precursor to the mark-666,” referring to a biblical passage in the book of Revelations alluding to the apocalypse. One individual gave a detailed religious objection:

The Archdiocese of New Orleans last week released a statement advising Catholics that the Johnson & Johnson vaccine is “morally compromised as it uses the abortion-derived cell line in development and production of the vaccine as well as the testing.”

Another threat to personal freedom derived from suspicion of government: “I feel as though it may be part of a bigger plan. I DO NOT trust our world leaders to do what is best for anyone but themselves”; “I don't trust government. The fact those who get the vaccine maybe dead in 10 years from the timed released effects of the vaccine.” Several people feared that magnets, microchips, or other foreign objects would be inserted during inoculation: “I heard there’s a chip in it”; “I’ve seen people putting magnets where they received the shot, and the magnets are sticking to them”; “Bill Gates tracking implants”; “I heard that it leaves this metal thing in your arm.”

3.2.6. COVID-19 denial

COVID-19 denial, the idea that COVID-19 is either not that dangerous or a large-scale hoax, was also found among respondents. Some did not seem to comprehend the seriousness of the virus: “I see no need to be vaccinated for something with such low risk of death and a high risk of adverse effects from the vaccination.” Others believed that the news had overstated its impact and included the media as part of a conspiracy-related theory: “It has been completely overblown”; “Too much false information being transmitted through the media to believe that it is even a real pandemic.” Another conspiracy-related theory emphasized political motivations: “The whole COVID-19B.S. was an attempt to get Trump from being reelected.”

3.3. Misinformation serving as antecedents to COVID-19 vaccine attitudes

The misinformation given in Study 2 as reasons for not getting the COVID-19 vaccine fell into six categories: medical, scientific, political, media, religious, and technology (Table 2 ). Medical misinformation informed comments related to individual health, individual deaths, an understanding of acquired immunity, and/or not “needing” the vaccine. Scientific misinformation informed responses related to vaccine development, ingredients, testing, recovery rate, statistics, claims that a vaccine was not a “true” vaccine, and the assertion that the vaccine does not guarantee that one will not contract the virus. Political misinformation was related to government, politicians, Dr. Anthony Fauci, economic impact, freedom, and choice. Comments about the media referred to stretching the truth, propaganda, lack of trust in media, and numbers not adding up. Religious misinformation was reflected in comments about the Bible, the “mark of the beast,” and the number 666. Technological misinformation occurred in comments about various forms of technology (microchips, magnets) and nefarious actors who employ them, as well as the possibility of hidden agendas and concealed power grabs.

Table 2.

Categories of misinformation fueling COVID-19 vaccine hesitancy.

Category Description Example Quote
Medical Related to impact on individual health, individual deaths, understanding of acquired immunity and/or not “needing” the vaccine “I have no desire to be a participant in an experimental drug that they are pushing on people.”
Scientific Related to vaccine development, ingredients, testing, recovery rate, statistics, not a “true” vaccine, and doesn’t 100 % prevent contracting COVID-19 “This is not science at work. They are not testing nor reporting final results.”
Political Related to government, politicians, Dr. Fauci, economic impact, freedom and choice “I don’t trust this governmental party in now. They have a country falling apart that they caused, and all they talk about is A SHOT IN THE ARM! That’s desperation for their evil plan…the evil government.”
Media Stretched the truth, propaganda, do not trust, numbers do not add up The media stretched the truth about COVID-19, and all the people that died did not die from COVID-19.”
Religious Related to the Bible, mark of the beast, and 666 “It is the mark of the evil one and will sentence one to eternal perdition.”
Technological Related to microchips, magnets, and DNA manipulation “Bill Gates tracking implants [in the vaccination].”

Beyond examining participants’ reasons for not getting the COVID-19 vaccine (Table 1) and the categories of misinformation fueling COVID-19 vaccine hesitancy (Table 2), it is valuable to examine their intersections and overlaps. By cross-referencing these two coding schema, several themes become evident, and all reflect the impact of nonmedical and nonscientific misinformation. It is to be expected that medical and scientific misinformation would be cited in discussing reasons for vaccine hesitancy, but it is surprising that political, media, and religious misinformation might be given as reasons to refuse such a preventative and common step against deadly disease (Table 3 ). Although both distrust of authorities and commercial profiteering have been identified in previous studies as reasons for vaccine hesitancy, our study participants cited political and media misinformation as a driver of distrust of authorities as well as commercial profiteering. Religious misinformation that focused on unforeseen future effects was also given as a reason for vaccine hesitancy. Fear of evil seemed to be linked with the COVID-19 vaccine in these individuals. Political misinformation also informed vaccine hesitancy related to both personal freedom and COVID-19 denial. These examples show how specific types of misinformation other than medical and scientific are related to the categories of vaccine hesitancy.

Table 3.

Examples of misinformation driving vaccine hesitancy.

Reasons for Vaccine Hesitancy Misinformation Type Example Quotes
Distrust of authorities/ commercial profiteering Political I don’t trust this government party in now [sic]. They have a country falling apart that they caused and all they talk about is A SHOT IN THE ARM! That’s desperation for their evil plan…the evil government
Political and media Go review all of those who have died post-vaccine, but mainstream media and Fauci and the CDC ignore those deaths or just consider them to be collateral damage.
Unforeseen future effects Religious and technological (1) The vaccine contains aborted baby cells. (2) The vaccine has a delayed-euthanasia effect. (3) The vaccine rearranges one’s DNA and RNA so that some demonically inspired scientist can claim that now one is made in its image rather than in the image and likeness of the Creator. (4) It is the mark of the evil one and will sentence one to eternal perdition. No thank you on all counts.
Personal freedom Political I don’t know about this whole Covid-19 thing. I don’t know if it’s a government thing to wear [sic] they’re moving money or is it a quarter [quota] they have to meet Dr. Fauci is now advocating that people will have to get an annual booster shot for a disease that is no more serious than the seasonal flu. We’re [sic] wrecked our economy for the interests of a small political class who doesn't care about us.
COVID-19 denial Political The whole covid 19 bs was an attempt to get trump from being reelected, that and the fraud worked—no, the link is I am NOT getting the unproven heart swelling crap

4. Discussion

The COVID-19 pandemic has brought anti-vaccine beliefs to the forefront of public discussion and confirmed that vaccine hesitancy is a public threat [1]. Our interview study in October and November of 2020 found mixed attitudes toward the COVID-19 vaccine, with positive attitudes that included feelings of hopefulness and confidence in doctors and science, and negative attitudes that included concerns about commercial profiteering and skepticism about overall effectiveness. The prevalence of reluctance and doubts about getting the forthcoming vaccine reflect findings that willingness to be vaccinated against COVID-19 was rapidly declining [2]. Six months later, in May–June 2021, our survey probed further into reasons why people remained unvaccinated despite increased access for most population groups, confirming findings that continued public distrust in vaccines among the unvaccinated was contributing to new waves of COVID-19 cases and deaths among the unvaccinated [3], [4].

Several themes identified in our two studies aligned with the categories of attitudes on the VAX scale [28], confirming earlier research in which mistrust of vaccine benefits and concerns regarding future unforeseen side effects were primary determinants of unwillingness to get a COVID-19 vaccine [29], [30]. However, our studies also identified new reasons for rejecting vaccines that were not included in the Vax scale: personal freedom and COVID-19 denial. These results demonstrate unique attitudes toward the COVID-19 vaccine during a crucial period of the global pandemic—a time when the potential for a vaccine was realized and vaccines were rolled out to the general population, but when unclear communication and poor understanding led to vaccine hesitancy driven by a larger range of social and political factors than found in previous studies. Misinformation was present and prominent during the COVID-19 pandemic [2], [3], [26], [27].

Our findings provide a comprehensive inventory of the forms of misinformation, including but not limited to medical and scientific [3], that have been previously identified separately in the literature: religious [15], political [16], [17], media [18], [19], and technological [20], [21]. By studying COVID-19 vaccine hesitancy from its origins during the period in which the vaccines were being tested and reviewed through the point when vaccines were widely available to the US adult population, we identified a more comprehensive set of vaccine attitudes in relation to COVID-19, as well as a classification scheme for misinformation that served as antecedents to COVID-19 vaccine hesitancy.

This work contributes to a new understanding of reasons why people choose not to be vaccinated against COVID-19, and it can provide guidance for planning public health and vaccination campaigns. Public health officials should consider directly refuting the various points of misinformation identified through our study: medical, scientific, political, media, religious, and technological. However, although combatting medical and scientific misinformation is important, focusing on facts alone will be insufficient to reduce people’s hesitancy toward vaccination. Public health officials can address medical and scientific misinformation in future pandemics or large-scale health crises, but trusted influential people and organizations in politics, the media, religion, and technology must do so too. Researchers should also explore ways to harness the power of the Internet and social media to positively influence vaccine uptake.

5. Limitations and future directions

The limitations of this study are as follows. At the time of interview data collection in October–November 2020, COVID-19 vaccines had not yet been approved by the US Food and Drug Administration. Nevertheless, this study provides a snapshot of attitudes during a critical period, examining attitudes toward the COVID-19 vaccines when their development, availability, and effectiveness were highly uncertain. The participants were all users of online survey platforms, which was crucial for data collection; but unfortunately, this sample of participants represented only a limited range of representative demographics. Given the use of qualitative methods, the lack of diversity, and the small sample size, our findings are illustrative rather than generalizable. Although the interview questions in Study 1 were tested in a pilot study, the survey questions for Study 2 were not piloted prior to their use. Further, the questions were not reviewed by external experts, although they were written in collaboration with one of the co-authors who holds a medical degree and a faculty appointment in a School of Nursing.

Future research should test the two new categories of vaccine hesitancy identified here along with the VAX scale [28]. By including personal freedom and COVID-19 denial as categories of vaccine hesitancy, researchers may be better able to understand how vaccine attitudes are directly and indirectly affecting public health. In addition, future studies should use larger, representative national samples, including participants across a wide range of age groups as well as other demographic groups, especially racial and ethnic minorities. The survey data were collected in May–June 2021, when three different vaccines (Pfizer, Moderna, and Johnson & Johnson) had been approved for US adults and were widely available in most areas. There is a need for longitudinal open-ended research tracking changes in public responses to new vaccines over time. Future research should systematically explore the relationship between the six COVID-19 vaccine attitudes and the six types of COVID-19 vaccine misinformation identified in this study. Such a study could identify correlations and, with a longitudinal design, might reveal causation between vaccine misinformation and vaccine attitudes.

6. Conclusion

The COVID-19 pandemic has highlighted new and more nuanced reasons for vaccine hesitancy driven by a complicated range of social and political factors. Beyond the four categories of the VAX scale, there are additional motivations for vaccine hesitancy. Personal freedom and COVID-19 denial are attitudes of particular interest, because they represent important yet understudied phenomena. Medical and scientific approaches may not be sufficient to combat misinformation based in religion, media, or politics; and public health officials may benefit from partnering with trusted, influential people and organizations with relevant expertise to address harmful misinformation that is helping to drive vaccine hesitancy.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Tara Zimmerman reports financial support was provided by National Science Foundation. Kenneth R. Fleischmann, Bo Xie, Min Kyung Lee reports financial support was provided by National Science Foundation.

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grants IIS-2027426 and CCF-2030859.

Appendix A. Semi-Structured interview questions

  • 1.
    From what source did you first learn about COVID-19?
    • (a)
      What was the information source? (Clarify source medium: TV, Facebook, NewsFeed)
    • (b)
      Did you trust the information you were hearing from this source? Why or why not?
    • (c)
      How confident are you in the information from this source? Why?
    • (d)
      What factors do you continue to use to assess the trustworthiness of that information?
    • (e)
      Did this information change your day-to-day life in any way? Please explain.
  • 2.
    What is the most recent health information you’ve heard about COVID-19?
    • (a)
      What was the information source?
    • (b)
      Did you trust the information? Why or why not?
    • (c)
      How confident are you in this information? Why?
    • (d)
      What factors do you continue to use to assess its trustworthiness?
    • (e)
      Has this information changed your day-to-day life in any way? Please explain.
  • 3.
    Please give me an example of COVID-19 health information that you trust.
    • (a)
      Why did you trust this information?
    • (b)
      What was the source of this information?
    • (c)
      How confident are you in this information? Why?
    • (d)
      Has your trust in this information changed over time?
    • (e)
      Did the source of the information contribute to your trust of this information? (if not already answered)
    • (f)
      Did this information change your day-to-day life in any way? Please explain. (if not already answered)
  • 4.
    Please give me an example of COVID-19 health information that you don’t trust.
    • (a)
      Why did you distrust this information?
    • (b)
      What was the information source?
    • (c)
      How confident are you in this information? Why?
    • (d)
      Has your distrust in this information changed over time?
    • (e)
      Did the source of the information contribute to your distrust of this information? (if not already answered)
    • (f)
      Did this information change your day-to-day life in any way? Please explain. (if not already answered)
  • 5.
    Please give me an example of COVID-19 health information that you originally distrusted but now you do trust.
    • (a)
      Why did you originally distrust this information?
    • (b)
      What led you to trust this information?
    • (c)
      What was the information source and did that affect your trust or distrust? (if not
    • (d)
      already answered)
    • (e)
      Did this information change your day-to-day life in any way? Please explain. (if
    • (f)
      not already answered)
  • 6.
    Could you please give me an example of COVID-19 information that you originally trusted but now do not trust.
    • (a)
      Why did you originally trust this information?
    • (b)
      What led you to distrust this information?
    • (c)
      What was the information source and did that affect your trust or distrust? (if not already answered)
    • (d)
      Did this information change your day-to-day life in any way? Please explain. (if not already answered)
  • 7.
    Is there a piece of COVID-19-related health information that you’re not sure whether to trust or distrust? Tell me about it.
    • (a)
      What factors lead you to trust this information?
    • (b)
      What factors lead you to distrust this information?
    • (c)
      What was the information source and how does that affect your trust or distrust?
    • (d)
      (if not already answered)
    • (e)
      How would trusting or distrusting this information change your day-to-day life? Please explain. (if not already answered) (20)
  • 8.
    Do you use social media? (If yes, continue; if no, first prompt with examples and if answer is still no, move to 9). Examples, if needed: Facebook, Twitter, Instagram, WhatsApp.
    • (a)
      When you encounter COVID-19 health information on social media, do you normally pay attention to who posted it?
      • i.
        Does that affect how you perceive that information?
    • (b)
      Have you encountered information you knew to be incorrect about COVID-19 on social media?
      • i.
        If so, how did you handle that?
    • (c)
      Have you seen any posts that have been marked as “Misinformation” or labeled as “Fact Checked by a 3rd party”?
      • i.
        Did that surprise you?
      • ii.
        How did that change your perception of the information?
      • iii.
        How did that change your perception of the person who posted it?
      • iv.
        Did you do anything about this?
    • (d)
      How has interacting with COVID-19 information on social media affected your relationships?
    • (e)
      Have any conflicts with others arisen from this?
      • i.
        If, so how have you managed that?
    • (f)
      When you consider sharing COVID-19 health information on social media, how do the potential reactions of individuals in your social network influence your decision to share?
    • (g)
      When you think about sharing COVID-19 information, do you think about how others might change their perceptions of you because of the information that you shared?
    • (h)
      Have any groups that you are involved with on social media influenced your views on COVID-19? Please explain.
    • (i)
      How have your personal beliefs and experiences influenced how you interact with COVID-19 content on social media?
  • 9.
    The first statement is, “The COVID-19 virus can be transmitted in areas with hot and humid climates.”
    • (a)
      Would you say that was true or false?
    • (b)
      Tell me more about why you feel that way.
  • 10.
    The second statement is, “Spraying and introducing bleach or another disinfectant into your body will not protect you against COVID-19 and can be dangerous.”
    • (a)
      Would you say that was true or false?
    • (b)
      Can you please tell me more about your answer?

Appendix B. Open-ended survey questions

Q29 What are the specific reasons that may have increased your interest in getting vaccinated for COVID-19?

Q30 Have you encountered any information that increased your interest in getting vaccinated for COVID-19? If yes, please describe the content of that information or put the link here.

Q31 What are the specific reasons that may have decreased your interest in getting vaccinated for COVID-19?

Q32 Have you encountered any information that decreased your interest in getting vaccinated for COVID-19? If yes, please describe the content of that information or put the link here.

Q51 What are the barriers, if any, for you to receive a COVID-19 vaccine? (please check all that apply).

  • There is no COVID-19 appointments available (1)

  • There is no COVID-19 vaccination providers near me (2)

  • I don’t know how to schedule a COVID-19 vaccine appointment (3) ›

  • I don’t know where to get a COVID-19 vaccine (4)

  • I don’t have a way to get to a COVID-19 vaccination site (Mobility) (5)

  • I don’t have a valid ID (6)

  • I don’t have health insurance (7)

  • Others (Please specify any other barriers you can think of) (8) ______________________________

Data availability

Data will be made available on request.

References

  • 1.McAteer J., Yildirim I., Chahroudi A. The VACCINES Act: deciphering vaccine hesitancy in the time of COVID-19. Clin Infect Dis. 2020;71(15):703–705. doi: 10.1093/cid/ciaa433. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.World Health Organization. Coronavirus disease (COVID-19) advice for the public: when and how to use masks. Updated December 2021. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/advice-for-public/when-and-how-to-use-masks.
  • 3.Loomba S., de Figueiredo A., Piatek S.J., de Graaf K., Larson H.J. Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nature Hum Behav. 2021;5(3):337–348. doi: 10.1038/s41562-021-01056-1. [DOI] [PubMed] [Google Scholar]
  • 4.Grewal M, Mushtaq A, Chopra T. “It’s worth a shot ... or is it?” Notes from the grassroots on vaccine hesitancy and bridging gaps. Infect Control Hosp Epidemiol Published online August 2, 2021. https://doi.org/10.1017/ice.2021.356. [DOI] [PMC free article] [PubMed]
  • 5.Tucker JA, Guess A, Barbera P, Vaccari C, Siegel C, Siegel A, et al. Social media, political polarization, and political disinformation: a review of the scientific literature. Published March 19, 2018. SSRN. https://doi.org/10.2139/ssrn.3144139.
  • 6.Starbird K. Disinformation’s spread: bots, trolls and all of us. Nature. 2019;571(7766):449. doi: 10.1038/d41586-019-02235-x. [DOI] [PubMed] [Google Scholar]
  • 7.Houli D., Radford M.L., Singh V.K. “COVID-19 is_”: the perpetuation of Coronavirus conspiracy theories via Google Autocomplete. Proc Assoc Inf Sci Technol. 2021;58(1):218–229. doi: 10.1002/pra2.450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.LaFrance A. The prophecies of Q: American conspiracy theories are entering a dangerous new phase. The Atlantic June 2020;26–3. https://www.theatlantic.com/magazine/archive/2020/06/qanon-nothing-can-stop-what-is-coming/610567.
  • 9.Douglas K.M., Uscinski J.E., Sutton R.M., Cichocka A., Nefes T., Ang C.S., et al. Understanding conspiracy theories. Polit Psychol. 2019;40(S1):3–35. [Google Scholar]
  • 10.Pennycook G., Epstein Z., Mosleh M., Arechar A., Eckles D., Rand D. In: Advances in Consumer Research Volume NA-48. Argo J., Lowrey T.M., Schau J.H., editors. Association for Computer Research; 2020. Understanding and reducing the spread of misinformation online; pp. 863–867. [Google Scholar]
  • 11.Silverman C. This analysis shows how viral fake election news stories outperformed real news on Facebook. BuzzFeed News. Published November 16, 2021. https://www.buzzfeednews.com/article/craigsilverman/viral-fake-election-news-outperformed-real-news-on-facebook.
  • 12.Vosoughi S., Roy D., Aral S. The spread of true and false news online. Science. 2018;359(6380):1146–1151. doi: 10.1126/science.aap955910. [DOI] [PubMed] [Google Scholar]
  • 13.Bierwiaczonek K., Kunst J.R., Pich O. Belief in COVID-19 conspiracy theories reduces social distancing over time. Appl Psychol Health Well Being. 2020;12(4):1270–1285. doi: 10.1111/aphw.12223. [DOI] [PubMed] [Google Scholar]
  • 14.Koltai K.S., Fleischmann K.R. Questioning science with science: the evolution of the vaccine safety movement. Proc Assoc Inf Sci Technol. 2017;54(1):232–240. doi: 10.1002/pra2.2017.14505401026. [DOI] [Google Scholar]
  • 15.Barua Z., Barua S., Aktar S., Kabir N., Li M. Effects of misinformation on COVID-19 individual responses and recommendations for resilience of disastrous consequences of misinformation. Prog Disaster Sci. 2020;8 doi: 10.1016/j.pdisas.2020.100119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Bilewicz M., Soral W. The politics of vaccine hesitancy: an ideological dual-process approach. Soc Psychol Pers Sci. 2022;13(6):1080–1089. doi: 10.1177/19485506211055295. [DOI] [Google Scholar]
  • 17.Fridman A., Gershon R., Gneezy A., Capraro V. COVID-19 and vaccine hesitancy: a longitudinal study. PLoS ONE. 2021;16(4):e0250123. doi: 10.1371/journal.pone.0250123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Gruzd A., Mai P. Going viral: how a single tweet spawned a COVID-19 conspiracy theory on twitter. Big DataSoc. 2020;7(2) doi: 10.1177/2053951720938405. [DOI] [Google Scholar]
  • 19.Romer D., Jamieson K.H. Conspiratorial thinking, selective exposure to conservative media, and response to COVID-19 in the US. Soc Sci Med. 2021;291 doi: 10.1016/j.socscimed.2021.114480. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Hornsey M.J., Finlayson M., Chatwood G., Begeny C.T. Donald Trump and vaccination: the effect of political identity, conspiracist ideation and presidential tweets on vaccine hesitancy. J Exp Soc Psychol. 2020;88 doi: 10.1016/j.jesp.2019.103947. [DOI] [Google Scholar]
  • 21.Ahmed W., Vidal-Alaball J., Downing J., López S.F. COVID-19 and the 5G conspiracy theory: social network analysis of Twitter data. J Med Internet Res. 2020;22(5):e19458. doi: 10.2196/19458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Chong M., Chen H. Racist framing through stigmatized naming: a topical and geo-locational analysis of #Chinavirus and #Chinesevirus on Twitter. Proc Assoc Inf Sci Technol. 2021;58(1):70–79. doi: 10.1002/pra2.437. [DOI] [Google Scholar]
  • 23.Costello M., Cheng L., Luo F., Hu H., Liao S., Vishwamitra N., et al. COVID-19: a pandemic of anti-Asian cyberhate. J Hate Stud. 2021;17(1):108. [Google Scholar]
  • 24.Hswen Y., Xu X., Hing A., Hawkins J.B., Brownstein J.S., Gee G.C. Association of “#covid19” Versus “#chinesevirus” With Anti-Asian Sentiments on Twitter: March 9–23, 2020. Am J Public Health. 2021;111(5):956–964. doi: 10.2105/AJPH.2021.306154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Nyhan B., Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33(3):459–464. doi: 10.1016/j.vaccine.2014.11.017. [DOI] [PubMed] [Google Scholar]
  • 26.Xie B.o., He D., Mercer T., Wang Y., Wu D., Fleischmann K.R., et al. Global health crises are also information crises: a call to action. J Assoc Inf Sci Technol. 2020;71(12):1419–1423. doi: 10.1002/asi.24357. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Imhoff R., Lamberty P. A bioweapon or a hoax? The link between distinct conspiracy beliefs about the Coronavirus disease (COVID-19) outbreak and pandemic behavior. Soc Psychol Personal Sci. 2020;11(8):1110–1118. doi: 10.1177/1948550620934692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Martin L.R., Petrie K.J. Understanding the dimensions of anti-vaccination attitudes: the Vaccination Attitudes Examination (VAX) scale. Ann Behav Med. 2017;51(5):652–660. doi: 10.1007/s12160-017-9888-y. [DOI] [PubMed] [Google Scholar]
  • 29.Paul E., Steptoe A., Fancourt D. Attitudes towards vaccines and intention to vaccinate against COVID-19: implications for public health communications. LancetReg Health Eur. 2021;1 doi: 10.1016/j.lanepe.2020.100012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Rosenthal S., Cummings C.L. Influence of rapid COVID-19 vaccine development on vaccine hesitancy. Vaccine. 2021;39(52):7625–7632. doi: 10.1016/j.vaccine.2021.11.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Verma N, Fleischmann KR, Zhou L, Xie B, Lee MK, Rich K, et al. Trust in COVID‐19 public health information. J Assoc Inf Sci Technol Published online September 20, 2022. https://doi.org/10.1002/asi.24712. [DOI] [PMC free article] [PubMed]
  • 32.Xie B, Charness N, Fingerman K, Kaye J, Kim MT, Khurshid A. When going digital becomes a necessity: ensuring older adults’ needs for information, services, and social inclusion during COVID-19. J Aging Soc Policy 2020;32(4–5):460–70. https://doi.org/10.1080/08959420.2020.1771237. [DOI] [PMC free article] [PubMed]
  • 33.Norman CD, Skinner HA. eHEALS: the eHealth Literacy Scale. J Med Internet Res 2006;8(4):e27. https://doi.org/10.2196/jmir.8.4.e27. [DOI] [PMC free article] [PubMed]
  • 34.Faulkner SL, Trotter SP. Theoretical saturation. In: Matthes J, ed. The International Encyclopedia of Communication Research Methods. Wiley; 2017. https://doi.org/10.1002/9781118901731.iecrm0250.
  • 35.Low J. A pragmatic definition of the concept of theoretical saturation. Sociol Focus. 2019;52(2):131–139. doi: 10.1080/00380237.2018.1544514. [DOI] [Google Scholar]
  • 36.Daniel BK. Student experience of the maximum variation framework for determining sample size in qualitative research. In: Stacey A, ed. Proceedings of the 18th European Conference on Research Methodology for Business and Management Studies. Academic Conferences and Publishing International Limited; 2019:92–100.
  • 37.Rowlands T., Waddell N., McKenna B. Are we there yet? A technique to determine theoretical saturation. J Comp Inf Syst. 2016;56(1):40–47. doi: 10.1080/08874417.2015.11645799. [DOI] [Google Scholar]
  • 38.Chandler J., Rosenzweig C., Moss A.J., Robinson J., Litman L. Online panels in social science research: expanding sampling methods beyond Mechanical Turk. Behav Res Methods. 2019;51(5):2022–2038. doi: 10.3758/s13428-019-01273-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Verma N., Shiroma K., Rich K., Fleischmann K.R., Xie B., Lee M.K. In: Diversity, Divergence, Dialogue: 16th International iConference 2021: Proceedings, Part II. Toeppe K., Yan H., Chu S.K.W., editors. Springer International; 2021. Conducting quantitative research with hard-to-reach-online populations: using Prime Panels to rapidly survey older adults during a pandemic. [Google Scholar]
  • 40.Flanagan J.C. The critical incident technique. Psychol Bull. 1954;51(4):327–358. doi: 10.1037/h0061470. https://www.apa.org/pubs/databases/psycinfo/cit-article.pdf [DOI] [PubMed] [Google Scholar]
  • 41.Wethington E., McDarby M.L. In: The Encyclopedia of Adulthood and Aging. Whitbourne S.K., editor. Wiley; 2015. Interview methods (structured, semistructured, unstructured) [DOI] [Google Scholar]
  • 42.Corrente M., Bourgeault I. SAGE Research Methods: Doing Research Online. SAGE Publications; 2022. Innovation in transcribing data: meet otter.ai. [DOI] [Google Scholar]
  • 43.Braun V., Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. doi: 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
  • 44.Litman L., Robinson J., Abberbock T. TurkPrime.com: a versatile crowdsourcing data acquisition platform for the behavioral sciences. Behav Res Methods. 2017;49(2):433–442. doi: 10.3758/s13428-016-0727-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Krippendorff K. 4th ed. SAGE Publications; 2018. Content Analysis: An Introduction to Its Methodology. [Google Scholar]
  • 46.Freelon D. ReCal2: Reliability for 2 coders. http://dfreelon.org/utils/recalfront/recal2/.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data will be made available on request.


Articles from Vaccine are provided here courtesy of Elsevier

RESOURCES