One Year of COVID-19 Vaccine Misinformation on Twitter: Longitudinal Study

Abstract

Background

Vaccinations play a critical role in mitigating the impact of COVID-19 and other diseases. Past research has linked misinformation to increased hesitancy and lower vaccination rates. Gaps remain in our knowledge about the main drivers of vaccine misinformation on social media and effective ways to intervene.

Objective

Our longitudinal study had two primary objectives: (1) to investigate the patterns of prevalence and contagion of COVID-19 vaccine misinformation on Twitter in 2021, and (2) to identify the main spreaders of vaccine misinformation. Given our initial results, we further considered the likely drivers of misinformation and its spread, providing insights for potential interventions.

Methods

We collected almost 300 million English-language tweets related to COVID-19 vaccines using a list of over 80 relevant keywords over a period of 12 months. We then extracted and labeled news articles at the source level based on third-party lists of low-credibility and mainstream news sources, and measured the prevalence of different kinds of information. We also considered suspicious YouTube videos shared on Twitter. We focused our analysis of vaccine misinformation spreaders on verified and automated Twitter accounts.

Results

Our findings showed a relatively low prevalence of low-credibility information compared to the entirety of mainstream news. However, the most popular low-credibility sources had reshare volumes comparable to those of many mainstream sources, and had larger volumes than those of authoritative sources such as the US Centers for Disease Control and Prevention and the World Health Organization. Throughout the year, we observed an increasing trend in the prevalence of low-credibility news about vaccines. We also observed a considerable amount of suspicious YouTube videos shared on Twitter. Tweets by a small group of approximately 800 “superspreaders” verified by Twitter accounted for approximately 35% of all reshares of misinformation on an average day, with the top superspreader (@RobertKennedyJr) responsible for over 13% of retweets. Finally, low-credibility news and suspicious YouTube videos were more likely to be shared by automated accounts.

Conclusions

The wide spread of misinformation around COVID-19 vaccines on Twitter during 2021 shows that there was an audience for this type of content. Our findings are also consistent with the hypothesis that superspreaders are driven by financial incentives that allow them to profit from health misinformation. Despite high-profile cases of deplatformed misinformation superspreaders, our results show that in 2021, a few individuals still played an outsized role in the spread of low-credibility vaccine content. As a result, social media moderation efforts would be better served by focusing on reducing the online visibility of repeat spreaders of harmful content, especially during public health crises.

Introduction

The global spread of the novel coronavirus (SARS-CoV-2) over the last 2 years affected the lives of most people around the world. As of December 2021, over 330 million cases of COVID-19 were detected and 5.5 million deaths were recorded due to the pandemic [1]. In the United States, COVID-19 was the third leading cause of death in 2020 according to the National Center for Health Statistics [2]. Despite their socioeconomic repercussions [3,4], nonpharmaceutical interventions such as social distancing, travel restrictions, and national lockdowns have proven to be effective at slowing the spread of the coronavirus [5-7]. As the pandemic evolved, pharmaceutical interventions such as vaccinations and antiviral treatments became increasingly important to manage the pandemic [8,9].

Less than one year into the pandemic, we witnessed the swift development of COVID-19 vaccines, expedited by new mRNA technology [10]. Both Pfizer-BioNTech [11] and Moderna [12] vaccines, among others, obtained emergency authorizations in the United States and Europe by the end of 2020, and governments began to distribute them to the public immediately. Mounting evidence shows that vaccines effectively prevent infections and severe hospitalizations, despite the emergence of new viral strains of the original SARS-CoV-2 virus [13,14]. It was estimated that the US vaccination program averted up to 140,000 deaths by May 2021 [15] and over 10 million hospitalizations by November 2021 [16].

The widespread adoption of vaccines is extremely important to reduce the impact of this highly contagious virus [17]. However, as of December 2021, when supplies were no longer limited, only 62% of US citizens had received two doses of COVID-19 vaccines [18]. Unvaccinated or partially vaccinated individuals still face risks of infection and death that are much higher than the risks for individuals who completed their vaccination cycle [19]. The geographically uneven vaccination coverage of the population can also lead to localized outbreaks and hinder governmental efforts to mitigate the pandemic [20].

Worldwide, most people are in favor of vaccines and vaccination programs, but a proportion of individuals are hesitant about some or all vaccines. Vaccine hesitancy describes a spectrum of attitudes, ranging from people with small concerns to those who completely refuse all vaccines. Previous literature links vaccine hesitancy to several factors that include the political, cultural, and social background of individuals, as well as their personal experience, education, and information environment [21]. Ever since public discourse moved online, concerns have been raised about the spread of false claims regarding vaccines on social media, which may erode public trust in science and promote vaccine hesitancy or refusal [22-25].

After the outbreak of the COVID-19 pandemic, a massive amount of health-related misinformation—the so-called “infodemic” [26]—was observed on multiple social media platforms [27-30], undermining public health policies to contain the disease. Online misinformation included false claims and conspiracy theories about COVID-19 vaccines, hindering the effectiveness of vaccination campaigns [31,32].

A few recent studies reveal a positive association between exposure to misinformation and vaccine hesitancy at the individual level [33] as well as a negative association between the prevalence of online vaccine misinformation and vaccine uptake rates at the population level [34]. Motivated by these findings, the aim of this study was to investigate the spread of COVID-19 vaccine misinformation by analyzing almost 300 million English-language tweets shared during 2021, when vaccination programs were launched in most countries around the world.

There are several studies related to the present work. Yang et al [30] carried out a comparative analysis of English-language COVID-19–related misinformation spreading on Twitter and Facebook during 2020. They compared the prevalence of low-credibility sources on the two platforms, highlighting how verified pages and accounts earned a considerable amount of reshares when posting content originating from unreliable websites. Muric et al [35] released a public data set of Twitter accounts and messages, collected at the end of 2020, which specifically focused on antivaccine narratives. Preliminary analyses show that the online vaccine-hesitancy discourse was fueled by conservative-leaning individuals who shared a large amount of vaccine-related content from questionable sources. Sharma et al [36] focused on identifying coordinated efforts to promote antivaccine narratives on Twitter during the first 4 months of the US vaccination program. They also carried out a content-based analysis of the main misinformation narratives, finding that side effects were often mentioned along with COVID-19 conspiracy theories.

Our work makes two key contributions to existing research. First, we studied the prevalence of COVID-19 vaccine misinformation originating from low-credibility websites and YouTube videos, which was compared to information published on mainstream news websites. As described above, previous studies either analyzed the spread of misinformation about COVID-19 in general (during 2020) or focused specifically on antivaccination messages and narratives. They also analyzed a limited time window, whereas our data capture 12 months into the rollout of COVID-19 vaccination programs. Second, we uncovered the role and the contribution of important groups of vaccine misinformation spreaders, namely verified and automated accounts, whereas previous work either focused on detecting users with a strong antivaccine stance or inauthentic coordinated behavior.

Considering these contributions, we addressed the first research question (RQ):

RQ1: What were the patterns of prevalence and contagion of COVID-19 vaccine misinformation on Twitter in 2021?

Leveraging a data set of millions of tweets, we identified misinformation at the domain level based on a list of low-credibility sources (website domains) compiled by professional fact-checkers and journalists, which is an approach that has been widely adopted in the literature to study unreliable information at scale [37-41]. Additionally, we considered a set of mainstream and public health sources as a baseline for reliable information. We then compared the volume of vaccine misinformation against reliable news, identified temporal trends, and investigated the most shared sources. We also explored the prevalence of misinformation that originated on YouTube and was shared on Twitter [30,42,43].

Analogous to the role of virus superspreaders in pandemic outbreaks [44], recent studies suggest that certain actors play an outsized role in disseminating misleading content [30,39,42]. For example, just 10 accounts were responsible for originating over 34% of the low-credibility content shared on Twitter during an 8-month period in 2020 [45]. To examine how vaccine misinformation was posted and amplified by various actors on social media, we addressed a second RQ:

RQ2: Who were the main spreaders of vaccine misinformation?

Specifically, we analyzed two types of accounts. First, we investigated the presence and characteristics of users who generated the most reshares of misinformation [45,46], with a specific focus on the role of “verified” accounts. Twitter deems these accounts “authentic, notable, and active” (see [47]). Second, we investigated the presence and role of social bots (ie, social media accounts controlled in part by algorithms). Previous studies showed that bots actively amplified low-credibility information in various contexts [38,48,49].

These findings deepen our understanding of the ongoing pandemic and generate actionable knowledge for future health crises.

Methods

Twitter Data Collection

On January 4, 2021, we started a real-time collection of tweets about COVID-19 vaccines using the Twitter application programming interface (API). The tweets were collected by matching relevant keywords through the POST statuses/filter v1.1 API endpoint [50]. This effort is part of our CoVaxxy project, which provides a public dashboard [51] to visualize the relationship between online (mis)information and COVID-19 vaccine adoption in the United States [52].

To capture the online public discourse around COVID-19 vaccines in English, we defined as complete a set as possible of English-language keywords related to the topic. Starting with covid and vaccine as our initial seeds, we employed a snowball sampling technique to identify co-occurring relevant keywords in December 2020 [52,53]. The resulting list contained almost 80 keywords. We show a few examples in Textbox 1; the full list can be accessed through the online repository associated with this project [54]. To validate the data collection procedure, we examined the coverage obtained by adding keywords one at a time, starting with the most common terms. Over 90% of the tweets contained at least one of the three most common keywords: “vaccine,” “vaccination,” or “vaccinate.” This indicates that the collected tweets are very relevant to the topic of vaccines.

Sample keywords employed to collect tweets about vaccines.

• covid19vaccine

• covidvaccine

• coronavirusvaccine

• vaccination

• covid19 pfizer

• pfizercovidvaccine

• oxfordvaccine

• getvaccinated

• covid19

• moderna

• vaccine

• covid19 pfizer

• mrna vaccinate

• covax

• coronavirus moderna

• vax

In this study, we analyzed the data collected in the period from January 4 to December 31, 2021. This comprises 294,081,599 tweets shared by 19,581,249 unique users, containing 8,160,838 unique links (URLs) and 1,287,703 unique hashtags. Figure 1 shows the daily volume of vaccine-related tweets collected.

Figure 1.

Time series of the daily number of vaccine-related tweets shared between January 4 and December 31, 2021. The median daily number of tweets is 720,575.

To comply with Twitter’s terms of service, we are only able to share the tweet IDs with the public, accessible through a public repository [54]. One can “rehydrate” the data set by querying the Twitter API or using tools such as Hydrator [55] or twarc [56].

Identifying Online Misinformation

We identified misinformation in our data set using two approaches. Following a common method in the literature [37-41], the first approach identified tweets sharing links to low-credibility websites that were labeled by journalists, fact-checkers, and media experts for repeatedly sharing false news, hoaxes, conspiracy theories, unsubstantiated claims, hyperpartisan propaganda, click-bait, and so on. Specifically, we employed the Iffy+ Misinfo/Disinfo list of low-credibility sources [57]. This list is mainly based on information provided by the Media Bias/Fact Check (MBFC) website [58], an independent organization that reviews and rates the reliability of news sources. Political leaning was not considered for determining inclusion in the Iffy+ list. Instead, the list includes sites labeled by MBFC as having a “Very Low” or “Low” factual-reporting level and those classified as “Questionable” or “Conspiracy-Pseudoscience” sources. The 674 low-credibility sources in the Iffy+ list also include fake-news websites flagged by BuzzFeed, FactCheck.org, PolitiFact, and Wikipedia.

To expand our list of low-credibility sources, we also employed news reliability scores provided by NewsGuard [59], a journalistic organization that routinely assesses the reliability of news websites based on multiple criteria. NewsGuard assigns news outlets a trust score in the range of 0-100. While it considers outlets with scores below 60 as “unreliable,” we adopted a stricter definition and only considered outlets with a score ≤30 as low-credibility sources. This yielded a list of 1181 websites, which we cannot disclose to the public since the NewsGuard data are proprietary. By combining the Iffy+ and NewsGuard lists, we obtained a total of 1718 low-credibility sources.

We tested the reliability of this domain-based approach to identify misinformation through a qualitative approach similar to that adopted in previous studies [38,60]. We randomly chose 50 low-credibility links in our data set and manually coded them as either “factual,” “misinformation,” or “unverified.” Two authors independently visited the actual web page of each link and researched its content to determine if it was accurate. A link was coded as “factual” if all claims within the article were corroborated by other sources. The “unverified” label was utilized for links that could no longer be accessed (eg, because the web page no longer exists). All other links were coded as “misinformation.” In the event of coding disagreements, authors shared and discussed what they learned during their independent research to reach an agreement on a single label. At the end of this procedure, 7 links were coded as “factual,” 38 as “misinformation,” 4 as “unverified,” and a single article was excluded as it appeared to be a personal blog post. We also note that of the 7 articles labeled as “factual,” 6 were from state propaganda outlets with a selection bias (eg, sputniknews.com or rt.com).

As a second approach, we analyzed links to YouTube videos shared on Twitter that might contain misinformation. We extracted unique video identifiers from links shared in the collected tweets and queried the YouTube API for the video status using the Videos:list endpoint. In light of recent YouTube efforts to remove antivaccine videos according to their COVID-19 policy [61] and their updated policy [62], we considered videos to be suspicious if they were not publicly accessible. Previous research shows that inaccessible videos contain a high proportion of antivaccine content, such as the

“Plandemic” conspiracy documentary [30]. The efficacy of this approach to identifying videos that contain antivaccine content is further supported in research that analyzed available videos shared by users that had also shared an inaccessible video [63]. The authors found that the majority of available videos tweeted by these users promulgated an antivaccine or antimandate stance. As some estimates suggest that it takes an average of 41 days for YouTube to remove videos that violate their terms [43], we checked the status of videos in March 2022, at least 2 months after the last video was posted on Twitter.

Sources of Reliable Information

We curated a list of reliable, mainstream sources of vaccine-related news as our baseline to interpret the prevalence of misinformation and characterized its spreading patterns [30]. In particular, we considered websites with a NewsGuard trust score higher than 80, resulting in a list of 2765 sources. We also included the websites of two authoritative sources of COVID-19–related information, namely the US Centers for Disease Control and Prevention (CDC) [64] and the World Health Organization [65]. In the rest of the paper, we use “low credibility” and “mainstream” to refer to the two sets of sources.

Link Extraction

Identifying low- and high-credibility links and YouTube links required extracting the top-level domains from the URLs embedded in tweets and matching them against our lists of web domains. Shortened links occurred frequently in our data set; therefore, we identified the most prevalent link-shortening services (see Textbox 2 for the list) and obtained the original links through HTTP requests.

List of URL-shortening services considered in our analysis.

bit.ly, dlvr.it, liicr.nl, tinyurl.com, goo.gl, ift.tt, ow.ly, fxn.ws, buff.ly, back.ly, amzn.to, nyti.ms, nyp.st, dailysign.al, j.mp, wapo.st, reut.rs, drudge.tw, shar.es, sumo.ly

Bot Detection

To measure the level of bot activity for different types of information, we employed

BotometerLite [66], a publicly available tool that can efficiently identify likely automated accounts on Twitter [67]. For each Twitter account, BotometerLite generates a bot score in the range of 0-1, where a higher score indicates that the account is more likely to be automated. BotometerLite evaluates an account by inspecting the profile information that is embedded in each tweet. This enabled us to perform bot analysis at the level of tweets in our data set.

Ethical Considerations

This research is based on observations of public data with minimal risks to human subjects. The study was thus deemed exempt from review by the Indiana University Institutional Review Board (protocol 1102004860). Data collection and analysis were performed in compliance with the terms of service of Twitter.

Results

Prevalence and Contagion of Online Misinformation

To address RQ1, we compared the prevalence of tweets that linked to domains in our lists of low-credibility and mainstream sources over time. We carried out a similar analysis for suspicious YouTube videos. As shown in Figure 2A-B, we observed a significant increasing trend in the daily prevalence of low-credibility information over time and a significant opposite trend for mainstream news. This is further confirmed in Figure 2C, which shows the daily ratio between the volumes of tweets linking to low-credibility and mainstream news. A significant increasing trend was observed, suggesting that the public discussion about vaccines on Twitter shifted over time from referencing trustworthy sources in favor of low-credibility sources. The peak in July corresponds to a time when the prevalence of mainstream news was particularly low (see Figure 2B). During this period, we also observed a burst of reshares for content originating from Children’s Health Defense (CHD), the most prominent source of vaccine misinformation (further discussed below).

Figure 2.

Timelines of prevalence of vaccine information on Twitter. Trends were evaluated with the nonparametric Mann-Kendall test. Colored bands correspond to a 14-day rolling average with 95% CIs. (A) Daily number of vaccine tweets sharing links to news articles from low-credibility sources. There is a significant increasing trend (P<.001). (B) Daily number of vaccine tweets sharing links to news articles from mainstream sources. There is a significant decreasing trend (P<.001). (C) Ratio between the volumes of tweets sharing links to low-credibility and mainstream sources. There is a significant increasing trend (P<.001). (D) Daily percentage of tweets sharing links to inaccessible YouTube videos among all tweets sharing links to YouTube. There is a significant decreasing trend (P<.001).

During the entire period of analysis, we found that misinformation was generally less prevalent than mainstream news, as shown in Figure 3A. However, we observed that low-credibility content tended to spread more through retweets compared to mainstream content, as shown in Figure 3B. This indicated that while low-credibility vaccine content was less prevalent overall, it had greater potential for contagion through the social network, suggesting that it might have only spread through a subsection of the population.

Figure 3.

Comparisons between prevalence of tweets linking to mainstream and low-credibility sources. (A) Daily percentage of vaccine tweets and retweets that share links to low-credibility news sources (median: 1.31%) and mainstream news sources (median: 7.53%). The distributions are significantly different according to a two-sided Mann-Whitney test (P<.001). (B) Distributions of the proportion of tweets linking to low-credibility sources (median: 89.19%) and mainstream sources (median: 67.96%) that are retweets. The distributions are significantly different according to a two-sided Mann-Whitney test (P<.001).

Overall, the fraction of vaccine-related tweets linking to YouTube videos was very small (daily median: 0.52%). However, a nonnegligible proportion of these posts (daily median: 10.95%) shared links to inaccessible videos, with a larger prevalence in the first half of 2021 (a peak of 45% was observed in July) and a significant decreasing trend toward the end of the year (see Figure 2D).

Most Popular Misinformation Sources

Looking at different sources of news about vaccines, Figure 4A shows the 20 most shared websites. We note three unreliable sources in this ranking: childrenshealthdefense.org, thegatewaypundit.com, and zerohedge.com. The most popular low-credibility source was the website of the CHD organization, an antivaccine group led by Robert F Kennedy Jr that became very popular during the pandemic as an alternative and natural medicine site [46,68]. This source was banned from Facebook and Instagram for repeatedly violating their guidelines against spreading medical misinformation in August 2022 [69]. Accounting for approximately 0.30% of all vaccine tweets, the prevalence of CHD was comparable to that of reputable sources such as washingtonpost.com and reuters.com, and was roughly twice the prevalence of CDC links (0.16%). As shown in Figure 4B, CHD tweets were much more widely shared than other low-credibility sources, most of which accounted for less than 0.05% of all shared tweets. CHD accounted for approximately 18% of all tweets linking to low-credibility sources, whereas the aggregated 20 most shared sources generated approximately 61% of all such tweets. Nevertheless, the total fraction of tweets sharing low-credibility news about vaccines accounted for only 1.5% compared to approximately 7.8% of tweets that linked to mainstream sources (see Figure 4C).

Figure 4.

Top sources of vaccine content. (A) The top 20 news sources ranked by percentage of vaccine tweets. (B) The top 20 low-credibility news sources ranked by percentage of vaccine tweets. (C) Percentages of all vaccine tweets linking to low-credibility and mainstream news sources.

Superspreaders of Misinformation

Recent work reveals that accounts disseminating a disproportionate amount of low-credibility content—so-called “superspreaders”—played a central role in the digital misinformation crisis [30,39,42,45,46]. These contributions also show that “verified” accounts often act as superspreaders of unreliable information. Therefore, we further investigated the role of such accounts to address RQ2.

Figure 5 shows that over time, verified accounts represented approximately 15% of those that posted vaccine content, but were consistently responsible for approximately 43% of that content. When focusing on the low-credibility content, verified accounts represented an even smaller proportion of accounts, less than 6%. Still, they were responsible for approximately 34% of retweets. These findings highlight a stunning concentration of impact and responsibility for the spread of vaccine misinformation among a small group of verified accounts. While there were substantially fewer verified accounts sharing low-credibility vaccine content (n=828) compared to those sharing vaccine content in general (n=98,612), Figure 6 shows that verified accounts tended to receive more retweets when posting low-credibility content than general vaccine content.

Figure 5.

Comparisons between percentages of original posters who are verified accounts and of retweets earned by verified accounts for different categories of vaccine content. Each data point is a daily proportion. The median daily proportions of verified accounts among posters of vaccine content, low-credibility news, and inaccessible YouTube videos are 15.4%, 5.6%, and 4.5%, respectively. The median daily proportions of retweets earned by verified posters of vaccine content, low-credibility news, and inaccessible YouTube videos are 43.1%, 34.2%, and 13.2%, respectively. All distributions are significantly different from each other according to two-sided Mann-Whitney tests (P<.001).

Figure 6.

Distributions of the mean numbers of retweets earned by verified accounts when sharing vaccine content (median 3.82), low-credibility news (median 9.43), and links to inaccessible YouTube videos (median 1). We display the complementary cumulative distributions in the main plot because the distributions are broad. In fact, the box plots (inset) have many outliers. All distributions are significantly different from each other according to two-sided Mann-Whitney tests (P<.001).

The top 25 accounts are ranked by the number of retweets to their posts linking to low-credibility sources in Figure 7. Eleven of these misinformation superspreaders are accounts that have been verified by Twitter, some of which are associated with untrustworthy news sources (eg, @zerohedge, @BreitbartNews, and @OANN). The top superspreader, Robert Kennedy Jr (@RobertKennedyJr), earned approximately 3.45 times the number of retweets of the second most-retweeted account (@zerohedge). Kennedy was identified as one of the pandemic’s “disinformation dozen” [42,46]. His influence fueled the high prevalence of links to the CHD website within our data set (as shown in Figure 4). His verified account had approximately 3.8 times more followers than the unverified @ChildrensHD account (416,200 vs 109,800, respectively, as of April 24, 2022). Retweets of Mr. Kennedy’s tweets singularly accounted for 13.4% of all retweets of low-credibility vaccine content. A robustness check removing this account from the data yielded consistent results for all analyses reported in this section.

Figure 7.

Top 25 accounts ranked by the number of retweets earned when sharing links to low-credibility news websites. Colors indicate whether accounts are verified (orange) or not (blue).

We also investigated the role of verified users in sharing suspicious videos from YouTube. As shown in Figures 5 and 6, we found that verified accounts do not play as central a role in spreading this content as found for content from low-credibility domains.

Role of Social Bots

To address RQ2, we also inspected the role of likely automated accounts in spreading COVID-19 vaccine misinformation. As mentioned in the Methods section, we employed BotometerLite [67] to calculate a bot score for all the accounts posting a tweet in our data set. We did not observe notable temporal trends in the activity of likely bots over time; therefore, Figure 8 shows the distributions of daily average bot scores for tweets sharing vaccine content, links to low-credibility sources, and inaccessible YouTube videos.

Figure 8.

Comparison between the daily average bot score of tweets sharing different categories of vaccine content. The median daily average bot scores of accounts sharing vaccine content, low-credibility news, and inaccessible YouTube videos are 0.22, 0.25 and 0.26, respectively. All distributions are significantly different from each other according to two-sided Mann-Whitney tests (P<.001).

We observed that tweets sharing links to low-credibility sources had significantly higher bot-activity levels than those of vaccine tweets overall. In addition, the daily average bot scores for tweets sharing inaccessible YouTube videos were even higher than those linking to low-credibility sources.

This analysis was carried out at the tweet level, meaning that if a bot-like account tweeted more times, it made a larger contribution. We observed similar results when performing the analysis at the account level by considering the contribution of each account once.

Discussion

We investigated COVID-19 vaccine misinformation spreading on Twitter during 2021 following the rollout of vaccination programs around the world. Leveraging a source-based labeling approach, we identified millions of tweets sharing links to low-credibility and mainstream news websites. While low-credibility information was generally less prevalent than mainstream content over the year, we observed an increasing trend in the reshares of unreliable news during the year and an opposite, decreasing trend for reliable information. Our data mostly capture English-language conversations, which could originate from different countries. However, our aggregate analysis could not disentangle the infodemic trends and peaks associated with different countries as observed in prior work [29].

Focusing on specific news sources, we noticed three low-credibility websites with volumes of reshares comparable to those of reliable sources. Alarmingly, the most prominent source of vaccine misinformation, CHD, earned more than twice the number of tweets as those linking to the CDC. Looking at users who earned the most retweets when sharing low-credibility news about vaccines, we observed the presence of many verified accounts. In particular, the verified user who earned the most retweets was Robert Kennedy Jr, the founder of CHD.

Given the increase in misinformation over time and the outsized role of a small group of verified users, we hypothesize that financial incentives may play an important role [68,70]. Low-credibility websites monetize visitors through donations, advertising, and merchandise. Our finding that vaccine misinformation tended to spread more through retweets compared to mainstream news suggests that misinformation content lends itself to such exploitation. Amplification by automated accounts may also have played a role in increasing levels of misinformation, as we found these accounts to be significantly more active at sharing low-credibility news and inaccessible YouTube videos compared to vaccine-related content overall. However, we did not find a trend of increased levels of automated sharing over time.

There are a number of limitations to our study. The source-based approach to identify low-credibility information at scale is not perfect. Credible sources may occasionally report inaccuracies and low-credibility sources often publish a mixture of reliable and unreliable information. Our analysis based on a sample of articles suggests that approximately 76% of articles from low-credibility sources do contain false or misleading content.

While we cannot publicly disclose NewsGuard ratings, they are available to researchers upon agreement, which should ensure reproducibility. We elected to include NewsGuard data because these data are more comprehensive and up-to-date, and the methodology is better documented compared to other ratings such as those from Iffy+ Misinfo/Disinfo. Nevertheless, it would be possible to repeat our analysis using only the free ratings from Iffy+ Misinfo/Disinfo since, as the literature suggests, the ratings are highly correlated [71,72]. In fact, we observed a high overlap between our lists of top sources; for example, 17 of the top 20 sources in Figure 4B are also present in the Iffy+ Misinfo/Disinfo list. More importantly, over 86% of the total number of low-credibility tweets identified with the merged list originate from websites contained in the Iffy+ Misinfo/Disinfo list alone. This suggests that the results are robust with respect to the ratings source.

Similar limitations exist with respect to labeling inaccessible YouTube videos as “low credibility.” For example, some of these videos may be inaccessible due to restricted access or copyright violations. An uploader’s choice to restrict access to a video may serve as a way to circumvent content moderation policies or could be unrelated to antivaccination efforts. However, in the context of the vaccination discussion on Twitter, examinations of videos and their Twitter posters suggest that most inaccessible videos are likely antivaccination in their orientation [63]. In addition, not all accessible videos contain accurate information about vaccines. YouTube may fail to identify content that should be removed according to its own policies. As such, analyses of inaccessible videos should be treated more as lower-bound estimates.

Another limitation is that the BotometerLite algorithm we employed to detect automated accounts is not perfect and may not accurately classify social bots [67]. We investigated whether bot-like behavior, as identified by BotometerLite, is associated with suspicious activity on Twitter. We used Twitter’s Compliance API [73] to check all accounts for suspension from the platform as of November 2022. We observed a significant positive correlation between the BotometerLite score, binned into 40 equal intervals, and the proportion of accounts suspended (Pearson r=0.93, P<.001). This suggests that the classifier reliably reveals behaviors that eventually lead to suspension on the platform.

Perhaps most importantly, Twitter users may not be very representative of the real-world population across a range of demographic groups [74], although information circulating around Twitter can have a great influence over the news media agenda [75]. Further studies should consider multiple social media platforms simultaneously, especially those with upward adoption trends [76].

Despite these limitations, our findings help map the landscape of online vaccine misinformation and can guide the design of intervention strategies to curb its spread. The presence of misinformation around COVID-19 vaccines on Twitter shows that there was an audience for this type of content, which might reflect a deeper distrust of medicine, health professionals, and science [77]. In a context of widespread uncertainty such as the COVID-19 pandemic, trust is critical for overcoming vaccine hesitancy, and recent research shows how online misinformation fueled vaccine hesitancy and refusal sentiment [25,34].

Our findings reveal the presence of a small number of main producers and repeat spreaders of low-credibility content. Given that these superspreaders played key roles in disseminating vaccine misinformation, a straightforward strategy could be to deplatform them [78,79], as shown by recent studies in different contexts [79-81] and as has been done by major platforms in notable cases such as Alex Jones [82] and Donald Trump [83].

While social media platforms have legal rights to regulate online conversations, the decisions to deplatform public figures should be made with caution. In fact, past interventions have sparked a vivid debate around free speech and caused many users to migrate to alternative platforms [79,81]. It is also unclear whether reducing the supply of false information and increasing the supply of accurate information can “cure” the problem of vaccine hesitancy [32]. An alternative path of action could be to reduce the financial incentives of those who profit from the spread of misinformation. Our results also show that vaccine misinformation is more viral than other kinds of information. Other effective approaches to reduce its spread include lowering the visibility of certain content (“down-ranking”) or not showing that content to users (“shadow banning”), as well as adding warning labels to content that is potentially harmful or inaccurate [84,85]. Platforms should partner with policymakers and researchers in evaluating the impacts of such different interventions [86].

There are several interesting RQs that are outside the scope of the present work, but that could be addressed by future research. For instance, further investigations could address the impact of Twitter’s removal of users due to the January 6 riots on the spread of misinformation in the following months. Other studies could investigate how the CHD organization shifted its antivaccination narratives from children to a broader COVID-19 vaccination campaign and remained the most popular source for the antivaccination movement. Future work could also analyze exposure to low-credibility information, which is more difficult to measure compared to the sharing patterns quantified in this study. This would allow answering the question of whether the spread of low-credibility information was confined to a limited group of people or reached a wide audience. Finally, it is still unclear how governmental and societal changes might have affected conversations around vaccines during the COVID-19 pandemic compared to the (anti)vaccination debate in previous years.

All in all, we believe our work provides actionable insights for addressing the online spread of vaccine misinformation. Such insights can be beneficial during the ongoing pandemic and future health crises.

Abbreviations

API

application programming interface

CDC

Centers for Disease Control and Prevention

CHD

Children's Health Defense

MBFC

Media Bias/Fact Check

RQ

research question