Abstract
Background
Global efforts toward the development and deployment of a vaccine for COVID-19 are rapidly advancing. To achieve herd immunity, widespread administration of vaccines is required, which necessitates significant cooperation from the general public. As such, it is crucial that governments and public health agencies understand public sentiments toward vaccines, which can help guide educational campaigns and other targeted policy interventions.
Objective
The aim of this study was to develop and apply an artificial intelligence–based approach to analyze public sentiments on social media in the United Kingdom and the United States toward COVID-19 vaccines to better understand the public attitude and concerns regarding COVID-19 vaccines.
Methods
Over 300,000 social media posts related to COVID-19 vaccines were extracted, including 23,571 Facebook posts from the United Kingdom and 144,864 from the United States, along with 40,268 tweets from the United Kingdom and 98,385 from the United States from March 1 to November 22, 2020. We used natural language processing and deep learning–based techniques to predict average sentiments, sentiment trends, and topics of discussion. These factors were analyzed longitudinally and geospatially, and manual reading of randomly selected posts on points of interest helped identify underlying themes and validated insights from the analysis.
Results
Overall averaged positive, negative, and neutral sentiments were at 58%, 22%, and 17% in the United Kingdom, compared to 56%, 24%, and 18% in the United States, respectively. Public optimism over vaccine development, effectiveness, and trials as well as concerns over their safety, economic viability, and corporation control were identified. We compared our findings to those of nationwide surveys in both countries and found them to correlate broadly.
Conclusions
Artificial intelligence–enabled social media analysis should be considered for adoption by institutions and governments alongside surveys and other conventional methods of assessing public attitude. Such analyses could enable real-time assessment, at scale, of public confidence and trust in COVID-19 vaccines, help address the concerns of vaccine sceptics, and help develop more effective policies and communication strategies to maximize uptake.
Keywords: artificial intelligence, COVID-19, deep learning, Facebook, health informatics, natural language processing, public health, sentiment analysis, social media, Twitter, infodemiology, vaccination
Introduction
The imminent availability of COVID-19 vaccines poses a pressing need to continually monitor and better understand public sentiments in order to develop baseline levels of confidence in them among the general public and enable the identification of early warning signals of loss in confidence [1]. This will help address the concerns of vaccine sceptics [2-4] and develop the required public trust in immunization [5,6] to realize the goal of generating herd immunity [7].
Traditionally, governments use surveys to understand public attitude; however, these typically have limitations including small sample sizes, closed questions, and limited spatiotemporal granularity. In order to overcome these limitations, we argue that social media data can be used to obtain more, real-time insights into public sentiments and attitudes with considerable spatiotemporal granularity. Over half of the worldwide population, including approximately 70% the populations of the United Kingdom and the United States, are active social media users, and social media usage has significantly increased during the pandemic; for instance, Facebook usage increased by 37%. Since social media data are largely unstructured, they are amenable to the application of established artificial intelligence (AI) techniques such as machine learning, deep learning (DL) [8], and natural language processing (NLP) [9] to extract topics and sentiments from social media posts.
Sentiment analysis involves categorizing subjective opinions from text, audio, and video sources [9] to determine polarities (eg, positive, negative, and neutral), emotions (eg, anger, sadness, and happiness), or states of mind (eg, interest vs disinterest) toward target topics, themes, or aspects of interest [10]. A complementary approach, termed stance detection [11], assigns a stance label (favorable, against, and none) to a post on a specific predetermined target, which in itself may not be referred to or be the target of opinion in the post. Such approaches are currently underutilized in health care research. In particular, there is significant untapped potential in drawing on AI-enabled social media analysis to inform public policy research.
Methods
Ethics
Since the data analyzed in this study were completely in the public domain, no ethics review was necessary. We conducted a thorough assessment of the privacy risk that our study posed to individuals, in accordance with previous reports [12,13], to ensure compliance with relevant sections of the General Data Protection Regulation. We strived to comply with best practices for user protection [14,15], ensuring that nonpublic material is not included in our data set. Further, to comply with privacy laws and social network policies in accordance with the General Data Protection Regulation to collect data from Twitter [16] and Facebook CrowdTangle [17] platforms, we have not shared or published direct tweets or posts by individuals, quotes from individuals, or names or locations of users who are not public organizations or entities.
Data Sources
We used data from both Facebook and Twitter, two of the most popular and representative social media platforms [16]. We used Facebook posts and tweets that were posted in English in the United Kingdom and the United States from March 1 to November 22, 2020. Facebook posts were obtained through the CrowdTangle platform [17] and Twitter posts from a publicly available Twitter API. We used hydrated tweets from the global COVID-19 data set [18], which collects up to 4,400,000 tweets per day (including retweets) and up to 1,100,000 cleaned tweets without retweets. The total number of tweets hydrated and used in this study was >158,000,000. Facebook posts and tweets were thematically filtered for both COVID-19– and vaccine-related keywords and then geographically filtered for the United Kingdom and the United States. The first step in filtering COVID-19–related keywords involved widely used terms from the data set of Banda et al [18] (Multimedia Appendix 1). Vaccine-related terms used for second step filtering were selected by our team: “vaccine,” “vaccination,” “immunise,” “immunize,” “immunisation,” and “immunization.” A 2-step thematic filtering process was applied using these keywords before processing and analysis.
Analysis
The filtered data set was initially preprocessed (eg, removing links, hashtags, and stop words) and a new hierarchical hybrid ensemble–based AI model was developed for thematic sentiment analysis. This utilized an average weighting ensemble [19] of 2 lexicon-based methods: Valence Aware Dictionary for Sentiment Reasoning (VADER) [20] and TextBlob [21]. These were combined with a pretrained DL-based model, Bidirectional Encoder Representations from Transformers (BERT) [22], using a rule-based ensemble method (Figure 1).
Figure 1.
Hierarchical hybrid-ensemble–based artificial intelligence model and data pipeline for thematic sentiment analysis. BERT: Bidirectional Encoder Representations from Transformers, VADER: Valence Aware Dictionary for Sentiment Reasoning.
A random 10% sample of Facebook posts and tweets was then manually annotated by the team and screened against our hybrid ensemble AI model’s sentiment classifications for refinement and validation. The hybrid ensemble model was optimized on the basis of the validation results, with sensitivity and specificity analysis revealing that the lexicon-based methods provided generally better accuracy for positive sentiments, and the BERT model generally provided better accuracy for neutral and negative sentiments, as illustrated using normalized confusion matrices (Multimedia Appendix 1).
VADER and TextBlob were combined though weighted averaging (VADERx0.45 + TextBlobx0.55), with TextBlob assigned a marginally higher weight of 0.55 owing to its performance. The weighted averaged output from the lexicons was combined with the output of the BERT model, using a final rule–based ensemble. The final output was combined through “If” and “Else” statements on the basis of the model’s output sentiments.
A number of established NLP techniques were used to analyze the processed data (Multimedia Appendix 1). Specifically, in addition to analyzing averaged sentiment trends and their geospatial mappings in the United Kingdom and the United States, we statistically analyzed the trends with Pearson correlation coefficient (r) and compared the findings with those of independent surveys. Sentiment word cloud and N-gram analyses were performed for specific periods of interest, around points of inflexion on sentiment trend graphs, to identify topics of discussion and to gain insight into the positive and negative content of online discourses. The analysis was also carried out over the entire study period to identify underlying themes and topics. Findings were validated, and further insights were obtained through manual reading of randomly selected posts around target points of interest by our team. Relevant social media data sets and outputs were anonymized, and statistical aggregates were made openly accessible for transparency and reproducibility (additionally through a publicly available dashboard [23]).
Results
Temporal Sentiment Trends
Monthly volume trends of the filtered Facebook posts and tweets in the United Kingdom and the United States for the target study period are shown in Multimedia Appendix 1. Figure 2 shows the averaged (weekly) positive, negative, and neutral Facebook sentiments in March-November 2020 in the United Kingdom and the United States. We identified topics of discussion on points of interest in the graphs. These are referred to in our descriptive analysis of the graphs below, and some are highlighted in Figures 2 and 3. It was interesting to note that the difference between the averaged positive and negative sentiment trends was more pronounced on Facebook than on Twitter.
Figure 2.
Averaged weekly trends in Facebook sentiments for (A) the United Kingdom and (B) the United States.
Figure 3.
Average weekly trends in Twitter sentiments for (A) the United Kingdom and (B) the United States.
For the United Kingdom, positive sentiments on Facebook displayed the most prominent trend (Figure 2A), showing a steady increasing trend since May 2020, corresponding to the initiation of vaccine trials and the recruitment of the first trial volunteer. We observed a peak in mid-August 2020, which was potentially associated with news on vaccine development in the United Kingdom and Russia. Negative sentiments on Facebook displayed an inverse trend to that of positive sentiments, and discourse was centered around vaccine conspiracies and halting of trials. For the United States, positive sentiments on Facebook displayed the most prominent trend, showing a small peak in August 2020, which was associated with posts relating to research on the COVID-19 vaccine in Russia. Moreover, negative sentiments on Facebook displayed a slight increase in mid-September 2020, which was associated with posts relating to the accelerated development of the COVID-19 vaccine (Figure 2B). More recently, from mid-October 2020, the trend of positive sentiments on Facebook in the United Kingdom and the United states increased, partly because of announcements from Pfizer Inc and Moderna Inc on successful vaccine trials [24].
Figure 3 illustrates the positive, negative, and neutral sentiments on Twitter from March to November 2020 for the United Kingdom and the United States. Figure 3A shows that in the United Kingdom, positive sentiments on Twitter displayed the most prominent trend, showing a small peak at the end of April and July of 2020, the former related to the first human vaccine trial. The negative sentiment trend on Twitter displayed peaks in July and October of 2020, simultaneously with the United Kingdom opting out of the European Union vaccination scheme and halting of the phase III vaccine trials at the University of Oxford owing to safety concerns [25]. Figure 3B shows that in the United States, positive sentiments on Twitter displayed the most prominent trend, showing major peaks from end-September to end-November of 2020, which was related to claims by ex-President Donald Trump regarding a vaccine being ready in a few weeks and an increase in Twitter discourse due to his reference to the “herd mentality.” We observed a small peak in the negative trend graph in mid-September 2020, which was related to halting of the phase III vaccine trial at the University of Oxford.
For both the United Kingdom and the United States, we observed a marked increase in the positive sentiment trend, since end-October 2020, which was related to recent breakthrough announcements by Pfizer Inc and Moderna Inc. Analysis of social media conversations indicated public optimism, with trial results being hailed as “good” and “amazing” and with “hope” prevailing for the “new year” (Multimedia Appendix 1). A notable peak in the negative sentiment trends for both countries, in approximately mid-October 2020, was associated with the growing antivaccination movement and with concerns regarding “fake news” and “misinformation.”
Statistical Analysis of Sentiment Trends
Statistical analysis (results detailed in Multimedia Appendix 1) involved the assessment of the strength of the association between the predicted sentiment in the trend graph and the accuracy of the labeled data. Overall, regarding COVID-19 vaccines, we observed stronger sentiments on Twitter for the United States, with both positive and negative sentiments displaying stronger increasing and decreasing trends, respectively, compared to the United Kingdom. Public sentiments on Facebook reflected a reduction in positive sentiments and an increase in neutral sentiments in both the United Kingdom and the United States, with positive sentiments displaying a slightly stronger decreasing trend in the United Kingdom than in the United States.
Sentiment Word Clouds and Text N-Gram Analysis
We performed sentiment word cloud and text N-gram analyses for the entire study period to identify and analyze notable events that were of interest to social media users, and the findings are summarized in Multimedia Appendix 1 (some of these were also identified in the aforementioned analysis, on the sentiment trend graphs).
Geospatial Sentiment Analysis
A geospatial map of overall (averaged) sentiments at the state level in the United States is shown in Figure 4 (left), and it indicates that most states had a negative sentiment. The states with an overall negative sentiment toward COVID-19 vaccines were concentrated in the West and Midwest regions, namely Idaho, Kansas, New Hampshire, West Virginia, and Alabama. The states with an overall positive sentiment were in the East, namely Maine, Colorado, Georgia, and Hawaii.
Figure 4.
Geospatial mapping of averaged social media public sentiments in the United States (left) and the United Kingdom (right) toward COVID-19 vaccines (1 or green: positive sentiments, 0: neutral sentiments, and -1 or red: negative sentiments).
A geospatial map of averaged sentiments toward COVID-19 vaccines at the county level in the United Kingdom is shown in Figure 4 (right). In contrast with the United States, most counties in the United Kingdom had an overall positive sentiment toward COVID-19 vaccines. The counties with the most positive sentiments included Cornwall, Kent, East Sussex, Surrey, and Dorset in England and Aberdeenshire, Angus, and Stirlingshire in Scotland. Furthermore, the counties with the most negative sentiments were West Sussex, Somerset, North Yorkshire, and Durham in England.
Overall Averaged Sentiments
Overall averaged sentiments in the United Kingdom and the United States on Facebook and Twitter are shown in Figure 5 and in Multimedia Appendix 1.
Figure 5.
Overall averaged sentiments: (A) Twitter sentiments in the United Kingdom, (B) Twitter sentiments in the United States, (C) Facebook sentiments in the United Kingdom, and (D) Facebook sentiments in the United States.
Discussion
Principal Findings
We analyzed temporal variations in public sentiments toward COVID-19 vaccines in the United Kingdom and the United States. We identified, evaluated, and mapped the key events impacting positive, negative, and neutral sentiments to the temporal trends. We also mapped spatial variations in public sentiment to regions in the United Kingdom and states in the United States. Our geospatial maps can help identify areas with more negative sentiments toward COVID-19 vaccines, which can be further studied for potential interventions, to allay the underlying public fears and concerns.
Our findings indicate that online public discourse on Facebook and Twitter across the United Kingdom and the United States is evolving, with both complementary and contrasting insights obtained from the 2 popular platforms. Comparative analysis revealed that over the 9-month study period, averaged public sentiment toward COVID-19 vaccines has been mostly positive and similar in both the United Kingdom and the United States across both platforms (57.70% average across both platforms for the United Kingdom vs 56.80% for the United States). Positive sentiments were related to public opinions on vaccine development, related trials, and news related to vaccine availability.
On both platforms, overall averaged negative sentiments were found to be similar for the United Kingdom (22.50%) and the United States (24.10%). It is interesting to note that Twitter sentiments appeared more negatively biased, with the proportion of negative sentiments being almost 2-fold those on Facebook, for both the United Kingdom (27.95% vs 17.04%) and the United States (30.57% vs 17.73%), which potentially reflects their respective user demographics. This finding appears consistent with those of Waterloo et al [16], who reported that public opinions were often more negatively biased on Twitter than on Facebook, with public opinions being more positively biased on Facebook. Negative sentiments in our study were related to public apprehensions and concerns regarding delays or pauses in vaccine trials, vaccine safety, corporations, and governments influencing vaccine availability and rights exclusivity for economic benefits.
A comparative analysis with independent surveys was carried out. Our findings related to trends of averaged positive and negative sentiment across the United Kingdom and the United States were found to correlate broadly. In the United States, during the early stages of the pandemic, polling indicated that a significant minority had low trust in a vaccine; for example, a Yahoo News/YouGov survey in May 2020 [26] reported that only 55% of people in the United States intended to get vaccinated against COVID-19, while almost 1 in 5 (19%) individuals would not get vaccinated. A similar survey in July 2020 [27] reported that 42% of people in the United States would get vaccinated (27% would not get vaccinated), while a survey in September 2020 [28] reported that only 36% of people in the United States were certain they would get vaccinated (32% would not get vaccinated). More recently, an Axios-Ipsos survey in November 2020 reported, consistent with the findings of our social media analysis, a marked increase in the proportion of individuals who are likely to get vaccinated (51% were “very” or “somewhat” likely to take the first-generation vaccine; this proportion would increase to 70% if the vaccine was proven safe and effective by public health officials) [29].
In the United Kingdom, a YouGov survey in June 2020 [30] reported that 41% of respondents would “probably” or “definitely” get vaccinated, while 1 in 6 (16%) respondents would “definitely” or “probably” not get vaccinated. The survey also reported that individuals who used social media more than traditional media as their source of news were 9% less likely to be in favor of being vaccinated. A more recent YouGov survey in the United Kingdom in November 2020 [31], related to the Pfizer COVID-19 vaccine, reported that 67% of individuals were “very” or “fairly” likely to take the vaccine when available, and approximately 1 in 5 (21%) individuals were unlikely to take it. While there has been a slight increase in the proportion of individuals unlikely to take the vaccine, the proportion of those likely to get vaccinated has increased, indicating a reduction in the number of individuals who were previously unsure. This could be attributed in part to the recent announcements by vaccine manufacturers, and our results corroborate this finding with a marked increase in positive sentiments since mid-October 2020, in both the United Kingdom and the United States, across the 2 social media platforms. Further studies on changes in sentiments could further the current understanding of factors that have contributed to this, with particular focus on the impact of government education programs.
Limitations
It is important to consider the limitations of our data sources and techniques and the related challenges and opportunities they present for future research. While we attempted to gauge nationwide public sentiments in the United Kingdom and the United States by analyzing posts in English on both Facebook and Twitter, our data may not be representative of the broader population of both countries. Users are known to differ in their usage and preferences regarding social media platforms on the basis of their sociodemographics (eg, age, socioeconomic status, and political affiliation). Vaccines are likely to be preferentially targeted at older populations and possibly ethnic minorities, communities with historically lower rates of vaccine uptake [32,33]. Further exploration is therefore imperative to increase our understanding of the public perception toward vaccines and their underlying behavioral determinants [34]. Social network analysis [35,36] can be performed in conjunction with DL methods to effectively identify sources of fake news or misinformation and their social networks to help deal with infodemic challenges [37,38]. Demographic data including age, gender, race, and geographic origin can also be inferred from social media profiles of users by using AI techniques [39]. This can help categorize distinct groups and inform the development of demographic-level engagement and tailored communication strategies to promote diversity and inclusion in vaccination campaigns. These can also effectively account for the fact that there are genuine knowledge voids being filled by misinformation [34].
The technical limitations of our approach include challenges in determining the geographic location of users and issues relating to the accuracy of the AI techniques (eg, interpreting sarcasm and implicit context). Alternative deep neural networks [40-42] and fuzzy-based approaches [43,44] can be explored and potentially integrated as part of our ensemble model in an attempt to further refine the present findings. The 2-step keyword-based thematic filtering process and the use of geotagged posts in this study resulted in relatively small sample sizes. This could be improved by using more sensitive filtering and data-driven search mechanisms, network metadata (such as likes and retweets), and additional social media and web-based platforms. On account of the current study limitations, our approach should only be used in conjunction with other techniques for understanding public sentiments, such as focus groups, input from civil society organizations, surveys, and public consultations.
Future studies could consider conducting periodic public surveys over the period of interest being explored through social media analysis. This would ensure that both methodologies were informed by each other over the course of the study to enable more granular spatiotemporal analysis, thus allowing more robust comparisons from reciprocal findings and deeper insights for policymakers. These could also complement other qualitative methods, such as in-depth interviews and ethnographic studies, as part of mixed-study approaches. Manual annotation or labeling of datasets is imperative when training AI models for NLP tasks to ensure accuracy and generalizability. These can be affected by the skill of annotators and the proportion of the data set that is labeled. Confounding factors, such as political affiliations, should also be included in future studies, by applying further filters to screen strategies and through targeted demographic analysis, to further the current understanding of the underlying determinants of public sentiments. Attitudes toward different vaccine manufacturers could also be explored to identify and assess effective public engagement strategies to build support for ethical principles and maximize the uptake of the imminently available vaccines.
Conclusions
One of the main threats to the resilience of vaccination programs globally is the rapid and global spread of misinformation. Public confidence in COVID-19 vaccines can be exacerbated by unproven concerns regarding vaccine safety, which seed doubt and distrust. Furthermore, there have been cases where vaccine debates have been purposefully polarized, thus exploiting the doubting public and system weaknesses for political purposes, while waning vaccine confidence elsewhere may be influenced by a general distrust in the government and scientific elites. Recent surveys and polls in the United Kingdom and the United States have indicated the fragility of support for vaccination, which furthers the requirement for a better understanding of underlying public concerns and attitudes, both at scale and in real time. Retrospective analysis of 2 popular and most representative social media platforms in this study demonstrates the potential of AI-enabled real-time social media monitoring of public sentiments and attitudes to help detect and prevent such fears and also to enable policymakers to understand the reasons why some social groups may be reluctant to be vaccinated against COVID-19. This can inform more effective policy-making and promote participatory dialogue on complex vaccine deployment issues, under conditions of uncertainty, including decisions on prioritization and equitability, to help maximize the uptake of imminently available vaccines.
Acknowledgments
This study was funded by the Chief Scientist Office of the Scottish Government under its COVID-19 priority research program (COV/NAP/20/07) and is jointly led by co-Principal Investigators AH and AS. AS was supported by BREATHE - The Health Data Research Hub for Respiratory Health [MC_PC_19004], which is funded by the UK Research and Innovation Industrial Strategy Challenge Fund and delivered through Health Data Research UK. AH was supported by the UK Government’s Engineering and Physical Sciences Research Council grants (EP/T021063/1, EP/T024917/1). This study does not represent the views of the Scottish or UK Governments.
Abbreviations
- AI
artificial intelligence
- BERT
Bidirectional Encoder Representations from Transformers
- DL
deep learning
- NLP
natural language processing
- VADER
Valence Aware Dictionary for Sentiment Reasoning
Appendix
Supplementary Material as quoted in the main article.
Footnotes
Conflicts of Interest: AS is a member of the Chief Medical Officer’s COVID-19 Advisory Group of the Scottish Government and the UK Government’s New and Emerging Respiratory Virus Threats Risk Stratification Subgroup.
References
- 1.de Figueiredo A, Simas C, Karafillakis E, Paterson P, Larson HJ. Mapping global trends in vaccine confidence and investigating barriers to vaccine uptake: a large-scale retrospective temporal modelling study. Lancet. 2020 Sep 26;396(10255):898–908. doi: 10.1016/S0140-6736(20)31558-0. https://linkinghub.elsevier.com/retrieve/pii/S0140-6736(20)31558-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gellin B. Why vaccine rumours stick—and getting them unstuck. The Lancet. 2020 Aug;396(10247):303–304. doi: 10.1016/s0140-6736(20)31640-8. [DOI] [Google Scholar]
- 3.Horder J. Toll of vaccine hesitancy. Nat Hum Behav. 2020 Apr;4(4):335. doi: 10.1038/s41562-020-0874-1. [DOI] [PubMed] [Google Scholar]
- 4.Ten threats to global health in 2019. World Health Organization. [2021-10-26]. https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019,
- 5.Lazarus J, Ratzan SC, Palayew A, Gostin LO, Larson HJ, Rabin K, Kimball S, El-Mohandes A. A global survey of potential acceptance of a COVID-19 vaccine. Nat Med. 2021 Feb;27(2):225–228. doi: 10.1038/s41591-020-1124-9. http://europepmc.org/abstract/MED/33082575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bloom BR, Nowak GJ, Orenstein W. "When Will Have a Vaccine?" - Understanding Questions and Answers about Covid-19 Vaccination. N Engl J Med. 2020 Dec 03;383(23):2202–2204. doi: 10.1056/NEJMp2025331. [DOI] [PubMed] [Google Scholar]
- 7.Fontanet A, Cauchemez S. COVID-19 herd immunity: where are we? Nat Rev Immunol. 2020 Oct;20(10):583–584. doi: 10.1038/s41577-020-00451-5. http://europepmc.org/abstract/MED/32908300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Perez S. Report: WhatsApp has seen a 40% increase in usage due to COVID-19 pandemic. TechCrunch. 2020. Mar 26, [2021-03-17]. https://techcrunch.com/2020/03/26/report-whatsapp-has-seen-a-40-increase-in-usage-due-to-covid-19-pandemic/
- 9.Adeel A, Gogate M, Hussain A. Contextual deep learning-based audio-visual switching for speech enhancement in real-world environments. Information Fusion. 2020 Jul;59:163–170. doi: 10.1016/j.inffus.2019.08.008. [DOI] [Google Scholar]
- 10.Dashtipour K, Gogate M, Li J, Jiang F, Kong B, Hussain A. A hybrid Persian sentiment analysis framework: Integrating dependency grammar based rules and deep neural networks. Neurocomputing. 2020 Mar;380:1–10. doi: 10.1016/j.neucom.2019.10.009. [DOI] [Google Scholar]
- 11.Majumder N, Bhardwaj R, Poria S, Gelbukh A, Hussain A. Improving aspect-level sentiment analysis with aspect extraction. Neural Comput & Applic. 2020 Aug 17;:1–11. doi: 10.1007/s00521-020-05287-7. doi: 10.1007/s00521-020-05287-7. [DOI] [Google Scholar]
- 12.Al-Ghadir AI, Azmi AM, Hussain A. A novel approach to stance detection in social media tweets by fusing ranked lists and sentiments. Information Fusion. 2021 Mar;67:29–40. doi: 10.1016/j.inffus.2020.10.003. [DOI] [Google Scholar]
- 13.Zimmer M. “But the data is already public”: on the ethics of research in Facebook. Ethics Inf Technol. 2010 Jun 4;12(4):313–325. doi: 10.1007/s10676-010-9227-5. [DOI] [Google Scholar]
- 14.Williams ML, Burnap P, Sloan L. Towards an Ethical Framework for Publishing Twitter Data in Social Research: Taking into Account Users' Views, Online Context and Algorithmic Estimation. Sociology. 2017 Dec;51(6):1149–1168. doi: 10.1177/0038038517708140. https://journals.sagepub.com/doi/10.1177/0038038517708140?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Franzke AS, Bechmann A, Zimmer M, Ess CM. Internet Research: Ethical Guidelines 3.0. Association of Internet Researchers. 2020. [2021-03-17]. https://aoir.org/reports/ethics3.pdf.
- 16.ESRC Research Data Policy. Economic and Social Research Council. [2021-03-17]. https://esrc.ukri.org/files/about-us/policies-and-standards/esrc-research-data-policy/
- 17.Waterloo SF, Baumgartner SE, Peter J, Valkenburg PM. Norms of online expressions of emotion: Comparing Facebook, Twitter, Instagram, and WhatsApp. New Media Soc. 2018 May;20(5):1813–1831. doi: 10.1177/1461444817707349. https://journals.sagepub.com/doi/10.1177/1461444817707349?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Garmur M, King G, Mukerjee Z, Persily N, Silverman B. CrowdTangle Platform and API. Harvard Dataverse. 2019. [2021-03-17]. [DOI]
- 19.Banda J, Tekumalla R, Wang G, Yu J, Liu T, Ding Y, Chowell G. A large-scale COVID-19 Twitter chatter dataset for open scientific research -- an international collaboration. arXiv. . doi: 10.3390/epidemiologia2030024. Preprint posted online November 13, 2020 https://arxiv.org/abs/2004.03688. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Yoo BH, Kim J, Lee B, Hoogenboom G, Kim KS. A surrogate weighted mean ensemble method to reduce the uncertainty at a regional scale for the calculation of potential evapotranspiration. Sci Rep. 2020 Jan 21;10(1):870. doi: 10.1038/s41598-020-57466-0. doi: 10.1038/s41598-020-57466-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hutto CJ, Gilbert E. VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Proceedings of the International AAAI Conference on Web and Social Media; The 8th International Conference on Weblogs and Social Media; 2014; Ann Arbor, MI. 2014. http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf, [Google Scholar]
- 22.Loria S, Keen P, Honnibal M, Yankovsky R, Karesh D, Dempsey E. TextBlob: simplified text processing. Secondary TextBlob: simplified text processing. Textblob. [2021-03-17]. https://textblob.readthedocs.io/en/dev/#:~:text=TextBlob%20is%20a%20Python%20(2,classification%2C%20translation%2C%20and%20more.
- 23.Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv. . Preprint posted online May 24, 2019 https://arxiv.org/abs/1810.04805. [Google Scholar]
- 24.COVID-19 AI-powered dashboard for sentiment and opinion mining in social media platforms. Edinburgh Napier University. [2021-03-17]. http://covidtracker.cloud.
- 25.Doshi P. Peter Doshi: Pfizer and Moderna’s “95% effective” vaccines—let’s be cautious and first see the full data. BMJ Opinion. 2020. Nov 26, [2021-03-17]. https://blogs.bmj.com/bmj/2020/11/26/peter-doshi-pfizer-and-modernas-95-effective-vaccines-lets-be-cautious-and-first-see-the-full-data/
- 26.Mahase E. Covid-19: Oxford researchers halt vaccine trial while adverse reaction is investigated. BMJ. 2020 Sep 09;370:m3525. doi: 10.1136/bmj.m3525. [DOI] [PubMed] [Google Scholar]
- 27.Sanders L. Who will get a COVID-19 vaccine when it arrives. YouGov. 2020. May 11, [2021-03-17]. https://today.yougov.com/topics/health/articles-reports/2020/05/11/who-will-get-covid-vaccine.
- 28.Bruce G. Fewer Americans say they’ll get vaccinated for COVID-19. YouGov. 2020. Aug 05, [2021-03-17]. https://today.yougov.com/topics/politics/articles-reports/2020/08/05/vaccinated-covid-19.
- 29.Jaimungal C. Concerns over fast-tracked COVID-19 vaccine has Americans unsure about vaccination. YouGov. 2020. Sep 15, [2021-03-17]. https://today.yougov.com/topics/health/articles-reports/2020/09/15/concerns-over-fast-tracked-covid-19-vaccine-has-am.
- 30.Jackson C, Newall M, Yi J. COVID-19 one year later: American reemergence? Ipsos. 2021. Mar 09, [2021-03-17]. https://www.ipsos.com/en-us/news-polls/axios-ipsos-coronavirus-index.
- 31.YouGov/CCDH Survey Results. YouGov. [2021-03-17]. https://docs.cdn.yougov.com/5mkju0kxbj/CCDH_RESULTS_ 062620_PR%20%28002%29.pdf.
- 32.McDonnell A. How many Britons are willing to take a coronavirus vaccine? YouGov. 2020. Nov 16, [2021-03-17]. https://yougov.co.uk/topics/health/articles-reports/2020/11/16/how-many-britons-are-willing-take-coronavirus-vacc.
- 33.Forster AS, Rockliffe L, Chorley AJ, Marlow LAV, Bedford H, Smith SG, Waller J. Ethnicity-specific factors influencing childhood immunisation decisions among Black and Asian Minority Ethnic groups in the UK: a systematic review of qualitative research. J Epidemiol Community Health. 2017 Jun;71(6):544–549. doi: 10.1136/jech-2016-207366. http://jech.bmj.com/lookup/pmidlookup?view=long&pmid=27531844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Malik AA, McFadden SM, Elharake J, Omer SB. Determinants of COVID-19 vaccine acceptance in the US. EClinicalMedicine. 2020 Sep;26:100495. doi: 10.1016/j.eclinm.2020.100495. https://linkinghub.elsevier.com/retrieve/pii/100495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.COVID-19 vaccine deployment: Behaviour, ethics, misinformation and policy strategies. The Royal Society. 2020. Oct 21, [2021-03-17]. https://royalsociety.org/-/media/policy/projects/set-c/set-c-vaccine-deployment.pdf.
- 36.Block P, Hoffman M, Raabe IJ, Dowd JB, Rahal C, Kashyap R, Mills MC. Social network-based distancing strategies to flatten the COVID-19 curve in a post-lockdown world. Nat Hum Behav. 2020 Jun;4(6):588–596. doi: 10.1038/s41562-020-0898-6. [DOI] [PubMed] [Google Scholar]
- 37.Hung M, Lauren E, Hon ES, Birmingham WC, Xu J, Su S, Hon SD, Park J, Dang P, Lipsky MS. Social Network Analysis of COVID-19 Sentiments: Application of Artificial Intelligence. J Med Internet Res. 2020 Aug 18;22(8):e22590. doi: 10.2196/22590. https://www.jmir.org/2020/8/e22590/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Gallotti R, Valle F, Castaldo N, Sacco P, De Domenico M. Assessing the risks of 'infodemics' in response to COVID-19 epidemics. Nat Hum Behav. 2020 Dec;4(12):1285–1293. doi: 10.1038/s41562-020-00994-6. [DOI] [PubMed] [Google Scholar]
- 39.Abd-Alrazaq A, Alhuwail D, Househ M, Hamdi M, Shah Z. Top Concerns of Tweeters During the COVID-19 Pandemic: Infoveillance Study. J Med Internet Res. 2020 Apr 21;22(4):e19016. doi: 10.2196/19016. https://www.jmir.org/2020/4/e19016/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Brandt J, Buckingham K, Buntain C, Anderson W, Ray S, Pool J, Ferrari N. Identifying social media user demographics and topic diversity with computational social science: a case study of a major international policy forum. J Comput Soc Sc. 2020 Jan 07;3(1):167–188. doi: 10.1007/s42001-019-00061-9. [DOI] [Google Scholar]
- 41.Ieracitano C, Pantó F, Mammone N, Paviglianiti A, Frontera P, Morabito FC. Toward an Automatic Classification of SEM Images of Nanomaterials via a Deep Learning Approach. In: Esposito A, Faundez-Zanuy M, Morabito FC, Pasero E, editors. Neural Approaches to Dynamics of Signal Exchanges. Singapore: Springer; 2019. Sep 19, pp. 61–72. [Google Scholar]
- 42.Mahmud M, Kaiser MS, Hussain A, Vassanelli S. Applications of Deep Learning and Reinforcement Learning to Biological Data. IEEE Trans Neural Netw Learn Syst. 2018 Jun;29(6):2063–2079. doi: 10.1109/TNNLS.2018.2790388. [DOI] [PubMed] [Google Scholar]
- 43.Zhang X, Zhou M, Liu H, Hussain A. A Cognitively Inspired System Architecture for the Mengshi Cognitive Vehicle. Cogn Comput. 2019 Nov 15;12(1):140–149. doi: 10.1007/s12559-019-09692-6. [DOI] [Google Scholar]
- 44.Ali A, Li J, Kanwal S, Yang G, Hussain A, Jane O'Shea S. A Novel Fuzzy Multilayer Perceptron (F-MLP) for the Detection of Irregularity in Skin Lesion Border Using Dermoscopic Images. Front Med (Lausanne) 2020;7:297. doi: 10.3389/fmed.2020.00297. doi: 10.3389/fmed.2020.00297. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Material as quoted in the main article.