Lexicon-based sentiment analysis to detect opinions and attitude towards COVID-19 vaccines on Twitter in Italy

Abstract

The paper proposes a methodology based on Natural Language Processing (NLP) and Sentiment Analysis (SA) to get insights into sentiments and opinions toward COVID-19 vaccination in Italy. The studied dataset consists of vaccine-related tweets published in Italy from January 2021 to February 2022. In the considered period, 353,217 tweets have been analyzed, obtained after filtering 1,602,940 tweets with the word “vaccin”. A main novelty of the approach is the categorization of opinion holders in four classes, Common users, Media, Medicine, Politics, obtained by applying NLP tools, enhanced with large-scale domain-specific lexicons, on the short bios published by users themselves. Feature-based sentiment analysis is enriched with an Italian sentiment lexicon containing polarized words, expressing semantic orientation, and intensive words which give cues to identify the tone of voice of each user category. The results of the analysis highlighted an overall negative sentiment along all the considered periods, especially for the Common users, and a different attitude of opinion holders towards specific important events, such as deaths after vaccination, occurring in some days of the examined 14 months.

1. Introduction

The World Health Organization (WHO) declared COVID-19 a pandemic on March 12th, 2020. With its diffusion, there has been a widespread and global use of online social media for sharing and disseminating news and updates on the pandemic, but also for social interactions, especially during closures and quarantines. As a result, a large amount of data about COVID-19 has spread and is still spreading in real time on social media, and its analysis can provide important clues about the perception and concerns of people about the pandemic.

As early as a few weeks after the appearance of COVID-19, it had become evident that vaccination was the only viable solution to combat the pandemic. Thanks to a global effort, it was possible within about a year to obtain approval from WHO for the immediate use of several types of vaccines, such as PziferBioNTech, AstraZeneca and Moderna. For a vaccination campaign of this magnitude to be successful, public support emerged as crucial. At the same time, understanding the sentiment and opinion of people about vaccines, and their willingness to be inoculated, has become a key and critically important step for being able to undertake appropriate strategies to improve confidence toward immunization tools.

In this sense, the role of social media is prominent both in reflecting public perceptions toward real-life topics and in exposing the users themselves to negative feelings and misinformation that can influence individuals to the point of leading them, as in this case, to either hesitate or refuse the vaccine [1]. Through monitoring vaccine-related conversations on social media, however, it is possible to identify which factors contribute to improved trust in them. The relevance of social media within such public discussion is also due to the increasing presence of health professionals and medical authorities on social media, with an active role in the conversations. Overall, social media have an important role in the real-time monitoring of exposure to infectious diseases and related vaccines and communication by global health actors, such as medical and governmental institutions.

For these reasons, the present study aims to assess the feelings and opinions of Italian Twitter users toward COVID-19 vaccination by analyzing posts collected from Twitter with the help of Sentiment Analysis (SA) methods to address the following research questions:

• 1.Which is the interest in the COVID-19 vaccine?
• 2.Which are and how do the emphasis and tone of the discussion on COVID-19 vaccination vary?
• 3.How controversial is and how does the discussion on the topic of COVID-19 vaccination vary?
• 4.Which is and how does the public perception, in terms of positive or negative feelings, on COVID-19 vaccination vary?
• 5.What are the main events that determine such feelings? Are there peculiar daily events in relation to specific months?

To answer these research questions, the paper proposes a methodology to assess public perceptions about vaccination and to gain insight into sentiments and opinions toward COVID-19 vaccination by analyzing a dataset consisting of vaccine-related tweets published in Italy from January 2021 to February 2022.

The proposed methodology exploits the Natural Language Processing (NLP) tool NooJ 1 to perform a quantitative and qualitative analysis of the data extracted from social media messages, and Sentiment Analysis techniques to identify, extract, quantify, and study affective states and subjective information on the basis of polarity (positive, negative or neutral), and intensity of agreement and disagreement using a numerical rating scale. The qualitative text analysis of the published posts uses an inductive approach to classify the authors of tweets in four main classes, Common users, Media, Medicine, and Politics, considered interesting for our analysis. Categorization is performed by using the metadata extracted from the user’s profiles, in particular by exploiting the short bios published by them.

Sentiment Analysis relies on the feature-based model proposed by [2] and it is based on the evaluation of co-occurrences of some noun phrases and sentiment words into evaluative sentence structure by exploiting an Italian sentiment lexicon [3], combined with Finite-State transducers for pattern annotation and syntactic analysis. The approach, which takes advantage of corpora-based lexicon analysis and dictionaries containing terms classified according to the sentiment they express, allows for scoring of the constituent entities of the text, then aggregating those scores until a view of the predominant sentiment in the text is provided.

It is worth pointing out that, such kind of analysis can be exploited in monitoring and understanding attitudes and feelings of people in case of the outbreak of phenomenons that affect public opinion, such as natural disasters, emergency situations, and even political, cultural, social events.

To the best of our knowledge, this is the first large-scale study that uses the posts published on the social network Twitter in Italy over a period of 14 months, with the following objectives:

• 1.automatically classify Twitter users into four categories, Common users, Media, Medicine, and Politics, by exploiting the biographies published and freely available, analyzed with respect to sets of keywords able to recognize linguistic patterns specific to each category;
• 2.explore public perceptions with the goal of identifying feelings and opinions toward the COVID-19 vaccination;
• 3.study the evolution of these sentiments over time starting from the beginning of the vaccination campaign until its completion, after the boosting dose;
• 4.provide a focus on specific events originated around the vaccination such as the start of the mass vaccination campaign (March 2021), the first deaths due to the use of AstraZeneca (March–June 2021), the discontinuation of AstraZeneca (March 15th, 2021), the availability of the green pass certificate (July–December 2021).

The remainder of the paper is organized as follows. In Section 2, a brief review of studies pertaining to social media and COVID-19 is given. In Section 3, details are provided on the proposed methodology and the tools we used. In Section 4 the evaluation measures adopted for assessing the analysis results are described. In Section 5, the results obtained are shown and discussed. In Section 6 the findings obtained by our analysis are compared with related similar works. Finally, Section 7, concludes the paper and provides suggestions for directing future studies.

2. Related works

With the fast spread of the COVID-19 epidemic, there has been an explosion of scientific research work focused on analyzing the correlation between social media and vaccines and how the use of the former as a source of information affected the perception of the latter by users. To this end, the most significant studies conducted in this context are given below, both from a global perspective and with regard to Italy in detail.

Global perspective.

Several initial studies took advantage of the wider availability of English-language tweets globally, so as to analyze general population sentiment. Among the first, Saleh et al. [4] collected tweets from February 1st to December 11th, 2020, during the vaccine development phase, correlating demographic information with SA techniques, finding a generally fluctuating positive trend depending on news events. Instead, Lyu et al. [5] sought to identify key topics and sentiments in influencing the goal of achieving herd immunity by collecting tweets between March 11th, 2020 and January 31st, 2021, leveraging techniques such as Latent Dirichlet Allocation for SA and the emotion lexicon of the National Research Council of Canada. Yousefinaghani et al. [6] analyzed tweets between January 2020 and January 2021, ranking the sentiment as positive, negative or neutral using the software VADER, and highlighting positive and negative sentiments in correspondence with the starting of the vaccination campaign and the increasing uncertainty about vaccines delivery. Similarly, Mahdikhani [7] collected tweets from January 20th 2020 to May 29th 2021 and examined them using different techniques, showing how higher emotional intensity corresponds to higher popularity regardless of information content.

Xie et al. [8] examined the public perception about COVID-19 vaccines on Twitter in the US from March 5th, 2020 to January 25th, 2021 employing topic modeling and SA techniques and the geographic and demographic characteristics (e.g. age, gender, ethnicity, geolocation) of users. The authors observed that sentiments remained fairly stable, with the exception of two positive and negative peaks regarding vaccine development and availability. Similarly, Hu et al. [9] studied the spatio-temporal trends of sentiment and emotions in the US from March 1st, 2020, to February 28th, 2021, at the national and state levels, identifying key events as drivers of change in perception and related emotions. In addition, Chen and Crooks [10] proposed an approach for spatio-temporal SA based on word embedding, exploiting tweets from the US from January 2015 to July 2021 and showing a correlation between positive sentiment and actual vaccinated population.

Hussain et al. [11] analyzed public sentiment on social media (Twitter and Facebook) in the UK and the US toward the vaccine between March 1st, 2020 and November 22nd, 2020, using SA techniques of NLP and deep learning, identifying a general public optimism toward vaccine development, efficacy and testing, as well as an inherent concern about safety, economic feasibility and social control. Furthermore, Lanyi et al. [12] employed NLP techniques to identify key barriers to vaccine adoption from a collection of tweets geolocated in London, UK, and collected between November 30th, 2020 and August 15th, 2021. Clustering of topics with negative sentiment revealed concerns about vaccine safety and distrust of government and pharmaceutical companies, as well as general misinformation.

Niu et al. [13] studied the change in sentiment toward vaccination in Japan by collecting tweets from August 1st, 2020 to June 30th, 2021, highlighting the correlation between changes in sentiment and daily events (e.g., infections, deaths, vaccinations) with the following criticisms of people.

Finally, some studies have tried to provide a cross-country comparison (including Italy): for instance, Greyling and Rossouw [14] exploited SA techniques applied to tweets to analyze the trend of positive attitudes toward vaccines in 10 countries, while Aygün et al. [15] leveraged deep learning techniques to classify tweets in English and Turkish language data sets through different language models (mBERT, BioBERT, ClinicalBERT, and BERTurk).

Alamoodi et al. [16] reviewed the studies proposed in the literature that analyze hesitancy cases to different vaccines by employing Sentiment Analysis techniques. The authors considered a period of 11 years, starting from January 2010 until July 2021, including COVID-19 vaccination. The review pointed out that vaccine hesitancy is influenced by false information appearing on social media about their side effects. Thus, understanding people’s sentiments can help authorities to provide appropriate public health messages to reduce mistrust toward vaccines.

Melton et al. [17] studied sentiment analysis of COVID-19 vaccines expressed on Reddit and Twitter. The authors collected data in the US in the period from January 1, 2020, to March 1, 2022. In the study, sentiment classification is computed by creating a custom fine-tuned version of BERT model, DistilRoBERTa, with data manually labeled and augmented by back-translation. Reported results show that the average sentiment expressed on Twitter was more negative (54.8%) than positive, and the sentiment expressed on Reddit was more positive (62.3%) than negative. Although the average sentiment was found to vary between these social media platforms, both platforms displayed similar behavior related to the sentiment shared at key vaccine-related developments during the pandemic.

Umair and Masciari [18] analyzed Twitter data about COVID-19 vaccines using artificial intelligence-based NLP and geo-spatial methods. In particular, BERT has been used to classify sentiment polarity. The approach also aimed at discovering the relationship of vaccine features geographically by applying hotspot analysis and kernel density estimation to highlight the regions of positive, negative or neutral sentiments.

Turón et al. [19] presented an analysis of the mood evolution in Spain regarding the COVID-19 vaccination. Tweets posted between February and December 2021 have been studied by combining social network analysis and sentiment analysis based on the lexicon. A network where nodes are the users and edges correspond to the retweets between users is built. Then two pro- and anti-vaccine groups are detected by applying a community detection method and their evolution is shown concerning four periods related to political decisions and relevant news. Interestingly, the sentiment polarity is influenced by relevant events like vaccine approval, problems regarding AstraZeneca, booster dose, and mandatory vaccination, showing a strict analogy to what happened in Italy, as will be clear in the following sections.

Focus on Italy.

Among the first to study the social response to the epidemic in Italy, there is the work of Fernandez et al. [20]. Specifically, that study aimed to compare sentiment in the three Italian macro-regions of north, center and south, collecting tweets during March–May 2020 from the regions of Lombardy, Lazio and Sicily, respectively. The authors used SA techniques in order to study the spatio-temporal distribution of the emotions of joy, fear (found to be prevalent in the period analyzed) and anger, correlating them with new cases and deaths, and showing an overall positive sentiment due to the frontline intervention of health workers. Unfortunately, the study refers to an early phase of the pandemic and does not allow for generalizing the main findings to later months.

Instead, the work of Gesualdo et al. [21], as part of the European Joint Action on Vaccination, covers a broader period, analyzing tweets posted in Italy during the period from November 2019 to June 2020. The aim was to understand variations in conversations about vaccines between the pre-pandemic and early-pandemic phases. A multidisciplinary team analyzed tweets from different perspectives: type and position on vaccines, tone of voice, target population, and source of information. From a prevalence of “discouraging” stances in the pre-pandemic period, there was a shift to a majority of “promotional” stances between February and March 2020, with either polemical or complacent tones in both cases. Overall, there has been a profound change in Italy, with a huge increase in promotional or ambiguous positions on the vaccine discourse. In addition, also as part of the European Joint Action on Vaccination, Cheatham et al. [22] proposed a transformer-based machine learning model for analyzing the position on vaccines in Italian tweets, exploiting both the previously mentioned dataset (November 2019–June 2020) and a second dataset covering the period April–September 2021. The compared models aimed to identify which category the tweets belonged to (promotional, neutral, and discouraging), identifying which had better performance, with results obtained between about 62% and 73% in terms of accuracy and $F_1$ score.

Bellodi et al. [23] applied two NLP tools, FEEL-IT and SentIta, to a small set of social media data about COVID-19 vaccine and the booster shot, between the end of 2021 and the beginning of 2022. FEEL-IT detects emotions in Italian text. For the sentiment analysis study, they used SentIta, similarly to what we have done. The authors find out some significant insights about the prevalent emotions among users and proposed to combine the outputs of the tools in order to increase the classification performance of an opinion according to three possible sentiments (positive/neutral/negative). As regards the specific emotion in the collected opinions, results show that anger is the most spread emotion. Anger is mostly due to political aversion to deeds and decrees issued by the Italian government, especially related to the booster shot and the so-called “green pass”. Following anger, authors find fear, due to mainly the fear of adverse events caused both by the first shot and the booster shot.

The aim of Stracqualursi and Agati [24] was to assess public opinion and perception on COVID-19 vaccines in Italy using 71,000 tweets containing vaccines-related keywords from Italian Twitter users, over the period February 1st to May 31st 2021. To determine the prevalent sentiment, spatial and temporal sentiment analysis was performed using VADER, and findings showed that sentiment fluctuations were highly influenced by news of vaccines’ side effects. Furthermore, the study has investigated the opinions of Italians with respect to different vaccine brands. As a result, the Oxford-AstraZeneca vaccine was the least appreciated among people. Finally, the study also aimed at identifying the most popular topics about COVID-19 vaccination. To this purpose, the Dynamic Latent Dirichlet Allocation (DLDA) model has been used to detect three main topics of discussion, which remained stable over time: vaccination plan info, the usefulness of vaccinating, and concerns about vaccines (risks, side effects, and safety).

The methodological approach proposed in this paper exhibits several key novelties and differences compared to the described state-of-the-art methods. They can be summarized as follows:

• 1.The study analyzes a dataset covering a large period, consisting of vaccine-related tweets published in Italy from January 2021 to February 2022. Conversely, most of the related approaches focus on a very short time period. In particular, in the majority of the cases, the analyzed data is relative to the year 2020 (or at most the early months of 2021) when the vaccination campaign was just started. Therefore, the obtained findings are not representative of the whole vaccination campaign and, thus, do not reflect people’s perceptions and feelings in correspondence with key and representative events and happenings that characterized the COVID-19 vaccination campaign. On the opposite, this work studied the evolution of these sentiments over time, from the two months of January and February 2021 before the introduction of the mass vaccination to the complete implementation in Italy during the different phases of the pandemic (e.g., the first dose, second dose, booster).
• 2.The present study aims to assess the feelings and opinions of Italian Twitter users toward COVID-19 vaccination with specific reference to four different stakeholders, namely medical experts, politicians, media and news organizations, and Common users, and with respect to the analyzed period. The work described in this paper is the first study focusing on a specific analysis of different stakeholders. We performed a detailed quantitative and qualitative analysis of the corpus of tweets, which provided us with an in-depth understanding of the discourse on vaccines on Twitter in Italy and of the actors involved in it.
• 3.The study provides a focus on some very specific events that originated around the vaccination campaign: such as the start of the vaccination campaign (March 2021), the first deaths due to the use of AstraZeneca (March–June 2021), the discontinuation of AstraZeneca (March 15th, 2021), the second dose (April 2021), the starting of the vaccination for kids (December 2021), the availability of the green pass certificate (July–December 2021).
• 4.To the best of our knowledge, this is the first study performed in Italy on social media to assess the people’s perception towards the whole COVID-19 vaccination campaign.

3. Materials and methods

This section introduces the materials and methods used in this study. Section 3.1 provides information on the used data set, while Sections 3.2, 3.3 propose a qualitative and quantitative analysis of the preprocessed data, respectively. Finally, Section 3.4 introduces some aspects related to sentiment analysis.

3.1. Data set

The data set analyzed in this paper comes from the resources collected by the Italian Association of Computational Linguistics 2 since the beginning of the COVID-19 outbreak in 2020, after the initiatives of the Computational Linguistics community to contribute to the fight against COVID-19 by making at disposal data sets, tools, news, and publications to the research community

(Computational Linguistics and the Covid-19 Outbreak 3 ).

The data set, named $40wita$ [25] consists of tweets in Italian downloaded daily by the University of Turin since March 1st, 2020, selected from the $TWITA$ collection of Italian tweets [26] by using the keywords covid, covid19, covid-19, corona virus, coronavirus, quarantena (quarantine), auto-isolamento (self-isolation), iorestoacasa (istayathome), stateacasa (stayathome), COVID19Italia (COVID19Italy). The information fields in the dataset are as follows: id, text, language, screen_name, date, timestamp, year, month, day, hour, lat, lon, location_json, location, source, urls, description, statuses_count, followers_count, friends_ count and media. Table 1 shows two tweets extracted from the dataset along with their attributes. For more details see http://twita.di.unito.it/dataset/40wita.

In particular, in this paper the tweets gathered from January 1st, 2021 until February 28th, 2022 were considered. The number of tweets downloaded in this period was 1,602,940. However, because of the objective of the analysis, the tweets have been filtered by eliminating all those not containing the word $vaccin$. The number of tweets remaining after the filtering is 353,217.

Table 1.

Sample of tweets extracted from 40wita.

id	1344782378410130000	1344782403055670000
Tweets’ text	Coronavirus: 9 deaths and 527 new infections. Positivity is rising among children in Trentino https://t.co/6c8N38FQCe	The anti-covid vaccines have arrived in the Ragusa area: we start with Ragusa, Modica and Vittoria https://t.co/p6rGKAWpIlhttps://t.co/DUY6mn4sSZ
language	it	it
screen_name	infoitinterno	CorriereRagusa
date	01/01/2021	01/01/2021
timestamp	1609456078518	1609456084394
year	2021	2021
month	1	1
day	1	1
hour	0	0
lat	null	null
long	null	null
location_json	null	null
location	Italia	Ragusa
source	¡a href=“http://www.informazione.it” rel=“nofollow”¿Informazione¡/a¿	¡a href=“https://dlvrit.com/” rel=“nofollow”¿dlvr.it¡/a¿
urls	https://www.informazione.it/a/C4A0A7D8-ADC4-4317-8121-923EAE11B81D/Coronavirus-9-deaths-and-527-new-contagions-rising-the-positivity-among-children-in-Trentino	http://dlvr.it/RphjhR
description	From the Inside section of information.it. News and politics related to Italy	Corriere di Ragusa is the first online newspaper of the Province of Ragusa. It offers precise, punctual, complete and free information.
statuses_count	2653868	23808
followers_count	4277	2018
friends_count	4	2
media		https://twitter.com/CorriereRagusa/status/1344782403055673344/photo/1 photo

3.2. Qualitative analysis: Tweets and users classification

The quantitative and qualitative data that the 40wita dataset provides can be exploited to deeply analyze both tweet content and the characteristics of the users who posted that tweet. The metadata available for each tweet consists of the author’s id and name, publishing date and timestamp, tweet URL, short bio description of the author, number of followers and followees, partly used for the analysis that follows. A qualitative text analysis using inductive approaches [27] was performed to obtain a classification of the type of user in four main classes, deemed interesting for our analysis.

To this end, the $NLP$ tool $NooJ$ for language formalization and text processing has been used. $NooJ$ includes a tokenizer that does automatic text preprocessing based on the provided text resources.

Among the NooJ modules that have been developed, for more than twenty languages, by the international NooJ community, the Italian one can be any time integrated with other ad hoc resources, in form of electronic dictionaries and local grammars. The freely downloadable Italian resources include:

• •electronic dictionaries of simple and compound words;
• •an electronic dictionary of proper names;
• •an electronic dictionary of toponyms;
• •a set of morphological grammars;
• •samples of syntactic grammar.

The Nooj annotations can describe simple word forms, multiword units, and even discontinuous expressions.

To obtain an automatic annotation of users, the electronic dictionaries of simple and compound words, and of proper names have been integrated with large-scale lexicons specific to each domain of interest.

These resources contain Italian names of professions and unambiguous simple words and multiword expressions, annotated with the knowledge domains of reference. The sources used to build the lexicons are an annotated lexicon of multiword expressions from the Italian module of the tool NooJ and a selection from the $MultiWordNet$ 1.5.0 [28],4 a multilingual lexical database in which the Italian WordNet is strongly aligned with Princeton WordNet 1.6. In order to avoid ambiguous classification, only lexical items described with less than two domains and synsets have been selected and used for the analysis. More in detail, the dictionaries NooJ and MultiWordNet have been enriched with about 58,000 Italian word senses, 41,500 lemmas (both simple words and multiword units), and 32,700 synsets. Both the dictionaries have been converted in the NooJ format and each lemma has been enriched with morphological and inflectional properties (FLX) in order to perform lemmatization and pos tagging.

In order to perform the user classification, all the Twitter bios have been analyzed according to four sets of keywords, able to recognize linguistic patterns specific to the following categories, as highlighted in Table 2, with the strings in bold:

Table 2.

Example of four bios and the corresponding class assigned to users.

User	Italian bio	Translation	Class
*****	Maestra di scuola primaria, curiosa del mondo e degli umani, soprattutto se di piccola taglia.	Primary school teacher, curious about the world and humans, especially if they are small.	Common users
matteosalvini	Leader della Lega #primagliitaliani	Lega Leader ( #Italians first)	Politics
SkyTG24	News, video, fotogallery e la diretta web 24 ore su 24. Sky tg24 è sui canali 100 e 500 di Sky e anche sul canale 50 del digitale terrestre.	News, videos, photogallery and the live web 24 h a day. Sky tg24 is on Sky channels 100 and 500 and also on digital channel 50.	Media
RobertoBurioni	Medico, Professore di Virologia all’ Università San Raffaele, Milano - Direttore scientifico Medical Facts	Doctor, Professor of Virology at San Raffaele University, Milan - Scientific Director of Medical Facts	Medicine

• •Common users;
• •Media (local and national press organizations);
• •Medicine (healthcare workers end domain experts);
• •Politics (politicians and domain experts).

Such classes thus include both users who belong to specific categories on the basis of their work, e.g. medical doctors, virologists, and healthcare assistants, for the Medicine category; journalists for the press organization; leaders of the Parties or people who occupy political positions for the political class, and pages that specify the topics of the posts they publish, for instance Quotidiano online per i professionisti del settore healthcare e vetrina del magazine AboutPharma and Medical Devices edito da Hps, whose English translation is Online newspaper for healthcare professionals and a showcase of AboutPharma and Medical Devices magazine published by Hps.

This way all the Twitter users, from the subset of 40wita dataset, have been attributed to one of the four classes listed above.

Table 2 reports four users with the profile’s bio description in Italian and English, along with the class assigned to each of them. For instance, the very known Italian virologist Roberto Burioni has been assigned to the $Medicine$ class, as can be clearly induced from the terms appearing in his bio, such as Doctor and Virology.

The following examples Example 1, Example 2 illustrate how pattern matching works on the short bios published by users for the political and medical domains by showing some entries from the MultiWordNet dictionary converted into the NooJ format.

Example 1

Leader politico (political leader): $N$ + FLX = C540 + MULTIWORDNET + DOMAIN = Politics + SYNSET = nX07459694

Example 2

Virologia (virology): $N$ + FLX = N41 + MULTIWORDNET + DOMAIN = Medicine + SYNSET = nX04633565

In the examples Example 1, Example 2, $N$ indicates the part of speech (noun), and all the other codes mean that the lemmas are associated with specific properties formalized as follows: +PROPERTY=Value. This way, the FLX property let the tool inflect each word according to the inflectional paradigms associated with its values (e.g. C540); the DOMAIN property shows the knowledge domain the lemmas belong to (i.e. Medicine); and, in the end, the SYNSET property represents the MultiWordNet synonym sets: set of words that are considered to be interchangeable in some context without changing the truth value of the sentence in which they occur. Some words are associated with just one synset and some have several.

After the categorization of users in one of the four classes, all the tweets have been classified by assigning them to the same class of the tweet author. For completeness, Table 3 reports a summary of the number of users and tweets by category. Notice that the total number of tweets is 379,514 instead of 353,217 because of the overlap between users, as explained below. The table points out that class Politics is less active with respect to the other categories in contributing to the online discussion regarding vaccines.

Table 3.

Summary of the number of tweets by category.

	Common users	Media	Medicine	Politics
Tweets	200,864	146,898	21,017	10,735
Users	47,286	6,348	782	1,587

3.3. Quantitative analysis of the user classes

In this paper, the users who correctly received two annotations have been considered to belong to both categories and, therefore, their posts have been analyzed in both the user classes.

As regards the overlap among categories, it must be noticed that the 4% of the posts have been classified as Media and Medicine (16,280); the 0.81% are Media and Politics (2,879); the 0.15% Politics and Medicine (520); and the 0.06% belong to Medicine, Politics, and Media (213).

As concerns the frequency of publication of the different users, it should be pointed out that Medicine and Media are characterized by the highest values in terms of the average number of published tweets per user: respectively 26.9 and 23.1.

Moreover, Medicine and Media share a profile that reaches one of the highest levels of publication frequency, that is @infoitsalute with its more than 15,000 published tweets (evaluated at the time in which the dataset has been downloaded). Among all the categories, the profile with the highest publication frequency is, instead, @infoitinterno.

The total number of shared posts shows a different ranking: Common users rank first in post sharing with more than 200,864 tweets, followed by the categories of Media (146,898), Medicine (21,017) and, in the end, Politics (10,735). This, added to the previous remark, displays how, in the categories of Media and Medicine, few profiles share the largest amount of posts, while in the class of Common users many different profiles share a very huge number of posts. Politics, instead, is composed of a very small number of users with poor participation in the discussion.

As regards the followers reached by all the considered classes, the aggregate of the Common user’s audience reaches the number of 114,245,682 followers. In addition, all the profiles classified as Media have 81,082,716 followers while Politics and Medicine have in total 23,676,330 and 2,466,091 followers, respectively.

The average length of the tweets is 187 characters. The longest tweets are the ones written by the users which have been classified as belonging to Politics, with special reference to the overlaps among Medicine, Media, and Politics (more than 250 characters).

3.4. Sentiment Analysis

Sentiment Analysis (SA) consists of the automatic treatment of private states, personal feelings or beliefs toward entities, events, and their properties. The term private states refers to many dimensions related to the linguistic expression of subjectivity, e.g. opinions, evaluations, emotions, and speculations. The main difference between subjectivity and objectivity [2] regards the impossibility to directly observe or verify subjective language; but under any circumstances, they implies truth:

whether or not the source truly believes the information, and whether or not the information is, in fact, true, are considerations outside the purview of a theory of linguistic subjectivity [29, p. 291].

Therefore, the SA task does not claim to portray reality. Instead, it provides an overview of the stories and the narrations that people freely share on social media, review websites, forums, blogs, and many other situations of communicative and affective exchange.

The huge volume of unstructured data, which is continuously shared by Internet users, poses several challenges to information monitoring activities, that need strategies and tools capable of converting texts written in natural language into structured data, liable to be stored and queried into database tables. In this context, it is crucial to have at disposal automatically extracted, analyzed, and summarized data, which do not include only factual information, but also opinions and emotions. However, as pointed out in Liu [2], it is essential first to formalize the problem of SA and then define a model to extract the sentiment: their feature-based SA model analyzes each individual sentence to determine the targets inside the sentence on which the opinion has been expressed, and then if the opinion is either positive, negative, or neutral.

The proposed model exploits two approaches to be seen as two stages in a pipeline: at the base of the methodology there is the use of a lexicon through which candidates for SA are identified, then through feature analysis, which constitutes the focus, certain themes are explored based on the proximity of certain elements to words in the sentiment lexicon. In what follows, in Sections 3.4.1, 3.4.2, respectively, both the lexicon-based and feature-based methods, as specialized for the COVID-19 application domain, are illustrated.

3.4.1. Lexicon-based sentiment analysis: lexical and syntactic resources

The method used in this paper to perform feature-based SA is based on the evaluation of the co-occurrences of a specific selection of noun phrases and sentiment words into evaluative sentence structures [30], [31], [32], [33], [34]. This comes from the exploitation of an Italian sentiment lexicon, SentIta 5 [3] combined with the use of Finite-State Transducers for the syntactic analysis and the pattern annotation. SentIta includes more than 20,000 entries, enriched through taboo words [35], idioms [36], hashtag and emojis [37].

Table 4, Table 5 display examples from the mentioned resource and Table 6 shows details about the word composition of SentIta. Table 4 illustrates some examples of polarized words that express a semantic orientation [38], i.e., allow for carping positive or negative sentiment. Table 5 shows in detail that the entries of SentIta do not include only polarized words but also intensive words. This is a cue that allows the identification of higher (FORTE that stands for intense) and lower (DEB that is the abbreviation of DEBOLE and means weak) items, that can support the results provided by the SA.

Table 4.

SentIta Evaluation tag set.

Lexical item	Translation	Tag	Description	Score
indispensabile	essential	+POS+FORTE	Strongly Positive	＋3
promettente	promising	+POS	Positive	＋2
rassicurante	comforting	+POS+DEB	Weakly Positive	＋1
nuovo	new	–	Neutral	0
sospettoso	mistrustful	+NEG+DEB	Weakly Negative	−1
disinformato	misinformed	+NEG	Negative	−2
vergognoso	shameful	+NEG+FORTE	Strongly Negative	−3

Table 5.

SentIta Intensity tag set.

Lexical item	Translation	Tag	Description	Score
grave	severe	+FORTE	Stronger	+
ridotto	low	+DEB	Weaker	–

Instead, the word composition mechanism of SentIta intervenes each time the general lexical resource is applied to a data set that belongs to a given domain or topic, adjusting it through high-priority dictionaries and according to the lemmas whose polarity change for that specific domain or topic. A clear example of a polarity switch in the COVID-19 domain are the words positive and negative, which are often associated with the outcome of the COVID-19 swabs.

Table 6.

Composition of SentIta.

Category	Entries	Example	Translation
Adjectives	5,381	allegro	cheerful
Adverbs	3,693	tristemente	sadly
Compound Adv	774	a gonfie vele	full steam ahead
Idioms	577	essere in difetto	to be in fault
Nouns	3,215	eccellenza	excellence
Verbs	1,255	piacere	to like
Bad words	182	cazzo	shit
Total	15,077

3.4.2. Feature-based sentiment analysis

In feature-based SA the targets on which opinions can be expressed are the $objects$ and their $features$. An $object$ is an entity, such as a product, an organization, a topic, or an individual. An $object$ is characterized by a set of $attributes$, also called $properties$ or $features$. The opinion holder is the person or organization that formulates the opinion. Opinion holders are the authors of posts on social media. The orientation of an opinion on a feature $f$ of an object $o$ says whether the opinion is positive, negative, or neutral. Opinion orientation is also called sentiment orientation, polarity of opinion, semantic orientation.

Definition 3.1 Model of an Object —

An object $o$ is represented with a finite set of features $F=\{f_1,f_2,\dots ,f_n\}$. Each feature $f_i\in F$ is expressed with any one of a subset of words or phrases $W_i=\{w_{i1},w_{i2},\dots ,w_{im}\}$ which are synonyms of $f_i$, or it is indicated by any one of a finite set of feature indicator $I_i=\{i_{i1},i_{i2},\dots ,i_{iq}\}$ of $f_i$.

Opinions are defined as positive or negative views, attitudes, or appraisals about a topic, expressed by an opinion holder in a given time. They are represented by Liu [2] as the following quintuple:

Definition 3.2 Opinion —

An opinion is a quintuple $(o_j,f_{jk},o{o}_{ijkl},h_i,t_l)$, where $o_j$ is the object of the opinion, $f_{jk}$ are its features, $o{o}_{ijkl}$ are the positive or negative semantic orientation, $h_i$ the opinion holder, and $t_l$ the time in which the opinion is expressed.

The two following examples report two tweets extracted from the $40wita$ data set and make explicit the quintuple extracted from each tweet.

Example 3 Negative Opinion —

Non sta andando affatto bene #COVID19 #VaccinoAstrazeneca. #AstraZeneca, donna ricoverata a #Napoli in condizioni gravissime dopo il #vaccino. I familiari: “Non aveva patologie”

(English translation: It is not going well at all #COVID19 #VaccineAstrazeneca. #AstraZeneca, woman hospitalized in #Naples in very serious conditions after the #vaccine. Family members: “she had no pathologies”)

where:

• •the object of the opinion $o_j$ is COVID-19 vaccine;
• •its features are $f_{ij}=\{\mathit{AstraZeneca},\mathit{hospitalized}\}$;
• •the semantic orientation $o{o}_{ijkl}$ of the opinion is negative because the tweet refers to a hospitalization related to vaccination;
• •the opinion holder $h_i$ is a Common user;
• •the time $t_l$ in which the opinion is expressed is 2021-03-15 18:26:31.

Example 4 Weakly Negative Opinion —

@EMA_News ribadisce che i benefici del vaccino AstraZeneca nella prevenzione del COVID-19, con il rischio associato di ospedalizzazione e morte, superano i rischi di effetti collaterali

(English translation: @EMA_News insists that the benefits of the AstraZeneca vaccine in preventing COVID-19, with the associated risk of hospitalization and death, outweigh the risks of side effects) where:

• •the object of the opinion $o_j$ is COVID-19 vaccine;
• •its features are $f_{ij}=\text{{}$ Astrazeneca; preventing COVID-19, hospitalization, death, side effects};
• •the semantic orientation $o{o}_{ijkl}$ of the opinion is weakly negative because EMA recognizes the risks of the vaccine, but still assumes its validity in COVID-19 prevention;
• •the opinion holder $h_i$ is a Common user;
• •the time $t_l$ in which the opinion is expressed is 2021-03-15 18:27:07.

4. Evaluation indexes

Before showing the results of our analysis, the measures we used to evaluate the results we obtained are described. In particular, in order to answer the various research questions posed in the Introduction, several indices have been calculated from the constituent words of the tweets, to gain insight into the general opinion and to provide a specific indication for the classes of users introduced in Section 3.2. In detail, the following indexes were used:

• 1.the arithmetic mean of Semantic Orientation (SO);
• 2.the arithmetic mean of Polarized Word Intensity (PWI);
• 3.the Standard Score (SS) (also known as Z-Score).

Regarding the SO and PWI indices, they are based on SentIta. The semantic orientation is computed by considering the frequency of occurrence of all those elements in the lexicon, called sentiment markers, which have been attributed a sentimental meaning with a positive or negative orientation, as shown in Table 4 above. For PWI, the frequency of occurrence of the intensifiers and attenuators present in the tweets is exploited in order to provide an indication of the tone of voice used, that is, whether more impetuous or more subdued, as shown in Table 5.

The SS index, on the other hand, is used to provide an indication regarding the importance of the presence or absence of a given keyword (or a group of) in a portion of the dataset compared to its overall occurrence in the entire dataset. In particular, it is possible to start from the definition provided in statistics, before arriving at the specific formulation for language processing:

Definition 4.1 Standard Score or Z-Score —

Let X be the generic random variable distributed according to a mean $\mu$ and variance ${\sigma }^2$, the Z-score is defined as: $Z=\frac{(X-\mu )}{\sigma }$

This process, called standardization, leads back to a random variable with a “standard” distribution, that is, of zero mean and variance 1. It helps to tell whether a data value is greater or less than the mean and how far it is from the mean: more precisely, it indicates how many standard deviations a data point is from the mean. Consequently, if Z is greater than 3 or less than −3, this indicates that the data point is very different from the others (in the case of a normal distribution, about 99.7% of the data points are within $\pm 3\sigma$), meaning it may be an outlier. At this point the specific variant for language processing can be introduced:

Definition 4.2 Standard Score for Language Processing —

Let $AF$ be the absolute frequencies (of the markers of interest) in each part of the data set, RS the relative size of each part of the data set, and $EF=AF\ast RS$ the expected frequencies (of the markers of interest) in each part of the data set, $\sigma$ the standard deviation, the standard score is defined as : $SS=\frac{AF-EF}{\sigma }$

This SS can be automatically measured by the Nooj tool, which allows it to be done with reference to the individual word or to groups of words, i.e. the markers of interest. In this case, it was chosen to proceed by groups of words with a semantic grouping criterion: all words that share a similar SO (all those positive or negative) or similar PWI (all those with the same absolute value) were grouped together.

5. Results

This section shows the results of the analysis conducted on the data set described in Section 3.1 with the aim of understanding the opinions of Italian Twitter users regarding COVID-19 vaccination. Each sub-section tries to answer to the research questions posed in Section 1 with the help of the evaluation indices described in the previous section, when possible, for analyzing and discussing the results.

5.1. Which is the interest in the COVID-19 vaccine?

In order to understand the extent to which the public is interested in the COVID-19 issue, conversation volumes were analyzed. Specifically, Fig. 1, Fig. 2 show the number of tweets posted during the analyzed period with reference to the string vaccin in total and broken down by class, respectively.

Fig. 1.

Total tweets by month.

Fig. 2.

Total tweets per month by categories.

Common users and Media are the classes with the highest number of tweets and, even when taken individually, significantly detach the other two, Medicine and Politics. Overall, this result agrees with what was shown in Table 3 above regarding the qualitative composition of the data set. Interestingly, there is no single month in which the number of tweets from Politics exceeds that of Medicine, which with half the number of users has roughly twice as many tweets, with a ratio of about 4:1 of the number of tweets per user in favor of Medicine over Politics. This situation underscores two aspects: on the one hand, the cautious attitude of Politics, whose trend always follows that of Medicine without overwhelming it, and on the other hand, the strongly divulging attitude of the Medicine category.

The highest conversation volumes are observed in the months of:

• •January 2021, when the vaccination campaign began;
• •March 2021, when deaths were recorded immediately after the inoculation of vaccines;
• •November 2021 to January 2022, when the growth of the infection curve was observed.

Although in absolute terms the volume of tweets remains distributed roughly as previously illustrated, it is significant to observe the behavior of the Medicine category outside the months already highlighted. In particular, a good volume of conversation is present in the month of both May and June: during this period there is a strong discussion about the effects of vaccination and the precautions to be taken and decisions to be made for the continuation of the vaccination campaign. Similarly, volumes of conversation are particularly high in September and October: this highlights how specialists are already debating the future evolution of the infection and its likely growth based on their predictions.

What it is curious to note, although expected, mainly due to the contingent economic and psychological situation of the population, there is an absolute collapse in conversation volumes in July by all categories: corresponding to the summer vacations, the need to escape from the grim context of the pandemic becomes stronger, taking advantage of a probably lower rate of contagiousness due to rising temperatures.

Equally interesting to observe is how the number of posts in the media category is highest, with respect to the other categories, in the months of February 2021 and May 2021. In both cases, this can be seen as a precursor sign, in the first case of the imminent arrival of vaccines, thus the succession of announcements, and in the second case of the need to ignite the debate and keep attention high on how to evolve the vaccination campaign following some deaths. Also in the month of February 2022 the situation is similar: at that time, in Italy, the introduction of an enhanced green pass requirement for all workers over the age of 50 was being discussed.

5.2. Which are and how does the emphasis and tone of the discussion on COVID-19 vaccination vary?

Understanding the tone of the discussion inherent in the vaccination campaign, therefore the emphasis with which this topic is addressed, makes use of the indices introduced earlier. In particular, by resorting to SentIta markers, a division was made between tones with strong intensity and tones with weak intensity by exploiting PWI, thus regardless of semantic orientation. In other words, two frequencies of occurrence in the lexicon will be measured: strong terms, i.e. terms whose score is $\pm 3$, and weak terms, i.e. terms whose score is $\pm 1$. In general, intensity is associated with both strongly oriented words, such as catastrophe and miraculous, and simply intense words, such as high and rarely, from the lexicon of the sentiment SentIta.

Based on this, the standard score SS, shown in Fig. 3, Fig. 4 for the 4 categories, respectively, was calculated for the groups of words with the same intensity, strong or weak. The overall average intensity value, both strong and weak, is shown in Fig. 5.

Fig. 3.

SS for words with a strong tone by categories.

Fig. 4.

SS for words with a weak tone by categories.

Fig. 5.

Average SS (both strong and weak words).

The figures highlight how far the discussion deviates from what is, on average, the strength or weakness with which different positions are expressed. More specifically, remembering that the SS expresses how many standard deviations one departs from the mean value (in this case of the expected frequencies), it becomes important to emphasize how the interest is to be placed on the detachment from average strength or weakness in the speeches, therefore on values tending to be particularly positive (＋3) or negative (−3). Consequently, the most interesting points to note are those of December 2021 and January 2022: with the start of the new year in Italy, the debate regarding the reinforced green pass that will be introduced in the following month is ignited, and this accentuates the separation between those who are strongly for or against it and those who urge calm and control. Analyzing the temporal trend, the different nature of these two types of parties is evident: while those who had an extreme position (in one way or another) progressively heightened the tone of the discussion, those who instead seek to bring the discourse back to reasonableness are more inconstant (going from a strong presence in December 2021, decreasing in January 2022, then increasing again in February 2022) and intervene in a more pronounced manner when circumstances call for it. In essence, this dichotomy highlights a substantial difference in the population between the attitude of those who are more impulsive and impetuous, perhaps because they were impressed by the deaths, and those who are more thoughtful, perhaps because they are burdened with doubts. To better highlight this, word clouds for the four categories showing the first 50 most used words with strong and weak intensity have been produced in Fig. 6, Fig. 7, respectively. Fig. 6 points out that the words related to death, followed by serious and severe, are the most prevalent for all the categories, showing the great apprehension of people regarding death as side effect of the vaccine. On the other hand, Fig. 7 highlights the doubts and worries of the users that try to have a precautionary attitude towards a vaccine, though they are suspicious regarding its efficacy.

Fig. 6.

The 50 most frequent strong words for the categories Common users Media Medicine and Politics (read from top left to bottom right).

Fig. 7.

The 50 most frequent weak words for the categories Common users, Media, Medicine and Politics (read from top left to bottom right).

Although the view provided is partial because it does not take into account the past average, it is also interesting to note the situation in January 2021 after AIFA (Agenzia Italiana del Farmaco, i.e. Italian Medicines Agency) authorizes the BioNTech/Pfizer vaccine on December 22nd, 2020, and shortly after Moderna (January 7th, 2021) and AstraZeneca (January 30th, 2021): the tones are not yet particularly heated, rather the need to keep tones (both hope and distrust) calm in the speeches is evident.

5.3. How controversial is and how does the discussion on the topic of COVID-19 vaccination vary?

In order to understand the emotional, even contradictory, charge that emerged from the conversations during the period under analysis, the markers were used similarly. In detail, again taking advantage of the SentIta markers, it was possible to make a subdivision on the basis of semantic orientation SO. In this case, PWI was not employed, but rather a separation was made between positive and negative sentiment, regardless of the intensity.

From this, SS was calculated for each of the 4 categories for all positive (＋1, ＋2, ＋3) and negative (−1, −2, −3) word classes, as shown in Fig. 8, Fig. 9, respectively; in addition, positive and negative mean values are shown in Fig. 10. Finally, Fig. 11 provides an overview of the number of tweets that take either a positive or negative orientation, distinguishing them from those that maintain a neutral orientation, which will be discussed below.

Fig. 8.

SS of positive words by categories.

Fig. 9.

SS of negative words by categories.

Fig. 10.

Average SS (both positive and negative words).

Fig. 11.

Neutral and oriented tweets by month.

At the beginning of the vaccination campaign, the absence of both positive and negative words is relevant: this clearly shows how all users were waiting for the results of the vaccination campaign before expressing an opinion on it. The behavior of Politics is singular: it goes from a strong absence of positivity in January 2021 to peaks of presence around mid-year: probably the explanation lies at the beginning of the first proclamations of the success of the vaccination campaign and optimism in how the situation is being handled. In addition, from a general point of view, it is important to note the drop in negativity close to the summer season: avoiding such positions becomes a way to evade the context and justify the need for the upcoming vacation season. This attitude is shared by the media (most of all), then by politicians and ordinary users to a similar extent; the same cannot be said for the medical class whose position is probably caught in the grip of Politics (whereby absence of negativity and presence of positivity are observed), which for obvious economic reasons was pushing for the summer months to pass in a regime of normality. Overall, it can be described as a kind of media-driven suspension of judgment.

Immediately after the summer, with an upturn in the curve of contagions, both the curves of positive and negative words increase. In particular, positions expressing negativity begin to increase on average as early as August with a shared trend among different categories, first among all Medicine (already engaged in the discussion about the future rise of contagions) leading the others, probably distracted by summer vacations.

After that, Media and Common users tend to be rather negative during the fall, touching the highest peaks between November and December 2021, while Medicine, followed closely by Politics, reduce their rate of negativity. But while Medicine takes an observational stance in which optimism is unlikely to be exposed, Politics does not take the same view: initially (September 2021), it was particularly positive and then maintained a rather thoughtful stance in the aftermath.

The end of 2021 (November and December 2021) and January 2022 are comparable in amount of the volume of conversations to January 2021; moreover, the intensity of tone also tends to increase in those months, as seen above. Overall, the situation is quite controversial, increasing the presence of both positive and negative words. The topics, all of which are highly debated, relate, for example, to the third dose of vaccines, the extension of the vaccine to children, the growth of infections despite vaccines, the obligation for school personnel, the management of the Christmas vacations, and the introduction of the super green pass. Only Politics from substantial stagnation between November and December 2021 shifts to negativity in January 2022. This time the trend with respect to the Medicine category is rather anomalous, in fact it does not follow it at the end of 2021 when it is generally positive but aligns with negativism in early 2022. Media and Common users are rather positive in the run-up to the holiday season, and then tend to uncertain positions with the start of the new year as they await political decisions on debated topics.

Finally, Fig. 11 clearly shows the growth, in the periods defined as most controversial, of oriented tweets (whether positive or negative), thus a polarization on the part of the public tending toward one position or another. This figure helps to make an important distinction, despite the similarities illustrated, between January 2021 and January 2022: if in the first case, uncertainty was strong because there were still no results from the vaccine campaign, but there was still much debate about the evolution of the pandemic, in the second case (and generally during the second half of the year) the stances begin to be stronger. Basically, alignments of those in favor and against vaccination begin to form a hard core of the discourse, becoming attractors of those who are still undecided.

5.4. Which is and how does the public perception, in terms of positive or negative feelings, of COVID-19 vaccination vary?

By simultaneously exploiting the SO and PWI indices, it is possible to provide a quantitative indication of the value attributable to sentiment. Thus, it becomes evident numerically what the public’s perception is during the period under analysis. In particular, the results given by this SA take into account not only the presence of positive and negative words, but also the presence of any contextual element that may alter the overall scores, which vary in the range [−3,＋3], i.e. from maximum negativity to maximum positivity. In this regard, Fig. 12, Fig. 13 show the average monthly sentiment value for individual categories and altogether, respectively.

Fig. 12.

Average sentiment per month by categories.

Fig. 13.

Overall average sentiment by month.

Considering that 0 represents neutrality, it is evident that sentiment is always negative overall. Even the apparent positive spike in Politics and Media in May 2021 is of little relevance in this regard, 0.2 being practically neutral rather than positive. At that time, the first reports regarding the correlation between thrombotic events and vaccines began becoming available. Common users are undoubtedly the category in which the negative sentiment is most entrenched. Only in February 2021, before the start of the vaccination campaign, the hope induced is likely to alleviate this, although in contrast to the category Medicine evidently doubtful on the subject. Interestingly, Medicine and Politics are less pessimistic in the run-up to April and July 2021, in relation to the start of the vaccination campaign and the approach of the vacations, therefore of a likely reduction in the contagiousness index. Similarly, in September 2021, these two categories are more negative than usual, due to the probable increase in the curve of contagiousness after the return from vacations and the consequent economic problems. Another globally interesting aspect is that related to the attitude of the Media category, which in the first part of the year travels around neutrality, and then shifts steadily into the negative zone thereafter. Often acting as an amplifier or attenuator of the perception of events, the Media category tries to maintain, as much as possible, the role of a rather moderate narrator and communicator of facts. Still, it tends, especially in the second half of the year, to be weakly linked to the trend of common opinion, shifting from the role of “guide” to the role of “led”.

5.5. What are the main events that determine such feelings? Are there peculiar daily events in relation to specific months?

In this section, a focus on the three months of 2021, which showed significant sentiment scores and was characterized by important events, is presented. The results put in relation sentiment average scores reached by the considered categories (Common users, Media, Medicine and Politics) with specific events occurred in the same periods. What emerges from the analyses is that Common users, which present highly negative orientation all the time, do not seem to worsen their sentiment and have stronger reactions to events, while Media remains stable around neutral scores, as mentioned earlier. Differently from these two larger classes, Politics and Medicine show significant reactions to what happens. Therefore, all the details that regard the sentiment trend in the considered months are presented below.

5.5.1. Which are the main events in relation to specific months that determined positive or negative feelings in relation to vaccination?

The most interesting months from the point of view of the relationship between events and changes in attitude in the four categories analyzed are March, August and December 2021, discussed below.

March 2021.

Positive sentiment values are found in the class of Media at the beginning of March (see Fig. 14, Fig. 15), but they fall on March 3rd, 2021, when the first death of the month occurs in Campania, four days after an anti-covid vaccine. However, the values of the Media remain neutral for the whole month, while the other classes show slightly different trends: Medicine and Common users are permanently negative. The Medicine class have sentiment values which rise until they reach neutrality in conjunction with the AIFA rulings. The Politics class is the one that holds the highest sentiment values, with a positive score on March 6th, 2021, date in which provisions of the new DPCM Draghi comes into force, and with negative peaks in the days in which news report cases of death attributed to the AstraZeneca vaccine.

Fig. 14.

Sentiment trends in March.

Fig. 15.

Main events in March.

August 2021.

Another interesting month is August 2021 (see Fig. 16, Fig. 17). On August 4th, 2021, the 7th AIFA Report on the surveillance of COVID-19 vaccines is published and, on the same day, a Circular from the Ministry of Health clarifies the procedures for issuing the green pass. On August 5th, 2021, a Circular from the Ministry of Health provides instructions on the certification of temporary exemption from anti-Covid vaccination. All of these events are followed by negative spikes in all categories. On August 7th, 2021, there was a peak in the Medicine and Politics categories, coinciding with the announcement by AIFA regarding the approval of the use of monoclonals in the treatment of Covid-19 patients. On August 11th, 2021, the recommended quarantine and isolation measures are updated in light of the circulation of the new SARS-CoV-2 variants in Italy, in particular the Delta variant. This seems to be positively received by Politics and Medicine. On August 16th, 2021, a Ministerial Decree establishes the allocation of resources to schools for the purchase of goods and services during the years 2021–2022 for COVID-19 issues. On this date there is a positive response from the Politics class alone. On August 20th, 2021, AIFA denies a fake news concerning the authorizations of anti-COVID vaccines, which still remain valid. In particular, according to this totally false news, following the authorization by the Agency for the use of some monoclonal antibodies against COVID-19, the marketing authorization for COVID vaccines issued by the EMA would have ceased. On the same day, all categories recorded particularly negative values. On August 30th, 2021, with a Circular from the Ministry of Education, the verification of the green pass for school staff is regulated. Also in this case the sentiment values present negative scores in all the classes.

Fig. 16.

Sentiment trends in August.

Fig. 17.

Main events in August.

December 2021.

The month of December 2021 (see Fig. 18, Fig. 19) is probably the most significant period of the year in terms of SA. On December 1st, 2021, AIFA approved the Comirnaty vaccine for the age group 5–11 years, but this fact does not seem to significantly move the views of any class. On December 2nd, 2021, the Interior Ministry provides indications related to the implementation of the Super Green Pass decree, which enters into force on December 6th, 2021. On this last date, the Politics and Medicine classes present positive values, while Common users and Media remain on negative sentiment values. On December 8th, 2021, the COVID-19 bulletin was announced, presenting the highest number of new cases since April: this precipitates, in particular, the mood of the Medicine category. On December 16th, 2021, clarifications are provided regarding the indications for the use of the Comirnaty vaccine (BioNTech/Pfizer) for the age group 5–11 years. Furthermore, the day after, December 17th, 2021, changes are provided to the DPCM of June 17th, 2021, regarding the green pass and clarifications related to the vaccination obligation for school staff, which seem to be well received by the Medicine category, but not by the Politics. On December 22nd, 2021, AIFA approved the Nuvaxovid (Novavax) vaccine and on December 23rd, 2021, the Ministry of Health extended the validity of the anti-Covid vaccination exemption certificates. In the following days, a drop in sentiment can be observed in all classes, which rises slightly on Christmas Day, except for Medicine, which on that day presents a greater positivity. On New Year’s Eve, the decree law enters into force, indicating urgent measures to contain the spread of the COVID-19 epidemic and the provisions on health surveillance. In detail, the decree concerned the new rules on the reinforced green pass, on the quarantine and on the capacities of stadiums and entertainment venues. This causes a lowering of sentiment values on that day.

Fig. 18.

Sentiment trends in December.

Fig. 19.

Main events in December.

6. Discussion and comparison with related approaches

Comparing our results with related approaches is a difficult task because most of the time each study uses datasets obtained from tweets posted in different countries, and where the target period is not the same. In general, important findings are obtained as a result of the interpretation of vaccine-related posts in the COVID-19 pandemic in concomitance with the agenda of the country. For all these reasons, a fair comparison cannot be easily carried out.

The aim of the discussion reported in the following is to somehow assess if results observed in different studies and targeting different countries are more or less in line with the findings we obtained with our analysis.

Hu et al. [9] studied the spatio-temporal trends of sentiment and emotions in US from March 1st, 2020, to February 28th, 2021, at the national and state levels, identifying key events as drivers of change in perception and related emotions. The period of observation that overlaps with the one on which we focused our analysis is January–February 2021. For such a period, the overall sentiment trend follows the same negative trend we detected with our analysis. In particular, we observed several similar trends, like the moderate decrease in the sentiment score due to issues and questions about COVID-19 vaccine delivery, that actually was a worldwide concern. Another common behavior is detected around mid-February 2021, where an increased sentiment score was observed before the start of the vaccination campaign characterized by hope and trust in vaccines, and the announcements of purchased million quantity of COVID-19 vaccine doses from Pfizer and Moderna.

Lanyi et al. [12] employed NLP techniques to identify key barriers to vaccine adoption from a collection of tweets geolocated in London, UK, and collected between November 30th, 2020 and August 15th, 2021. With respect to this work, the time period overlap with the dataset we analyzed goes from January to August 15th, 2021. Authors have shown that clustering of topics with negative sentiment revealed concerns about vaccine safety and distrust of government and pharmaceutical companies, in line with the main reasons that we identified for the negative sentiments that we observed in Italy by analyzing our dataset (e.g., the first deaths in March).

Niu et al. [13] studied the change in sentiment toward vaccination in Japan by collecting tweets from August 1st, 2020 to June 30th, 2021. Similar to what we have done, the authors highlighted the correlation between changes in sentiment and daily events. According to our findings, authors reported that negative sentiment overwhelmed positive sentiment. In particular, the authors reported several negative peaks in Japan, reflecting the same findings we identified in Italy with our analysis like the one in February due to the severe infection situation and the local vaccination policy in Japan. Another relevant peak has been detected in June when the WHO suggested that children should not be vaccinated in the current stage on June 22, 2021.

The results of our analysis are in accordance with those reported in Aygün et al. [15] where, as in our case, social media data posted in Italy during the year 2021 are analyzed. In particular, in Aygün et al. [15] data from two social media, Twitter and Reddit, are reported. The study shows that sentiment reaches the highest number of positive polarity on both platforms in between March–April 2021, after that both began a fluctuating and gradual decline in sentiment near to early pandemic levels, especially on Twitter. Overall, policy and health issues stand out in negative tweets against vaccination in Italy with a substantial number of vaccine-focused tweets that are negative.

Yousefinaghani et al. [6] analyzed tweets between January 2020 and January 2021. The authors categorized the tweets into three categories positive, negative, and neutral. Similar to our analysis, the negative sentiments were related to a range of concerns, but the majority usually focused on vaccine development being time-consuming, doubts in vaccine safety or reaction to governments, political figures and manufacturers. On the other hand, positive tweets were usually about scientific breakthroughs, medical advice and spreading hope.

7. Conclusion

In this paper, a study on the sentiment trends that can be detected and monitored by exploring user-generated contents has been presented, dropped specifically into the scenario related to the COVID-19 pandemic in Italy. In order to perform the analyses, the method exploited here has been grounded on a sentiment lexicon, SentIta, applied to raw texts, together with syntactic insights. The peculiarity of the method regards the focus on the user’s classification into four significant categories, through the addition of a feature-based approach. This way it has been possible to automatically monitor volumes of conversations, sentiment scores, and words usages during the analyzed period. Opinion holders, such as the users that come from the Media, Medicine, and Politics, present differences in terms of opinion expressed and, if compared with Common users, show the relevance of a preliminary analysis and classification to understand who is leading the discussion and who is following it and why.

It is worth pointing out that such kind of analysis can support government and health institutions in the choice of vaccination policies, besides COVID-19, that take into account people’s sentiment in response to important events that can occur, thus promoting targeted interventions tailored to people’s moods and reactions.

The major limitation, as well as possible extension in future work, concerns the period analyzed: a greater view of the pre-pandemic phase could change some balances and show usual trends in some categories that by profession must maintain certain positions.

Overall, all the results show that SA alone cannot be truly explanatory unless it is accompanied by a broader view of the phenomena analyzed. This undoubtedly concerns the classification of users, but also the analysis of users’ tone of voice and attitude. This work, as far as we know, is the first investigation providing such a comprehensive insight into the discussion around the COVID-19 vaccine campaign in Italy.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.