Abstract
COVID-19 vaccine hesitancy is still widespread. During the pandemic, the internet has been the preferred channel for health-related information, especially for less-educated citizens who tend to be the most hesitant about vaccination. A well-structured web communication strategy could help both to overcome vaccine hesitancy and to ensure equity in healthcare service access. This study investigated how the various regional and local health authorities in Italy used their institutional websites to inform users about COVID-19 vaccinations between March and April 2021. We browsed 129 institutional websites, checking the availability, quality and quantity, actionability and readability of information using a literature-based common grid. Descriptive statistics and statistical tests were performed. The online public dissemination of COVID-19 vaccination information in Italy was fragmented, both across and within regions. The side effects of vaccinations, were often not reported on the websites, thus missing an opportunity to enhance vaccination uptake. More focus should also be placed on readability, since readability indexes showed that they were difficult to understand. Our research revealed that several actions could be implemented to enhance online communication on COVID-19 vaccination. For instance, simplifying texts can make them more understandable and the information reported actionable.
Keywords: Covid-19, Vaccine, Communication, Online information, Readability, Italy
1. Introduction
WHO declared COVID-19 a pandemic on 11 March 2020 [1], and Italy was one of the first countries affected, registering the highest incidence of confirmed COVID‐19 cases, outside China, on 22 March 2020 [2].
Considering this situation, global health authorities were strongly committed to controlling COVID-19 outbreaks and requiring citizens, for instance, to use personal protective equipment and restricting their freedom of movement. Over the past centuries, control of epidemics (i.e., poliomyelitis and smallpox) has been successfully achieved via the use of vaccines [3]. Therefore, the rapid development of a safe and effective vaccine against COVID-19 quickly became an urgent health priority for worldwide institutions [4]. The COVID-19 vaccine set a record time by reaching the market in less than a year after the declaration of the COVID-19 pandemic, and in early December 2020 the WHO decided the first mass vaccination program should be started [5].
Despite the availability of an innovative and safe vaccine, COVID-19 vaccine hesitancy is still widespread [6]. Vaccine hesitancy refers to delays in acceptance or refusal of vaccines regardless of the availability of vaccination services [7,8]. Vaccine hesitancy is not new, since it has existed for several decades in many countries, presenting a serious concern for global public health communities [9]. Italy registered an alarming collapse in immunization coverage from 2010 to 2014 and Italian public health authorities tried to counter this through a National Immunization Prevention Plan (2017–2019) and through a new law (n.119/2017) that increased the number of mandatory vaccinations [10], [11], [12]. The targets of the abovementioned plan were only partially reached. Furthermore, in Italy there was a significant drop both in the demand and supply of vaccines during the pandemic. It is thus questionable whether the COVID-19 pandemic would have been able to fix the problem of vaccine hesitancy in Italy, as hypothesised by Harrison and Wu [3].
Gerussi and colleagues [13] conducted a study to assess the attitude towards COVID-19 vaccines among COVID-19 patients admitted to hospital in northern Italy. The results showed that more than half of the respondents in the selected cohort were hesitant or undecided towards COVID-19 vaccines (59.2%). A lower vaccine hesitancy was found among Italian students [10]. Indeed, about 20% of the students reported a low intention to vaccinate. Such results are in line with a survey involving 12,322 residents in the entire country [14], which revealed that 17.6% of the respondents were unwilling to be vaccinated. It also revealed that receiving clear information on COVID-19 vaccination could increase citizens’ propensity to receive a vaccine [14]. Similarly, it appears that people with high/medium frequency of access to reliable information are more willing to undergo vaccinations [15].
In line with this evidence, the European center for Disease Prevention and Control (ECDC) stated that good communication leads to significant results in promoting vaccine acceptance [16]. Bell and colleagues showed that clear communication and transparency on how COVID-19 vaccines were developed and tested, as well as on vaccine safety and efficacy, are key to alleviating COVID-19 vaccine hesitancy [17]. In order to increase access to health services, health information should be designed in such a way that is easily understandable and readable for all the citizens, especially for the less educated [18], who are historically considered the most hesitant about vaccination [19]. Lower education levels emerged as predictors of higher COVID-19 vaccine hesitancy in the United Kingdom [20] and in the United states [21]. The level of education also seems to affect the propensity of Italian citizens to receive COVID-19 vaccination [14,22]. Considering that individuals with lower levels of education are more likely to use the Internet as a source for health information [23], the internet represents a key channel for informing citizens about COVID-19 vaccination and could play a significant role in overcoming vaccine hesitancy.
2. Materials and methods
2.1. Study setting
The Italian National Health System (NHS) is a universal decentralized Beveridge system, financed by general taxation. It covers nineteen regions and two autonomous provinces (APs) [24]. The healthcare system is organized and administered at three levels: national, regional, and local. The national government is responsible for the identification of core health benefits that should be equally granted across the country and distributes funds to the regions. Regional governments supervise, organize and supply healthcare services jointly with preventive and health promotion services (e.g., vaccination services) [25]. The local level guarantees the delivery of primary, secondary, and tertiary healthcare services, together with preventive and health promotion services.
The Italian central government has designed and published national recommendations for COVID-19 vaccination [26,27]. These guidelines detail the order of the availability of the COVID-19 vaccinations according to the different priority categories. Further information is shown in Table 1 . The regions adopted national guidelines within their regional COVID-19 vaccination plans.
Table 1.
Recommendations on COVID-19 Vaccination target groups by the Italian Government. The Plan was adopted through decree-law on 12 March 2021.
| COVID-19 Vaccination Interim recommendations on COVID-19 Vaccination target groups |
|---|
This table reports the priority categories exactly as listed in the National Strategic Plan of Italy for the supply of the COVID-19 Vaccine. The Plan was designed by the Ministry of Health, Special Commissioner for Emergency, ISS (National Institute of Health), AGENAS (National Agency for Regional Health Services) and AIFA (Italian Medicines Agency), according to two main risk factors (age and pathological conditions).STEP 1: Priority target groups
|
Table 1. Recommendations on COVID-19 vaccination target groups by the Italian Government.
This study focuses on the analysis of COVID-19 vaccination information gathered from the regional websites and local Health Authorities (LHAs). The list of websites was taken from the Ministry of Health [28]. A total of 108 LHA and 21 regional websites were analyzed.
The aim of this analysis was to explore four dimensions related to the online communication of Italian health institutions: availability, quantity and quality, usability, and readability of information.
From a citizen's perspective, we thus analysed whether correct information on the COVID-19 vaccination was present on institutional websites, whether it was easy to find, whether it was possible to make a vaccine reservation through this information, and how readable this information was. We also focused on the variability in the policy approaches of the Italian institutions.
2.2. Operationalization and common grid
According to Juran and colleagues [29], the quality of information refers to the degree of the information's usefulness or the “fitness for use” in a particular context. In our study, this degree of usefulness was defined according to the Ministry of Health recommendations on COVID-19 Vaccination, which indicated target groups and priorities (Box 1). In addition, information on where and how to get vaccinated was included in this category (see Appendix).
The definition for actionability given by Shoemaker and Wolf [30] refers to the extent to which individuals from diverse backgrounds and with different levels of health literacy can understand what they can do based on the information presented. We thus focused on the information on how to make a reservation and on the booking procedures themselves (i.e., online, via mobile app) for a COVID-19 vaccine.
The selected websites were analyzed through a common grid [See Table 3, Attachment 1.], based on our previous studies [25,31] and adapted according to the Italian guidelines for COVID-19 vaccination [26,27]. The grid was populated by thirty variables, grouped into three macro areas, according to the online communication dimensions of interest. In the first macro area, the availability of information was explored to understand whether users can easily find the information about vaccinations. The quantity and quality of information were investigated in the second macro area. The focus was on practical information for citizens (i.e., who can access vaccination, when they can get vaccinated, where the vaccination hubs are located, and information on side effects). In the third macro area the focus was on the actionability of the information. We also investigated how the vaccination was organized at regional and local levels for the different categories of citizens. Vaccination booking procedures for citizens over 80 years old were mapped and analyzed, since this group was the main target of COVID-19 vaccination at the time of the study.
The websites were analyzed by three researchers in our group. Any disagreement that occurred during data collection was debated by all the authors until a consensus was reached.
2.3. Readability analysis
Texts concerning COVID-19 were collected throughout the study period. Furthermore, COVID-19 vaccination information found on national institutional websites (e.g., Ministry of Health, Italian National Institute of Health, and Italian Drug Agency) were collected since most LHAs and regional websites redirected users there.
A readability analysis was conducted to investigate the quality and linguistic complexity of the texts, aimed at understanding if websites were easy versus difficult-to-read for citizens. Two well-established indexes were adopted to assess the text quality; the Gulpease index [32] and the READ-IT index [33]. In addition, the New Basic Italian Vocabulary (NBIV) was used to assess the lexical complexity of the gathered texts [34].
The Gulpease Index reaches a score of 100 when the text is easy to read, while as the text gets more difficult the score decreases. An index score above 80 signals that the text is easily readable by less educated people (primary school), while a score below 40 suggests that the text is comprehensible only to medium-high educated people (high school).
The READ-IT index relies on sophisticated Natural Language Processing (NLP) technologies, that screen a wide range of linguistic factors affecting the readability of a text [35], [36], [37]. READ-IT provides an evaluation of the level of difficulty of a text and ranges between 0 and 100: the more difficult the readability, the higher the score.
The percentage of NBIV words is calculated as a percentage of adopted words from the NBIV which includes the most familiar Italian words. When the percentage of NBIV words is higher than 80%, the text is regarded as easy to understand.
2.4. Data analysis
Excel (Microsoft Inc., Redmond, Washington, USA) was used to collect information according to the above-mentioned common grid (see Appendix 1). Descriptive and inferential statistics (analysis of variance, t-test, chi2) were performed using STATA15. Specifically, variability analysis among regions and geographical areas in Italy (North, center, South and Islands) were carried out.
As for the readability analysis, the READ-IT index was calculated through READ–IT, a classifier based on Support Vector Machines using LIBSVM (Library for Support Vector Machines) [38]. This tool, given the set of features described above and a training corpus, creates a statistical model using the feature statistics extracted from the training corpus. All readability analyses were carried out using a corpus of texts linguistically annotated using LinguA [33,39]
3. Results
3.1. Availability of information
The analysis showed that 76% of regional websites and 72% of LHA websites did not include a linkable call to action (CTA) to the COVID19 vaccination. In contrast, Tuscany's regional website [40] showed a button on the Home Page to invite users to discover more about vaccination.
A total of 57% of the regional websites published information to explain the characteristics of COVID-19 . More than a third of LHA websites did not provide citizens with COVID-19 information at all. Furthermore, a statistically significant difference was found both among regions (p = 0.000) and among areas (p = 0.001), with northern and central regions (such as, Veneto and Emilia Romagna) generally providing more accurate information, while southern regions (such as, Calabria) and the islands (such as Sardinia) presented quantitively and qualitatively poorer information. Thus, the further south the region, the lower the availability of COVID19 information on the institutional websites.
The researchers also tested the availability of COVID-19 vaccination information by inserting the following query in the websites’ search bar: ‘vaccine(s)’ to find out results related to the Coronavirus vaccine. A total 28% of regional and 13% of LHA websites did not show results, while most of the websites reported substantial findings in the primary section of the search result page (66% of regional websites, 57% of LHAs ones).
3.2. Quantity and quality of information
High variability emerged from the analysis of LHA websites regarding the presence of information on the category of citizens that had access to a vaccination. A total of 46% of the websites reported extensive information for all categories, while 42% mentioned information only on a cluster of citizens’ categories.
Fifteen out of the 21 regional websites provided citizens with comprehensive information on the different steps in the vaccination procedure, specifying the exact date when different citizens’ categories could obtain a vaccine (e.g., Veneto[41]). The remaining regional websites did not report information or reported it partially, relating only to the users that could obtain a vaccine in the period of the analysis (e.g., Campania [42]). LHA’ websites showed homogeneous results on the time-period when certain citizen groups could obtain their vaccine.
As for the place of vaccination, 57% of the regional websites provided wide-ranging information on the location of vaccination hubs. For example, Friuli Venezia Giulia [43] created an easy-to-read infographic, providing its citizens with detailed information on the various vaccination hubs in the area, using geolocation. However, 9% indicated only the type of facility where the service was provided, without any other information (e.g., Marche [44]), while 34% reported no information.
Similarly, most of the LHA websites (44%) provided citizens with detailed information on where they could get the vaccination, while 37% did not present any information, and 17% reported it partially. Information on the different booking methods for COVID-19 vaccination were also checked. A total of 17 out of 21 regional websites (81%) dedicated a section of their website to inform citizens on the booking methods (e.g., Lazio [45]). In contrast, several regional websites reported no (9.5%) or limited (9.5%) information on how to book vaccinations.
Comprehensive information on different booking methods for COVID-19 vaccination was found in more than half (54%) of the LHA websites, while about 17% reported it partially.
A statistically significant difference emerged both among regions and LHAs in terms of information on user categories, time and place for vaccination and booking methods, with the northern and central regions generally performing better (p = 0.001).
High variability was observed in relation to the presence of information on the adverse effects of the different types of COVID-19 vaccines. Fig. 1 highlights that most of the regional and LHA websites did not give information on possible side effects. About 57% of LHA websites did not provide any kind of information, 10% reported partial information, with an ‘Information Note’ on the different vaccines, without a specific section with a detailed explanation of side effects.
Fig. 1.
The left-side map shows regional websites’ results regarding the information on COVID-19 vaccination side effects. The right-side bar chart shows LHAs websites’ results on side effects per region.
Fig. 1. The map shows the results of regional websites regarding information on the side effects of Covid. The right-side bar chart shows LHA website results on side effects per region.
3.3. The actionability of information
Search queries regarding information on COVID-19 booking procedures were performed on regional websites since the vaccine booking procedures are established by the regional authorities and then adopted by LHAs. The different booking procedures adopted by the Italian regions are available in the Appendix. The analysis focused on booking procedures for citizens over 80 years old, since these citizens were the main target of COVID-19 vaccination during the study period. The analysis showed that regions adopted different strategies.
We found that 19% of regions (e.g., Veneto) adopted a top-down approach, meaning that the Regional Health System telephoned the citizens to give them an appointment date. On the other hand, most of the regions (e.g., Apulia) chose a bottom-up approach, thus requiring citizens to make their own reservation via multiple platforms (e.g., regional booking portal, tollfree number).
3.4. Readability analysis
Texts concerning COVID-19 vaccination collected across websites were found to be difficult to read on average (Fig. 2 ). Overall, the Gulpease index ranged from a minimum value of 41.57 and a maximum of 81.44, with a mean value of 52. Regional website texts scored between 67.9 and 44.7, signaling that those texts are particularly difficult to read by less educated people. Similarly, the regional means of the LHA websites scored between 60.6 and 46.3, being easily comprehensible only by medium-high educated people. The same occurred for texts collected on national institutional websites.
Fig. 2.
GulpEase indexes for COVID-19 vaccination texts of regional, LHAs’ and institutional websites. LHAs’ text results are reported as a mean of the scores of the LHAs’ populating the region.
A statistically significant difference among regions (p = 0.000) was found, with the southern regions generally performing worse.
Fig. 2. Gulpease indexes for COVID-19 vaccination texts of regional, LHA and institutional websites. LHAs text results are reported as a mean of the scores of the LHAs in the region.
The analysis also showed that most of the texts published online by the regions with a higher percentage of low educated people [46] were difficult to read for people with a primary school diploma according to the Gulpease Index [32] (see Table 2 ).
Table 2.
Gulpease Index (Regional websites) in relation to the percentage of people aged 25 to 64 with primary school diploma or no title.
| REGION | GULPEASE INDEX | % OF LOW EDUCATED PEOPLE |
|---|---|---|
| ABRUZZO | 67 | 3,6 |
| AOSTA VALLEY | 53,3 | 3 |
| APULIA | 53,2 | 9,5 |
| BASILICATA | N/A | 5,6 |
| BOLZANO | 58,7 | 2,4 |
| CALABRIA | 44,7 | 10,5 |
| CAMPANIA | 47,3 | 9,9 |
| EMILIA ROMAGNA | 61,2 | 2,7 |
| FRIULI VENEZIA GIULIA | 49,4 | 2,3 |
| LAZIO | 52,7 | 3,1 |
| LIGURIA | 67,9 | 2,6 |
| LOMBARDY | 53,2 | 2,8 |
| MARCHE | 46,9 | 3,4 |
| MOLISE | 48,5 | 4,3 |
| PIEDMONT | 47,1 | 3,4 |
| SARDINIA | 50,4 | 5,6 |
| SICILY | 52 | 8,4 |
| TRENTO | 55,1 | 1,7 |
| TUSCANY | 54,9 | 3,8 |
| UMBRIA | 50,9 | 3 |
| VENETO | 48,7 | 2,6 |
Table 2. Gulpease Index (Regional websites) in relation to the percentage of people aged 25 to 64 with primary school diploma or no school leaving certificate at all.
The mean value of the READ-IT index was 88.86. Regional website text scores ranged from 55.4 to 99.9, while the regional means of the LHA texts ranged from 53.8 to 99.9 with a higher variability across territories (see Fig. 3 - Appendix 2). These findings show that the analyzed texts present complex linguistic characteristics, such as multiple subclauses, complex verbal predicate structures, non-canonical orders of sentence constituents, and embedded sequences of subordinate clauses.
Fig. 3.
Global READ-IT indexes for COVID-19 vaccination texts of regional, LHAs’ and institutional websites. LHAs’ text results are reported as a mean of the scores of the LHAs’ populating the region.
The mean percentage of words in the NBIV was 66.01%. The use of NBIV words in the COVID-19 vaccination regional website text ranges from 73.8% to 51.4%. LHA texts concerning COVID-19 vaccination showed a slightly better performance, where the regional means were between 74.6% and 57.4%. However none of the analyzed texts contained only words belonging to the basic vocabulary. Similar scores were reached by National institutional websites, ranging from 74% of the Italian National Institute of Health (ISS) to 61% of the Italian National Agency for Regional Healthcare Services (AGENAS) (see Fig. 4 - Appendix 2).
Fig. 4.
NBIV words percentage (in terms of different lemmas) for COVID-19 vaccination texts of regional LHAs’ and institutional websites. LHAs’ text results are reported as a mean of the scores of the LHAs’ populating the region.
4. Discussion
As Italian citizens consider whether or not to receive COVID-19 vaccination, it is important to understand how the related information is available and how accessible it is for them. This study provides key findings on how to perform online health communication as clearly as possible in emergency settings, such as the COVID19 pandemic. We analysed key factors such as quality of content and readability since the topic was new and the easiness-to-read is even more important in this situation.
An increasing number of people choose the internet when looking for health information, even more so for a new and unknown vaccine such as the COVID-19 one [14]. The current study investigated the availability, quantity, quality, actionability and readability of COVID-19 vaccine information on 129 institutional Italian websites in Italy. The readability analysis was also conducted on national institutional websites (Ministry of Health, ISS, AGENAS and Italian Drug Agency).
Significant variability in COVID-19 information disclosure was observed across the analyzed websites. Previous research on online waiting time information disclosure as well as on health digital services provided evidence of significant variability both within and between region/LHA websites in Italy [25,47].
Our findings have concrete implications with regards to how to disclose COVID-19 information online. As concerns the availability of information, few websites presented a linkable call to action (CTA) in the Home Page. In fact, CTAs are the gateways to visitor action on a website. Inbound marketers firmly believe that CTAs are key to increasing user traffic and exploration [48], [49], [50]. Additionally, most of the websites analyzed reported COVID-19 vaccination information in the primary section of the search result page. However, a limited percentage of regional and LHAs websites still did not provide citizens with substantial results. Search bars are the most intuitive way to search for information through websites, therefore, a poorly functioning search bar could represent a barrier to accessing the information [51]. CTAs and search tool bars are two key aspects of online communication and efforts are needed to improve these tools.
As for the quantity and quality of information, almost half of the LHA websites did not give detailed information on COVID-19 vaccination sites. Providing citizens with partial or no information could represent a barrier to accessing health services [14]. We suggest that LHAs should provide detailed information on the citizens’ categories, time, and place for the COVID-19 vaccination on their websites, since complete information fosters healthcare access by making the set of choices transparent to the citizens [18].
The study revealed that few websites reported thorough information on the side effects of COVID-19 vaccination. Indeed, transparent communication of COVID-19 vaccine side effects is fundamental to increase the vaccination rate [52,53]. In fact, international organizations have highlighted the importance of good communication on the side effects related to COVID-19 vaccination to reduce vaccine hesitancy [16,17,50]. Furthermore, studies in Italy have revealed that receiving clear information on COVID-19 vaccines could increase the propensity of Italian citizens to get vaccinated [14,22]. Additionally, a clear communication of side effects is recognized as a way to alleviate COVID-19 vaccine hesitancy [17,54]. About 20% of Italian citizens are not willing to be vaccinated and these percentages were reported as being higher in a study conducted to assess the attitude towards COVID-19 vaccine among COVID-19 patients admitted to hospital in a city in northern Italy [10,14]. More than half of the respondents in the selected cohort were hesitant or undecided about COVID-19 (59.2%) vaccines [13]. The same studies revealed that receiving clear information on COVID-19 vaccine could increase citizens’ propensity to obtain vaccination [14,22]. In the light of these findings, institutional websites should provide in-depth information on COVID-19 side effects, instead of just providing the citizens with an informed consent form, which is a very technical and difficult-to-read document.
The actionability of information focuses on COVID-19 vaccine booking procedures. Indeed, clear information on booking methods is fundamental to provide citizens with the instructions to gain access to vaccination services. The analysis showed that most regional and LHA websites reported extensive information on vaccine booking procedures. Differences emerged in the procedures that citizens needed to follow to book vaccination appointments, specifically for citizens over 80 years old. Most of the regions adopted a bottom-up approach, requiring citizens to make their own reservation, while only a few used a top-down hierarchical system despite the literature suggesting this method as the best for reaching high coverage vaccination levels [55]. However, COVID-19 vaccination rates for over 80-year-olds in Italy showed different trends in regions with the same booking approaches. Indeed, by adopting a top-down approach, the Veneto region immediately registered a wide vaccination coverage, while Tuscany lagged far behind for a while. In contrast, regions adopting bottom-up approaches (e.g., Lazio, Emilia Romagna) promptly reached high coverage levels.
Furthermore, COVID-19 represents a significant opportunity to redesign vaccination booking procedures and services, considering that online booking is increasingly popular and well-known for multiple services (e.g., holidays, travel, outpatient visits) [56]. Additional studies are required in order to assess the performance of different regional booking systems and organization with regard specifically to COVID-19 vaccination coverage.
The readability analysis showed that online information on COVID-19 vaccination is difficult to understand for citizens with low and medium levels of education as texts are characterized by difficult words and sentences. These results are similar to those observed in previous studies on waiting time information texts published on public healthcare organization websites [25,31]. Beyond this linguistic complexity, the negative readability results could be partially explained by the absence of a healthcare glossary for measuring the text readability (i.e., specific terms, most recurrent words), thus overestimating the difficulty of texts at the grammatical level [57]. Healthcare institutions at the various levels should cooperate with all the major healthcare stakeholders (e.g., professionals, patients) to create a basic healthcare vocabulary composed of specific terms that fall within the cultural and linguistic knowledge of the average citizen.
With a few exceptions (e.g., Bolzano), readability emerged as a topic worthy of attention since low readability levels could raise equity issues in terms of understandability and use of online information. Providing exhaustive and easy-to-read information could represent the keystone for reducing reluctancy and increase acceptability to vaccination among citizens and specifically for those with a lower level of education. Those citizens are indeed the most likely to search for healthcare information on the internet [23] and to be hesitant towards vaccination [14,22], but experience issues processing information and consequently have a greater difficulty in accessing healthcare services. For this reason, we expected that the readability of COVID-19 vaccine online information to be particularly high in those regions populated by higher percentages of lower educated people. However, the analysis showed different results. Difficulties in understanding and using online vaccination information for this population group is notable, since citizens with the lowest education levels were observed as the most reluctant to be vaccinated [14,20,21]. This phenomenon could represent a barrier for the country in reaching full COVID-19 vaccination coverage. L'avete già detto diverse volte
In a context of full and accessible online information for all users, the COVID-19 pandemic could represent an opportunity to reach high vaccination coverage, as hypothesized by Harrison and Wu [3].
Given these findings, access to vaccination services should be given comprehensive coverage. This includes communication and considering the characteristics, needs and preferences of different groups of users [58]. National, regional, and local institutions should revise COVID-19 texts to improve their readability. Texts should be simplified, with shortened sentences, using familiar words and bullet points. The same analytical tool used for the readability analysis is also available online, in a demo version1 .
Using this online tool, the above-mentioned editing/screening process could be implemented enabling texts to be analysed, sentence by sentence. Furthermore, it identifies problems with the structure of the text in real time, as well as with the choice of words. The text can thus be immediately changed and re-evaluated using the same tool.
Overall, our findings reveal that the public disclosure of COVID-19 vaccination information is variable and fragmented in Italy. The risk is an increase in inequity in terms of information access.
Nevertheless, the results of this study do not support the hypothesis that the differences in online communication of the COVID-19 vaccination can be explained by the differences in geographical areas, and nor by the characteristics of the populations of each region (i.e., mean age). However, we might have expected information on COVID-19 vaccinations to be compiled and presented on the basis of the different composition of the regions’ population (i.e., in terms of age and level of education, which are predictors of vaccine hesitancy). For instance, regions with lower mean educational levels, or with relatively high numbers of non-Italian speaking citizens, or of a higher age, should make more effort to provide easy-to-access, easy-to-read and easy-to-use information. However, this was not the case.
This suggests that national, regional, and local institutions could cooperate with citizens to design a plan for COVID-19 vaccination online communication to guarantee citizens equal access to such information. A list of performance indicators on COVID-19 vaccination are already published at national level to observe regional differences in vaccination coverage across different target groups according to the national guidelines for COVID-19 vaccination [59]. However, specific performance indicators on COVID-19 vaccination information should be created as well as shared and endorsed by politicians, professionals, and managers. Publishing performance information could support healthcare systems in improving text quality and readability via benchmarking and reputation [60]. Furthermore, a process of collegial benchmark competition, which encourages the identification of best practices and continuous peer learning, should be adopted when performance information is publicly reported, specifically in the healthcare dimension since it has been proven to improve performance in Italy.
Our study presents some limitations and represents opportunities for future research. First, our study focused only on public institutional websites. However, several private accredited providers and public hospital institutions operate across the country and provide COVID-19 vaccination information via their online websites. Future research could analyze the online communication of the above-mentioned institutions, also including website traffic data. This information is not included in the current study due to the unavailability at the time.
Secondly, the results of the study represent the state of the art at a specific time. Websites are constantly evolving organisms, so it is important to keep the study up to date. In fact, there is a constant comparison between key stakeholders, and keeping this study up to date could lead to novel, integrated, and improved online information within short time periods. Furthermore, the Italian case clearly showed that COVID-19 information can change overnight (e.g., AstraZeneca vaccine case) [61]. However, we did not investigate possible correlations between information and vaccination rates. Future studies should investigate whether the quality, quantity, availability, and readability of information produce a positive effect in terms of outcomes.
Third, the analysis did not consider additional online sources of COVID-19 vaccination information, such as social media. Thus, the study is not fully representative of all online information sources, especially considering that healthcare organizations are increasingly using such channels for communicating with patients, and that social media have been widely used during the pandemic resulting in an emotional and behavioral response of populations [62].
Finally, the study focused on a single country, Italy. Further studies could replicate the analysis in other countries to check the state of the art in vaccination communication in other contexts and health organizations.
Furthermore, a comparative evaluation of the Italian practices aimed at identifying good practices was beyond the scope of this work. Nevertheless, further research could focus on the identification of good practices for the online disclosure of vaccination information in Italy. Several studies were identified that provided the framework for the identification of good practices [63,64].
5. Conclusions
Clear and extensive online communication of COVID-19 information is expected to foster vaccination uptake and reduce vaccine hesitancy. While citizens may be very interested in COVID-19 vaccination information, there are still some barriers that characterize Italian health communication strategies. Overall, the variability of information both across regions and within LHAs populating the same region (i.e., the regional website clearly states the places for vaccination, while LHAs’ operating in the same region do not; the regional/LHA website presents information on citizens eligible for vaccination, the time and place of vaccination, but gives no information on the possible side effects) and the low readability of COVID-19 vaccination online information are examples of these barriers. These findings suggest there is no national, regional or local coordination regarding COVID-19 vaccine communication. Furthermore, this study revealed that vaccination information and, consequently, the related services could have not been accessible for lower educated population groups in Italy. These findings could act as a driver to reduce some barriers for citizens in exploring, understanding, and using vaccination information for accessing the relative service.
CRediT authorship contribution statement
L. Pirrotta: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Writing – original draft, Project administration. E. Guidotti: Conceptualization, Methodology, Formal analysis, Investigation, Writing – original draft, Supervision, Project administration. C. Tramontani: Conceptualization, Methodology, Investigation. E. Bignardelli: Conceptualization, Visualization, Funding acquisition. G. Venturi: Formal analysis, Writing – original draft. S. De Rosis: Formal analysis, Writing – review & editing, Supervision.
Declaration of Competing Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
Acknowledgements
The authors acknowledge AGENAS for the constant support and useful comments, the AGENAS Director Domenico Mantoan, Professor Sabina Nuti and all the researchers from the Management and Healthcare (MeS) Laboratory, Institute of Management, Scuola Superiore Sant'Anna, for their continuous research inputs and precious insights. The authors would also like to thanks the Behavioural Insights Unit (BIU) of the MeS laboratory and all regional administrators and their staff for discussing the results.
Funding
This research was co-founded by the National Agency for Regional Health Services (AGENAS) and the Management and Healthcare Laboratory (MeS), Institute of Management, Scuola Superiore Sant'Anna.
Footnotes
http://www.ilc.cnr.it/dylanlab/apps/texttools/?tt_user=guest
Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.healthpol.2022.08.001.
Appendix. Supplementary materials
References
- 1.Ducharme J. Time; 2020. World Health Organization Declares COVID-19 a ‘Pandemic.’ Here's What That Means.https://time.com/5791661/who-coronavirus-pandemic-declaration/ Published Online First. [Google Scholar]
- 2.Angelico R., Trapani S., Manzia T.M., et al. The COVID-19 outbreak in Italy: initial implications for organ transplantation programs. Am J Transplant. 2020;20:1780–1784. doi: 10.1111/ajt.15904. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Harrison E.A., Wu J.W. Vaccine confidence in the time of COVID-19. Eur J Epidemiol. 2020;35:325–330. doi: 10.1007/s10654-020-00634-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kim Y.C., Dema B., Reyes-Sandoval A. COVID-19 vaccines: breaking record times to first-in-human trials. npj Vaccines. 2020;5:19–21. doi: 10.1038/s41541-020-0188-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.World Health Organization (WHO). Coronavirus disease (COVID-19): vaccines. 2020.https://www.who.int/news-room/q-a-detail/coronavirus-disease-(covid-19)-vaccines (accessed 4 Oct 2021).
- 6.Lazarus J.V., Ratzan S.C., Palayew A., et al. A global survey of potential acceptance of a COVID-19 vaccine. Nat Med. 2021;27:225–228. doi: 10.1038/s41591-020-1124-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.European Centre for Disease Prevention and Control. Vaccine hesitancy. https://www.ecdc.europa.eu/en/immunisation-vaccines/vaccine-hesitancy.
- 8.Pullan S., Dey M. Vaccine hesitancy and anti-vaccination in the time of COVID-19: a Google Trends analysis. Vaccine. 2021;39:1877–1881. doi: 10.1016/j.vaccine.2021.03.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.WHO. Ten threats to global health in 2019. 2019.
- 10.Barello S., Nania T., Dellafiore F., et al. Vaccine hesitancy’ among university students in Italy during the COVID-19 pandemic. Eur J Epidemiol. 2020;35:781–783. doi: 10.1007/s10654-020-00670-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Odone A., Bucci D., Croci R., et al. Vaccine hesitancy in COVID-19 times. An update from Italy before flu season starts. Acta Biomed. 2020;91:1–5. doi: 10.23750/abm.v91i3.10549. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Signorelli C., Guerra R., Siliquini R., et al. Italy's response to vaccine hesitancy: an innovative and cost effective National Immunization Plan based on scientific evidence. Vaccine. 2017;35:4057–4059. doi: 10.1016/j.vaccine.2017.06.011. [DOI] [PubMed] [Google Scholar]
- 13.Gerussi V., Peghin M., Palese A., et al. Vaccine Hesitancy among Italian Patients Recovered from COVID-19 Infection towards Influenza and Sars-Cov-2 Vaccination. Vaccines (Basel) 2021;9:172. doi: 10.3390/vaccines9020172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mantoan D., Nuti S., Cantarelli P., et al. Il vaccino e la vaccinazione contro il COVID-19 : la propensione della popolazione italiana ad aderire alla campagna vaccinale. 2021.
- 15.Caserotti M., Gavaruzzi T., Girardi P., et al. Who is likely to vacillate in their COVID-19 vaccination decision? Free-riding intention and post-positive reluctance. Prev Med (Baltim) 2022;154 doi: 10.1016/j.ypmed.2021.106885. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.European Centre for Disease Prevention and Control (ECDC). Catalogue of interventions addressing vaccine hesitancy. 2017. https://ecdc.europa.eu/sites/portal/files/documents/Catalogue-interventions-vaccine-hesitancy.pdf.
- 17.Bell S., Clarke R., Mounier-Jack S., et al. Parents’ and guardians’ views on the acceptability of a future COVID-19 vaccine: a multi-methods study in England. Vaccine. 2020;38:7789–7798. doi: 10.1016/j.vaccine.2020.10.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Thiede M. Information and access to health care: is there a role for trust? Soc Sci Med. 2005;61:1452–1462. doi: 10.1016/j.socscimed.2004.11.076. [DOI] [PubMed] [Google Scholar]
- 19.Tabacchi G., Costantino C., Napoli G., et al. Determinants of European parents’ decision onthe vaccination of their children against measles,mumps and rubella: a systematic review andmeta-analysis. Hum Vaccin Immunother. 2016;12:1909–1923. doi: 10.1080/21645515.2016.1151990. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Robertson E., Reeve K.S., Niedzwiedz C.L., et al. Predictors of COVID-19 vaccine hesitancy in the UK household longitudinal study. Brain Behav Immun. 2021;94:41–50. doi: 10.1016/j.bbi.2021.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Khubchandani J., Sharma S., Price J.H., et al. COVID-19 Vaccination Hesitancy in the United States: a Rapid National Assessment. J Community Health. 2021;46:270–277. doi: 10.1007/s10900-020-00958-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Il Sole 24 ore. Pandemia e vaccini: il ruolo della comunicazione. 2021.
- 23.Mesh G., Mano R., Tsamir J. Minority status and health information search: a test of the social diversification hypothesis. soc. 2012;75:854–858. doi: 10.1016/j.socscimed.2012.03.024. [DOI] [PubMed] [Google Scholar]
- 24.Tediosi F., Gabriele S., Longo F. Governing decentralization in health care under tough budget constraint: what can we learn from the Italian experience? Health Policy (New York) 2009;90:303–312. doi: 10.1016/j.healthpol.2008.10.012. [DOI] [PubMed] [Google Scholar]
- 25.De Rosis S., Guidotti E., Zuccarino S., et al. Waiting time information in the Italian NHS: a citizen perspective. Health Policy (New York) 2020;124:796–804. doi: 10.1016/j.healthpol.2020.05.012. [DOI] [PubMed] [Google Scholar]
- 26.Ministero della Salute. Vaccinazione anti-SARS-CoV-2/COVID-19; Raccomandazioni ad interim sui gruppi target della della vaccinazione anti-SARS-CoV-2/COVID-19. 2021.
- 27.Ministero della Salute . PIANO STRATEGICO; 2020. Vaccinazione anti-SARS-CoV-2/COVID-19. [Google Scholar]
- 28.Ministero della Salute. Elenco Aziende sanitarie locali e Strutture di ricovero. 2020.http://www.salute.gov.it/portale/documentazione/p6_2_8_1_1.jsp?id=13 (accessed 22 Apr 2021).
- 29.Juran J. The Free Press; New York: 1992. Juran on quality by design: the new steps for planning quality into goods and services. [Google Scholar]
- 30.Shoemaker S.J., Wolf M.S BC. Developement of the Patient Education Material Assessment Tool (PEMAT) Physiol Behav. 2018;176:139–148. doi: 10.1016/j.pec.2014.05.027.Development. [DOI] [Google Scholar]
- 31.De Rosis S., Ferre F., Guidotti E., et al. Dall'individuazione alla valorizzazione delle best practice. L'esperienza del sistema di valutazione dei servizi sanitari regionali a supporto della diffusione dell'eccellenza. 2019. Digitalizzazione nella prenotazione e pagamento delle visite specialistiche; pp. 175–182. [Google Scholar]
- 32.Pietro Lucisano e Maria Emanuela Piemontese. GULPEASE: una formula per la predizione della difficoltà dei testi in lingua italiana. 1988.
- 33.Orletta F.D., Montemagni S., Venturi G., et al. READ - IT : assessing Readability of Italian Texts with a View to Text Simplification. Comput Linguist. 2011:73–83. [Google Scholar]
- 34.De Mauro T. Nuovo vocabolario di base della lingua Italiana. 2016:1–18.www.dizionario.internazionale.it/nuovovocabolariodibase.
- 35.Dell'Orletta F., Montemagni S., Venturi G. Assessing document and sentence readability in less resourced languages and across textual genres. ITL - Int J Appl Linguist. 2014;165:163–193. doi: 10.1075/itl.165.2.03del. [DOI] [Google Scholar]
- 36.Brunato D., Venturi G. Le tecnologie linguistico-computazionali nella misura della leggibilità di testi giuridici. Inform e Dirit. 2014;XXIII:111–142. [Google Scholar]
- 37.Venturi G., Bellandi T., Dell'Orletta F., et al. NLP-Based Readability Assessment of Health-Related Texts: a Case Study on Italian Informed Consent Forms. 2015;:131–41. doi:10.18653/v1/w15-2618.
- 38.Chang C.C., Lin C.J. LIBSVM: a Library for support vector machines. ACM Trans Intell Syst Technol. 2011;2:1–39. doi: 10.1145/1961189.1961199. [DOI] [Google Scholar]
- 39.Attardi G., Informatica D., Orletta F.D., et al. Reverse revision and linear tree combination for dependency parsing. 2009;:261–4.
- 40.Regione Toscana. https://www.regione.toscana.it/home (accessed 20 Mar 2021).
- 41.Regione Veneto. https://www.regione.veneto.it/ (accessed 22 Mar 2021).
- 42.Regione Campania. https://www.regione.campania.it/ (accessed 22 Mar 2021).
- 43.Regione Friuli Venezia Giulia. http://www.regione.fvg.it/rafvg/cms/RAFVG/ (accessed 21 Mar 2021).
- 44.Regione Marche. https://www.regione.marche.it/ (accessed 20 Mar 2021).
- 45.Regione Lazio. http://www.regione.lazio.it/ (accessed 20 Mar 2021).
- 46.ISTAT. http://dati.istat.it/ (accessed 1 Jun 2021).
- 47.De Rosis S. The online communication of healthcare organisations in the ‘post-truth’ era: an analysis of 167 websites in Italy. Int J Integr Care. 2018;18:249. doi: 10.5334/ijic.s2249. [DOI] [Google Scholar]
- 48.Meyerson M. Entrepreneur Press; 2015. Success secrets of the online marketing superstars. [Google Scholar]
- 49.Goward C.You should test that: conversion optimization for more leads, sales and profit or the art and science of optimized marketing. 2012.
- 50.Kevin W. Sullivan. The Medical Marketer's Guide: success Strategies for Group Practice. 1990.
- 51.Wilson L. Emerald Publishing Limited; 2019. 30-Minute Search engine optimisation (SEO) actions. [Google Scholar]
- 52.Quinn S.C., Jamison A.M., Freimuth V. Communicating effectively about emergency use authorization and vaccines in the COVID-19 pandemic. Am J Public Health. 2021;111:355–358. doi: 10.2105/AJPH.2020.306036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Cohen A.F., van Gerven J., Burgos J.G., et al. COVID-19 vaccines: the importance of transparency and fact-based education. Br J Clin Pharmacol. 2020;86:2107–2110. doi: 10.1111/bcp.14581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Palm R., Bolsen T., Kingsland J.T. The Effect of Frames on COVID-19 Vaccine Hesitancy. medRxiv. 2021 doi: 10.1101/2021.01.04.21249241. 2021.01.04.21249241. [DOI] [Google Scholar]
- 55.Salmon D.A., Dudley M.Z., Glanz Jason M, et al. Vaccine Hesitancy: causes, Consequences, and a Call to Action. Am J Prev Med. 2015:49. doi: 10.1016/j.amepre.2015.06.009. [DOI] [PubMed] [Google Scholar]
- 56.Zhang X., Guo X., Lai K.H., et al. From offline healthcare to online health service: the role of offline healthcare satisfaction and habits. J Electron Commer Res. 2017;18:138–154. [Google Scholar]
- 57.Smith C.A., PatientsLikeMe Wicks PJ. Consumer health vocabulary as a folksonomy. AMIA Annu Symp Proc. 2008:682–686. [PMC free article] [PubMed] [Google Scholar]
- 58.Berry L.L. Service innovation is urgent in healthcare. AMS Rev. 2019;9:78–92. doi: 10.1007/s13162-019-00135-x. [DOI] [Google Scholar]
- 59.Laboratorio Management e Sanità. Sistemi di Valutazione della Performance. https://performance.santannapisa.it/pes/start/start.php.
- 60.Gusmano M.K., Helderman J.K. Frontiers of health policy research. Heal Econ Policy Law. 2019;14:135–140. doi: 10.1017/S1744133119000070. [DOI] [PubMed] [Google Scholar]
- 61.Il Sole 24 Ore. Vaccini, il caso AstraZeneca fa diminuire le somministrazioni: -17% nel fine settimana. https://www.ilsole24ore.com/art/vaccini-caso-astrazeneca-fa-diminuire-somministrazioni-17percento-fine-settimana-AE4dyHQ?refresh_ce=1.
- 62.De Rosis S., Lopreitea M., Puliga M., et al. The early weeks of the Italian Covid-19 outbreak: sentiment insights from a Twitter analysis. Health Policy (New York) 2021;Volume 125:987–994. doi: 10.1016/j.healthpol.2021.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Borghini A., Corazza I., Nuti S. Learning from excellence to improve healthcare services: the experience of the maternal and child care pathway. Int J Environ Res Public Health. 2021;18:1–10. doi: 10.3390/ijerph18041481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.De Rosis S., Guidotti E., Zuccarino S. Dall'individuazione alla valorizzazione delle best practice. L'esperienza del sistema di valutazione dei servizi sanitari regionali a supporto della diffusione dell'eccellenza. 2019. L'esperienza della regione Puglia; pp. 183–189. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.