Reliance on scientists and experts during an epidemic: Evidence from the COVID-19 outbreak in Italy

Abstract

Research suggests trust in experts and authorities are important correlates of compliance with public health measures during infectious disease outbreaks. Empirical evidence on the dynamics of reliance on scientists and public health authorities during the early phases of an epidemic outbreak is limited. We examine these processes during the COVID-19 outbreak in Italy by leveraging data from Twitter and two online surveys, including a survey experiment. We find that reliance on experts followed a curvilinear path. Both Twitter and survey data showed initial increases in information-seeking from expert sources in the three weeks after the detection of the first case. Consistent with these increases, knowledge about health information linked to COVID-19 and support for containment measures was widespread, and better knowledge was associated with stronger support for containment policies. Both knowledge and containment support were positively associated with trust in science and public health authorities. However, in the third week after the outbreak, we detected a slowdown in responsiveness to experts. These processes were corroborated with a survey experiment, which showed that those holding incorrect beliefs about COVID-19 gave no greater – or even lower – importance to information when its source was stated as coming from experts than when the source was unstated. Our results suggest weakened trust in public health authorities with prolonged exposure to the epidemic as a potential mechanism for this effect. Weakened responsiveness to expert sources may increase susceptibility to misinformation and our results call for efforts to sustain trust in adapting public health response.

Highlights

• •Reliance on scientists and experts initially increased during Italian COVID-19 outbreak.
• •Trust in science and health authorities predicted better knowledge about COVID-19.
• •Responsiveness to information from experts weakened over time.
• •Prolonged exposure to the epidemic weakened trust in public health authorities.

1. Introduction

Scientific research has brought multifarious benefits to people's daily lives, and public trust in science and in experts should be a natural extension of science's cultural achievements (Barber, 1990). Aligned with this, surveys show that people across the world report fairly high levels of trust in science and scientists (Gallup, 2019; Pew Research Center, 2019). Moreover, reliance on science and on experts is essential to the functioning of modern, highly differentiated societies where knowledge is specialized and complexity is constantly growing (Luhmann, 1979), and when individuals lack the knowledge to make decisions and evaluate risks associated with a hazard (Siegrist & Cvetkovich, 2000). These conditions are especially relevant in the context of an outbreak of a new infectious disease, such as that of COVID-19, as research shows that concepts linked to infectious diseases are often poorly understood by the public (Zingg & Siegrist, 2012) while the outcomes of recommended behavioural measures for disease control are not clearly nor immediately visible (Betsch, 2020; Redelmeier & Shafir, 2020).

Empirical research analysing the relationship between public health and public trust in the context of crises remains limited, especially at the onset of new infectious diseases. Existing literature indicates that variables linked to trust in health authorities and government institutions (e.g. national and local governments) are important correlates of citizens’ compliance with public health policies, restrictions and guidelines in the context of epidemics (Blair et al., 2017; Gilles et al., 2011; Siegrist & Zingg, 2014; Vinck et al., 2019). However, sustaining trust can be challenging in times of uncertainty and risk (Larson & Heymann, 2010; Van Bavel et al., 2020). For example, in the early days of the Ebola epidemic in Western Africa in 2013, the lack of trust in healthcare providers led affected families to hide sick family members (Larson, 2016). Based on these experiences, the World Health Organization (WHO) cited the lack of trust in the health system as a major driver of the failure of the containment of the later Ebola outbreak (World Health Organization, 2019).

Understanding how people rely on scientists and experts for information and their responsiveness to guidance from experts in the context of an unfolding epidemic is, therefore, crucial. Yet, little is known about how reliance and information-seeking, as well as public trust in experts, evolves during an epidemic, especially in its early stages (Siegrist & Zingg, 2014; Van Bavel et al., 2020). While on one hand a shared external threat may result in increased reliance on experts and trust in institutions, individuals may respond to an unknown threat with suspicion, skepticism and by developing conspiracy theories to explain its cause (Sibley et al., 2020; Siegrist & Zingg, 2014; Van Bavel et al., 2020). The theoretical predictions from existing literature are ambiguous and indicate the need for further empirical research.

This paper empirically examines the dynamics of reliance on scientists and health authorities in real-time in the weeks leading up to the first peak of the COVID-19 outbreak in Italy by leveraging complementary sources of digital trace data from the social media platform Twitter, and survey data including a survey experiment. First, using Twitter and rapid online survey data collected via the popular messaging app, Telegram, we examine how the willingness to consult experts and authoritative sources of information regarding COVID-19 evolved during the first weeks after initial disease outbreak in Italy. Second, using a separate, pre-registered survey experiment fielded to a geographically-targeted online pool recruited via Facebook in Northern Italy in areas that were affected early and most significantly by the virus, we analyse beliefs and misperceptions about health information linked to COVID-19. We assess if people's willingness to modify misperceptions with respect to the virus differed when the source of the same information was experimentally manipulated as coming from experts or not.

After the first detection of the disease in Wuhan, China in late December 2019, within three months COVID-19 – the causative pathogen of which was a coronavirus ultimately named SARS-CoV-2 – had spread to well over one hundred countries, resulting in the WHO declaring it a global pandemic on March 11, 2020 (Sohrabi et al., 2020). As of mid-September 2020, over 31 million confirmed cases of the virus and over 960,000 deaths globally had been reported to the WHO (World Health Organization, 2020). Italy was the first European country to experience a significant COVID-19 outbreak, with the detection of the first case on the February 21, 2020 in the province of Lodi in the region of Lombardy in northern Italy. While each province in Italy had confirmed cases of the virus by mid-March 2020, the diffusion of the outbreak in the country in the weeks leading up to the first peak was very heterogeneous with the majority of cases being concentrated in Lombardy in the north of the country (Odone, Delmonte, Scognamiglio, & Signorelli, 2020). Italian authorities implemented draconian measures including a nationwide ‘lockdown’ on March 10 to tackle the COVID-19 outbreak.1 As the pandemic continues to unfold globally, empirical evidence on how reliance on scientists and experts has evolved in the context of COVID-19, as well as on the impact of the disease on public trust in science and public health authorities, is needed. This study contributes to understanding these processes by drawing on the Italian context.

2. Background and hypotheses

Reliance and trust in experts and institutions is important in contexts where individuals lack the knowledge to make decisions and are unable to evaluate and understand the risks associated with a hazard (Siegrist & Cvetkovich, 2000). A public health crisis induced by the emergence of a new infectious disease such as the spread of COVID-19 provides such a setting as existing research indicates that concepts linked to infectious diseases (e.g. herd immunity) are often poorly understood by the public (Zingg & Siegrist, 2012). Furthermore in the absence of immediate pharmaceutical interventions, the management of infectious diseases requires significant behavioural change, even though the outcomes of such recommended behavioural measures are often not immediately nor easily perceived (Betsch, 2020; Van Bavel et al., 2020).

Although the above discussion suggests that trust in science, health experts and institutions are important for the management of pandemics, the theoretical predictions on how the dynamics of trust are affected during an infectious disease outbreak are ambiguous. The source model of group threat suggests that individuals may respond to an external threat by strengthening in-group ties (Greenaway & Cruwys, 2019). Following from this, we may expect greater trust and responsiveness to authorities, as well as social solidarity in response to a crisis (Sibley et al., 2020). On the other hand, individuals may feel the need to explain a large event with proportionally large causes, resulting in a response of suspicion, skepticism, or acceptance of conspiracy theories (Leman & Cinnirella, 2007). Individuals may be more drawn to conspiracy theories when their psychological needs are otherwise frustrated and they are more socially excluded, as might occur during lockdowns and with self-isolation measures (Van Bavel et al., 2020).

The empirical literature on the dynamics of trust in science and experts in the context of public health crises is scant. There is some evidence that crises can increase generalized trust and trust in authorities. In their study of the Ebola outbreak in West Africa in the three most affected countries of Guinea, Liberia, and Sierra Leon, Flückiger et al. (2019) found that trust in central government (parliament and president) and police increased in regions with higher exposure to the epidemic. Shupp et al. (2017) found that people who were affected by a tornado showed an increased level of general trust but also an increased level of trust in authorities and civil servants. Evidence on how long these changes in trust last however is mixed. For example, Calo-Blanco et al. (2017) showed that trust and social cohesion increased after a large-scale earthquake but slowly weakened as environmental conditions improved over time. In contrast, Aassve et al. (2020) found that the spread of the Spanish flu of 1918 is negatively and significantly correlated with generalized trust in the United States today.

The above-mentioned studies examine trust outcomes after the crisis; how these changes occur as these events are underway or shortly after they emerge is less well understood. An exception here is van der Weerd et al. (2011) who analysed the dynamics of trust in government during the H1N1 pandemic in the Netherlands and found that it decreased during the outbreak. The reasons recorded in their survey for weakening trust changed during the course of the pandemic, with the most reported reason being the perception that information was incomplete or withheld, to a belief in the later stage that the threat had been exaggerated by the government. Heterogeneity in the intensity of the outbreak may also affect dynamics of trust as suggested by Fong and Chang (2011) who found that trust was not a significant predictor of community actions during the SARS outbreak in Taiwan where larger outbreaks occurred. These findings suggest that a perceived inability to control an outbreak may erode trust in authorities.

In light of the above, we hypothesize that, on average, attention to scientific sources of information increases as a first reaction to the outbreak through an unconditional reliance on experts when facing a new threat. However, this effect may begin to decrease over time, following a reversed U-shape curve, as the epidemic continues to spread, and as frustration against experts as well as authorities sets in who are perceived as unable to stop the diffusion of the disease. At the individual level, we expect trust in science and public health authorities to be positively correlated with better knowledge about health information, as well as acceptance of containment policies.

3. Data and Methods

Quantitative, rapid data collection in the context of newly emerging infectious disease outbreaks is challenging, when face-to-face contact is restricted and changes likely to occur in short time spans (Blair et al., 2017; Geldsetzer, 2020). We employed multiple and complementary approaches of online data collection, including digital trace and survey data, to collect evidence on real-time dynamics of reliance on experts focusing particularly on scientists and public health authorities in the context of the first wave of the COVID-19 outbreak in Italy. Through social media engagement on Twitter, and through an online survey fielded on Telegram that asked respondents about information-seeking behaviours from different sources, we draw on two complementary measures of public engagement and information-seeking behaviours about COVID-19 in the six weeks following initial detection of cases in Italy. Through a second survey with an embedded survey experiment fielded in the second-half of March 2020, we analyse the relationship between trust in science, health knowledge about COVID-19 and support for COVID-19 containment measures. Our survey experiment examines responsiveness to expert information in modifying incorrect beliefs linked to COVID-19. In what follows, we provide information on each of these modes of data collection.

3.1. Twitter analysis

Twitter has been a widely-used medium for information-dissemination from scientific and health authorities during the COVID-19 pandemic across the world (Pollett & Rivers, 2020). We used the Twitter API to obtain a corpus of tweets that would be informative of the debate on COVID-19 in Italy during the first weeks of the Italian epidemic. To generate such a corpus, we started by selecting the most representative hashtags for the period between February 26 and April 15, 2020. Chen et al. (2020) find that #coronavirus is by far the most used hashtag for discussions on the pandemic globally; given our focus on Italy, we looked for hashtags specifically related to the Italian discussion. Hence, among the ten hashtags most often co-occurring with #coronavirus, we picked the two explicitly related to COVID-19 and to Italy: #coronavirusitalia and #covid19italia.2

After retrieving all geolocated tweets within Italy containing the above-mentioned hashtags, we identified most mentioned users in this corpus, and classified such users into different categories (e.g. scientists, health authorities, media, politicians as shown in Table 1).3 We allocated these categories as they were broadly representative of the different institutions and organisations involved in the discussion around COVID-19, and allowed us to compare dynamics of interest in experts with others who were also active in these discussions. These categories also broadly concur with the categories used in the Telegram survey that we fielded (described in the next section) in which we asked respondents who they sought information from in relation to COVID-19.

Table 1.

Summary statistics of retweets from Italian COVID-19 Twitter sample.

	Mean	Std. dev.	Min.	Max.	N
Authorities	62.75	403.74	0	10313	1169
Health authorities	298.29	1727.90	0	57753	2905
Journalists	67.40	181.29	0	2145	1153
Media	1.74	1.74	0	17	614
Other	237.18	425.06	0	5654	427
Politicians	292.74	454.15	0	7268	1565
Scientists	135.75	462.29	0	7864	1649
Total	191.63	1025.13	0	57753	9268

Note: first four columns: distribution of retweets at the tweet level, disaggregated by the category of the account which produced the tweet; last column: number of tweets considered.

We then downloaded all tweets posted by the three most mentioned accounts in each of the different categories in the period of time considered.4 In our analyses, we were interested particularly on retweets of the tweets within two categories of experts – scientists and health authorities.5 We focus on retweets as it is a reasonable measure of engagement, and retweets can generally be interpreted to signal agreement/endorsement with content. We note that however some of these retweets may have been retweets with comments, where the behaviour may either be endorsement or disagreement. Unfortunately we cannot distinguish between retweets with and without comments in a straightforward way. In any case, it is worth noting that disagreeing retweets might still contribute to a diffusion of the scientists’ tweeted content, possibly among readers who will agree with it. We also verified that the number of mentions received by each account in the time interval considered strongly correlated with both its number of followers later in the year as of November 13, 2020, and to the number of geolocalized tweets posted in Italian with the #coronavirus hashtag. These checks provide further support that the attention dynamics we captured were directed to users who were involved in the broader COVID-19 discussion in Italy, and not only those involved in its initial phases.

By focusing on three users per category we were able to focus on only the most popular accounts, all of which were from publicly-recognizable entities: this, together with a manual inspection, also safeguarded our analysis from the issue of automated accounts (or bots). This selection criterion resulted in 9268 tweets and a total of 2,032,772 retweets. The distribution of retweets is skewed, with 13 tweets being retweeted more than 8000 times, 12 of which were posted by the WHO. Summary statistics of retweets per category for the Twitter dataset are reported in Table 1.

Retweets are analysed by estimating Equation (1) for each category i using ordinary least squares (OLS) regression with standard errors robust to heteroskedasticity.

$$retweet\_coun{t}_i={\beta }_0+{\beta }_{1}time+{\beta }_{2}tim{e}^2+{\beta }_{3}total+{\epsilon }_i$$ (1)

where total controls for general Twitter interest in COVID-19 in Italy, measured as the total number of per-day retweets in our original (hashtag-defined) sample. We estimate Eq. (1) for each category with both a linear and a quadratic term to assess the attention dynamics over time that they received, net of general interest as captured by the total term.

3.2. Continuous survey via Telegram

Rapid response online surveys enable swift and timely measurement of public perceptions and responses in the context of fast-moving disease outbreak, and overcome low-response rates associated with telephone surveys and challenges associated with face-to-face data collection in epidemic contexts (Geldsetzer, 2020). We administered a rapid response online survey about information-seeking behaviour in four waves by sending an invitation in a popular Telegram channel, covid19, dedicated to the diffusion of news (in Italian) about the pandemic. Hence, participants to our survey had self-selected for receiving information on COVID-19. In the survey, we asked them about their desire (on a scale from 1 to 8) to receive information about the novel coronavirus from different sources, specifically: 1) doctors and scientists, 2) the government and local administration (authorities), 3) health authorities (e.g. WHO), and 4) celebrities (from show-business and sports). During the days elapsed between the first and the fourth wave, the channel grew from around 50,000 to around 70,000 members.

We obtained 9025 participant/wave combinations, which reduce to 8798 after dropping incomplete responses (see Table 2 for per-wave sample size). The exact wording of the questions asked during the 4 waves of the data collection via Telegram is reported in the Supplementary Information S6.1. The first three waves were conducted entirely within the Telegram app through a bot that interacted with participants asking the questions in sequence. In the fourth wave of the Telegram survey, in addition to the usual questions, an abbreviated form of the survey experiment (described in the next section) was run on the Telegram pool by directing users to a Google Form survey.

Table 2.

Summary statistics of Telegram sample.

	Count	%
Wave: 1	2103	23.9
Wave: 2	1164	13.2
Wave: 3	1597	18.2
Wave: 4	3934	44.7
Educ.: Prim./middle school	489	10.1
Educ.: Diploma	2549	52.4
Educ.: Degree	1501	30.9
Educ.: Master or higher	325	6.7
Age: below 14	40	0.5
Age: 14–29	6034	68.6
Age: 30–44	1722	19.6
Age: 45–64	963	10.9
Age: 65+	39	0.4
Gender: female	3196	36.3
Gender: male	5602	63.7

Note: Descriptive statistics (unweighted) for responses in the Telegram survey (N = 8,798). Education level was reported differently, and is not included, for wave 4. Further details are available in Supplementary Information S4.2. Notice that in the “Gender” question, responses with “prefer not to say” were automatically excluded because of the impossibility of correctly imputing post-stratification weights.

Table 2 shows the sample by demographic characteristics (see further information on the same in Supplementary Information S4.2). Respondents in the Telegram survey were predominantly young, more likely to be male, although the demographic composition of the sample widened across the waves to cover a wider age range and included more female participants. Data obtained from Telegram are analysed by OLS. To account for the demographic biases of our sample, we use post-stratification weights for age, gender and education to re-weight the sample to match the Italian population along these dimensions. Population data on age, gender and educational distributions were obtained from the Italian Census (ISTAT, 2020). We do this via an iterative proportional fitting (raking) procedure whereby the iterative process is repeated until the difference between the sample margins and the known population margins is smaller than a specified tolerance value (Fienberg et al., 1970): this was done via the Python package ipfn (Forthomme, 2017).

Fig. S3 in Supplementary Information S4.2 displays the density of responses in the Telegram survey across Italy based on the ZIP code of respondents. While each Italian region is covered, the responses are most frequent in Lombardy, followed by Veneto, the two regions that were most affected by the first wave of COVID-19 outbreak in Italy. Further details on the Italian outbreak and its mortality toll are shown in Fig. 1 and S11.

Fig. 1.

Sampling strategy: Regions and provinces in Northern Italy selected together with number of positive COVID-19 cases by province as for March 17, 2020

Note: Every point corresponds to a respondent. Data have been geolocalized based on respondents' ZIP codes. Province-level data on positive COVID-19 cases for Lombardy and Veneto are available at the following link: https://github.com/pcm-dpc/COVID-19. We specifically targeted two early outbreak areas including 10 municipalities (in red) in Lombardy and 1 in Veneto, as well as municipalities bordering with these initial outbreak municipalities for a total of 66 municipalities (46 in Lombardy and 20 in Veneto). We then separately targeted an area in the province of Bergamo (BG) which has been severely affected by the epidemic and the municipalities bordering this area for a total of 32 municipalities. . (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

3.3. Survey experiment

We recruited a geographically targeted online sample using the Facebook advertisement platform to field a rapid response survey with an embedded survey experiment. Our sample was geographically targeted to recruit respondents from 15 provinces in Lombardy6 and Veneto,7 the two regions in the north of the country that experienced the earliest and most significant outbreaks in Italy. This is clear from Fig. 1 showing province-level data on positive COVID-19 cases by March 17, 2020. More specifically, our sampling strategy included a detailed geographical targeting of some specific municipalities (in red in Fig. 1) within those regions (in the provinces of Lodi, Padova, and Bergamo).8

Examples of recruitment via Facebook for survey research have emerged in the health and social sciences as it enables swift and demographically diverse recruitment given Facebook's large user base that is broadly representative of the general population when used with appropriate post-stratification weights (Ramo & Prochaska, 2012; Zhang et al., 2018; Schneider & Harknett, 2019; Kalimeri et al., 2020; Grow et al., 2020). Facebook is the most widely used online social media platform in Italy, and over 60% of the Italian adult population are Facebook users.9 Respondents that opted in and clicked the advertisement were enrolled into our subject pool. To minimise topical self-selection (Lehdonvirta, Oksanen, Räsänen, & Grant, 2020) (e.g recruiting respondents with unusually greater interest in the coronavirus), our recruitment ad avoided any mentions to the content of our survey and only implied that survey respondents were sought for a subject pool for ongoing research projects in the social sciences. Subsequently, the survey link was sent out to all those who entered the subject pool through the ad, and was administered via the survey platform Qualtrics. The survey ran from March the 17th to March the 30th and we received a total of 994 completed responses. The exact wording of the questions administered is reported in the Supplementary Information S6.2. Our sample featured good geographical representation from the areas that we sought respondents. Nevertheless, we unwittingly recruited a sample of respondents even outside the initially targeted areas, as shown in Fig. S6 in Supplementary Information S4.3.

The survey questionnaire asked respondents about their socio-demographic characteristics, the perceived importance of containment measures that had been implemented by the Italian government, as well as questions about trust in science and the Italian Institute for Public Health. Furthermore, we investigated knowledge about health information linked to the coronavirus. This health information was based on widely available content on the website of Italian Institute for Public Health (Istituto Superiore di Sanità).10 The four questions (measured on a 0–10 point Likert scale with 10 indicating complete accuracy) we asked were: 1) Are younger people also at risk of contracting the coronavirus? 2) Are antibiotics helpful in preventing the new coronavirus infection? 3) Is it safe to receive parcels from China or other countries where the virus has been identified? 4) Is washing hands really useful in preventing the coronavirus infection?

Our treatment manipulation in the survey experiment proceeded in three steps. We first asked respondents for an answer (to one of the four questions above), and then exposed them to information on the same topic by using directly relevant quotations from the website of the Italian Institute for Public Health. When providing them this current information, we randomised the framing either (i) quoting the text as coming from public health experts, or (ii) providing the same statement without any source. Finally, we asked if the subject wanted to change their responses to their original answers once the new information was provided. This exogenous treatment manipulation allows us to answer the question of whether the information source changes the propensity to adjust the respondents’ beliefs in cases when initial beliefs were wrong. The exact wording of the treatment manipulation is reported in Supplementary Information S6.2. Participants in the online survey were offered a modest payment.

The data from the survey experiment were analysed using the following model:

$$Willin{g}_i=\alpha +\beta I_i+\gamma Z_i+{\epsilon }_i$$ (2)

Where $Willin{g}_i$ is a dummy indicating whether the respondent is willing to update their beliefs (when wrong), $I_i$ is a dummy variable indicating whether the source of information was indicated, and $Z_i$ are a set of control variables, including gender, age, educational attainment (secondary, bachelor, master and higher), marital status (married, cohabiting, divorced, widow), parental status, employment status (employed, retired, student, homemaker, other), mathematical skills, political self-placement on a left-right scale, two dummies for Lombardy and Veneto – the two most represented regions–, and latitude. The estimated models also include measures of trust in science and in the Institute for Public Health (on a scale of 0–100) as described in Table 3. A scatter plot for trust measures is provided in Fig. S10 in Supplementary information S4.3.

Table 3.

Summary statistics of variables used in the analysis of survey with embedded survey experiment (unweighted).

Variable	Mean	Std. Dev.	Min.	Max.	N
Support for containment measures: Dependent variable forTable 5
Imp. of social distancing: non-positive	9.23	1.509	0	10	985
Imp. of social isolation: positive	9.871	0.676	0	10	987
Imp. of social distancing: elderly	9.593	1.155	0	10	987
Willing to update (if wrong): Dependent variable for survey experiment (Fig. 6andTable S7)
Willing to update: Youths	0.129	0.335	0	1	319
Willing to update: Antibiotics	0.342	0.475	0	1	380
Willing to update: Parcels	0.168	0.374	0	1	507
Willing to update: Hands	0.09	0.286	0	1	357
Wrong beliefs: Dependent variable forTable 6
At least once wrong	0.818	0.386	0	1	994
Wrong: Youths	0.321	0.467	0	1	994
Wrong: Antibiotics	0.469	0.499	0	1	810
Wrong: Parcels	0.514	0.5	0	1	984
Wrong: Hands	0.36	0.48	0	1	990
Survey Experiment Manipulation: Treatment variable
Info: experts	0.494	0.5	0	1	995
Trust measures
Trust: National Inst. Pub. Health	85.933	16.299	0	100	985
Trust: science	93.36	10.378	12	100	987
Trust science (avg)	89.704	11.629	20.5	100	986
Covariates
Single	0.602	0.49	0	1	994
Married	0.2	0.4	0	1	994
Cohabitation	0.148	0.355	0	1	994
Divorced	0.047	0.212	0	1	994
Widow	0.003	0.055	0	1	994
Secondary	0.318	0.466	0	1	994
PhD	0.046	0.21	0	1	994
Bachelor	0.248	0.432	0	1	994
Master	0.359	0.48	0	1	994
Lower Secondary	0.028	0.166	0	1	994
Other	0.032	0.177	0	1	994
Homemaker	0.019	0.137	0	1	994
Unemployed	0.133	0.34	0	1	994
Employed	0.5	0.5	0	1	994
Retired	0.019	0.137	0	1	994
Student	0.297	0.457	0	1	994
Math skills	0.965	0.183	0	1	979
Has children	0.225	0.418	0	1	994
Gender (male = 1)	0.345	0.476	0	1	992
Age	32.87	11.616	18	74	993

Note: Coviariates pertain to the full sample. Survey experiment analysis on willingness to update beliefs when wrong was conducted on the sub-sample that answered questions incorrectly.

In addition to the survey experiment, we also analysed the relationship between the trust variables, knowledge of health information and support for containment measures collected in the survey (the results of this analysis are depicted in Table 5, Table 6). Table 3 shows the unweighted summary statistics for the variables used in the empirical analysis. Equation (2) was estimated via both OLS and Logit models (marginal effects), with robust standard errors clustered at the province-level.

Table 5.

Relationship between holding wrong beliefs, trust in science, and support for containment measures (weighted).

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
	Imp. of social distancing: non-positive			Imp. of social isolation: positive			Imp. of social distancing: elderly
	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se
At least once wrong	−0.513***	−0.519***	−0.487***	−0.099***	−0.096***	−0.090***	0.010	−0.014	0.014
	(0.118)	(0.108)	(0.115)	(0.024)	(0.020)	(0.022)	(0.126)	(0.118)	(0.121)
Trust: science	0.033***			0.007**			0.023***
	(0.006)			(0.002)			(0.004)
Trust: National Inst. Pub. Health		0.023***			0.005**			0.014*
		(0.004)			(0.002)			(0.005)
Trust science (avg)			0.032***			0.007***			0.021***
			(0.005)			(0.002)			(0.005)
Covariates	✓	✓	✓	✓	✓	✓	✓	✓	✓
Constant	0.253	0.713	−0.126	7.324***	7.350***	7.193***	8.483***	8.996***	8.282***
	(8.223)	(7.840)	(8.018)	(1.214)	(1.119)	(1.174)	(1.908)	(2.063)	(2.004)
N.	769	769	769	769	769	769	768	768	768
$R^2$	0.266	0.274	0.282	0.082	0.091	0.092	0.174	0.162	0.176

Note: Weighted OLS. Standard errors clustered at the province-level reported in parentheses. Covariates include: gender (male = 1), age, educational attainment (secondary, bachelor, master and higher), marital status (married, cohabiting, divorced, widow), parental status, employment status (homemaker, employed, retired, student, other), mathematical skills, self-placement of a left-right scale, a dummy for Lombardy, a dummy for Veneto, and Latitude. Table S5 in Appendix S4.5 depicts the coefficients for the covariates used in the model. + p < 0.10, *p < 0.05, *p < 0.01, *p < 0.001.

Table 6.

Relationship between trust and knowledge of health information linked to the coronavirus (weighted).

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)
	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se	b/se
	Wrong: Youths			Wrong: Antibiotics			Wrong: Parcels			Wrong: Hands
Trust: science	−0.008+			−0.011***			−0.007***			−0.007**
	(0.005)			(0.002)			(0.002)			(0.002)
Trust: National Inst. Pub. Health		−0.008***			−0.004*			−0.002+			−0.002
		(0.002)			(0.002)			(0.001)			(0.001)
Trust science (avg)			−0.013***			−0.008**			−0.005*			−0.005**
			(0.002)			(0.002)			(0.002)			(0.002)
Covariates	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Constant	2.895+	4.005***	4.279***	2.621*	2.224+	2.533*	1.210	0.960	1.197	4.822**	4.712**	5.057***
	(1.596)	(1.152)	(1.082)	(1.195)	(1.180)	(1.189)	(1.402)	(1.460)	(1.518)	(1.406)	(1.441)	(1.380)
N.	937	935	935	773	773	773	932	930	930	937	935	935
$R^2$	0.328	0.374	0.391	0.301	0.273	0.288	0.256	0.238	0.246	0.188	0.160	0.173

Weighted OLS. Standard errors clustered at the province-level reported in parentheses. Covariates include: gender (male = 1), age, educational attainment (secondary, bachelor, master and higher), marital status (married, cohabiting, divorced, widow), parental status, employment status (homemaker, employed, retired, student, other), mathematical skills, self-placement of a left-right scale, a dummy for Lombardy, a dummy for Veneto, and Latitude. Table S6 in Appendix S4.5 depicts the coefficients for the covariates used in the model. + p < 0.10, *p < 0.05, *p < 0.01, *p < 0.001.

Our sample was younger (median age in census = 45.8, median age in the Facebook pool = 29), had greater secondary-degree education (share of the population holding at least a secondary degree in the census = 65%, share of the population holding at least a secondary degree in the Facebook pool = 97%) and had a greater share of women (share of women in the census = 49%, share of women in the Facebook pool = 65%) compared with the Italian general population based on the Census (ISTAT, 2020).11 To account for these demographic biases, our analyses applied post-stratification weights by age, gender and education to conform our sample along these characteristics to the Italian population.12 As with the Telegram analyses, we computed weights via an iterative proportional fitting procedure (Fienberg et al., 1970) implemented in Stata through the command ipfweight (Bergmann, 2011).

4. Results

4.1. Reliance on scientists and experts during an epidemic

Using the Twitter dataset, we study how social media attention given to scientists and health authorities evolved between the end of February, when the first case of the virus was identified in Italy, until mid-April, when Italy was widely believed to have passed the (first) peak.13 We measured the average number of retweets for tweets by the three most mentioned scientists (Roberto Burioni, Ilaria Capua, Pierluigi Lopalco) and health authorities (Ministry of Health, Italian Red Cross and WHO) to assess how attention given to these sources for COVID-19 content changes over the period.

Fig. 2 and Table 4 show the results of OLS regressions as described in Section 3.1 in the Data and Methods section. Fig. 2 and columns 1–2 of Table 4 show that social media attention given to scientists and health authorities at first increased with the disease outbreak but then began to turn around in mid-March, shortly after the nationwide lockdown implemented by Italian authorities corresponding to the uptick in the number of cases and deaths as shown in Fig. S11 in Appendix S5. This analysis controls for the total number of retweets, i.e. the changes in total volumes of retweets within our sample over the period, as a proxy for general interest in information related to COVID-19. These results thus indicate a relative shift away from scientists and health authorities for information and content linked to COVID-19 by the end of the period (April 2020) rather than a generalized decline in COVID-19 interest (the total control variable is not statistically significant in Table 4). Furthermore, the curvilinear pattern of changes in social media attention does not emerge for media, politicians and other categories, see columns 4–6 of Table 4.

Fig. 2.

Evolution of retweets of scientists and health authorities during the COVID-19 outbreak in Italy.

Note: Marginal effects plot. OLS. Blue: Telegram survey waves. Red: Italian lockdown. Green: Survey Experiment. Robust standard errors. Covariates in the model include: time, time squared, total number of retweets. Results depicted in this figure are retrieved from Table 4. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Table 4.

Evolution of retweets of COVID-19 tweets from different categories of accounts over time during the initial weeks of the outbreak in Italy (February 26 to April 15, 2020).

	(1)	(2)	(3)	(4)	(5)	(6)
	Scientists	Authorities	Health Authorities	Media	Politicians	Other
Time	1.281+	4.616**	11.284***	−0.024	−1.111	0.538
	(0.738)	(1.639)	(3.054)	(0.187)	(1.716)	(2.695)
Time $\times$ Time	−0.033*	−0.092**	−0.258***	0.000	0.045	0.010
	(0.015)	(0.034)	(0.063)	(0.002)	(0.036)	(0.050)
Total	0.000	0.000	0.003	−0.000*	0.001	0.000
	(0.000)	(0.001)	(0.002)	(0.000)	(0.001)	(0.001)
Constant	26.707***	13.144	26.968	2.453	122.709***	60.160**
	(6.117)	(12.336)	(24.753)	(3.621)	(12.950)	(20.944)
N.	8480	2771	668	537	4846	2483
R2	0.002	0.003	0.047	0.009	0.002	0.001

OLS. Standard errors robust to heteroskedasticity reported in parentheses. + p < 0.10, *p < 0.05, *p < 0.01, *p < 0.001.

Our findings are also supported by a difference-in-differences approach, where daily averages of retweets across categories are subtracted from each tweet's retweets count (see Section S2 in the Supplementary Materials for details). This corroborates the observation that dynamics captured by Table 4 are indeed specific to individual categories and do not reflect general interest dynamics of the debate on the Italian pandemic.

Similar patterns are observed in the data we collected via an opt-in rapid response survey on the popular messaging app Telegram. The first wave of this survey took place on February the 27th, 6 days after the discovery of the first case in Italy. The other waves were conducted on March 5, March 13 (just after the lockdown in Italy and corresponding to the descending path in the scientists panel on Fig. 2), and March 20. Data from this survey allow us to directly track changes in self-reported preferences of individuals over time.

Fig. 3 and Table S3 in Supplementary Information S4.2 summarise the results. Already in the first wave of the survey at the end of February, we observe that interest in receiving information from scientists and health authorities was higher than from other sources (authorities and celebrities). For each of the three categories of scientists, government authorities, and health authorities, there was a sizeable and significant overall increase of interest over time. In contrast, a decreasing pattern emerged for celebrities. However, both categories of experts – scientists and health authorities – featured a U-shaped pattern in changes in information-seeking from them. Interest in these sources increased until the third wave of the survey but then started decreasing. Interest kept increasing instead for authorities, perhaps unsurprisingly given the importance that public bodies bear when emergency laws are being put in place.

Fig. 3.

Summary of longitudinal evidence from Telegram survey

Note: Y-axis refers to responses to questions about desiring information on COVID-19 from specific sources. Bottom: average response for each wave (notice the different scale in the celebrities plot); top: difference between each wave and the first, with 95% confidence intervals. Post-stratification weights are applied to each wave to conform to the Italian population along demographic characteristics (age, gender, and education). N = 8798.

An analysis of within-user variance of trust over time is not feasible due to the extreme unbalancedness of our panel: out of the 7695 participants in our survey, we obtained only 21 with complete responses from each of the four waves. Nevertheless, in Fig. S4 we show that the main dynamics of interest over time are robust to the changes in the sample composition, by comparing responses across subsequent waves for those respondents appearing in two consecutive waves (N = 779).

4.2. Public health knowledge, compliance and willingness to update wrong beliefs

The survey experiment was administered from March 17th to March 30th, i.e. corresponding to the part of Fig. 2 where retweets of scientists and health authorities is descending, and when a slowdown in information-seeking from scientists and health authorities also emerges in the Telegram survey between waves 3 and 4.

Three weeks after the identification of the first cases in Italy by the second-half of March 2020, public health messages on the importance of social distancing and the isolation of positive cases had been widely received by the public, and mean scores on containment support measures were high, as shown in Fig. 4.14 Knowledge about health information linked to the coronavirus was also generally widespread in our sample, although this varied for different types of information, as shown in Fig. 5. Using a score of 10 (on a scale of 0–10) for complete accuracy, around 50% of the respondents knew that younger people are also at risk of contracting the coronavirus, and more than 70% knew that washing hands is important in preventing the coronavirus infection. Knowledge about other, more technical questions was comparatively less accurate – just over 30% knew that antibiotics are not helpful in treating the infection, and around 30% knew that it is safe to receive parcels from countries where the virus had been identified.

Fig. 4.

Support for containment measures: distribution of responses to importance of social distancing and the isolation of positive cases questions (weighted)

Note: Higher values indicate greater support. Scale from 0 to 10.

Fig. 5.

Knowledge of coronavirus-related questions (weighted)

Note: Measured on a 0–10 scale with 10 indicating complete accuracy.

Those who were better informed about the coronavirus were also more supportive of containment policies for two out of three questions as indicated by the negative coefficients on the at least once wrong variable in Table 5. Trust in science and the Institute for Public Health showed statistically significant positive associations with support for containment measures, net of a range of socio-demographic controls, as shown in Table 5. Trust variables also showed positive and statistically significant associations with knowledge outcomes, as shown in Table 6. While the trust measures were consistently associated with both types of outcomes in our models, demographic covariates showed weak or inconsistent associations with these outcomes (see Tables S5 and S6). In a survey fielded at a similar time to ours, Barari et al. (2020) also found little socio-demographic heterogeneity in containment support measures during this period. This suggests that in the early phases of an epidemic outbreak in the face of a novel and unknown threat, psychological factors (e.g. trust in science and health authorities) may be better at explaining variation in these outcomes than socio-demographic factors (e.g. age, education).

The results of our exogenous treatment manipulation are depicted in Fig. 6. The bars represent marginal effect plots of the treatment effect for each one of the four questions that we asked respondents. Fig. 6 (and Table S7 in Supplementary Information S4.3) shows that those holding incorrect beliefs about the disease gave no different (for the question about antibiotics being effective in preventing the coronavirus infection and that on the importance of washing hands) or significantly lower (by 17 percentage points for the question about younger people also being at risk of contracting coronavirus and by 16 percentage points about receiving parcels from China or countries with an outbreak) importance to information when the source of such information was explicitly stated as coming from scientific experts.15 These results are also confirmed by a simplified version of the same survey experiment nested within the last wave of the Telegram survey (see Fig. S5).

Fig. 6.

Information and willingness to update wrong beliefs based on exposure to treatment

Note: OLS. Marginal effect plots of treatment effects (reference to experts in the information received). Buffers represent 95% confidence intervals. Standard errors clustered at the province-level. Covariates (as described in Table 3 include: gender, age, dummies for educational attainment (secondary, bachelor, master and higher), dummies for marital status (married, cohabiting, divorced, widow), a dummy for having children, dummies for employment status (homemaker, employed, retired, student, other), mathematical skills, self-placement of a left-right scale, average distance from outbreak, average trust in science, a dummy for Lombardy, a dummy for Veneto, and Latitude. Wording of the questions reported in Supplementary Information S6.2.

We further examined if weakened trust in science or health authorities is a potential mechanism for this effect. We augmented our dataset by linking information on the prevalence of COVID-19 cases16 in the respondent's municipality using their zipcodes. The prevalence data at the localised municipality level were only available for Lombardy, so we focus only on this region for this analysis of the relationship between intensity of exposure and trust outcomes. COVID-19 case counts data were retrieved from an official website of the Lombardy region, and made available on GitHub, by the ondata/covid19italia project.17 See Battiston & Gamba, 2020 for a study employing analogous data from Lombardy. This analysis, shown in Table 7, reveals a curvilinear relationship between intensity of exposure to COVID-19 and trust in the Institute for Public Health, with higher levels of trust among those with greater number of cases in their municipality, but reduced trust with continued exposure. Continued exposure is proxied by prevalence one week before the interview.

Table 7.

Determinants of trust in experts (Lombardy only).

	(1)	(2)	(3)
	Trust: National Inst. Pub. Health	Trust: science	Trust science (avg)
	b/se	b/se	b/se
Prevalence	11730.073*	4374.695	8789.752*
	(5362.750)	(3714.764)	(3969.607)
Prevalence_1w	−11265.879*	−4330.631	−8367.633*
	(5570.714)	(3721.199)	(4111.286)
Covariates	✓	✓	✓
Constant	109.021***	107.900***	114.428***
	(8.867)	(15.203)	(7.584)
N.	463	465	463
$R^2$	0.325	0.269	0.384

OLS. Covariates include: gender (male = 1), age, educational attainment (secondary, bachelor, master and higher), marital status (married, cohabiting, divorced, widow), parentalstatus, employment status (homemaker, employed, retired, student, other), mathematical skills, self-placement of a left-right scale. + p < 0.10, *p < 0.05, *p < 0.01, *p < 0.001.

5. Discussion

Drawing on Twitter and rapid response online survey data, this study shows how reliance on scientists and experts evolved in the early phases of the Italian COVID-19 outbreak. Shortly after the identification of the first cases of COVID-19 in Italy in February 2020, both Twitter and Telegram data pointed to initial increasing attention to and information-seeking from scientists and health authorities. These findings are consistent with preliminary evidence on initial increases in information-seeking behaviour using internet search data from the US in early stages of COVID-19 (Bento et al., 2020). However, in both Telegram and Twitter we found a stall in this increase, and in particular on Twitter, where data are available over a longer period of time we detected declines in social media attention to scientists and health authorities after mid-March 2020 taking the form of a reversed U-shape.

The implications of the willingness to consult expert sources in the early weeks of the epidemic outbreak, as suggested by the Telegram survey and Twitter data, was also reflected in knowledge outcomes about health information linked to the coronavirus examined in a separately fielded online survey to a sample recruited via Facebook in the last two weeks of March. This survey showed generally high levels of public understanding of information about the disease as well as support for containment measures, consistent with other survey evidence from this time (Barari et al., 2020), and also found that trust in science and public health authorities were positively associated with both health knowledge and containment support measures. Better knowledge about health information linked to coronavirus was also correlated with greater support for containment measures, suggesting an important role for health literacy and awareness in fostering public health compliance (Paakkari & Okan, 2020). However – echoing the signs of a declining levels of attention to scientists and health authorities in Twitter and Telegram – our survey experiment found no different – or even lower – willingness to modify misperceptions when the source of the information was explicitly stated as coming from public health experts.

Our findings, across the different, complementary data sources collected across different online platforms provide empirical evidence for weakening attention and responsiveness to scientists and health authorities with continued exposure to the COVID-19 pandemic. Further analyses focusing on Lombardy, for which we had municipality-level prevalence data, suggested weakened trust in public health authorities with continued exposure to the outbreak as a potential mechanism for this effect of weakened responsiveness to experts. It is plausible that other mechanisms also underlie this effect. As the novelty of the virus wanes, behavioural fatigue may have affected declining attention to scientists and health authorities (Redelmeier & Shafir, 2020). Increasing negative emotions about the virus may result in people relying on emotions rather than information in shaping beliefs about the virus (Van Bavel et al., 2020). Potential frustration with the inability to control the epidemic could explain the emerging distrust of experts and health authorities that we detected. In addition to epidemic exposure, a potentially fragmented scientific communication due to the often heterogeneous positions about the coronavirus among experts themselves is also a plausible mechanism for weakening reliance on experts and emerging skepticism. Support for this is suggested by a separate poll run during this period (Observa Science in Society, 2020) that found that nearly half of the Italian public saw the diversity of opinions given by experts as creating confusion. Further empirical research is needed to better understand these mechanisms of shifting public responsiveness to experts in the context of an unfolding pandemic.

We acknowledge limitations in our study. First, our study relies on online data from social media and messaging platforms. While online samples provide the opportunity for high-frequency, cost-efficient measurement in the context of a fast-evolving epidemic when face-to-face data collection is restricted, and overcome challenges associated with low response rates in telephone surveys, they are likely to suffer from issues of self-selection and demographic representativeness. We used post-stratification weights to adjust both surveys for demographic characteristics of age, gender and education. Further, when recruiting respondents via Facebook we took steps to avoid topical self-selection of users only interested in coronavirus or public health issues, and administered the survey separately to respondents on a different platform outside of Facebook (Qualtrics). For robustness, we also placed an abbreviated version of the survey experiment on Telegram, which also yielded similar findings to the results obtained from the Facebook sample. We acknowledge nonetheless that social media samples may be selective in other ways in terms of psychological or behavioural characteristics that our covariates might not fully capture. Information exchange on social media spaces, for example, may be ideologically segregated or polarised where individuals sort themselves into ‘echo chambers’, as has been shown for Twitter in the case of the vaccination debate in Italy (Cossard et al., 2020), or COVID-19 discussions in the US (Jiang et al., 2020). Other work however has indicated that ideological segregation is less clear for current events or crises (Barberá et al., 2015), as could plausibly be the case in the early phases of an outbreak of a new infectious disease. We acknowledge potential self-selection of respondents in the Telegram survey pool, as these are from individuals who had self-selected onto the channel to receive information on the virus. It is possible that this group may be more likely to experience fatigue and frustration dynamics resulting in waning attention and responsiveness to scientists and health authorities earlier. However, our finding of weakened responsiveness to expert sources was also confirmed through a survey experiment with randomization in a different pool, and the broad consistency of dynamics emerging across different online platforms provides support for a greater generalisability of our findings. Lastly, although we were able to explore weakened trust as a potential mechanism for a subset of our sample, we are unable to empirically assess the role of other potential mechanisms described above.

Despite these limitations, our study provides novel empirical evidence for how although in the face of a new threat, reliance on scientists and experts initially increases, this increase may be short-lived. Weakening attention to scientists – and indifferent or weakened responsiveness to scientific information as suggested by our survey experiment – are likely to increase susceptibility to misinformation in the context of a pandemic that has also been described as an “infodemic” (Zarocostas, 2020), and are important to guard against. Even when a pandemic is underway, our results point to the importance of trust in science, which emerges as a resilient predictor of both public health knowledge and containment support. To sustain public trust in science throughout a crisis, our study points to the need for clear, sustained and transparent channels of information communication from scientific authorities to the public to anticipate and guard against frustration.

Author contributions

All authors contributed equally to this work. Author names are listed alphabetically.

Ethical statement

Our study received ethical approval from University of Oxford Central University Research Ethics Committee (Reference number: R68769/RE001) and has been run in accordance with rules and procedures of the Centre for Experimental Social Sciences (CESS) at Nuffield College in the University of Oxford. The study was funded by a grant awarded from the Nuffield College academic fund (AF-20-3). Open access support provided through the Leverhulme Centre for Demographic Science. The funder had no role in shaping the research design and results reported.

Appendix A. Supplementary data

The following is the supplementary data to this article: