An experimental study on the effects of electronic medicinal product information on vaccine hesitancy

Abstract

Although immunization through vaccination is one of the most successful public health interventions, coverage of some vaccination programs has decreased in recent years due to increased vaccine hesitancy. Cognitive biases have been shown to play an important role in affecting vaccine hesitancy. In this study, we conducted a randomized controlled trial (N = 2000, N = 1000 from Spain and N = 1000 from Bulgaria), where subjects were randomly assigned to one experimental condition. The experimental conditions differed by whether electronic product information (ePI) was presented to the subjects and by the type of information that was made more salient to the patient. The current study showed that the provision of digital information in the form of ePI has important consequences for achieving high vaccination rates. The main result suggests that providing vaccination information in the form of ePI can increase patients’ vaccine hesitancy. This effect remained when positive and/or negative information in the ePI was made more salient to the patients. Additionally, we observe that vaccine hesitant individuals spend less time reading ePI. We conclude, by relating the current study to the relevant literature, that salience and information overload could be the main driver of vaccine hesitancy in the context of this study.

Introduction

Immunization through vaccination is considered to be a global health success story, saving millions of lives every year and being a highly cost-effective strategy¹. Estimations suggest that the vaccination of each birth cohort prevents approximately 42,000 deaths and 20 million cases of disease, with net savings of nearly $14 billion in direct costs and $69 billion in total societal costs in the US alone². Vaccines reduce the risks of getting a disease by working with the individual’s natural defences to build protection. Nonetheless, although scientific evidence is clear about immunization being one of the most successful public health interventions, actual coverage has decreased due to the disruptions caused by the COVID-19 pandemic and vaccine hesitancy has increased during the years following the pandemic³. Vaccine hesitancy, the choice to delay or refuse available vaccines, is a major obstacle for the effectiveness of ongoing vaccination programs. Therefore, it was recently declared by the WHO as one of the top 10 health threats to global public health⁴. Mounting evidence shows that vaccine hesitancy is determined by a mix of demographic, structural, social, and behavioural factors^5,6. For example, on the individual level, several studies have shown a higher vaccine hesitancy rate among people with high educational level and higher socio-economic status. This suggests that vaccine hesitancy is an issue that cuts across socio-demographic determinants^7,8. Vaccination programs only achieve their promised benefits when enough individuals follow vaccination recommendations. There is a mix in the trends of vaccine uptake among vaccination programs with some increasing while others stagnating or even dropping. For example, vaccine coverage for the birth dose of Hepatitis B has been increasing from low levels over the last years, while the third dose coverage was at the same level in 2023 as in 2015 with a decrease since its peak in 2019 [Hepatitis B vaccination coverage]. The vaccine for measles shows a similar pattern with the same uptake for the first dose in 2023 as in 2009 and with a slight decrease in recent years. At the same time, the second dose shows an increasing trend from a lower level [Measles vaccination coverage]. Vaccine hesitancy is an issue for the vaccination programs whose uptake is stagnating or decreasing. In response to non-increasing vaccination rates, the WHO Immunization Agenda 2030 was launched in November to increase the uptake of vaccines for diseases like haemophilus influenzae type b, hepatitis B, human papillomavirus, measles, and many others¹.

It is important to better understand how individuals perceive and process information concerning vaccination before adapting their behaviours to the specific vaccination programs. People commonly use automatic heuristics when dealing with complex information^9,10. This leads to bias in decision making as documented by Kahneman and Tversky who pointed out that people rely on a limited number of heuristic principles which reduce the complex task of assessing probabilities and predicting values. In situations of extreme uncertainty, human brains struggle to organize partial and jumbled information into a coherent interpretation, being vulnerable to cognitive biases, leading sometimes to a decrease in cognitive capacity and bias in decision making¹¹. In general, these heuristics are quite useful, but sometimes they lead to severe and systematic errors. When facing complex decision-making processes, individuals use intuitive, automatic, and often unconscious strategies to address an issue more easily. For example, when deciding to follow the vaccination program, research has shown that individuals are more averse to take a risk associated with an action – inoculation of a possibly ‘unsafe’ vaccine—than to take the risk associated with inaction—taking the chances of contracting a vaccine-preventable disease, this is known as omission bias⁷. Additionally, individuals show a misunderstanding of co-occurrence with causation, also labelled as co-incidence bias, attributing every event occurring after immunization to be caused by vaccination¹². Research has also shown that individuals seek information that confirm their initial beliefs about vaccines^13,14. This is known as confirmation bias. An increasing number of studies show that heuristics and cognitive biases affect vaccination decision making, also in the COVID-19 pandemic (see, e.g.^15–17). However, research examining how to influence these heuristics and cognitive biases to reduce the hesitancy to get vaccinated is limited. This can be translated into an effective communication strategy for national or supranational organizations that want to facilitate vaccination programs in order to improve citizens’ health.

Considering all the above-mentioned elements, vaccine hesitancy must be considered a cognitively mediated process^18–20, that leads to several behavioural outcomes, from hesitancy to refusal, among a variety of people. In line with this, recent studies have found that behavioural factors such as risk^21,22, time²³, and social preferences can predict vaccine hesitancy in addition to socio-demographic characteristics such as socio-economic status²⁴. Individuals can range from the genuinely uncommitted, with no definite opinion, little knowledge, and interest in vaccines, who randomly forget or delay some vaccines to the well informed, very interested and committed individuals, covering multiple different socio-demographic characteristics. To this end, significant differences have been observed in how populations of different countries perceive vaccines. For example, according to the Eurobarometer flash report on Attitudes on vaccination against COVID-19 from 2022 [Eurobarometer], it is found that Spain is the second country in the EU with the lowest vaccine hesitancy and highest acceptance of country-wide vaccine strategies, while Bulgaria is consistently ranked the lowest on these aspects among the EU countries. In addition, finding trustworthy information about vaccines is most difficult in Bulgaria while much easier in Spain according to the national populations participating in the Eurobarometer flash report.

A bias that has been found to strongly affect patient behaviour is salience. Salience bias refers to the fact that individuals pay more attention and overweigh aspects that stand out in the decision environment. Such aspects can for example be highlighted information or even to present information that is normally not presented to individuals. Making such information stand out affects patients’ behaviour since the salient information has an increased importance for their decisions. Salience bias has considerable theoretical and empirical support^25–28 and it has been found to be important for the health domain. For example, by making survival or mortality probabilities salient has been found to affect patients’ selection of treatment, and providing, and thus highlighting, information of personal health benefits increased COVID-19 vaccination rates^15,29. However, making information more salient by increasing the amount of information available to individuals does not always translate in an increase in prevention behaviours such as willingness to adhere to treatment and intention to vaccinate. Several studies suggest that increasing information exposition and salience can decrease preventive behaviours and increase vaccine hesitancy in certain individuals and contexts. This can be due to negative information becoming more salient to the patient since this information, which they are typically not exposed to, is presented to them. It has been found that increased information on the potential negative effects of the vaccine increases the salience and thus mental availability about the negative event or an increase in risk perception towards the vaccine, even if framed positively^11,30,31. This can in turn lead to increased vaccine hesitancy³².

A phenomenon related to salience is information overload which occurs when individuals display worsened decision making because of the volume of the information exceeding an individual’s processing capacity^33,34. Information overload has been observed to cause individuals to be more stressed and confused, to abstract information with incorrect interpretations, ignore further information input, and have reduced accuracy in decision-making³³. This is partially explained by the fact that information overload seems to alter the way information is processed, by increasing individuals’ tendency towards greater heuristic information processing and less systematic information processing³⁵. Specific to the healthcare context, it has been observed that information overload can have detrimental impact on patients’ decision-making processes such as negatively affect the use of self-management educational materials³⁶, treatment adherence, as well as increasing depression and anxiety³⁷. Similar effects on decision making caused by information overload have also been observed in the specific context of vaccine hesitancy. For example, a recent study carried out during the COVID-19 pandemic, highlighted that when individuals are overloaded with vaccine information retrieved online, they tend to develop higher anxiety through cyberchondria, and an increase in the perceived risk of the vaccine, that translated into an increased vaccine hesitancy³⁸. Evidence on the topic is however mixed, as it has been found that exposure to the so called ‘infodemics’—a rapid and far-reaching spread of both accurate and inaccurate information, generally causing information overload—had a positive relationship with intent to vaccinate and a negative effect only when paired with belief in vaccine misinformation³⁹. This inconsistency in findings seems to suggest that belief in misinformation, and therefore quality and trustworthiness of the source of information could exert a stronger influence on vaccine hesitancy than pure information overload. Instead, recent evidence paints a different picture. First, concerning the role of trustworthiness of the source, several studies observed that information sources which are traditionally considered as the most reliable and trustworthy in the health context (such as medical experts and healthcare workers) seem to be less effective in mitigating the negative effects of vaccine scepticism and counterbalance vaccine hesitancy when compared to other less traditional health information sources such as celebrity endorsement³⁸, and newspapers⁴⁰.

A product information leaflet (PIL) and its electronic version, electronic product information (ePI), is intuitively one of the most reliable and trustworthy sources of information available in the healthcare context. The aim of the PIL and ePI is to provide patients with trustworthy information about the medicine or vaccine for the patients to make an informed decision. They are not designed to stimulate a certain behaviour such as increased adherence to treatment. However, providing PILs to patients have been found to have negative effects due to information overload. For example, Bapat et al.⁴¹, found that longer PILs with excessive and complex information can lead to higher anxiety and lower comprehension. Another more recent study highlighted that reading the PIL aroused anxiety and decreased adherence in some patients and that many indicated that the PIL did not provide them with the needed information⁴². Nevertheless, even when increasing the clarity of the information to give patients the information they need, there is the risk of overloading them and hinder their prevention behaviours. The study conducted by Kelly et al.⁴³, highlighted that more clarifying information is not always better and may lead to cognitive overload, inhibiting comprehension. It observed that additional qualitative context about warnings and side effects resulted in lower comprehension of side effect information.

It is important to generate further knowledge on the potential behavioural effects of providing ePI to patients. Therefore, the main objective of the current study is to investigate the effects that presenting and changing the salience of ePI has on vaccine hesitancy. The study is part of the European funded IHI project Gravitate-Health [https://www.gravitatehealth.eu/]. The mission of Gravitate-Health is to equip and empower citizens with digital information tools that make them confident, active, and responsive in their patient journey, specifically encouraging safe use of medicines for better health outcomes and quality of life. To achieve this, the Gravitate Health solution, the G-lens, is developed to, among other things, provide ePI and offer a direction for patients to access trustworthy and up-to-date information that better meets their individual needs, especially for specific vulnerable groups. ePI for both medications and vaccines is envisioned to be provided to patients with the overarching goal of improving adherence to medication treatment. The G-lens aims to focus different information, thus making it more salient, depending on the needs of the patient. In line with the general aims of Gravitate-Health, we study how the provision and increased salience of positive and negative information in an ePI affects vaccine hesitancy. While the increased salience of positive and negative information is not considered specifically by the Gravitate Health solution it is information that is relevant to, and thus satisfy the needs of, all patients.

Results

Comparing the share of vaccine hesitant subjects across the five conditions generates the main result of the study: Providing ePI, made more salient or not, increases vaccine hesitancy. Figure 1 shows the share of vaccine hesitancy by the five conditions:No ePI—No information presented, ePI: ePI presented, ePI + Pos: ePI presented and positive information made more salient, ePI + Neg: ePi presented and negative information made more salient, and ePI + Pos & Neg: ePI presented and both positive and negative information made more salient. Results from chi-squared tests show that the observed differences are statistically significant (p < 0.01 for comparisons between the “No ePI” condition and other conditions except for the “ePI + Pos & Neg” condition, in which case p < 0.05). Importantly, there are no statistically significant differences between any of the conditions in which the ePI is provided. Consequently, increasing the salience of positive and/or negative information seems to have a limited effect on vaccine hesitancy.

Fig. 1.

Share of vaccine hesitant subjects by condition. Notes: Whiskers show 95% confidence intervals. The five different conditions are: No ePI—No information presented, ePI: ePI presented, ePI + Pos: ePI presented and positive information made more salient, ePI + Neg: ePi presented and negative information made more salient, and ePI + Pos & Neg: ePI presented and both positive and negative information made more salient.

The result that providing ePI increases vaccine hesitancy remains significant when controlling for a series of variables. Table 1 shows the average marginal effects from three logit estimations using vaccine hesitant as the dependent variable. Model 1 shows the simplest specification, only including dummy variables for the different conditions. Consequently, these results closely mimic what was previously displayed in Fig. 1. Model 2 controls for levels of risk perception (RP), whether a subject has a high risk for experiencing severe illness of the flu (High risk), and risk preferences (Risk taking) and time preferences (Patience) as well as social status. Four types of risk perceptions are measured: How dangerous the patient perceives the flu to be (Flu severity), the risks related to the flu if vaccinated (flu vaccinated), the risks related to the flu if not vaccinated (flu not vaccinated) and the risk of experiencing adverse effects (adverse effects). Finally, Model 3 controls for socio-demographic background factors. Specifically, country (Spain), sex (Female), age, and additional socio-demographic variables including level of education (Edu), living situation (Living), work situation (Work), and household composition (Household). Categorical variables have been transformed into dummy variables in the regression for Edu, Living, Work, and Household. Table 1 does not show the estimates of the additional socio-demographic variables. However, the full logit output is presented in the supplementary material.

Table 1.

Average marginal effects from logit estimation with vaccine hesitant as the dependent variable.

Variable	Model 1	Model 2	Model 3
ePI	0.1027***	0.0633**	0.0591**
	(0.032)	(0.028)	(0.027)
ePI + Pos	0.0976***	0.0766***	0.0718***
	(0.032)	(0.028)	(0.027)
ePI + Neg	0.1266****	0.1004****	0.0880***
	(0.032)	(0.027)	(0.026)
ePI + Pos & Neg	0.0676**	0.0601**	0.0523**
	(0.032)	(0.027)	(0.026)
RP—Flu severity		− 0.0314****	− 0.0384****
		(0.007)	(0.007)
RP—flu not vaccinated		− 0.0965****	− 0.0860****
		(0.008)	(0.008)
RP—flu vaccinated		0.0449****	0.0462****
		(0.009)	(0.008)
RP—adverse effects		0.0715****	0.0636****
		(0.008)	(0.008)
High risk		0.0138	− 0.0232
		(0.02)	(0.021)
Social status		− 0.0158****	− 0.0121**
		(0.005)	(0.005)
Risk taking		− 0.0127***	− 0.0109**
		(0.004)	(0.004)
Patience		− 0.0061	− 0.0016
		(0.004)	(0.004)
Trust		− 0.0179****	− 0.0168****
		(0.004)	(0.004)
Spain			− 0.1596****
			(0.018)
Age			0.0036****
			(0.001)
Female			− 0.0138
			(0.018)
Additional socio-demographic variables			Yes
N	2000	2000	2000
Pseudo R-squared	0.01	0.22	0.29

Average marginal effects from three logit estimations using vaccine hesitancy as the dependent variable. Model 1 includes dummy variables for the different conditions. Model 2 controls for levels of risk perception (RP), whether a subject has a high risk for experiencing severe illness of the flu (High risk), and risk preferences (Risk taking) and time preferences (Patience) as well as Social status. Four types of risk perceptions are measured: How dangerous the patient perceives the flu to be (Flu severity), the risks related to the flu if vaccinated (flu vaccinated), the risks related to the flu if not vaccinated (flu not vaccinated) and the risk of experience adverse effects of the vaccine (adverse effects). Additionally, Model 3 controls for socio-demographic background factors. Specifically, country (Spain), sex (Female), Age, and additional socio-demographic variable including level of education, living situation, work situation, and household composition. **** p<0.001, *** p< 0.01, ** p<0.05, * p<0.1.

Table 1 shows that while the inclusion of control variables (Model 2 and Model 3) decreases the estimated effects of ePI provision, the main result remains statistically significant: ePI provision increases vaccine hesitancy. Additionally, risk perceptions, country (Spain), and trust have strong and highly significant correlations with subjects being vaccine hesitant. We also find that older individuals are more vaccine hesitant [A balance check, using an ANOVA test, shows that the treatment groups are balanced w.r.t age, p = 0.871]. Subjective social status and risk-seeking are negatively correlated with vaccine hesitancy. Vaccine hesitant subjects perceive flu as less dangerous (RP—Flu severity) and the risks related to flu when not being vaccinated as lower (RP—flu not vaccinated) relative the non-hesitant subjects. At the same time, vaccine hesitant subjects perceive higher risks related to flu when being vaccinated (RP—flu vaccinated), suggesting they believe the vaccine to be less effective, and of adverse effects from the vaccine (RP adverse effects) compared to non-hesitant subjects.

By splitting the condition effects by the two countries studied; Spain and Bulgaria, it seems that the increase in vaccine hesitancy due to ePI provision is mainly driven by the subjects from Spain and that overall vaccine hesitancy is lower in Spain than in Bulgaria. The last finding confirms previous studies⁶. Figure 2 shows the share of vaccine hesitant subjects split by condition and country. The differences are statistically significant (p < 0.05) between the “No ePI” condition and all other conditions except for the “ePI x Pos & Neg” condition. For Bulgaria, only the “ePI” condition is statistically different from the “No ePI” condition (p < 0.05).

Fig. 2.

Share of vaccine hesitant subjects split by whether from Spain or Bulgaria by condition. Notes: Whiskers show 95% confidence intervals. The five different conditions are: No ePI—No information presented, ePI: ePI presented, ePI + Pos: ePI presented and positive information made more salientd, ePI + Neg: ePi presented and negative information made more salient, and ePI + Pos & Neg: ePI presented and both positive and negative information made more salient.

However, a different pattern emerges when we re-run Model 3 of Table 1 separately by each country and thus control for the additional variables collected. The results are presented in Table 2. Now, all treatment effects become significant at least at the 10% level for Bulgaria. In fact, the ePI and ePI + Neg treatments have the largest effects and are significant at the 5% level. For Spain, only the ePI + Neg is significant at the 5% level and ePI + Pos at the 10% level. ePI + Neg in Spain shows the overall largest effect of a 9.62% point increase in vaccine hesitancy. These results suggest that vaccine hesitancy increases by providing ePI information in Bulgaria, while in Spain the provision of information and highlighting of negative information has the bigger impact. The full specifications can be found in the supplementary material.

Table 2.

Average marginal effects from logit estimation with vaccine hesitant as the dependent variable run by Spanish and Bulgarian participants separately.

Variable	Spain	Bulgaria
ePI	0.0360	0.0807**
	(0.042)	(0.035)
ePI + Pos	0.0692*	0.0595*
	(0.041)	(0.032)
ePI + Neg	0.0962**	0.0642**
	(0.041)	(0.032)
ePI + Pos & Neg	0.0394	0.0553*
	(0.041)	(0.033)
RP—Flu severity	− 0.0503****	− 0.0280***
	(0.011)	(0.01)
RP—flu not vaccinated	− 0.1008****	− 0.0620****
	(0.012)	(0.011)
RP—flu vaccinated	0.0420***	0.0483****
	(0.012)	(0.011)
RP—adverse effects	0.0682****	0.0606****
	(0.012)	(0.01)
High risk	0.0081	− 0.0416
	(0.032)	(0.026)
Social status	− 0.0141*	− 0.0126**
	(0.007)	(0.006)
Risk taking	− 0.0127**	− 0.0054
	(0.006)	(0.006)
Patience	− 0.0067	0.0004
	(0.006)	(0.005)
Trust	− 0.0172***	− 0.0156***
	(0.006)	(0.005)
Age	0.0067****	0.0009
	(0.001)	(0.001)
Female	− 0.0240	− 0.0058
	(0.028)	(0.022)
Additional socio-demographic variables	Yes	Yes
N	1000	1000
Pseudo R-squared	0.25	0.3

Average marginal effects from two logit estimations using vaccine hesitancy as the dependent variable. The first model is run on the Spanish participants and the second model on the Bulgarian participants. Both models include dummy variables for the different conditions and controls for levels of risk perception (RP), whether a subject has a high risk for experiencing severe illness of the flu (High risk), and risk preferences (Risk taking) and time preferences (Patience) as well as Social status. Additionally, four types of risk perceptions are measured: How dangerous the patient perceives the flu to be (Flu severity), the risks related to the flu if vaccinated (flu vaccinated), the risks related to the flu if not vaccinated (flu not vaccinated) and the risk of experience adverse effects of the vaccine (adverse effects). Finally, control variables for socio-demographic background factors. Specifically, sex (Female), Age, and additional socio-demographic variable including level of education, living situation, work situation, and household composition. **** p<0.001, *** p<0.01, ** p<0.05, * p<0.1.

We now deepen the behavioural analysis by looking at the time the participants spent reading the different ePI pages. Furthermore, we split the analysis by whether subjects indicated that they were vaccine hesitant or not to increase our understanding of the differences between the two groups. The analysis shows that vaccine hesitant individuals spend less time reading the ePI. Figure 3 shows the average time spent reading the ePI by whether a subject is hesitant or not towards getting vaccinated against the flu. Sub-figure A. shows the average time calculated for all six pages. Sub-figure B. all pages presenting positive and/or negative information (pages 2, 3, and 5). Sub-figure C shows the result for the pages showing no positive or negative information (pages 1, 4, 6). The sub-figures on the lower row show the average time spent reading when there is D. only positive information (page 2), E. only negative information (page 5), and F. positive and negative information on the same page (page 3).

Fig. 3.

Average time spent reading the ePI by whether a subject is hesitant or not towards getting vaccinated against flu. Notes: Whiskers show 95% confidence intervals.

Mann–Whitney U tests confirm that observed differences are statistically significant for all pages, and for pages where there are positive and/or negative information presented as well as when there is no such information (p < 0.001 for all comparisons except in the last case in which p < 0.05). Computing log-transformations, to obtain normally distributed variables, and applying t-tests generate the same results. Moreover, hesitant spend less time reading the information regardless of if the pages contain positive, or negative, or positive and negative information (p < 0.01 for all comparisons except om the last case in which p < 0.05).

Discussion

Vaccination is considered one of the most successful public health interventions, preventing at least 10 million deaths between 2010 and 2015¹. Despite this, vaccination rates and confidence in vaccines have recently reported to decrease as vaccine hesitancy has increased^44,45. To this end, the effects of providing patients with information about vaccination has not been explored deeply in the academic literature, and the studies that focus on the topic are scarce. A plausible reason for this is that patients are not provided with information of vaccines to the same extent as compared to other medications. However, this gap is important to fill given the importance of vaccines and the developments in electronic provision of healthcare information as demonstrated by the Gravitate-Health project.

The aim of this study, as part of the Gravitate-Health project, is to start filling this gap in the literature by exploring how vaccine ePI provided and made more salient to patients affect vaccine hesitancy. Additionally, it aims to provide directions for effective information provision to reduce the levels of vaccine hesitancy since information provision has been observed as an efficient way to improve adherence to treatment and improve health outcomes, but only when it is done correctly³⁹. The current study shows that the provision of digital vaccine information in the form of ePI has important consequences for achieving high vaccination rates. It was found that presenting ePI in the way proposed in this study increases vaccine hesitancy. Additionally, making positive and/or negative information more salient to the patients did not alter this effect. This further underlines the need for well-designed and targeted strategies, such as educational and decision-aiding tools, to reduce vaccine hesitancy.

Salience bias is a potential explanation for the result that information provision increases vaccine hesitancy. ePI is typically not presented to patients before deciding whether to get vaccinated. Therefore, presenting ePI to the patients may constitute a salience effect as information, and in particular negative information, becomes more salient than in the situation where no such information is given. This would be in line with previous studies showing that presenting information on the potential negative effects of the vaccine increases the salience and thus mental availability about the negative event^11,30,31 Similarly, it has been found that providing information about the negative effects of the disease, thus making it salient, improves vaccine attitudes⁴⁶. Moreover, this salience effect would be stronger than the salience effect of making either negative and/or positive information more salient by highlighting it since no differences were found between these conditions in the total sample. Amplifying the visual salience, such as by highlighting, has been found to have effect on consumer behaviour in shopping situations^47–49. The discrepancy in findings between these studies and ours could be explained by that increased visual salience is more effective when the consequences of the decisions are small and when the environment is full of distractions, such as in a shopping situation. However, for decisions with large consequences, such as those concerning one’s health, changes in visual salience have smaller effects on behaviour. When splitting the sample by the Spanish and Bulgarian populations, the findings suggest that the salience effect of presenting ePI is stronger in Bulgaria. For Spain, the largest effect was found when ePI was coupled with the salience of negative information. This could be explained by prior attitudes as Bulgarians have reported to be much more sceptical to vaccines than people from Spain and that Spanish people find trustworthy vaccine information easier [Eurobarometer]. Due to this, Bulgarians could be less used to, and more sceptical of, vaccine information and react more to it when being presented and salient. Information overload is another potential explanation to the observed negative effect of presenting ePI about vaccines to patients. ePI is typically (very) long and contains information that is hard to understand and not expressed in a format accessible to the general population. Consequently, presenting ePI can result in information overload which in turn makes the patients more vaccine hesitant. Similar findings have been reported when patients retrieve large amounts of vaccine information online³⁸.

Vaccine hesitant individuals have generally spent less time reading the ePI in this study. This is in line with previous findings in the literature suggesting that it could be a mechanism to cope with information overload, as individuals tend to avoid and withdraw from information when experiencing overload. Specifically, there seems to be two general pragmatic strategies for coping with information overload when it arises: filtering information, as reading only parts of it, and avoiding or withdrawing from the information^50,51. Both strategies can be observed as less time spent reading the information. This result could be interpreted as information overload increases vaccine hesitancy as found in the literature³⁸. Prior attitudes towards the ePI is another potential explanation for this result. Participants rejecting the vaccine may engage less with ePI, as they may be more likely to distrust the source of the information, such as scientists and governmental agencies, and already decided not to act⁵². This is closely related to confirmation bias, which would occur when vaccine hesitant individuals, being sceptical towards the information, would seek information that confirm their initial beliefs and consequently read less of the ePI^13,14.

Finally, this study also observed that vaccine hesitant individuals display higher levels of risk perception towards the vaccine, replicating previous results observed in the literature, where the fear of the side effects of vaccines explained (together with other factors) 54% of the variance in vaccine hesitancy³². Specifically, our study highlighted that vaccine hesitant individuals perceive flu as less dangerous, and the risks related to flu when not being vaccinated as lower. At the same time, they perceive higher risks related to flu conditional of being vaccinated and of adverse effects from the vaccine. Interestingly, risk perception did not change at all between conditions. This confirms previous evidence that even when information is framed in a positive way and is intended to reduce risk perception, the mere fact that information is presented leads people to assume that a behaviour involves substantial risk, because of pragmatic conversational norms⁵³. Additionally, people systematically overestimate the likelihood of low probability events described in semantic terms such as “low risk” or “people may occasionally experience”, and presenting semantic information about the risks of engaging in a new behaviour makes people less likely to engage in that behavior³¹. Consequently, the effects on vaccine hesitancy of information provision and increased salience does need seem to stem from a change in risk perceptions.

Methods

An online experiment was conducted to better understand how the provision and increased salience of digital information related to a flu vaccine, in the form of an ePI, affected vaccine hesitancy. The experiment consisted of the following three stages:

1. Read ePI of flu vaccine (if provided)

2. State willingness to get vaccinated against flu

3. Fill out a questionnaire to collect a series of determinants and control variables.

The experiment was conducted as a randomized controlled trial in which subjects were randomly assigned to one out of five conditions. The conditions were the following:

• No ePI—No information presented.

• ePI—ePI presented.

• ePI + Pos—ePI presented and positive information made more salient.

• ePI + Neg—ePi presented and negative information made more salient.

• ePI + Pos & Neg—ePI presented and both positive and negative information made more salient.

The experimental conditions differed by whether ePI was presented to the subjects and the type of text of the ePI that was made more salient: Either none, positive, negative or both. Therefore, Stage 1 differs between conditions while stages 2 and 3 are the same across all conditions.

A piece of text was made more salient by putting a box around it with a light green background and a dark green border. Healthcare professionals determined which information of the ePI was considered positive and negative. The ePI was divided into seven pages, each of which was presented on a separate screen. The subjects who were presented with ePI had to spend at least 3 s on each page before they could continue to the next. The time a subject spent at each page is measured in the experiment. The ePI and its possible salience is presented in the supplementary material.

After being exposed to the ePI (or no information), the subjects answered whether they were willing to get vaccinated. Three options were given: 1) No, 2) unsure, and 3) yes. Following this, the questionnaire started with a series of questions regarding risk perceptions from Martinelli and Veltri¹⁸. From this, four types of risk perceptions were constructed: How dangerous the patient perceives the flu to be (Flu severity), the risks related to the flu when vaccinated (flu vaccinated), the risks related to the flu when not vaccinated (flu not vaccinated) and the risk of experiencing adverse effects (adverse effects). Subjects also provided medical information: if they had been diagnosed with a chronic disease, were pregnant, or diagnosed with Diabetes type 1. This information, together with age (65 +) allowed for the identification of individuals at high risk of flu. Thereafter, sociodemographic information was collected on the following dimensions: sex, age, level of education, social status, living situation, work situation, and household composition. Finally, measures for time, and risk preferences as well as trust were included from Falk et al.⁵⁴. The full questionnaire is presented in the supplementary material.

The recruitment and programming of the experiment was done by an online panel provider. A total number of 2000 subjects were recruited, 1000 from Spain and 1000 from Bulgaria to ensure a variation in the degree of vaccine hesitancy as it has been found to be low in Spain and high in Bulgaria⁶. A representative sample with respect to age and gender of the adult population of each country was recruited. Table 3 shows the sample characteristics by gender and age that were used for the sampling for Bulgaria and Spain respectively.

Table 3.

Sample characteristics by gender and age for Bulgaria and Spain.

Source: Eurostat.

	Male	Female	18–24	25–54	55–65	Total
Bulgaria	485 (48.5%)	515 (51.5%)	138 (13.84%)	649 (64.88%)	213 (21.28%)	1000
Spain	490 (49%)	510 (51%)	150 (14.95%)	654 (65.37%)	197 (19.68%)	1000

Quantitative methods will be employed to analyse the effects of ePI provision and salience. The main measure of interest is the intention to get vaccinated against the flu. Subjects not answering “Yes” to whether they would get vaccinated are classified as vaccine hesitant. The share of vaccine hesitant subjects is analysed by testing for differences between conditions using chi-squared tests. Logit estimation with vaccine hesitant as the dependent variable will accompany the analysis to investigate the robustness of the results to the inclusion of the control variables measured in the questionnaire. Thereafter, the time spent reading each section is analysed to get a deeper understanding of the behavioural effects of ePI provision and salience. All pages of the ePI except the last that contains contact information to the distributors are included in the analysis. Being continuous non-normally distributed variables, differences in time spent reading between vaccine hesitant and non-hesitant subjects is tested using Mann–Whitney U tests. The experimental protocol was approved by the ethical committee of the Open University of Catalonia (UOC) and the experiment was conducted in accordance with relevant guidelines and regulations. Informed consent was obtained from all subjects prior to participating in the experiment.

Conclusion

The current study shows that providing patients with electronic information about vaccines has important effects on vaccine hesitancy. The main result suggests that providing vaccination information in the form of ePI can increase patients’ vaccine hesitancy. Moreover, this effect remained when positive and/or negative ePI was made more salient to the patients. These results may stem from that displaying ePI to patients increases the salience of negative information since patients are typically not provided information regarding vaccines. Alternatively, the effect could stem from information overload as the amount of information provided in the experiment may be higher than for other medications. The current study is, thus, among the first to provide evidence of the potential negative behavioural effects of vaccine information provision.

Future research on how provision and increased salience of ePI affect patient behaviour is needed. Considering the evidence presented in this study, care must be taken when presenting patients with medical information, and further research should aim to understand the optimal design of ePI for vaccination patients. Future research should aim at disentangling salience bias from information overload, and possibly other cognitive biases, to understand if any is the driver of vaccine hesitancy. In addition, understanding how ePI provision and salience affects other outcome variables such as trustworthiness, persuasiveness, and informativeness would be crucial to understand through which channels the effect works. Efforts should also be made to explore eventual concurrent strategies and approaches to adopt when providing additional information to vaccination patients. Such strategies could be to give them education material or other communication tools, to ensure that the increased amount of information does not negatively affects patient adherence to treatment. Projects such as Gravitate-Health are now more than ever crucial to better understand the most optimal strategies for providing better and more useful information to patients, to improve patient outcomes.

Author contributions

Ana Roca-Umbert, Francisco Lupiáñez-Villanueva, and Jim Ingebretsen Carlson developed the experimental design for the study. Jim Ingebretsen Carlson ran the experiment and conducted the data analysis. The article was written by Federico Puppo, Frans Folkvord, Ana Roca-Umbert, Francisco Lupiáñez-Villanueva, and Jim Ingebretsen Carlson. Jim Ingebretsen Carlson had the overall responsibility of the study.

Funding

This paper was funded by the Gravitate-Health project. The Gravitate-Health project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (IMI JU) under grant agreement No 945334. The IMI JU receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA and Datapharm Limited.

Data availability

The data that support the findings of this study are available from the corresponding author, [Jim Ingebretsen Carlson], upon reasonable request.

Declarations

Competing interests

The authors declare no competing interests.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-025-96092-6.