The effect of mandatory COVID-19 certificates on vaccine uptake: synthetic-control modelling of six countries

Abstract

Background

Mandatory COVID-19 certification (showing vaccination, recent negative test, or proof of recovery) has been introduced in some countries. We aimed to investigate the effect of certification on vaccine uptake.

Methods

We designed a synthetic control model comparing six countries (Denmark, Israel, Italy, France, Germany, and Switzerland) that introduced certification (April–August, 2021), with 19 control countries. Using daily data on cases, deaths, vaccinations, and country-specific information, we produced a counterfactual trend estimating what might have happened in similar circumstances if certificates were not introduced. The main outcome was daily COVID-19 vaccine doses.

Findings

COVID-19 certification led to increased vaccinations 20 days before implementation in anticipation, with a lasting effect up to 40 days after. Countries with pre-intervention uptake that was below average had a more pronounced increase in daily vaccinations compared with those where uptake was already average or higher. In France, doses exceeded 55 672 (95% CI 49 668–73 707) vaccines per million population or, in absolute terms, 3 761 440 (3 355 761–4 979 952) doses before mandatory certification and 72 151 (37 940–114 140) per million population after certification (4 874 857 [2 563 396–7 711 769] doses). We found no effect in countries that already had average uptake (Germany), or an unclear effect when certificates were introduced during a period of limited vaccine supply (Denmark). Increase in uptake was highest for people younger than 30 years after the introduction of certification. Access restrictions linked to certain settings (nightclubs and events with >1000 people) were associated with increased uptake in those younger than 20 years. When certification was extended to broader settings, uptake remained high in the youngest group, but increases were also observed in those aged 30–49 years.

Interpretation

Mandatory COVID-19 certification could increase vaccine uptake, but interpretation and transferability of findings need to be considered in the context of pre-existing levels of vaccine uptake and hesitancy, eligibility changes, and the pandemic trajectory.

Funding

Leverhulme Trust and European Research Council.

Introduction

Some countries have either introduced or are considering mandatory COVID-19 certification using proof of at least two doses of an approved vaccine, negative test (usually in the past 48 h), or recovery certificate to show recent natural infection (usually within the past 6 months; eg, RT-PCR or antigen test).1, 2, 3 Certification has been introduced for international travel, and, in domestic contexts, managing entry into settings such as nightclubs, large events, hospitals, gyms, or indoor hospitality (appendix p 3).³ Several media reports have linked certification to increased vaccine uptake,4, 5, 6 yet evidence remains anecdotal. Self-reported surveys suggest certification reduces vaccine intentions,⁷ and a systematic review concluded that the quality and quantity of studies was low, with a lack of conclusive evidence on whether certification would affect vaccine uptake.⁸

Certain groups, such as younger people and men have lower levels of COVID-19 vaccine uptake, often attributed to complacency or lower perceptions of risk from COVID-19, while some minority ethnic people have lower uptake because of a lack of trust in authorities and government.9, 10 A relevant question is whether certification could be an additional policy to increase uptake for those with vaccine complacency or hesitancy or, conversely, whether certification could reduce uptake due to ethical, trust, and privacy concerns. Vaccines provide not only protection for the immunised, but reduce transmission and risk of serious illness and death, and the chain of infection is broken only when sufficient numbers are vaccinated.¹¹ Given lack of empirical evidence on vaccine uptake and multiple authorities considering whether certificates should be introduced, we conducted the first empirical study to assess the relationship between the introduction of COVID-19 certification on observed vaccine uptake from April to September, 2021.

Methods

Study design and data

Data on COVID-19-related health indicators were used from Our World in Data,12, 13 which harmonises daily information on indicators such as cases, deaths, and vaccinations. We linked these to the Oxford COVID-19 Government Response Tracker,¹⁴ which provides daily country information on the implementation and stringency of non-pharmaceutical interventions. Age-specific analyses used European Centre for Disease Prevention and Control (ECDC) data on weekly age-specific vaccine doses. For all sources, we used the data on vaccine doses (and infections) in each country up until Nov 8, 2021, examining the timing of COVID-19 certification starting on April 21, to Sept 23, 2021.

Research in context.

Evidence before this study

The introduction of COVID-19 certification or vaccine passports, particularly for domestic use, has been linked to reduced self-reported vaccine intentions in some contexts, yet national media and health offices report increases in vaccinations. We examined all English literature until Oct 15, 2021, on Web of Science, Scopus, PubMed, and preprint servers (medRxiv and bioRxiv) with the terms “vaccination certificate”, “COVID-19 certificate”, “health passport”, “health certificate”, “health pass”, “digital health pass”, “health code app”, and “vaccination”, “immunization”, and “immunisation”. We were unable to identify empirical studies that examined the effect of the implementation of mandatory COVID-19 certification on vaccine uptake.

Added value of this study

To our knowledge, this is the first rigorous empirical analysis of the relationship between the introduction of COVID-19 certification and vaccine uptake. Self-reported surveys suggest certification reduces vaccine intentions in certain contexts, and a previous systematic review found a lack of conclusive evidence on whether certification would affect vaccine uptake.

Implications of all the available evidence

Our study provides evidence that mandatory COVID-19 certification restricting access to certain settings can influence vaccine uptake for those groups affected by the intervention. Given higher vaccine complacency and hesitancy in certain groups, such as younger people (<30 years), this intervention could be an additional policy lever to increase vaccine uptake and population-level immunity. Future studies examining more countries and variation by eligibility criteria and factors beyond age are warranted.

COVID-19 certification

We selected six countries (Denmark, France, Germany, Israel, Italy, and Switzerland) that implemented COVID-19 certification, where sufficient data and time after the incident of at least 40 days were available to examine effects. Details on when each intervention was announced, introduced, and the setting, rules, or exemptions can be found in the appendix (p 3).

A country was considered as implementing mandatory certificates if certificates were required for access to at least some frequently used public venues such as restaurants, or cultural events (appendix p 3). Some countries initially only applied certification rules to specific settings (eg, Switzerland for large events since July 1, 2021). In the main analysis, we focused only on general access restrictions and examined different settings in a secondary analysis. In some cases, certificates were introduced gradually across different settings, such as in Denmark where restrictions started in some settings on April 6 (eg, hairdressers), when most public places were closed, but later (April 29) restrictions were extended to include most public places.

Outcomes

The main outcome was new COVID-19 vaccination doses administered daily per capita, using a 7-day smoothed rolling average.¹³ For age-specific analyses, we added data from ECDC (appendix p 21). In secondary analyses, we tested the effect of certificates on daily reported COVID-19 cases. All health-related outcomes were adjusted by (age-specific) population (per million; appendix p 10).

Because age-specific data from ECDC are only available for somewhat larger age groups, we compiled original data on detailed age-specific vaccination rates directly from national sources for France,¹⁵ Israel,¹⁶ Italy,¹⁷ and Switzerland.¹⁸ Because data from Israel did not have age-specific counts of the total population, we supplemented this gap with population information.¹⁹ For Switzerland, age-specific vaccination data are only available on a weekly basis. We therefore applied the same method as described for the ECDC data (appendix p 21) to distribute vaccinations across the week.

Statistical analysis

We used a synthetic control method, which constructs a synthetic control country for each country that introduced mandatory COVID-19 certification.20, 21 The method re-weights non-treated countries from the potential pool of control units in a way that the average pretreatment trend (and other selected characteristics) is similar to the trend and characteristics of the single treated unit. More precisely, the method first calculates the importance of several covariates for the outcome and subsequently computes weights, which minimises the difference between the treatment and control groups in the importance-weighted covariates. The average from the synthetic control group provides a counterfactual trend of the outcome for the treated unit, and predicts how the vaccination rates in the treatment country would have developed in the absence of an intervention (introduction of certificates).

To weight the available control countries in a way that they closely resembled the respective treatment country, we selected several time-constant and time-varying characteristics. The time-constant country characteristics we used were (1) median age, (2) proportion of individuals aged 70 years and older, (3) life expectancy, (4) gross domestic product per capita, and (5) population density. The time-varying characteristics we used were (1) pretreatment outcome (1–7 days, 8–14 days, 15–21 days, and 22–28 days before intervention); (2) stringency sub-indices of 12 non-pharmaceutical interventions (1–21 days before intervention), including school closures, workplace closures, cancelled public events, restricted gatherings, public transport closures, stay-at-home orders, restricted internal movement, restricted international travel, protecting the elderly, testing policy, contact tracing, and face coverings;¹⁴ (3) daily COVID-19 cases (1–7 days, 8–14 days, 15–21 days, and 22–28 days before intervention); and (4) proportion of individuals with at least two COVID-19 vaccine doses (1–21 days before intervention). In our main analysis on vaccination rates, we backdated the intervention by 20 days to allow for potential anticipation effects linked to the planned introduction. Results without backdating were, however, qualitatively similar (appendix p 24).

In secondary analyses we examined reported cases of COVID-19 as the outcome, including the following variables (instead of daily cases and proportion fully vaccinated): (1) cumulative vaccinations per 100 inhabitants (1–7 days, 8–14 days, 15–21 days, and 22–28 days before intervention) and (2) daily tests per capita (8–21 days before intervention).

The following countries that did not introduce certification in this period were included in the pool of control countries: Austria, Belgium, Canada, Croatia, Czech Republic, Finland, the UK, Ireland, Lithuania, Luxembourg, the Netherlands, Poland, Portugal, Slovakia, Slovenia, Spain, Sweden, Turkey, and the USA. Control countries in Europe and North America were chosen as comparators given their relative similarity in the time-constant and time-varying traits described and availability of vaccines. After additional sensitivity checks, we excluded Norway from the control pool because of changes in vaccine eligibility (appendix p 6). Analyses were done using the R (version 4.0.5) package gsynth, and results visualised using ggplot.

To provide an estimate of the underlying uncertainty of our synthetic control group, we followed Vagni and Breen's work²² and calculated bootstrapped 95% CIs. Here, we used non-parametric bootstrapping with percentile intervals and 1000 bootstrap runs (appendix p 13). We also did robustness checks by randomly resampling 50% of the included covariates for reweighting the control countries (appendix p 14), and by conducting placebo studies (appendix p 18).²⁰ These additional analyses provide support for the main conclusions.

In secondary analyses, we used the generalised synthetic control method²³ to estimate an average effect across our selected cases in comparison with the remaining 163 countries included in Our World in Data. The generalised synthetic control method uses the information of control units and the pretreatment trends of the treated units to impute the expected outcome of the treated units after receiving the intervention.²⁴ As covariates, we included the predictors already described on a daily basis and a two-way fixed effects model to account for country-specific differences and time-specific shocks.

Role of the funding source

The funders of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report.

Results

Figure 1 shows the development over time of new daily vaccinations per million for the treated country (with certification) and the synthetic control group (19 countries with no certification), as well as the difference in daily vaccine doses per million before and after the introduction of mandatory certification. France had a lower rate of daily vaccinations per capita compared with other similar countries until 20 days before the intervention (appendix p 26). When mandatory certificates were announced in France on July 23, 2021, daily vaccinations started to increase to an above-average level and remained so after certificates were required (figure 1).

Figure 1.

Daily new vaccinations in France, Israel, and Italy before and after the introduction of a mandatory COVID-19 certificate for various settings compared with a reweighted synthetic control group

Data are from Our World in Data²⁵ and Oxford COVID-19 Government Response Tracker.²⁶ Shaded areas represent 95% CIs based on a non-parametric bootstrap procedure (1000 runs). The solid vertical line indicates the introduction of COVID-19 certification and the dashed vertical line indicates the 20-days' backdated intervention.

We quantified the relative and absolute gains in vaccine uptake of the anticipation effect of the introduction of certification (sum over 20 days before) and afterwards (sum over 40 days after; appendix p 26). The anticipation effect in France meant that vaccine uptake exceeded the control country by 55 672 (95% CI 49 668–73 707) vaccines per million population or, in absolute terms, 3 761 440 (3 355 761–4 979 952) doses (appendix p 26). Vaccine uptake in France up to 40 days after certification exceeded the control country by 72 151 (37 940–114 140) vaccines per million population, or 4 874 857 (2 563 396–7 711 769) doses (appendix p 26). Confidence bounds, however, indicate more certainty for the anticipation effect.

For Israel, we found similar results before re-implementing COVID-19 certification (figure 1). Although Israel started from a very low level of vaccinations 60 days before the intervention, it reached the control average before the intervention (figure 1). The re-implementation coincided with a sharp increase in daily vaccinations directly afterwards and 40 days after implementation rates were still more than 5000 doses per million inhabitants higher than they were in the synthetic control country without the intervention (figure 1). The anticipation effect was small, but we found an increase of 246 733 (95% CI 219 083–268 990) vaccines per million capita in total up to 40 days after or, in absolute terms, 2 168 728 (1 925 688–2 364 362) doses (appendix p 26).

For Italy, we also found a strong anticipation effect (41 629 [95% CI 24 942–53 132] doses per million population) before the announcement of COVID-19 certification, followed by a decrease slightly below the average of the synthetic control group (figure 1; appendix p 26). At 30 days after implementation, daily doses in Italy were 1370 doses (1177–2421) greater than those in the synthetic control group, again suggesting a positive relationship between certification and vaccine uptake (figure 1).

In Switzerland, approximately 1 month before the introduction of certificates, vaccination levels were lower than in control countries and daily doses were above average briefly before the intervention (appendix p 26). Again, vaccination rates exceeded the average of the control countries for 40 days after introduction of certification (47 380 [95% CI 9870–78 627] doses per million population; appendix p 26). Results based on different estimation methods support the long-lasting upward shift after certification (appendix p 23).

Country-specific differences are apparent given diverse contexts and reasons for introducing certification, which was largely to increase testing (Denmark) or in response to increasing cases (France, Germany, and Belgium) or low vaccine uptake (Italy and France). In Denmark and Germany, we did not find a clear effect of certification on vaccination rates (appendix p 8). In Denmark, vaccine certifications were introduced in times of limited vaccine supply and vaccination rates were greater than in other countries, with the aim to encourage regular testing. The certificate was discontinued from Sept 10, but was re-introduced in November due to rising cases. For Germany, vaccination rates increased 3 weeks after the intervention, but absolute differences were small (appendix p 9). Germany has a federal structure of decision making, which resulted in location-specific rules and exemptions from certification that were based on local incidence rates (appendix p 4). Germany was also in the midst of a federal election, with two leading candidates publicly opposing certification.⁶ Denmark and Germany did not start from a below-average level of vaccine uptake compared with control countries before the intervention (appendix p 26), indicating that the greatest leverage of certification is if vaccination rates are below average.

Secondary analyses on COVID-19 cases found that the effect of certificates on reported cases was difficult to assess on the basis of available data (appendix p 10). For some countries (France, Germany, Italy, and Switzerland), we found a reduction in cases after the intervention, whereas for others (Israel and Denmark) we found a continued increase above the rate in control countries (appendix p 10). The context of the pandemic trajectory when implementing the measure was important because many countries introduced certification during a period of increasing cases (figure 2 ).

Figure 2.

COVID-19 cases by selected countries that introduced COVID-19 certification, March 1 to Nov 1, 2021

Vertical lines indicate dates of introduction of COVID-19 certification. Certification was implemented on April 21, 2021, in Denmark (most restrictions lifted August–September); on July 29, 2021, in Israel (re-introduction of the certificate); on Aug 6, 2021, in Italy; on Aug 9, 2021, in France; on Aug 23, 2021, in Germany; and on Sept 13, 2021, in Switzerland.

Countries had different age-specific vaccine roll-outs (appendix p 6). We repeated previous analyses for France and Italy, but with age-specific vaccination rates (appendix p 26). Both countries showed a strong effect of certification (anticipation and afterwards) for those aged 18–24 years and 25–49 years, with smaller effects in older age groups (especially in France; appendix p 26), but we note that these older groups also had a longer eligibility period to be vaccinated compared with younger people. In France, we found a short period of increased uptake among those aged 18–24 years after intervention. In Italy, the youngest age group (18–24 years) had an increase in daily vaccinations directly before and after the intervention and another increase 2–3 weeks after certification (appendix p 26). We also compiled more detailed age-specific data for France, Israel, Italy, and Switzerland (figure 3 ). This analysis suggests that those younger than 20 years and aged 20–29 years old had increased uptake (figure 3).

Figure 3.

Daily new vaccinations by age group in France, Israel, Italy, and Switzerland before and after the introduction of a mandatory COVID-19 certificate

Data are from an open platform for French public data,¹⁵ Government Database Israel,¹⁶ Extraordinary Commissioner for the COVID-19 Emergency Italy,¹⁷ and the Federal Office of Public Health Switzerland.¹⁸ The vertical lines represent the date of implementing mandatory COVID-19 certificates.

One obvious concern is that our results are an artifact of age-related vaccine roll-out and are explained by the expanded eligibility to younger age groups during this period. To examine this possibility, we linked our findings to the timing of age-related eligibility (appendix p 6). For France, children aged 12 years and older were eligible for vaccination on June 15, 2021, much earlier than the introduction of the certificate in August. In Italy, vaccines were available for children aged 12 years and older on Aug 16, which could partially explain the increase in daily vaccines from mid-August, 2 weeks after certification was introduced (appendix p 3). However, eligibility does not explain the pronounced increase among 20–29-year-olds (figure 3), eligible for a longer period. The two youngest age groups (12–19 years and 20–29 years) in Italy continued to have the highest vaccination rates among all age groups after the intervention (figure 3). Findings were similar for both doses, but also for first dose only (appendix p 22).

Although the sharp increase in vaccinations among the youngest age groups in Israel could be attributed to second doses following the first spike, the intervention appears to be an uptake of first doses among 12–15-year-olds who became eligible on June 6. Eligibility for vulnerable 5–11-year-olds was introduced at the end of July, but this could not explain the sharp increase after certification was introduced. Although children aged 12–15 years were eligible on June 6, 2021, by the end of June only 2–4% of this group were vaccinated.²⁷ After a surge in school infections, at the end of June the Israeli Prime Minister urged citizens to vaccinate their children, warning that the allotted doses would expire on July 9. Health-care providers reported that appointments for 12–15-year-olds tripled, suggesting increased infections and a public announcement of urgency and waste might have prompted vaccinations.²⁷ We found an increase in vaccinations around the introduction of certificates (July 29) for first doses only, which seems independent of the earlier announcement (appendix p 22). Although smaller, we found an anticipation effect and an increase in uptake lasting approximately 40 days after the intervention for first doses in Israel (appendix p 22).

In Switzerland, vaccines were approved for 12–15-year-olds on June 4, 2021, considerably earlier than the introduction of stricter certification in mid-September. The roll-out depended on the Canton—eg, Zurich started to vaccinate those aged 12–15 years on June 25, and Luzern only at the end of July. This increased roll-out might have affected vaccine uptake in the initial introduction of targeted certification (July 7), but was not likely to have affected uptake after the main intervention date (mid-September). Moreover, the staged introduction across Cantons is unlikely to be responsible for the small spike around the introduction of the first certification stage in early July.

Switzerland first introduced some access restrictions (events >1000 participants and nightclubs), and later extended restrictions to general situations and activities (appendix p 7). These earlier access restrictions appear to have only affected vaccine uptake in those younger than 20 years, while uptake did not change among older groups (figure 4 ). Extending these restrictions to more general activities continued to affect uptake in those younger than 20 years, but also uptake among older age groups (30–39 years and 40–49 years; figure 4).

Figure 4.

Daily new vaccinations by age group in Switzerland before and after the introduction of a targeted and general mandatory COVID-19 certificate

Data are from the Federal Office of Public Health Switzerland.¹⁸ The vertical dashed line indicates targeted access restrictions (events with >1000 participants and nightclubs) and the vertical solid line indicates the introduction of general access restrictions (events with >30 participants, entire hospitality sector, and leisure activities).²⁸

Discussion

Mandatory COVID-19 certification was associated with a sharp increase in vaccination rates before implementation and had a long-lasting effect, with above-average rates after implementation. The effect was related to the average pre-intervention levels of vaccine uptake and reasons for introducing certification, with countries that had lower-than-average levels of uptake showing more pronounced effects (France and Israel), particularly in some age groups. The effect of certification was smaller in Italy, with above-average vaccination rates 20–40 days before intervention, but comparatively lower rates thereafter. We found no obvious effect in Germany, which already had an average uptake similar to that of control countries, or in Denmark, where certification was introduced in times of limited vaccine supply and the aim was largely to increase testing.

We found an age effect related to where certificates were introduced, with some settings (eg, nightclubs and large events) disproportionately affecting younger people. However, age-based changes in vaccine eligibility criteria must also be considered. Age-specific analysis revealed that particularly those younger than 20 years had the highest increased uptake after certification was introduced, followed by those aged 20–29 years, for both first and second vaccine doses. When certification was introduced in particular settings (nightclubs and events with >1000 people), increased uptake was found mainly in those younger than 20 years in Switzerland. When certification was extended to broader settings (events >30 people, the hospitality sector, and leisure activities) increases in daily vaccinations were also observed in those aged 20–49 years. The effect of certificates on COVID-19 infections was difficult to assess, but encouraging people who attend higher-risk venues such as nightclubs or large events to be vaccinated would logically reduce transmission risk.

Limitations of this study include not having access to granular daily age-based uptake for all countries nor the ability to examine confounders such as ethnicity or socioeconomic status. Certification was also introduced at different phases during the pandemic for different reasons and across diverse regional and national-level conditions that had varying levels of age eligibility, supply, vaccine hesitancy, trust, enforcement, and variation in infections and mortality. Vaccine hesitancy is also highly context-specific and influenced by the historical experiences of different sociodemographic groups.²⁹ A UK survey concluded that mandatory vaccine certificates were unlikely to affect vaccination intentions in those aged 18–24 years.⁷ The UK, however, has higher vaccine confidence compared with that in France³⁰ and Italy.³¹ Certificates were also introduced for different goals such as countering vaccine hesitancy in France and Italy, and in Israel to incentivise vaccine uptake with the use of certification for desirable events or settings.³² The generalisability of our findings should therefore be interpreted in the context of pre-existing levels of vaccine uptake, vaccine hesitancy, levels of trust in government and vaccinations, and the pandemic trajectory.

The introduction of mandatory certificates raises multiple ethical issues such as a risk of exacerbating inequalities among certain ethnic or socioeconomic groups that have lower uptake and trust in authorities. Certificates might also generate inequalities in contexts where differential age-specific roll-outs exist for vaccines or boosters. Certificates also have the potential to create a digital divide if passes are largely electronic or cannot be used across devices or national borders, and might raise data privacy and linkage concerns.¹ Although we found that certification increased vaccine uptake in certain settings and groups, COVID-19 certification alone will not increase vaccine uptake among all groups. Other measures such as geographically targeted vaccine drives³³ or peer-to-peer and community dialogue within low-trust groups to generate understanding might be more effective for certain groups.29, 33 COVID-19 certification is only part of a constellation of multiple policy levers that could be used to counter vaccine complacency and hesitancy and increase uptake.⁹ Younger people and certain groups such as men and those from low socioeconomic backgrounds have shown increased levels of vaccine complacency because of a lower perception of the risk of COVID-19,9, 10, 31 suggesting that mandatory certification could be one mechanism to increase uptake among those groups to reach population-level immunity and protect the broader population.

Data sharing

All data are publicly available and listed in this Article.

Declaration of interests

We declare no competing interests.