Quantifying the Population-Level Effect of the COVID-19 Mass Vaccination Campaign in Israel: A Modeling Study

Abstract

Background

Estimating real-world vaccine effectiveness is challenging as a variety of population factors can impact vaccine effectiveness. We aimed to assess the population-level reduction in cumulative severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) cases, hospitalizations, and mortality due to the BNT162b2 mRNA coronavirus disease 2019 (COVID-19) vaccination campaign in Israel during January–February 2021.

Methods

A susceptible-infected-recovered/removed (SIR) model and a Dynamic Survival Analysis (DSA) statistical approach were used. Daily counts of individuals who tested positive and of vaccine doses administered, obtained from the Israeli Ministry of Health, were used to calibrate the model. The model was parameterized using values derived from a previous phase of the pandemic during which similar lockdown and other preventive measures were implemented in order to take into account the effect of these prevention measures on COVID-19 spread.

Results

Our model predicted for the total population a reduction of 648 585 SARS-CoV-2 cases (75% confidence interval [CI], 25 877–1 396 963) during the first 2 months of the vaccination campaign. The number of averted hospitalizations for moderate to severe conditions was 16 101 (75% CI, 2010–33 035), and reduction of death was estimated at 5123 (75% CI, 388–10 815) fatalities. Among children aged 0–19 years, we estimated a reduction of 163 436 (75% CI, 0–433 233) SARS-CoV-2 cases, which we consider to be an indirect effect of the vaccine.

Conclusions

Our results suggest that the rapid vaccination campaign prevented hundreds of thousands of new cases as well as thousands of hospitalizations and fatalities and has probably averted a major health care crisis.

During the second half of December 2020, Israel launched a national vaccination campaign to promote coronavirus disease 2019 (COVID-19) vaccine use. This campaign was based on the BNT162b2 mRNA COVID-19 vaccine (Pfizer-BioNTech, Mainz, Germany) and was planned to include a large proportion of the Israeli adult population in a short time interval [1].

Vaccine effectiveness against symptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection at days 14–20 after the first dose and at 7 days following the second dose was 47%–57% and 92%–94%, respectively [2–6]. Postlicensure effectiveness studies are crucial to determine the population-level impact of a vaccine and determine the total impact of direct and indirect effects of the vaccine [7, 8]. But these studies are prone to limitations derived from their observational design and potential bias introduced by case ascertainment, surveillance, and data quality [9].

Estimating real-world vaccine effectiveness is challenging. A variety of population factors can impact vaccine effectiveness, including differences between vaccinated and unvaccinated individuals related to health-seeking behaviors and access to health care, prior health conditions, or demographic characteristics [10]. Effectiveness studies are not randomized, meaning vaccinated and unvaccinated individuals may be fundamentally different across these demographic and socioeconomic factors. Potential selection bias in the administration of the vaccine is typically unknowable, and careful statistical controls must be included to account for confounding [11]. Population differences may also bias observational studies because of differential exposure to infection or differences in access to care and health-seeking behaviors. Observational studies are not blinded, so vaccinated individuals may change behaviors that mitigate the probability of infection. Cases of disease reported to a surveillance system are not random and may reflect any number of biases [11].

Challenges related to delays in case reporting, weekend effects, censoring and truncation, and uneven geographic or population vaccination rollout can all impact estimates of effectiveness that rely on high-quality surveillance data.

Equally crucial is understanding the indirect effects of the vaccine. Indirect effects occur through 2 main mechanisms. First, vaccination can reduce symptoms and viral shedding, rendering infected individuals less infectious than unvaccinated individuals [12]. The recent SIREN study clearly demonstrated this effect with the BNT162b2 mRNA COVID-19 vaccines [9]. Second, vaccination can reduce the number of infected people in the population, thereby reducing the risk of infection among susceptible individuals. Effectiveness studies typically compare outcomes of vaccinated and nonvaccinated individuals and systematically underestimate the combined direct and indirect protective benefits to vaccinees [13, 14]. Quantifying the indirect effects of vaccination typically requires more time, innovative study design, and higher-quality data collection [12].

Though much has been written on methods for evaluating vaccine effectiveness [7, 15], most observational studies continue to use traditional statistical approaches that compare incidence rates in the vaccinated vs unvaccinated population. Studies do not account for confounding caused by other population-level mitigation strategies like lockdowns, business and school closures, or travel restrictions, which fundamentally alter the epidemiology of the disease under study. However, no matter how many potential confounding variables are controlled for, traditional statistical models cannot usually fully account for the dynamically changing biases and complex interactions/uncertainties present in any particular study. They also cannot fix problems with poor-quality or incomplete surveillance data. Mathematical models [16, 17] are another avenue by which to explore the population-level impact of the vaccine, including both direct and indirect effects [14]. Building a valid vaccination model for SARS-CoV-2 is particularly challenging because the changing dynamics of both infection and vaccination must be accounted for, reflecting the race between continuous spread of infection and the vaccination efforts restricted by logistics and supply limitations.

A viable mathematical model of SARS-CoV-2 vaccination has to take into account various complex interactions between multiple factors affecting the dynamics of the epidemic, like the initial disease prevalence, the compliance with nonpharmaceutical interventions (NPIs), the rate of growth or decay of infection at various times, the speed of the vaccine rollout, and its targeting and uptake [18]. In addition, it is important to assess the effect of vaccination not only in terms of efficacy and effectiveness, but also in estimations of the averted SARS-CoV-2 infections and COVID-19-related hospitalizations and fatalities, both in the vaccinated and unvaccinated populations. The magnitude of averted cases depends not only on the efficacy, but also on other factors such as disease incidence, degree of implementation of NPIs, compliance with vaccination recommendations, etc. Estimation of direct and indirect COVID-19-related burden averted following vaccination rollout may better characterize the benefit of the vaccination campaign beyond what randomized controlled trials and observational studies provide.

Due to a lack of data from randomized longitudinal trials, we developed a mathematical model that could be used with observational data to quantify the effect of vaccination as the infection spreads and public health countermeasures (eg, lockdowns and social distancing) are implemented.

Using an extended version of the standard compartmental susceptible-infected-recovered/removed (SIR) model and a Dynamic Survival Analysis (DSA) statistical approach [19] to estimate its parameters, we aimed to assess the population-level reduction in cumulative SARS-CoV-2 cases due to the BNT162b2 mRNA COVID-19 vaccination campaign in Israel. We used the SIR model [20] for disease transmission with 2 additional compartments for individuals vaccinated with only 1 dose and those vaccinated with 2 doses. Data on daily counts of individuals who tested positive and daily numbers of vaccine doses administered were used to calibrate the model. The statistical methodology to infer the parameters of the compartmental model is based on the DSA approach [19, 21], which combines classical dynamical systems theory and survival analysis. The DSA approach applies a simple algebraic manipulation to the SIR equations and allows us to apply tools from survival analysis to population-level epidemic data. The DSA approach accounts for changes in SARS-CoV-2 infections due to confounding effects of lockdown and other mitigation strategies, while simultaneously accounting for data-related challenges. This approach is particularly appropriate for the Israeli context as the effect of the vaccination campaign was slowed by a resurgence of COVID-19 cases, largely due to the rapid circulation of the B.1.1.7 variant [1, 22]. Consequently, January and February 2021 saw the highest rates of COVID-19-related fatalities and hospitalization of patients with severe disease.

The full effect of the vaccine has been difficult to estimate because it was launched simultaneously with nonpharmacological measures such as school closure and national lockdown. One of our model parameters accounts for the effective removal of individuals from the susceptible pool due to vaccination and NPIs such as lockdown. This parameter is learned empirically using the DSA method. The population-level effect of vaccines is then computed by setting this specific parameter to 0. This approach provides an objective and standardized way of assessing the population-level effect of vaccination in that it can be generalized to other populations.

Here, we use the DSA method to estimate the number of cases, hospitalizations, and fatalities prevented during the first 2 months of the mass vaccination campaign in Israel. We also estimate the indirect effect of vaccination in the adult population on the incidence of new SARS-CoV-2 cases in unvaccinated children. We used 2 methods to quantify the population-level impact on the reduction in cumulative SARS-CoV-2 infections due to rapid vaccination. Approach 1 simulates population-level daily counts of positive tests based on the model and known testing patterns when no vaccines are administered. We then compare this simulated number to the actual number of known positive tests to estimate the vaccine-attributable reduction in cases. We expect these simulated estimates to be higher because they assume no mitigation measures were enacted except for vaccination. However, this approach fails to separate the effect of vaccination from the confounding effect of lockdown and other preventive measures that occurred simultaneously on the vaccine rollout in Israel. Approach 2 parameterizes the model using values derived from a phase of the pandemic during which similar lockdown and other measures were implemented. In this second approach, we used the daily case counts from September 1, 2020, to November 1, 2020, a time window that saw a surge in cases followed by a strict lockdown. We expected these simulated estimates to be lower because the NPIs are explicitly incorporated into the model through parameterization, thereby reducing the estimated overall cumulative cases and attributing a smaller reduction in cumulative cases to the vaccine.

METHODS

Surveillance and Data

COVID-19 Cases

Daily counts of COVID-19 cases and fatalities attributed to COVID-19 were obtained from Ministry of Health reports and sites [23, 24].

COVID-19 Vaccinees

Daily counts of COVID-19 vaccinees (BNT162b2 mRNA COVID-19 vaccine) were obtained from the Ministry of Health reports and sites.

Population

The age-specific breakdown of the Israeli population was obtained from the Israel Central Bureau of Statistics [25].

Setting and Population

The vaccination campaign was launched on December 20, 2020. A timetable of the vaccination campaign and the relevant nonpharmacologic measures used to slow COVID-19 spread are detailed in the Supplementary Data.

Mathematical Model

We used the standard SIR compartmental model for disease transmission, along with 2 additional compartments for individuals vaccinated with only 1 dose and those vaccinated with 2 doses. A detailed description is provided in the Supplementary Data and Supplementary Table 1.

Patient Consent

The design of this work conforms to standards currently applied in Israel, and, according to the guideline of the Ministry of Health, this study is considered exempt from institutional review board approval as de-identified data from public sources were used.

RESULTS

Figure 1 shows the SARS-CoV-2 epidemic curve from March 1, 2020, through the end of the study period (February 28, 2021). There was a significant outbreak beginning in late August 2020, which was controlled through the second national lockdown. An additional large outbreak (“third wave”) began in December and coincided with the beginning of the mass vaccination campaign (Figure 2B). Since the start of the mass vaccination of the Israeli population (December 20, 2020) through the end of the study period (February 28, 2021), a total of 399 565 individuals contracted SARS-CoV-2 infection in Israel, with 7217 COVID-19-associated hospitalizations for moderate to severe conditions and 2681 COVID-19-associated deaths. During the study period, a gradual decline in the weekly number of COVID-19-associated hospitalizations was observed (beginning on January 17, 2021), as well as a gradual decline of weekly fatalities (Figure 2A & B) associated with SARS-CoV-2 infections beginning on January 24, 2021 (Figure 2B).

Figure 1.

Daily case counts of SARS-CoV-2-positive tests. Daily numbers of SARS-CoV-2-positive samples tested during March 2020–February 2021 are shown. The “epidemic waves” of COVID-19 in Israel are depicted; major time points, including lockdown periods, social restrictions, and the start of vaccinations, are noted. Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Figure 2.

SARS-CoV-2 infections, hospitalizations, fatalities, and vaccinations by date. A, Daily counts of positive SARS-CoV-2 tests. B, Time series and counts of the BNT162b2-mRNA COVID-19 vaccine first dose administered. Vaccinations began on December 19, 2020. C, Daily counts of COVID-19 moderately to severely ill hospitalizations. In order to account for potential changes in the definition of moderate, severe, and critical cases, we have combined the counts. D, Daily counts of COVID-19 fatalities. Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Figure 3A compares the actual number of cumulative SARS-CoV-2 cases in the entire Israeli population with the estimated number of cumulative cases under the no vaccination scenarios following the 2 approaches described above. The actual and estimated cumulative cases and 75% confidence bounds are shown in Table 1. Given the huge amount of uncertainty (as seen in the figures), we used 75% confidence bounds because 90% or 95% confidence bounds, which are more standard, would be too wide to be useful for our purpose. The purple dotted line (and the blue shaded regions indicating 75% confidence bounds) shows the no intervention scenario (Approach 1), while the black dashed lines (and the gray shaded region indicating 75% confidence bounds) correspond to the no vaccination scenario in which the parameters are trained on data from September 1, 2020, to November 1, 2020 (Approach 2). The solid red line indicates the actual number of cumulative cases in the population over the study period. Table 1 indicates for the total population a reduction of 913 057 (75% CI, 128 043–1 442 984) SARS-CoV-2 infections under Approach 1 and 648 585 (75% CI, 25 877–1 396 963) under Approach 2. The corresponding values per 1 million population were 98 708 and 70 117 averted SARS-CoV-2 infections under Approaches 1 and 2, respectively.

Figure 3.

Actual cumulative numbers, and calculated numbers under the no vaccination scenario, of SARS-CoV-2 infections, hospitalizations, and fatalities. Weekly numbers of SARS-CoV-2 infections, hospitalizations, and deaths are shown during January–February 2021, including the actual numbers and the calculated numbers of the no vaccination scenario utilizing the 2 different approaches. A, The actual and calculated numbers of the no vaccination scenario of SARS-CoV-2 weekly positive tests are shown for the whole population. B, The actual and calculated numbers of the no vaccination scenario of weekly SARS-CoV-2-positive tests are shown for children aged 0–19 years, demonstrating the indirect effect of vaccination on the young population (<20 years of age). C, The actual and calculated numbers of the no vaccination scenario of weekly SARS-CoV-2 hospitalizations are shown, demonstrating the effect of vaccination on hospitalizations for moderate to severe conditions. D, The actual and calculated numbers of the no vaccination scenario of weekly SARS-CoV-2 deaths are shown, demonstrating the effect of vaccination on cumulative fatality. The solid red curve corresponds to actual case counts. The purple dotted line (and blue shaded regions indicating 75% confidence bounds) shows the no intervention scenario modeled under Approach 1. The black dashed line (and gray shaded region indicating 75% confidence bounds) corresponds to the no vaccination scenario modeled under Approach 2. Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Table 1.

Actual Cumulative Number of COVID-19 Cases in the Total Population, in the Population <20 Years of Age, Hospitalizations, and Mortality and the Population-Level Effect of Vaccination on the Estimated Cumulative Positive Tests Under No Intervention and the Estimated Reduction in Cases With Vaccination, Following 2 Different Simulation Approaches

Estimate	Actual Cumulative Number as of 2/28/2021	Approach 1		Approach 2
		Estimated Cumulative Positive Tests Under No Intervention (75% CI)	Estimated Reduction in Cases With Vaccination (75% CI)	Estimated Cumulative Positive Tests Under No Intervention (75% CI)	Estimated Reduction in Cases With Vaccination (75% CI)
Total population	774 045	1 687 102	913 057	1 422 630	648 585
		(902 088–2 217 029)	(128 043–1 442 984)	(799 922–2 171 008)	(25 877–1 396 963)
Children (age <20 y)	500 286	1 155 005	654 719	663 722	163 436
		(614 395–1 522 481)	(114 109–1 022 195)	(439 237–933 509)	(0–433 233)
Hospitalization	16 941	39 784	22 843	33 042	16 101
		(21 850–52 247)	(4909–35 306)	(18 951–49 976)	(2010–33 035)
Mortality	5778	13 167	7389	10 901	5123
		(7140–17 356)	(1362–11 578)	(6165 16 593)	(388–10 815)

Abbreviation: COVID-19, coronavirus disease 2019.

Figure 3B and Table 1 show the actual number of cumulative SARS-CoV-2 infections in the population <20 years of age and the estimated number of cumulative cases under the 2 simulation approaches. We consider this to be an indirect effect of the vaccine as the population age <20 years was not eligible for vaccination until the end of February. Again, the solid red line indicates the actual number of cumulative cases in this population (500 286). The indirect effect of the vaccine equates to an estimated reduction of 654 719 (75% CI, 114 109–1 022 195) under Approach 1 and 163 436 under Approach 2 (75% CI, 0–433 233). The corresponding values per 1 million pediatric population (aged 0–19 years) were 198 400 and 49 526 SARS-CoV-2 infections averted under Approaches 1 and 2, respectively. The simulated daily SARS-CoV-2-positive tests in the entire population and among the younger population (<20 years of age) on no interventions and no vaccination regimes are shown in Supplementary Figure 1.

Figure 3C and Table 1 show the effect of vaccination on COVID-19-related moderate to severe hospitalizations. As of February 28, 2021, the cumulative number of hospitalizations was 16 941 (red line). Under Approach 1 (purple dotted line), the reduction in hospitalizations was estimated at 22 843 (75% CI, 4909–35 306), and under Approach 2, it was 16 101 (75% CI, 2010–33 035). The corresponding values per 1 million population were 2470 and 1741 averted hospitalizations under Approaches 1 and 2, respectively.

Figure 3D and Table 1 show the effect of vaccination on the cumulative number of COVID-19-related fatalities. Approach 1 estimates a reduction of 7389 deaths (75% CI, 1362–11 578), while Approach 2 estimates a reduction of 5123 deaths (75% CI, 388–10 815).

The corresponding values per 1 million population were 799 and 554 averted deaths associated with SARS-CoV-2 infections under Approaches 1 and 2, respectively. The simulated population-level daily SARS-CoV-2 hospitalizations and mortality under the no intervention and no vaccination regimes are shown in Supplementary Figure 2.

A comparison of simulated cumulative count of positive tests and daily counts of positive tests with true daily counts is shown in Figure 4. In the figure, the solid red lines show the actual trajectories. The means of the simulated trajectories are shown as a broken line in purple.

Figure 4.

Comparison of actual SARS-CoV-2 infections and simulated counts during January 8–January 28, 2021. Comparison of simulated cumulative counts of positive counts and daily counts of positive tests with true counts is shown. As of January 8, 2021, the cumulative count of positive tests was 480 338. As demonstrated in the figure, the simulated trajectories lie close to the true counts of cumulative positive tests. The solid red line shows the actual trajectories. The means of the simulated trajectories are shown as a broken line in purple. The shaded blue regions indicate 75% confidence around the mean trajectory. A, Comparison of the cumulative counts of positive counts against the true counts. B, Comparison of the fitted daily counts of positive tests against true counts of daily positive tests. The dips in the counts are due to weakened effects. Even though the true trajectory of daily counts of positive tests is unsmooth, the simulated counts of daily positive tests are generally close to it. In particular, the true trajectory lies entirely within the 75% confidence bounds, indicating a good fit of the model to the data. Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

As of January 8, 2021, the cumulative count of positive tests was 480 338. As seen in Figure 4, the simulated trajectories lie close to the true counts of cumulative positive tests (Figure 4A) and daily counts of positive tests (Figure 4B). The true trajectory lies entirely within the 75% confidence bounds (shaded blue regions), indicating a good fit of the model to the data. This comparison demonstrates good matching between the simulated and the actual curves of daily counts. Further comparisons are provided in Supplementary Figures 3–10.

DISCUSSION

This study evaluated the impact of the BNT162b2 mRNA vaccine in the Israeli population utilizing a mathematical model that enumerated the number of averted COVID-19 cases as a result of the mass vaccination in Israel. Under Approach 2, which parameterizes the model using values derived from a phase of the pandemic during which similar lockdown and other preventive measures were implemented, the estimated number of cases averted during the study period was 70 117/1 000 000, with estimated prevention of 1741/1 000 000 hospitalizations for moderate to severe conditions and 554 fatalities per 1 million population. We also evaluated the indirect effect in children, who during the study period were not yet vaccinated but would be offered protection from widespread vaccination of adults. As children may be less susceptible to COVID-19 infection and less infectious than adults (at least with the pre-B.1.1.7 circulating SARS-CoV-2 variants), interaction with adults may have been a major driver to SARS-CoV-2 infection among children, and we therefore hypothesized that the prevention of SARS-CoV-2 infections in adults would be accompanied by a decline in pediatric COVID-19 cases [26–28]. In line with this hypothesis, the results of the study revealed that under Approach 2, a total of 163 436 COVID-19 cases in children aged 0–19 were averted (averted rate: 49 526 cases per 1 million pediatric population) due to vaccination of the adult population.

Our study adds another important avenue for understanding BNT162b2 mRNA vaccine effectiveness and its impact on population-level infection rates. It should be pointed out that the use of a highly effective vaccine does not necessarily result in the prevention of many new cases, as if new cases could have been prevented by other means such as altered public behavior, the effect of the vaccine may not be apparent. Traditional observational studies, which use the same type of surveillance data we use here, are sensitive to problems with data quality and often cannot adequately account for changes in SARS-CoV-2 infection due to confounding effects of mitigation strategies. The DSA modeling approach directly accounts for the impact of additional mitigation measures in the parameterization of the model (especially in Approach 2), thereby providing a novel method for enumerating the effectiveness of the vaccine in reducing excess morbidity. This approach also provides estimates of uncertainty, which can strengthen inferences when data quality is an issue. Moreover, despite being relative, the lockdown measures introduced at different phases of the pandemic will have different effects on the overall rate of infection [27]. However, this can be properly captured by a modeling-based analysis such as ours but not necessarily by empirical studies.

One of the major lessons of this study has been the vast importance of a rapid vaccination campaign as results suggest that a slower pace of vaccination in Israel could have resulted in the addition of hundreds of thousands of new cases as well as thousands of hospitalizations and fatalities. Such a large number of hospitalizations would have resulted in a major health care crisis like those seen in other countries [29, 30].

There are several limitations that should be mentioned. This study is based on a mathematical model that is sensitive to initial parameterization and therefore prone to inherent errors in assumptions. Another limitation is that the concurrent nonpharmacologic measures implemented during the vaccination campaign could have potentially averted some and even most of these cases without the vaccination campaign. However, the parameters used in approach 2 were derived from a phase of the pandemic during which similar mitigation measures were implemented—for example, September 2020, when there was a national lockdown and school closure. When we assessed model fit for this early segment of the time series, the close match between the actual trajectories of positive cases and the simulated ones indicated a very good fit of the model to the observed data. Thus, extrapolation of the model into a later time period and comparing these estimates with real-world data demonstrated the model’s reliability.

We assume a “mass action” mode of disease transmission in this study. The empirical analysis (Figure 4) confirms that this assumption is acceptable for our current purpose. In a sense, one may think about our analysis as a way of averaging the agent-based dynamics, which despite being more realistic is also difficult to calibrate from empirical data. As we are concerned with an overall population-level effect, the use of an average transmission network appears acceptable in our case [31–35].

The main strength of our study is that it is based on a reliable national database and is in line with the recent data that show the real-life vaccine effectiveness in the Israeli population. This report illustrates the effect of the rapid implementation of COVID-19 vaccination at a national scale and suggests that the accompanying models serve as a paradigm for other national COVID vaccination programs.