Abstract
Background
As mass vaccination campaigns against coronavirus disease 2019 (Covid-19) commence worldwide, vaccine effectiveness needs to be assessed for a range of outcomes across diverse populations in a noncontrolled setting. In this study, data from Israel’s largest health care organization were used to evaluate the effectiveness of the BNT162b2 mRNA vaccine.
Methods
All persons who were newly vaccinated during the period from December 20, 2020, to February 1, 2021, were matched to unvaccinated controls in a 1:1 ratio according to demographic and clinical characteristics. Study outcomes included documented infection with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), symptomatic Covid-19, Covid-19–related hospitalization, severe illness, and death. We estimated vaccine effectiveness for each outcome as one minus the risk ratio, using the Kaplan–Meier estimator.
Results
Each study group included 596,618 persons. Estimated vaccine effectiveness for the study outcomes at days 14 through 20 after the first dose and at 7 or more days after the second dose was as follows: for documented infection, 46% (95% confidence interval [CI], 40 to 51) and 92% (95% CI, 88 to 95); for symptomatic Covid-19, 57% (95% CI, 50 to 63) and 94% (95% CI, 87 to 98); for hospitalization, 74% (95% CI, 56 to 86) and 87% (95% CI, 55 to 100); and for severe disease, 62% (95% CI, 39 to 80) and 92% (95% CI, 75 to 100), respectively. Estimated effectiveness in preventing death from Covid-19 was 72% (95% CI, 19 to 100) for days 14 through 20 after the first dose. Estimated effectiveness in specific subpopulations assessed for documented infection and symptomatic Covid-19 was consistent across age groups, with potentially slightly lower effectiveness in persons with multiple coexisting conditions.
Conclusions
This study in a nationwide mass vaccination setting suggests that the BNT162b2 mRNA vaccine is effective for a wide range of Covid-19–related outcomes, a finding consistent with that of the randomized trial.
Mass vaccination campaigns using newly approved vaccines against the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)1,2 are beginning in many parts of the world. Randomized clinical trials of mRNA-based vaccines reported efficacies for preventing coronavirus 2019 (Covid-19) in the range of 94%2 to 95%.1
Although randomized clinical trials are considered the “gold standard” for evaluating intervention effects, they have notable limitations of sample size and subgroup analysis, restrictive inclusion criteria, and a highly controlled setting that may not be replicated in a mass vaccine rollout. For example, the phase 3 trial of the BNT162b2 mRNA vaccine against Covid-19 included 21,720 persons who were randomly assigned to the vaccinated group, which permitted estimates of vaccine efficacy in only a small number of subpopulations.1 Moreover, patients with chronic diseases were included only if the conditions were deemed stable by the investigators.3 It is also important to see whether in a scaled-up vaccination program such factors as suboptimal adherence to vaccination schedules and vaccine-handling logistics influence vaccine effectiveness. Postauthorization analyses can thus meet the urgent need to evaluate the effectiveness of Covid-19 vaccines across diverse populations with a wide range of coexisting conditions, in the midst of imperfect adherence to vaccination protocols and the challenges of cold-chain maintenance and vaccine-deployment logistics.
We leveraged the integrated data repositories of Israel’s largest health care organization to evaluate Covid-19 vaccine effectiveness for five outcomes: documented SARS-CoV-2 infection, symptomatic Covid-19, hospitalization, severe illness, and death. Using this observational data set, we evaluated the effectiveness over time and in subpopulations defined by age, sex, and coexisting conditions.
Methods
Study Population
We analyzed data from Clalit Health Services (CHS), the largest of four integrated health care organizations in Israel, which insures 4.7 million patients (53% of the population). A description of the CHS data repositories used for this study is provided in the Supplementary Appendix (Supplementary Methods 1), available with the full text of this article at NEJM.org. Information on authors’ access to these repositories as well as authors’ contributions to the study is provided in Supplementary Methods 2. This study was approved by the CHS institutional review board. The study was exempt from the requirement for informed consent.
Study Design
We designed this observational study to emulate a target trial of the causal effect of the BNT162b2 vaccine on Covid-19 outcomes.4 Eligibility criteria included an age of 16 years or older, not having a previously documented positive SARS-CoV-2 polymerase-chain-reaction (PCR) test, and being a member of the health care organization during the previous 12 months.
Population groups in which internal variability in the probability of exposure or the outcomes is high and controlling for the high variability is not feasible (e.g., high variability in infection risk among patient-facing health care workers in dedicated Covid-19 wards as compared with administrative staff) were excluded. Such population groups are persons not having a documented geostatistical living area, those who have had interactions with the health care system during the preceding 3 days that may indicate the start of symptomatic disease and may preclude vaccination, nursing home residents, persons medically confined to the home, or health care workers.
Each day during the period from December 20, 2020, to February 1, 2021, all newly vaccinated persons were matched in a 1:1 ratio to unvaccinated controls. For each person, follow-up ended at the earliest of the following events: occurrence of an outcome event, death unrelated to Covid-19, vaccination (for unvaccinated controls), vaccination of the matched control (for vaccinated persons), or the end of the study period. Newly vaccinated persons were eligible for inclusion in the study, even if they had previously been selected as a control.
We matched vaccine recipients and controls on variables associated with the probability of both vaccination and infection or severity of Covid-19: age, sex, sector (general Jewish, Arab, or ultra-Orthodox Jewish), neighborhood of residence (since disease activity and vaccination uptake vary greatly across defined geostatistical areas), history of influenza vaccination during the preceding 5 years (0, 1 or 2, 3 or 4, or ≥5 vaccinations), pregnancy (a potential risk factor for severe Covid-195 and associated with the rate of vaccination owing to evolving vaccination guidelines for pregnant women), and the total number of coexisting conditions that had been identified by the Centers for Disease Control and Prevention (CDC) as risk factors for severe Covid-19 as of December 20, 2020.6,7 (See Supplementary Methods 3 for additional information about the matching process. The protocol and statistical analysis plan are available at NEJM.org.)
The five outcomes of interest were documented SARS-CoV-2 infection confirmed by positive PCR test, documented symptomatic Covid-19, hospital admission for Covid-19, severe Covid-19 (according to National Institutes of Health criteria)8 and death from Covid-19. Each of these outcomes includes the outcomes that follow it. In a supplementary analysis, we also evaluated an additional outcome, SARS-CoV-2 infection without documented symptoms, as an imperfect proxy for asymptomatic infection (since mild symptoms may not be documented).
Table S1 provides details on definitions of variables. Persons with missing data for smoking status or body-mass index (BMI) were dropped from the analysis.
Statistical Analysis
Covariate balance after matching was evaluated with the use of a plot of the mean differences between variable values (standardized for continuous variables) for the vaccinated and unvaccinated groups, with a difference of 0.1 or less considered to be acceptable.9 Survival curves for the vaccinated and unvaccinated groups were estimated with the Kaplan–Meier estimator.10 We considered three periods: days 14 through 20 after the first dose of vaccine, days 21 through 27 after the first dose (administration of the second dose was scheduled to occur on day 21 after the first dose), and day 7 after the second dose until the end of the follow-up. For each period, we used the Kaplan–Meier estimator with daily outcome and censoring events to compute the probability (“risk”) of the outcome during the period, using matched pairs in which both persons were still at risk at the beginning of the period. We then calculated risk ratios for vaccination as compared with no vaccination and estimated the vaccine effectiveness as one minus the risk ratio. We estimated the vaccine effectiveness only in analyses in which there were more than 10 instances of an outcome across the two groups.
The period immediately after the first dose, when immunity is gradually building,1 was excluded in the main analyses because the risk ratio is expected to be close to 1 during this period. In secondary analyses, we considered the periods from day 0 through day 20 and day 0 through day 27, to avoid a potential selection bias in the main analyses that were restricted to persons whose data remained uncensored at the beginning of each period (see Supplementary Methods 4).11-13 We also conducted a sensitivity analysis in the 6 days after the second dose of vaccine among those who received a second dose. A further sensitivity analysis estimated the hazard ratio each day for the documented SARS-CoV-2 infection outcome.
We performed an additional sensitivity analysis to assess the potential for selection bias due to informative censoring. In this analysis, data on controls who were subsequently vaccinated were censored only after 7 days (i.e., after the period with little or no vaccine effect) plus the median time from documented Covid-19 diagnosis to the outcome being studied.
We calculated 95% confidence intervals using the percentile bootstrap method with 500 repetitions. Analyses were performed with the use of R software, version 4.0.2.
Results
Study Population
Of 1,503,216 CHS members who were vaccinated, 1,163,534 were eligible for the study and 596,618 were matched to unvaccinated controls (Figure 1). Matched persons were younger than the eligible population overall and had a lower prevalence of chronic conditions because there was a smaller pool of potential unvaccinated matches for older vaccine recipients, owing to high vaccination rates in the older population (Table S2 and Fig. S1). The baseline characteristics of the matched persons are shown in Table 1. All variables were well balanced between the study groups (Fig. S2). About 0.6% of persons with missing data on smoking status or body-mass index were dropped from the analysis (Figure 1). Data for 44% of the unvaccinated controls and their matched pairs were censored when the controls received the vaccine.
Figure 1. Study Population and Cohort Enrollment Process, December 20, 2020, to February 1, 2021.
The 1,503,216 persons vaccinated before February 1, 2021, were also required to be without a documented SARS-CoV-2 PCR-positive result before the vaccination date. Absolute numbers and percentage changes are shown for each inclusion and exclusion criterion. The exclusion process was gradual and occurred in phases; persons could have had more than one reason for exclusion. The same exclusion criteria were applied to the unvaccinated persons for each index date in which they were considered for matching. The chart focuses on the vaccinated population. CHS denotes Clalit Health Services.
Table 1. Demographic and Clinical Characteristics of Vaccinated Persons and Unvaccinated Controls at Baseline.*.
| Characteristics | Unvaccinated Controls (N=596,618) |
Vaccinated Persons (N=596,618) |
|---|---|---|
| Median age (IQR) — yr | 45 (35–62) | 45 (35–62) |
| Age group — no. (%) | ||
| 16 to 39 yr | 213,090 (35.7) | 213,090 (35.7) |
| 40 to 49 yr | 130,752 (21.9) | 130,752 (21.9) |
| 50 to 59 yr | 85,609 (14.3) | 85,609 (14.3) |
| 60 to 69 yr | 88,153 (14.8) | 88,153 (14.8) |
| 70 to 79 yr | 56,946 (9.5) | 56,946 (9.5) |
| ≥80 yr | 22,068 (3.7) | 22,068 (3.7) |
| Sex — no. (%) | ||
| Female | 298,059 (50.0) | 298,059 (50.0) |
| Male | 298,559 (50.0) | 298,559 (50.0) |
| Population sector — no. (%) | ||
| General Jewish | 463,234 (77.6) | 463,234 (77.6) |
| Arab | 120,896 (20.3) | 120,896 (20.3) |
| Ultra-Orthodox Jewish | 12,488 (2.1) | 12,488 (2.1) |
| No. of risk factors according to CDC criteria — no. (%) | ||
| 0 | 338,384 (56.7) | 338,384 (56.7) |
| 1 | 140,779 (23.6) | 140,779 (23.6) |
| 2 | 55,766 (9.3) | 55,766 (9.3) |
| 3 | 29,273 (4.9) | 29,273 (4.9) |
| ≥4 | 32,416 (5.4) | 32,416 (5.4) |
| No. of influenza vaccinations during preceding 5 yr — no. (%) | ||
| 0 | 351,141 (58.9) | 351,141 (58.9) |
| 1 or 2 | 116,200 (19.5) | 116,200 (19.5) |
| 3 or 4 | 50,441 (8.5) | 50,441 (8.5) |
| ≥5 | 78,836 (13.2) | 78,836 (13.2) |
| CDC “certain” risk criteria — no. of persons (%) | ||
| Cancer | 11,946 (2.0) | 11,595 (1.9) |
| Chronic kidney disease | 40,568 (6.8) | 40,587 (6.8) |
| Chronic obstructive pulmonary disease | 12,667 (2.1) | 11,131 (1.9) |
| Heart disease | 39,165 (6.6) | 38,913 (6.5) |
| Solid-organ transplantation | 495 (0.1) | 435 (0.1) |
| Obesity: BMI, 30 to 40 | 100,584 (16.9) | 105,476 (17.7) |
| Severe obesity: BMI, ≥40 | 9,856 (1.7) | 8,920 (1.5) |
| Pregnancy | 1,508 (0.3) | 1,508 (0.3) |
| Sickle cell disease | 98 (<0.1) | 109 (<0.1) |
| Smoking | 118,733 (19.9) | 97,881 (16.4) |
| Type 2 diabetes mellitus | 66,198 (11.1) | 65,343 (11.0) |
| CDC “possible” risk criteria — no. of persons (%) | ||
| Asthma | 32,114 (5.4) | 29,814 (5.0) |
| Cerebrovascular disease | 18,392 (3.1) | 17,792 (3.0) |
| Other respiratory disease | 2,198 (0.4) | 2,014 (0.3) |
| Hypertension | 101,017 (16.9) | 103,028 (17.3) |
| Immunosuppression | 15,823 (2.7) | 16,180 (2.7) |
| Neurologic disease | 25,897 (4.3) | 24,111 (4.0) |
| Liver disease | 11,109 (1.9) | 9,699 (1.6) |
| Overweight: BMI, 25 to 30 | 203,296 (34.1) | 212,778 (35.7) |
| Thalassemia | 3,764 (0.6) | 3,967 (0.7) |
| Type 1 diabetes mellitus | 2,309 (0.4) | 2,406 (0.4) |
The 86,601 persons who were first recruited as unvaccinated controls and then, after vaccination, were re-recruited as vaccinated persons appear in both groups. BMI denotes body-mass index (the weight in kilograms divided by the square of the height in meters), CDC Centers for Disease Control and Prevention, and IQR interquartile range.
Vaccine Effectiveness
During a mean follow-up of 15 days (interquartile range, 5 to 25), 10,561 infections were documented (0.6 infections per 1000 person-days), of which 5996 (57%) were symptomatic Covid-19 illness, 369 required hospitalization, 229 were severe cases of Covid-19, and 41 resulted in death. Hospitalizations, severe disease, and death occurred at increasing time spans from diagnosis (median times, 1, 5, and 11 days, respectively; see Fig. S3). Of persons who had 21 or more days of follow-up, 96% received a second dose of vaccine (95% of whom received it before day 24).
Figure 2 shows the cumulative incidence curves for the included outcomes, and Table 2 shows the estimated vaccine effectiveness for the main outcomes and time periods. During the period from 14 to 20 days after the first dose, the estimated vaccine effectiveness for documented infection was 46% (95% confidence interval [CI], 40 to 51); symptomatic Covid-19 illness, 57% (95% CI, 50 to 63); hospitalization, 74% (95% CI, 56 to 86); severe illness, 62% (95% CI, 39 to 80); and death, 72% (95% CI, 19 to 100). During the period from 21 to 27 days after the first dose, the estimated effectiveness for these outcomes was 60% (95% CI, 53 to 66), 66% (95% CI, 57 to 73), 78% (95% CI, 61 to 91), 80% (95% CI, 59 to 94), and 84% (95% CI, 44 to 100), respectively. In the follow-up period starting 7 days after the second dose, the vaccine effectiveness for documented infections, symptomatic illness, hospitalization, and severe disease was 92% (95% CI, 88 to 95), 94% (95% CI, 87 to 98), 87% (95% CI, 55 to 100), and 92% (95% CI, 75 to 100), respectively. The daily value for one minus the hazard ratio for the documented infection outcome is included in Figure S4; it is consistent with a gradual daily increase in vaccine effectiveness.
Figure 2. Cumulative Incidence of the Five Outcomes.
Cumulative incidence curves (1 minus the Kaplan–Meier risk) for the various outcomes are shown, starting from the day of administration of the first dose of vaccine. Shaded areas represent 95% confidence intervals. The number at risk at each time point and the cumulative number of events are also shown for each outcome. Graphs in which all data are shown with a y axis scale from 0 to 100 (along with the data shown, as here, on an expanded y axis) are provided in Figure S8 in the Supplementary Appendix.
Table 2. Estimated Vaccine Effectiveness against Covid-19 Outcomes during Three Time Periods.*.
| Period | Documented Infection | Symptomatic Illness | Hospitalization | Severe Disease | Death | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1−RR | Risk Difference | 1−RR | Risk Difference | 1−RR | Risk Difference | 1−RR | Risk Difference | 1−RR | Risk Difference | |
| % (95% CI) | no./1000 persons (95% CI) | % (95% CI) | no./1000 persons (95% CI) | % (95% CI) | no./1000 persons (95% CI) | % (95% CI) | no./1000 persons (95% CI) | % (95% CI) | no./1000 persons (95% CI) | |
| 14 to 20 days after first dose | 46 (40–51) |
2.06 (1.70–2.40) |
57 (50–63) |
1.54 (1.28–1.80) |
74 (56–86) |
0.21 (0.13–0.29) |
62 (39–80) |
0.14 (0.07–0.21) |
72 (19–100) |
0.03 (0.01–0.07) |
| 21 to 27 days after first dose | 60 (53–66) |
2.31 (1.96–2.69) |
66 (57–73) |
1.34 (1.09–1.62) |
78 (61–91) |
0.22 (0.13–0.31) |
80 (59–94) |
0.18 (0.10–0.27) |
84 (44–100) |
0.06 (0.02–0.11) |
| 7 days after second dose to end of follow-up | 92 (88–95) |
8.58 (6.22–11.18) |
94 (87–98) |
4.61 (3.29–6.53) |
87 (55–100) |
0.22 (0.08–0.39) |
92 (75–100) |
0.32 (0.13–0.52) |
NA | NA |
Confidence intervals were estimated using the percentile bootstrap method with 500 repetitions. Estimates were calculated only for cells with more than 10 instances of an outcome across the two groups. NA denotes not available, and RR risk ratio.
Table 3 shows the estimated vaccine effectiveness for documented SARS-CoV-2 infection and Covid-19 outcomes in subpopulations defined by age, sex, and coexisting conditions. The estimates are consistent with similar effectiveness across age groups and slightly lower effectiveness among patients with multiple coexisting conditions.
Table 3. Estimated Vaccine Effectiveness against Covid-19 Outcomes in Subpopulations According to Characteristics at Baseline.*.
| Characteristic and Period | Documented Infection | Symptomatic Illness | ||
|---|---|---|---|---|
| 1−RR | Risk Difference | 1−RR | Risk Difference | |
| % (95% CI) | no./1000 persons (95% CI) | % (95% CI) | no./1000 persons (95% CI) | |
| Male sex | ||||
| 14 to 20 days after first dose | 41 (32 to 50) | 1.71 (1.22 to 2.21) | 52 (41 to 61) | 1.26 (0.90 to 1.62) |
| 21 to 27 days after first dose | 57 (48 to 65) | 2.25 (1.76 to 2.75) | 62 (49 to 72) | 1.30 (0.92 to 1.67) |
| 7 days after second dose to end of follow-up | 91 (80 to 96) | 7.33 (4.48 to 10.84) | 88 (71 to 98) | 2.90 (1.87 to 4.02) |
| Female sex | ||||
| 14 to 20 days after first dose | 50 (41 to 57) | 2.39 (1.84 to 2.86) | 60 (52 to 68) | 1.81 (1.43 to 2.19) |
| 21 to 27 days after first dose | 63 (55 to 71) | 2.38 (1.91 to 2.91) | 69 (58 to 78) | 1.38 (1.02 to 1.71) |
| 7 days after second dose to end of follow-up | 93 (88 to 97) | 9.75 (6.84 to 13.48) | 96 (90 to 100) | 6.22 (3.60 to 9.56) |
| Age, 16 to 39 yr | ||||
| 14 to 20 days after first dose | 49 (41 to 57) | 2.29 (1.74 to 2.88) | 57 (46 to 68) | 1.38 (0.99 to 1.80) |
| 21 to 27 days after first dose | 64 (54 to 72) | 2.80 (2.20 to 3.48) | 67 (52 to 78) | 1.27 (0.89 to 1.73) |
| 7 days after second dose to end of follow-up | 94 (87 to 97) | 8.72 (5.72 to 12.69) | 99 (96 to 100) | 4.06 (2.76 to 5.66) |
| Age, 40 to 69 yr | ||||
| 14 to 20 days after first dose | 47 (40 to 55) | 2.13 (1.69 to 2.66) | 59 (50 to 67) | 1.68 (1.32 to 2.05) |
| 21 to 27 days after first dose | 58 (49 to 67) | 2.19 (1.67 to 2.70) | 65 (53 to 74) | 1.38 (1.03 to 1.80) |
| 7 days after second dose to end of follow-up | 90 (82 to 95) | 8.96 (6.16 to 13.05) | 90 (75 to 98) | 5.01 (2.53 to 8.67) |
| Age, ≥70 yr | ||||
| 14 to 20 days after first dose | 22 (−9 to 44) | 0.81 (−0.28 to 1.89) | 44 (19 to 64) | 1.36 (0.48 to 2.36) |
| 21 to 27 days after first dose | 50 (19 to 72) | 1.40 (0.42 to 2.35) | 64 (37 to 83) | 1.35 (0.62 to 2.22) |
| 7 days after second dose to end of follow-up | 95 (87 to 100) | 6.10 (3.43 to 9.61) | 98 (90 to 100) | 4.77 (2.14 to 7.70) |
| No coexisting conditions | ||||
| 14 to 20 days after first dose | 49 (42 to 56) | 2.13 (1.69 to 2.59) | 55 (45 to 63) | 1.32 (0.98 to 1.67) |
| 21 to 27 days after first dose | 66 (58 to 73) | 2.49 (1.99 to 2.98) | 73 (62 to 82) | 1.27 (0.92 to 1.64) |
| 7 days after second dose to end of follow-up | 91 (83 to 96) | 7.67 (4.90 to 11.07) | 93 (78 to 100) | 3.54 (1.79 to 5.90) |
| One or two coexisting conditions | ||||
| 14 to 20 days after first dose | 43 (32 to 53) | 2.05 (1.41 to 2.73) | 57 (45 to 66) | 1.74 (1.25 to 2.24) |
| 21 to 27 days after first dose | 56 (45 to 65) | 2.43 (1.77 to 3.16) | 62 (47 to 73) | 1.56 (1.05 to 2.06) |
| 7 days after second dose to end of follow-up | 95 (88 to 98) | 10.53 (6.73 to 14.40) | 95 (88 to 100) | 6.21 (3.82 to 8.95) |
| Three or more coexisting conditions | ||||
| 14 to 20 days after first dose | 37 (12 to 55) | 1.60 (0.43 to 2.76) | 62 (43 to 77) | 2.19 (1.20 to 3.18) |
| 21 to 27 days after first dose | 37 (−1 to 62) | 1.03 (−0.03 to 2.02) | 47 (11 to 73) | 0.97 (0.16 to 1.86) |
| 7 days after second dose to end of follow-up | 86 (72 to 95) | 5.83 (3.16 to 9.03) | 89 (68 to 98) | 3.97 (1.41 to 6.68) |
| Obesity | ||||
| 14 to 20 days after first dose | 49 (32 to 65) | 2.50 (1.40 to 3.75) | 65 (48 to 79) | 2.31 (1.32 to 3.33) |
| 21 to 27 days after first dose | 48 (19 to 66) | 2.02 (0.69 to 3.25) | 50 (11 to 73) | 1.25 (0.18 to 2.27) |
| 7 days after second dose to end of follow-up | 95 (88 to 100) | 12.43 (6.03 to 20.70) | 98 (91 to 100) | 9.60 (4.03 to 17.39) |
| Type 2 diabetes mellitus | ||||
| 14 to 20 days after first dose | 25 (−10 to 51) | 1.17 (−0.36 to 2.74) | 48 (14 to 68) | 1.94 (0.49 to 3.28) |
| 21 to 27 days after first dose | 49 (2 to 78) | 1.29 (0.04 to 2.67) | 60 (10 to 84) | 1.18 (0.12 to 2.27) |
| 7 days after second dose to end of follow-up | 91 (71 to 100) | 6.85 (3.31 to 11.33) | 91 (68 to 100) | 5.06 (1.84 to 8.96) |
| Hypertension | ||||
| 14 to 20 days after first dose | 28 (2 to 49) | 1.12 (0.08 to 2.26) | 45 (16 to 64) | 1.33 (0.37 to 2.22) |
| 21 to 27 days after first dose | 45 (15 to 66) | 1.49 (0.42 to 2.53) | 59 (31 to 79) | 1.47 (0.60 to 2.39) |
| 7 days after second dose to end of follow-up | 93 (85 to 99) | 7.67 (4.35 to 11.72) | 95 (84 to 100) | 5.60 (2.97 to 8.92) |
Confidence intervals were estimated using the percentile bootstrap method with 500 repetitions. Estimates were calculated only for cells with more than 10 instances of an outcome across the two groups. RR denotes risk ratio.
The estimated vaccine effectiveness for the asymptomatic infection proxy was 29% (95% CI, 17 to 39) during the period from 14 to 20 days after the first dose, 52% (95% CI, 41 to 60) 21 to 27 days after the first dose, and 90% (95% CI, 83 to 94) 7 or more days after the second dose (Table S3 and Fig. S5).
Figure S6 shows a magnification of the cumulative incidence curve for the symptomatic illness outcome, showing the divergence of the curves starting around day 12. This is shown in comparison with the same curve from an analysis minimally matched (on age and sex only) that shows an earlier and wider separation of the curves.
Table S4 shows the sensitivity analyses of vaccine effectiveness across additional follow-up periods. Cumulative effectiveness estimates starting from day 0 were lower across all outcomes. Effectiveness estimates conditional on receipt of the second dose of vaccine were higher than unconditional estimates for days 21 through 27 after the first dose.
Table S5 and Figure S7 show the results of the sensitivity analysis in which data for persons who were enrolled as controls and were then vaccinated were censored at a delay (a number of days after the vaccination date, depending on the outcome). The estimates are similar to those of the main analysis. Table S6 details all analyses performed during the study, and Table S7 includes the life tables for the various outcomes.
Discussion
This study evaluates the effectiveness of the novel BNT162b2 mRNA vaccine1 against Covid-19 in a nationwide mass vaccination setting. Estimated vaccine effectiveness during the follow-up period starting 7 days after the second dose was 92% for documented infection, 94% for symptomatic Covid-19, 87% for hospitalization, and 92% for severe Covid-19. Estimated effectiveness during days 14 through 20 (after one dose) and days 21 through 27 (gradual shifting between the first and second vaccine doses) was 46% and 60% for documented infection, 57% and 66% for symptomatic Covid-19, 74% and 78% for hospitalization, 62% and 80% for severe Covid-19, and 72% and 84% for Covid-19–related death, respectively.
The first primary end point evaluated in the randomized trial of the BNT162b2 vaccine was symptomatic Covid-19. In both the randomized trial and our study, the cumulative incidence of symptomatic Covid-19 in the vaccinated and unvaccinated groups began to diverge around day 12 after the first dose.1 The estimated vaccine efficacy for symptomatic Covid-19 starting at day 7 after the second dose was 95% in the randomized trial, as compared with 94% in our study. The estimated efficacy between the first dose and the second dose was 52% in the trial, as compared with 29% in our study. This difference may reflect the high level of transmission in Israel during the study period,14 which affected both the vaccinated persons and the controls equally during the first 12 days after administration of the first dose. To eliminate this distortion, we estimated first-dose effectiveness of the vaccine against Covid-19 for the period from days 14 through 20; the estimated effectiveness was 57%.
The estimated effectiveness for documented infection during days 14 through 20 was 46% in our study. A relatively similar effectiveness of 51% was reported by Chodick et al.,15 who evaluated a cohort from another health care organization in Israel and used a different study design that compared infection among vaccinated persons at days 13 through 24 after the first dose against infection during days 0 through 12.
In the randomized trial, the estimated vaccine efficacy for severe Covid-19 (89% over the entire study period) was based on only 10 cases. Our study recorded 229 cases of severe Covid-19 — 55 in the vaccinated group and 174 in the unvaccinated group — resulting in an estimated effectiveness of 62% for days 14 through 20 after the first dose, 80% for days 21 through 27, and 92% for 7 or more days after the second dose.
The large sample size in our study also allowed us to estimate vaccine effectiveness for specific subpopulations that the randomized trial was not sufficiently powered to evaluate. In the trial, the estimated efficacy for Covid-19 among persons up to 55 years of age, older than 55 years, and 65 years or older 7 days after the second dose was 94 to 96%. We were able to study more granular age groups, and we estimated that the vaccine effectiveness was similar for adults 70 years of age or older and for younger age groups for the same time period.
The randomized trial estimated vaccine efficacy for patients with one or more coexisting conditions according to the Charlson comorbidity index16 and specifically for patients with obesity or hypertension. These measures do not provide clarity regarding effectiveness in patients with multiple coexisting conditions. We estimated vaccine effectiveness in relation to various numbers of coexisting conditions and found indications that effectiveness may be slightly lower among persons with higher numbers of coexisting conditions.
Two factors make the present study uniquely suited to evaluating the effectiveness of the BNT162b2 vaccine in a practical application: first, a rare combination of rich medical background data, Covid-19 PCR test results (for the documented infection outcome), and patient follow-up data in both community (for the symptomatic Covid-19 outcome) and inpatient (for all other outcomes) settings — CHS has maintained such an integrated data repository for over half the Israeli population, and has updated it daily, for more than two decades; and second, the rapid pace and high uptake of Covid-19 vaccine in Israel and the high disease rates during the vaccination campaign. On the other hand, the rapid pace of the vaccination campaign contributed to the frequent censoring of data for matched unvaccinated controls, especially among those over the age of 60 years (often only a few days after matching) and the corresponding reduction in the average follow-up period of the study.
Concerns have emerged regarding the possible resistance of SARS-CoV-2 variants to Covid-19 vaccines17,18 and neutralizing antibodies.19,20 During the study period, an increasing share of SARS-CoV-2 isolates in Israel — up to 80% in the days before data extraction — were of the B.1.1.7 variant.21 Thus, this study estimates an average effectiveness of the vaccine over multiple strains. Although we cannot provide a specific effectiveness estimate for the B.1.1.7 variant, the plateau observed during the later periods in the cumulative incidence curve for vaccinated persons suggests that the BNT162b2 vaccine is also effective for this variant, an observation consistent with previous reports that showed preserved neutralizing antibody titers.22 The B.1.351 variant was estimated to be rare in Israel at the time of data extraction.23
As with any observational study, our study may have been affected by residual confounding due to differences between vaccinated persons and unvaccinated controls, especially in terms of health-seeking behavior. We therefore performed rigorous matching on a wide range of factors that may be expected to confound the causal effect of the vaccine on the various outcomes. After the matching process, we found a consistent pattern of similarity between the groups in the days just before day 12 after the first dose (the anticipated onset of the vaccine effect), which thus serve as a “negative control”24 period (Figure 2, Fig. S6, and Table S7). This similarity occurred despite a temporary increase in events among unvaccinated controls during the very first days after the first vaccine dose, most likely stemming from the fact that persons who choose to be vaccinated on a specific day are feeling well at the time of vaccination. The similarity of the study groups in coexisting conditions and known risk factors for severe Covid-19 (Table 1 and Fig. S2) provides further evidence of exchangeability (i.e., absence of confounding). However, this rigorous matching process came at the cost of not including in the final cohort approximately 34% of the vaccinated persons who otherwise met the study’s eligibility criteria. Limited matching on age and sex only would have been insufficient to eliminate the early confounding (Fig. S6).
We also excluded population groups with high internal variability in the probability of vaccination or outcome, such as health care workers, persons confined to the home for medical reasons, and nursing home residents, to avoid residual confounding. Although the randomized trial was also less likely to include persons who were not healthy enough to comply with the scheduled visits and vaccination plan, it did not exclude health care workers.
To assess a possible selection bias that could stem from informative censoring, whereby controls who are vaccinated feel well around the time of vaccination, we performed a sensitivity analysis in which they were kept in the unvaccinated group for a period of time that was set differently for each outcome (Fig. S7 and Table S5). This analysis showed results similar to those of the main analysis, which suggests that any such bias was small in our analysis.
Finally, the date of onset of symptoms was not available for the analysis. Instead, for infection outcomes, the date was set to the date of swab collection for the first positive PCR test. Given that there was likely to have been a time gap between the onset of symptoms and swab collection, the observed divergence of the cumulative incidence plots for the infection outcomes between the vaccinated persons and unvaccinated controls may be slightly delayed. In parallel, there might be an underestimation of the vaccine effectiveness at each time window, since the estimate actually reflects a narrower window for the vaccine to be active. Because SARS-CoV-2 PCR testing is highly accessible in Israel and can be done without referral in a matter of hours, we estimate this potential time gap and thus the vaccine effectiveness underestimation to be small. In interpreting the effectiveness estimates for more severe outcomes, longer median gaps should be kept in mind (Fig. S3): 1 day for hospitalization, 5 days for severe Covid-19, and 11 days for Covid-19 death.
This study estimates a high effectiveness of the BNT162b2 vaccine for preventing symptomatic Covid-19 in a noncontrolled setting, similar to the vaccine efficacy reported in the randomized trial. Our study also suggests that effectiveness is high for the more serious outcomes: hospitalization, severe illness, and death. Furthermore, the estimated benefit increases in magnitude as time passes. These results strengthen the expectation that newly approved vaccines can help to mitigate the profound global effects of the Covid-19 pandemic.
Acknowledgments
We thank Daniel Nevo, for his advice on analysis, and Yatir Ben-Shlomo, Galit Shaham, and Uriah Finkel, for their assistance with the project.
Protocol
Supplementary Appendix
Disclosure Forms
This article was published on February 24, 2021, at NEJM.org.
Owing to data privacy regulations, the raw data for this study cannot be shared.
Footnotes
Dr. Lipsitch receives support from the Morris–Singer Fund.
Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.
References
- 1.Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020;383:2603-2615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med 2020;384:403-416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.ClinicalTrials.gov. Study to describe the safety, tolerability, immunogenicity, and efficacy of RNA vaccine candidates against COVID-19 in healthy individuals. 2020. (https://clinicaltrials.gov/ct2/show/NCT04368728).
- 4.Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol 2016;183:758-764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Zambrano LD, Ellington S, Strid P, et al. Update: characteristics of symptomatic women of reproductive age with laboratory-confirmed SARS-CoV-2 infection by pregnancy status — United States, January 22–October 3, 2020. MMWR Morb Mortal Wkly Rep 2020;69:1641-1647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Centers for Disease Control and Prevention. People with certain medical conditions. February 3, 2021. (https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-with-medical-conditions.html).
- 7.Centers for Disease Control and Prevention. Older adults at greater risk of requiring hospitalization or dying if diagnosed with COVID-19. December 13, 2020. (https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/older-adults.html).
- 8.National Institutes of Health. Coronavirus disease 2019 (COVID-19) treatment guidelines. 2021. (https://www.covid19treatmentguidelines.nih.gov/). [PubMed]
- 9.Harder VS, Stuart EA, Anthony JC. Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychol Methods 2010;15:234-249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc 1958;53:457-481. [Google Scholar]
- 11.Lipsitch M, Goldstein E, Ray GT, Fireman B. Depletion-of-susceptibles bias in influenza vaccine waning studies: how to ensure robust results. Epidemiol Infect 2019;147:e306-e306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Gallagher T, Lipsitch M. Postexposure effects of vaccines on infectious diseases. Epidemiol Rev 2019;41:13-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hernán MA. The hazards of hazard ratios. Epidemiology 2010;21:13-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kaslow DC. Force of infection: a determinant of vaccine efficacy? January 26, 2021. (https://www.medrxiv.org/content/10.1101/2021.01.21.21250235v1.full). preprint. [DOI] [PMC free article] [PubMed]
- 15.Chodick G, Tene L, Patalon T, et al. The effectiveness of the first dose of BNT162b2 vaccine in reducing SARS-CoV-2 infection 13-24 days after immunization: real-world evidence. January 29, 2021. (https://www.medrxiv.org/content/10.1101/2021.01.27.21250612v1). preprint. [DOI] [PMC free article] [PubMed]
- 16.Charlson M, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol 1994;47:1245-1251. [DOI] [PubMed] [Google Scholar]
- 17.Mahase E. Covid-19: Novavax vaccine efficacy is 86% against UK variant and 60% against South African variant. BMJ 2021;372:n296-n296. [DOI] [PubMed] [Google Scholar]
- 18.Johnson & Johnson announces single-shot Janssen COVID-19 vaccine candidate met primary endpoints in interim analysis of its phase 3 ENSEMBLE trial. New Brunswick, NJ: Johnson & Johnson, January 29, 2021. (https://www.jnj.com/johnson-johnson-announces-single-shot-janssen-covid-19-vaccine-candidate-met-primary-endpoints-in-interim-analysis-of-its-phase-3-ensemble-trial). [Google Scholar]
- 19.Wang P, Liu L, Iketani S, et al. Increased resistance of SARS-CoV-2 variants B.1.351 and B.1.1.7 to antibody neutralization. bioRxiv 2021. January 26 (Preprint).33532778 [Google Scholar]
- 20.Xie X, Zou J, Fontes-Garfias CR, et al. Neutralization of N501Y mutant SARS-CoV-2 by BNT162b2 vaccine-elicited sera. bioRxiv 2021. January 7 (Preprint).33442691 [Google Scholar]
- 21.Efrati I. Israel to extend COVID vaccine drive to anyone over 16 starting Thursday. Haaretz. February 3, 2021. [Google Scholar]
- 22.Muik A, Wallisch A-K, Sänger B, et al. Neutralization of SARS-CoV-2 lineage B.1.1.7 pseudovirus by BNT162b2 vaccine–elicited human sera. Science 2021. January 29 (Epub ahead of print). [DOI] [PMC free article] [PubMed]
- 23.Hoffman M. Coronavirus: 80 cases of South African variant discovered in Israel. Jerusalem Post. February 2, 2021. (https://www.jpost.com/breaking-news/coronavirus-80-cases-of-south-african-variant-discovered-in-israel-657452).
- 24.Lipsitch M, Tchetgen Tchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology 2010;21:383-388. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.