Effectiveness of mRNA BNT162b2 COVID-19 vaccine up to 6 months in a large integrated health system in the USA: a retrospective cohort study

Abstract

Background

Vaccine effectiveness studies have not differentiated the effect of the delta (B.1.617.2) variant and potential waning immunity in observed reductions in effectiveness against SARS-CoV-2 infections. We aimed to evaluate overall and variant-specific effectiveness of BNT162b2 (tozinameran, Pfizer–BioNTech) against SARS-CoV-2 infections and COVID-19-related hospital admissions by time since vaccination among members of a large US health-care system.

Methods

In this retrospective cohort study, we analysed electronic health records of individuals (≥12 years) who were members of the health-care organisation Kaiser Permanente Southern California (CA, USA), to assess BNT162b2 vaccine effectiveness against SARS-CoV-2 infections and COVID-19-related hospital admissions for up to 6 months. Participants were required to have 1 year or more previous membership of the organisation. Outcomes comprised SARS-CoV-2 PCR-positive tests and COVID-19-related hospital admissions. Effectiveness calculations were based on hazard ratios from adjusted Cox models. This study was registered with ClinicalTrials.gov, NCT04848584.

Findings

Between Dec 14, 2020, and Aug 8, 2021, of 4 920 549 individuals assessed for eligibility, we included 3 436 957 (median age 45 years [IQR 29–61]; 1 799 395 [52·4%] female and 1 637 394 [47·6%] male). For fully vaccinated individuals, effectiveness against SARS-CoV-2 infections was 73% (95% CI 72–74) and against COVID-19-related hospital admissions was 90% (89–92). Effectiveness against infections declined from 88% (95% CI 86–89) during the first month after full vaccination to 47% (43–51) after 5 months. Among sequenced infections, vaccine effectiveness against infections of the delta variant was high during the first month after full vaccination (93% [95% CI 85–97]) but declined to 53% [39–65] after 4 months. Effectiveness against other (non-delta) variants the first month after full vaccination was also high at 97% (95% CI 95–99), but waned to 67% (45–80) at 4–5 months. Vaccine effectiveness against hospital admissions for infections with the delta variant for all ages was high overall (93% [95% CI 84–96]) up to 6 months.

Interpretation

Our results provide support for high effectiveness of BNT162b2 against hospital admissions up until around 6 months after being fully vaccinated, even in the face of widespread dissemination of the delta variant. Reduction in vaccine effectiveness against SARS-CoV-2 infections over time is probably primarily due to waning immunity with time rather than the delta variant escaping vaccine protection.

Funding

Pfizer.

Introduction

In a pivotal randomised controlled trial, the BNT162b2 mRNA vaccine (tozinameran, Pfizer–BioNTech) showed 95% or greater efficacy against symptomatic and severe COVID-19 disease due to SARS-CoV-2.¹ In the early months after its introduction, BNT162b2 has been shown to be highly effective in the real-world setting and to have had a large public health effect on reducing infections, hospital admissions, and deaths at a time when the alpha (B.1.1.7) variant was the predominant strain in Israel,2, 3, 4 the USA,5, 6, 7, 8 Canada,⁹ the UK,10, 11, 12, 13, 14, 15, 16 and Qatar.17, 18

The continual emergence of SARS-CoV-2 variants has raised concern that COVID-19 vaccines could have reduced effectiveness against new viral strains; however, BNT162b2 has shown robust amounts of neutralising antibodies against all variants of concern evaluated to date.19, 20, 21 Moreover, confirmatory, real-world studies have shown high effectiveness of two doses of BNT162b2 against COVID-19, especially severe disease, caused by variants of concern alpha,3, 17 beta (B.1.351),17, 22 and delta^{9,14–16,23,24} in various settings.

After global transmission of the delta variant in June and July, 2021, reports describing reduced effectiveness of BNT162b2 (and other COVID-19 vaccines) against SARS-CoV-2 infections caused by the delta variant began to surface from Israel,²⁵ Qatar,²³ and the USA.26, 27

The emergence of the delta variant, however, might not be the primary driver of reported declines in effectiveness against SARS-CoV-2 infections and increasing rates of breakthrough infections among individuals who are fully vaccinated.²³ In Israel, Qatar, and the USA, for example, widespread dissemination of the delta variant also coincided with the time period during which many individuals at high risk who were fully vaccinated first (eg, health-care workers, individuals who were immunocompromised, and older people) were approaching 6 months since the receipt of their second dose. Thus, waning of vaccine-induced immunity, which was observed in the pivotal randomised controlled trial before the emergence of the delta variant,²⁸ is an important factor to consider in the context of reported declines in effectiveness.

Research in context.

Evidence before this study

After global transmission of the delta (B.1.617.2) variant in June and July, 2021, reports describing reduced effectiveness of BNT162b2 (and other COVID-19 vaccines) against SARS-CoV-2 infections caused by the delta variant began to surface from Israel, Qatar, and the USA. Vaccine effectiveness studies in the setting of widespread prevalence of the delta variant, however, have not adequately differentiated the effect of the variant from potential waning immunity on observed reductions in effectiveness against SARS-CoV-2 infections. To help answer this urgent public health question, we evaluated overall and variant-specific real-world effectiveness of BNT162b2 against SARS-CoV-2 infections and COVID-19-related hospital admissions by time since vaccination among members of a large integrated health-care system in the USA up until 6 months after full vaccination.

Added value of this study

Our variant-specific analysis suggests that reductions in BNT162b2 effectiveness over time are likely to be primarily due to waning vaccine effectiveness rather than the delta variant escaping vaccine protection given that effectiveness against delta variant infections was more than 90% within 1 month of full vaccination, reductions in effectiveness in infections by time since being fully vaccinated were observed irrespective of SARS-CoV-2 variant, and effectiveness against hospital admissions due to the delta variant was very high over the entire study period.

Implications of all the available evidence

Related to other findings from Israel, the USA, and other countries, our findings underscore the importance of monitoring vaccine effectiveness over time and suggest that booster doses are likely to be needed to restore the initial high amounts of protection observed early in the vaccination programme.

Vaccine effectiveness studies in the setting of high prevalence of the delta variant have not adequately differentiated the effect of the delta variant from potential waning immunity on observed reductions in effectiveness against SARS-CoV-2 infections. This distinction is essential to inform the need for booster doses and to establish what the antigenic composition of future vaccines should be. To help answer this urgent public-health question, we aimed to evaluate overall and variant-specific real-world effectiveness of BNT162b2 against SARS-CoV-2 infections and COVID-19-related hospital admissions by time since vaccination among members of a large integrated health-care system in the USA.

Methods

Study design and participants

In this retrospective cohort study, we analysed electronic health records from the Kaiser Permanente Southern California (KPSC) health-care system (CA, USA) to assess the effectiveness of the BNT162b2 vaccine against SARS-CoV-2 infections and COVID-19-related hospital admissions. The study population consisted of all KPSC members aged 12 years and older. The start of the study period corresponded to the date the first doses of BNT162b2 were administered to KPSC members. The test-negative design described in the study protocol will be performed in future work.

KPSC is an integrated health-care organisation with more than 4·7 million members, representative of the socioeconomic and racial and ethnic diversity of the area's population.²⁹ KPSC electronic health records integrate clinical data including diagnostic, pharmacy, laboratory, and vaccination history information across all settings of care. Care delivered to members outside of the KPSC system is also captured, as outside providers must submit detailed claims to KPSC for reimbursement by the health plan.

Participants were required to have 1 year or more of membership (allowing a 31-day gap during previous membership to allow for potential delays in renewal) to determine comorbidities and medical history. Patients with documentation requesting removal from all research studies were excluded. The study protocol was reviewed and approved by the KPSC institutional review board, which waived requirement for informed consent (number 12816).

Procedures

COVID-19 vaccines were provided to KPSC members at no cost following emergency use authorisation. Any COVID-19 vaccines administered to members outside of the KPSC system during the study period were captured using batch queries to the California Immunization Registry. California providers are required by law to report all COVID-19 vaccine administrations to the registry every 24 h. KPSC followed the state of California guidance in rolling out COVID-19 vaccines, first making vaccines available to health-care workers in December, 2020. Vaccines were then progressively made available to older people, individuals with underlying health conditions, and essential workers. By April, 2021, anyone aged 16 years or older was eligible to receive the vaccine. Those aged 12–15 years became eligible in May, 2021.

The primary exposure was full vaccination with BNT162b2, defined as receiving two doses of BNT162b2 with 7 days or more after the second dose. Individuals were considered partially vaccinated if they received only one dose with 14 days or more after the first dose or if they received two doses with less than 7 days after the second dose. Individuals were considered unvaccinated until receipt of their first dose of BNT162b2, or until censoring at disenrolment, death, or receipt of another COVID-19 vaccine.

Outcomes

Outcomes comprised SARS-CoV-2 infection defined as testing positive for SARS-CoV-2 via a PCR test from any sample (ie, bronchial lavage, nasopharyngeal or nasal swab, oropharyngeal swab, throat swab, saliva, sputum, or tracheal aspirate) in any clinical setting regardless of the presence of symptoms (see appendix p 1), and COVID-19-related hospital admission defined as a hospital admission with a positive SARS-CoV-2 PCR test that was conducted between 14 days before and 3 days after the date of hospital admission.

All PCR-positive SARS-CoV-2 laboratory specimens collected between March 4 and July 21, 2021, were processed for whole genome sequencing and viral lineage designation (appendix p 1). A small number of archived specimens (n=148) collected before March 4, 2021, were also included. For those with multiple positive samples, the first successfully sequenced sample was included in analyses.

Statistical analysis

Using descriptive statistics, we described the distribution of demographic and clinical characteristics of the study cohort by BNT162b2 vaccination status and history of SARS-CoV-2 infection. Among those who tested positive for SARS-CoV-2, we described study population characteristics by infecting strain (ie, delta, other variant, failed sequencing). Analyses of specimens that failed sequencing were not specified in the protocol but were added due to sufficient sample size and to better understand potential bias in the sequenced sub-sample. Median time since full vaccination was also described. Hazard ratios (HRs) with 95% CIs from an unadjusted Cox model with time-varying covariates were estimated comparing rates of SARS-CoV-2 infection and COVID-19-related hospital admissions among fully vaccinated and partially vaccinated individuals to those who were unvaccinated. BNT162b2 vaccination status was categorised as time-varying, with all participants entering the cohort as unvaccinated. Follow-up time was censored at the time of disenrolment from KPSC, death, receipt of any other newly licensed or investigational COVID-19 vaccine or prophylactic agent other than BNT162b2, or receipt of more than two doses of BNT162b2. Unexposed person-time consisted of follow-up time of those never vaccinated against COVID-19, as well as time contributed by participants before being vaccinated or censored. To assess durability, vaccine effectiveness was estimated at monthly intervals after participants were fully vaccinated with BNT162b2. Sufficient sample size allowed for monthly estimates rather than the 3-month intervals specified in the protocol. Calendar time was included in all models (crude and adjusted) as the underlying time scale to allow the baseline hazard to vary flexibly as vaccine eligibility, testing practices, non-pharmaceutical interventions, lockdown requirements, disease activity, and COVID-19 treatment changed over time. The estimated hazard for a model with time-varying covariates does not have the direct relationship with cumulative incidence that the standard Cox model does, as cumulative incidence depends on the entire history of the time-varying covariate for all patients. Thus, the vaccine effectiveness estimates from these models will not match a crude rate ratio calculated using events or person-time (appendix pp 7–8). With calendar time as the timescale, both unadjusted and adjusted models compare those who are unvaccinated on each calendar date to those who are vaccinated on that same date. The adjusted Cox model extends this, effectively comparing each vaccinated person on a given date to a person with the same covariates who is unvaccinated as of that date.

Adjusted HRs and 95% CIs were estimated by including all measured covariates in the Cox models with time-varying vaccination status. Variables included in the multivariable models were age, sex, race and ethnicity, previous PCR-positive SARS-CoV-2, previous health-care utilisation (inpatient, outpatient, emergency department, or virtual), body-mass index, acute myocardial infarction, congestive heart failure, cerebrovascular disease, peripheral vascular disease, organ transplant, diabetes, malignancy, renal disease, chronic obstructive pulmonary disease, hypertension, Charlson comorbidity index, influenza vaccination in the year before index date, pneumococcal vaccination in the 5 years before index date, and neighbourhood deprivation index³⁰ to capture differences in neighbourhood level socioeconomic status. The inclusion of all pre-specified covariates, as requested by the US Food and Drug Administration, differs from the backward selection method outlined in the protocol. Robust variance was computed to account for clustering introduced by including neighbourhood deprivation index in the model. For all models, vaccine effectiveness was calculated as: (1–HR) multiplied by 100%. Due to limitations in sample size, variant-specific vaccine effectiveness analyses were not stratified by age, were estimated only up to 4 months for SARS-CoV-2 infections, and were not stratified by month for COVID-19-related hospital admissions. Statistical comparisons of vaccine effectiveness by time since vaccination were made using Wald χ² tests for contrasts within Cox models. Vaccine effectiveness for delta and other variants could not be directly compared in the same regression model. The difference between delta variant vaccine effectiveness versus other variant vaccine effectiveness was compared using independent Z tests on the log HRs, which are conservative as the vaccine effectiveness for COVID-19 variants is positively correlated in the same population. All analyses were performed using SAS Enterprise Guide statistical software, version 7.1. This study was registered with ClinicalTrials.gov, NCT04848584.

Role of the funding source

The funder of the study approved the study design, and participated in data interpretation and writing of the report.

Results

The study period ran from Dec 14, 2020, to Aug 8, 2021. As of Dec 14, 2020, of 4 920 549 individuals assessed for eligibility there were 3 436 957 members of KPSC who fulfilled the inclusion criteria of age 12 years or older with membership of 1 year or longer who were included in the study cohort. Median age was 45 years (IQR 29–61), 1 799 395 [52·4%] participants were female and 1 637 394 [47·6%] were male. 1 390 587 (40·5%) participants were Hispanic, 1 108 456 (32·3%) were white, 399 186 (11·6%) were Asian or a Pacific Islander, and 276 199 (8·0%) were Black. In the year before the study start date, 74 284 (2·2%) of 3 436 957 participants had one or more positive SARS-CoV-2 PCR tests, and 543 628 (15·8%) had one or more negative PCR tests (table ).

Table.

Baseline characteristics

	BNT162b2 vaccination status				SARS-CoV-2 outcomes
	Unvaccinated* (n=2 290 189)	One dose plus <14 days (n=27 274)	One dose plus ≥14 days or two doses plus <7 days (n=76 205)	Two doses plus ≥7 days (n=1 043 289)	Uninfected (n=3 252 916)	SARS-CoV-2 infection (n=184 041)	COVID-19 hospital admission (n=12 130)	Total (N=3 436 957)
Age, years
12–15	104 918 (4·6%)	7164 (26·3%)	10 697 (14·0%)	78 843 (7·6%)	192 999 (5·9%)	8623 (4·7%)	45 (0·4%)	201 622 (5·9%)
16–44	1 038 609 (45·4%)	12 943 (47·5%)	35 876 (47·1%)	420 393 (40·3%)	1 417 518 (43·6%)	90 303 (49·1%)	2366 (19·5%)	1 507 821 (43·9%)
45–64	709 815 (31·0%)	5808 (21·3%)	20 709 (27·2%)	314 911 (30·2%)	990 866 (30·5%)	60 377 (32·8%)	4302 (35·5%)	1 051 243 (30·6%)
≥65	436 847 (19·1%)	1359 (5·0%)	8923 (11·7%)	229 142 (22·0%)	651 533 (20·0%)	24 738 (13·4%)	5417 (44·7%)	676 271 (19·7%)
Median	45 (29–61)	29 (15–45)	37 (21–54)	46 (29–62)	45 (29–61)	42 (29–57)	62 (49–74)	45 (29–61)
Sex
Male	1 115 148 (48·7%)	12 694 (46·5%)	36 843 (48·3%)	472 709 (45·3%)	1 552 606 (47·7%)	84 788 (46·1%)	6608 (54·5%)	1 637 394 (47·6%)
Female	1 174 921 (51·3%)	14 579 (53·5%)	39 355 (51·6%)	570 540 (54·7%)	1 700 146 (52·3%)	99 249 (53·9%)	5522 (45·5%)	1 799 395 (52·4%)
Other or unknown	120 (<0·1%)	1 (<0·1%)	7 (<0·1%)	40 (<0·1%)	164 (<0·1%)	4 (<0·1%)	0	168 (<0·1%)
Race and ethnicity
Hispanic	924 696 (40·4%)	14 683 (53·8%)	35 991 (47·2%)	415 217 (39·8%)	1 284 467 (39·5%)	106 120 (57·7%)	6691 (55·2%)	1 390 587 (40·5%)
Black	197 993 (8·6%)	3465 (12·7%)	6350 (8·3%)	68 391 (6·6%)	262 682 (8·1%)	13 517 (7·3%)	1201 (9·9%)	276 199 (8·0%)
White	759 438 (33·2%)	5563 (20·4%)	19 422 (25·5%)	324 033 (31·1%)	1 066 792 (32·8%)	41 664 (22·6%)	2752 (22·7%)	1 108 456 (32·3%)
Asian or Pacific Islander	226 149 (9·9%)	1734 (6·4%)	8355 (11·0%)	162 948 (15·6%)	385 995 (11·9%)	13 191 (7·2%)	1268 (10·5%)	399 186 (11·6%)
Other	52 505 (2·3%)	602 (2·2%)	1906 (2·5%)	25 431 (2·4%)	76 892 (2·4%)	3552 (1·9%)	117 (1·0%)	80 444 (2·3%)
Unknown	129 408 (5·7%)	1227 (4·5%)	4181 (5·5%)	47 269 (4·5%)	176 088 (5·4%)	5997 (3·3%)	101 (0·8%)	182 085 (5·3%)
Body-mass index, kg/m2
<18·5	62 618 (2·7%)	2127 (7·8%)	3953 (5·2%)	38 136 (3·7%)	103 360 (3·2%)	3474 (1·9%)	132 (1·1%)	106 834 (3·1%)
18·5–24·9	607 399 (26·5%)	8366 (30·7%)	22 675 (29·8%)	307 811 (29·5%)	907 630 (27·9%)	38 621 (21%)	1750 (14·4%)	946 251 (27·5%)
25·0–29·9	687 057 (30·0%)	7167 (26·3%)	21 499 (28·2%)	318 164 (30·5%)	978 156 (30·1%)	55 731 (30·3%)	3436 (28·3%)	1 033 887 (30·1%)
30·0–34·9	439 367 (19·2%)	4634 (17·0%)	13 359 (17·5%)	191 486 (18·4%)	605 962 (18·6%)	42 884 (23·3%)	3101 (25·6%)	648 846 (18·9%)
35·0–39·9	203 208 (8·9%)	2272 (8·3%)	6232 (8·2%)	86 551 (8·3%)	276 414 (8·5%)	21 849 (11·9%)	1803 (14·9%)	298 263 (8·7%)
≥40·0	137 456 (6·0%)	1497 (5·5%)	3854 (5·1%)	54 839 (5·3%)	181 492 (5·6%)	16 154 (8·8%)	1691 (13·9%)	197 646 (5·8%)
Unknown	153 084 (6·7%)	1211 (4·4%)	4633 (6·1%)	46 302 (4·4%)	199 902 (6·1%)	5328 (2·9%)	217 (1·8%)	205 230 (6·0%)
Comorbidities
Congestive heart failure	43 875 (1·9%)	218 (0·8%)	995 (1·3%)	20 120 (1·9%)	61 451 (1·9%)	3757 (2·0%)	1357 (11·2%)	65 208 (1·9%)
Coronary artery disease	26 661 (1·2%)	120 (0·4%)	568 (0·7%)	12 379 (1·2%)	37 662 (1·2%)	2066 (1·1%)	613 (5·1%)	39 728 (1·2%)
Peripheral vascular disease	179 305 (7·8%)	539 (2·0%)	3538 (4·6%)	96 772 (9·3%)	268 007 (8·2%)	12 147 (6·6%)	3316 (27·3%)	280 154 (8·2%)
Cerebrovascular disease	34 513 (1·5%)	147 (0·5%)	846 (1·1%)	16 661 (1·6%)	49 626 (1·5%)	2541 (1·4%)	730 (6·0%)	52 167 (1·5%)
Organ transplant	3111 (0·1%)	18 (0·1%)	63 (0·1%)	1638 (0·2%)	4408 (0·1%)	422 (0·2%)	160 (1·3%)	4830 (0·1%)
Diabetes with unknown glycated haemoglobin	25 942 (1·1%)	195 (0·7%)	725 (1·0%)	9648 (0·9%)	34 427 (1·1%)	2083 (1·1%)	329 (2·7%)	36 510 (1·1%)
Diabetes with glycated haemoglobin <7·5%	157 336 (6·9%)	814 (3·0%)	3693 (4·8%)	81 669 (7·8%)	229 185 (7·0%)	14 327 (7·8%)	2566 (21·2%)	243 512 (7·1%)
Diabetes with glycated haemoglobin ≥7·5%	86 318 (3·8%)	644 (2·4%)	2254 (3·0%)	38 732 (3·7%)	117 845 (3·6%)	10 103 (5·5%)	1966 (16·2%)	127 948 (3·7%)
Chronic obstructive pulmonary disease	204 050 (8·9%)	2338 (8·6%)	6298 (8·3%)	101 486 (9·7%)	295 394 (9·1%)	18 778 (10·2%)	2209 (18·2%)	314 172 (9·1%)
Renal disease	106 351 (4·6%)	420 (1·5%)	2137 (2·8%)	53 200 (5·1%)	154 006 (4·7%)	8102 (4·4%)	2579 (21·3%)	162 108 (4·7%)
Malignancy	52 934 (2·3%)	288 (1·1%)	1194 (1·6%)	27 092 (2·6%)	77 528 (2·4%)	3980 (2·2%)	792 (6·5%)	81 508 (2·4%)
Hypertension	465 109 (20·3%)	2637 (9·7%)	10 930 (14·3%)	231 754 (22·2%)	673 564 (20·7%)	36 866 (20·0%)	6227 (51·3%)	710 430 (20·7%)
Charlson comorbidity index
0	1 685 257 (73·6%)	22 609 (82·9%)	60 171 (79%)	743 248 (71·2%)	2 379 993 (73·2%)	131 292 (71·3%)	4460 (36·8%)	2 511 285 (73·1%)
1	303 977 (13·3%)	3213 (11·8%)	9266 (12·2%)	149 201 (14·3%)	437 558 (13·5%)	28 099 (15·3%)	2171 (17·9%)	465 657 (13·5%)
2	126 645 (5·5%)	713 (2·6%)	3047 (4·0%)	62 764 (6·0%)	182 559 (5·6%)	10 610 (5·8%)	1499 (12·4%)	193 169 (5·6%)
3	57 517 (2·5%)	254 (0·9%)	1240 (1·6%)	30 419 (2·9%)	85 034 (2·6%)	4396 (2·4%)	885 (7·3%)	89 430 (2·6%)
≥4	116 793 (5·1%)	485 (1·8%)	2481 (3·3%)	57 657 (5·5%)	167 772 (5·2%)	9644 (5·2%)	3115 (25·7%)	177 416 (5·2%)
Previous positive SARS-CoV-2 PCR test
1	47 993 (2·1%)	668 (2·4%)	1681 (2·2%)	18 356 (1·8%)	68 258 (2·1%)	440 (0·2%)	71 (0·6%)	68 698 (2·0%)
≥2	3827 (0·2%)	53 (0·2%)	116 (0·2%)	1590 (0·2%)	5537 (0·2%)	49 (<0·1%)	6 (<0·1%)	5586 (0·2%)
Previous positive SARS-CoV-2 serology
1	2466 (0·1%)	41 (0·2%)	56 (0·1%)	1231 (0·1%)	3764 (0·1%)	30 (<0·1%)	4 (<0·1%)	3794 (0·1%)
≥2	69 (<0·1%)	0	0	45 (<0·1%)	113	1 (<0·1%)	0	114 (<0·1%)

Data are n (%) or median (IQR). Characteristics of Kaiser Permanente Southern California members (n=3 436 957), by BNT162b2 vaccination status (as of Aug 8, 2021), and by SARS-CoV-2 outcomes (Dec 14, 2020, to Aug 8, 2021).

^*Unvaccinated group includes those not vaccinated with BNT162b2 as of Aug 8, 2021, and those vaccinated with other COVID-19 vaccines. Those vaccinated with COVID-19 vaccines other than BNT162b2 are censored in the vaccine effectiveness modelling at vaccination date.

During the study period, 184 041 (5·4%) of 3 436 957 participants were infected with SARS-CoV-2, among whom 12 130 (6·6%) were admitted to hospital. A higher proportion of the individuals infected with SARS-CoV-2 were younger (median age 42 years vs 45 years), Hispanic (57·7% vs 39·5%), and obese (>30 kg/m²; 43·9% vs 32·7%) than those who were not infected. Among those infected with SARS-CoV-2, a higher proportion of those who were admitted to hospital for COVID-19 were older, male, had comorbidities, and had greater previous health-care utilisation than those not admitted to hospital (table, appendix p 2).

Of 9147 specimens sent for whole genome sequencing, 236 were excluded from analyses (42 were the second sequenced samples from the same individual; 194 were the second failed samples from the same individual). Therefore, 8911 specimens were included for analyses and 5008 (56·2%) of 8911 had a sequence determined (appendix pp 3–4). We systematically submitted all PCR-positive specimens for sequencing starting March 4, 2021; however, the overall count of submitted specimens (n=8911) was 4·8% of all positive SARS-CoV-2 cases in the study (n=184 041). Specimens for which a sequence could not be determined were more likely to have high cycle threshold (Ct) values (appendix p 5). The median Ct values of sequenced N, ORF1ab, and S genes were 23·0 cycles for N, 23·3 cycles for ORF1ab, and 23·4 cycles for S; the median Ct values for specimens for which a sequence could not be determined were 30·7 cycles for N, 32·4 cycles for ORF1ab, and 28·8 cycles for S. Over the study period, 1422 (28·4%) of 5008 specimens for which a sequence could be determined were the delta variant. The proportion of sequenced specimens that were delta increased from 0·6% (seven of 1192) in April, 2021, to 86·5% (923 of 1067) in July, 2021 (figure 1 ). The distribution of comorbidities and previous health-care utilisation was generally consistent between the variant groups in our cohort (appendix pp 3–4).

Figure 1.

Distribution of variants from January to July, 2021

n=5008. Failed sequence counts are not included.

By Aug 8, 2021, 1 146 768 (33·4%) of 3 436 957 cohort members had received one or more doses of BNT162b2 (1 010 516 received ≥1 dose of mRNA-1273 [Moderna], 109 911 Ad26.COV2.S [Janssen], 2972 other COVID-19 vaccines or mixed regimens, and 1 166 790 remained unvaccinated). Of these, 1 043 289 (91·0%) of 1 146 768 patients were fully vaccinated, and 76 205 (6·6%) of 1 146 768 were partially vaccinated with BNT162b2 (table). Mean time since being fully vaccinated (7 days after second dose) was 3·4 months (SD 1·8); 752 562 (72·1%) of 1 043 289 of the fully vaccinated individuals were fully vaccinated at least 3 months before.

Over the entire study period, fully vaccinated individuals had an adjusted vaccine effectiveness of 73% (95% CI 72–74) against SARS-CoV-2 infections and 90% (89–92) against COVID-19-related hospital admissions (appendix pp 6–7). Stratified by age group, the vaccine effectiveness against infection of those who were fully vaccinated was 91% (95% CI 88–93) for those aged 12–15 years and 61% (57–65) for those aged 65 years and older (appendix p 6). The age stratified vaccine effectiveness against hospital admissions was 92% (95% CI 88–95) for those aged 16–44 years, and 86% (82–88) for those aged 65 years and older (appendix p 6).

Vaccine effectiveness against infection for the fully vaccinated decreased with increasing time since vaccination, declining from 88% (95% CI 86–89) during the first month after full vaccination to 47% (43–51) after 5 months (≥157 days after second dose, p<0·0001; figure 2A ; appendix p 9). Individuals aged 65 years and older had a vaccine effectiveness of 80% (95% CI 73–85) within 1 month after being fully vaccinated, decreasing to 43% (30–54; p<0·0001) at 5 months after full vaccination (figure 2A; appendix p 9). Among fully vaccinated individuals of all ages, overall adjusted vaccine effectiveness estimates for COVID-19 hospital admissions were 87% (95% CI 82–91) within 1 month after being fully vaccinated, and 88% (82–92) at 5 months after full vaccination, showing no significant waning (p=0·80; figure 2B; appendix pp 9–10).

Figure 2.

Adjusted estimated vaccine effectiveness against SARS-CoV-2 infection and hospital admissions

Vaccine effectiveness (95% CI) against SARS-CoV-2 infection (A) and COVID-19 hospital admission (B) by age group and number of months since being fully vaccinated with BNT162b2. *BNT162b2 authorised for those aged 12–15 years in May, 2021, limiting follow-up time for this age group.

Overall vaccine effectiveness against infection with the delta variant for the fully vaccinated was 75% (95% CI 71–78), while overall vaccine effectiveness for other variants was 91% (88–92; appendix pp 9–10). Estimates against both delta and other variants were high within 1 month after full vaccination (vaccine effectiveness against delta 93% [95% CI 85–97] vs other variants 97% [95–99]; p=0·29). At 4 months after full vaccination, vaccine effectiveness against delta infections declined to 53% (95% CI 39–65) and vaccine effectiveness against other variants declined to 67% (45–80; p=0·25). The difference in rate of decline in vaccine effectiveness between delta and other variants was not significant (p=0·30). For specimens in which a sequence could not be determined, adjusted vaccine effectiveness after full vaccination declined from 84% [95% CI 78–88]) at less than 1 month to 47% (30–59) after 4 months (figure 3 ; appendix pp 10–11). Among the fully vaccinated, vaccine effectiveness against hospital admissions was 93% (95% CI 84–96) for delta and 95% (90–98) for other variants. Effectiveness against hospital admissions was lower among specimens that failed sequencing (vaccine effectiveness 77% [95% CI 67–85]; appendix pp 10–11).

Figure 3.

Adjusted estimated vaccine effectiveness against SARS-CoV-2 infection by variant

Data are shown for number of months since being fully vaccinated with BNT162b2 with 95% CIs.

Discussion

This retrospective cohort study conducted in a large integrated health-care system showed that individuals who were fully vaccinated with BNT162b2 had 73% (95% CI 72–74) overall effectiveness against SARS-CoV-2 infections and 90% (89–92) effectiveness against COVID-19-related hospital admissions after a mean time since being fully vaccinated of 3·4 months. Effectiveness against SARS-CoV-2 infections waned during the 6 months of this study. Effectiveness against hospital admissions in all age groups did not wane over the duration of the study. These findings are consistent with preliminary reports from the Israel Ministry of Health and US Centers for Disease Control and Prevention showing reductions in effectiveness of BNT162b2 against infections 5 months or longer after being fully vaccinated, but consistently high estimates against COVID-19-related hospital admissions and severe disease up until July, 2021.24, 25, 26, 27 The most recent report from August, 2021, from Israel, however, suggests that some reduction in effectiveness against hospital admissions has been observed among older people (≥65 years) roughly 6 months after receiving the second dose of BNT162b2.³¹ Thus, long-term effectiveness data against severe outcomes should be continuously monitored in our study population and globally.

Effectiveness of BNT162b2 against infections caused by the delta variant, which became the predominant strain in KPSC by July, 2021, was 75% (95% CI 71–78) over the study period. Effectiveness against delta infections at 1 month after being fully vaccinated was high at 93% (85–97) but fell to 53% (39–65) up to 5 months after being fully vaccinated. Effectiveness against other (non-delta) variants within 1 month of being fully vaccinated was also high at 97% (95–99) and also waned, to 67% (45–80) up to 5 months after being fully vaccinated. Effectiveness against delta-related hospital admissions over the entire study period was high, at 93% (84–96) and was similar to effectiveness against hospital admissions for other (non-delta) variants. These findings are consistent with reports from the USA24, 26, 27 and Qatar.²³ Our variant-specific analyses suggest that reductions in vaccine effectiveness over time are likely to be primarily due to waning vaccine effectiveness rather than the delta variant escaping vaccine protection given that vaccine effectiveness against delta infections was more than 90% soon after vaccination, vaccine effectiveness against delta and other variants for hospital admissions was very high over the entire study period, and reductions in vaccine effectiveness against infection by time since being fully vaccinated were observed irrespective of the variant. We did not observe a difference in waning between variant types; however, the number of events at 3–4 months was low for analyses by variant. As such, analyses with longer follow-up to measure the rate of waning for the delta versus other variants are warranted. Related to our findings, studies from Canada⁹ and the UK14, 15 have shown high effectiveness of BNT162b2 against symptomatic COVID-19 caused by the delta variant in a vaccine schedule that separates the first and second doses by 2–3 months instead of 3 weeks. This longer interval between doses could lead to higher immunological responses;32, 33 however, duration of follow-up in these studies (<3 months)9, 14, 15 was insufficient to establish the effects of waning. Moreover, given the lower effectiveness after only one dose observed in our study and in other reports of one-dose effectiveness against variants of concern like beta or delta,14, 17, 23 delaying the second dose is not without risk.

Our results reiterate in a real-world US setting that vaccination with BNT162b2 remains an essential tool for preventing COVID-19, especially COVID-19-associated hospital admissions, caused by all current variants of concern. Along with other emerging evidence,9, 14, 15, 16, 23 our results suggest that despite early effectiveness of BNT162b2 against delta and other variants of concern, effectiveness against infection erodes steadily in the months after receipt of the second dose. Waning effectiveness and an increased number of infections 6–12 months after the second dose—along with the potential need for booster doses—was expected given that lower neutralising antibody titres during this time period have been observed in immunogenicity studies.34, 35, 36 Waning has been observed for both mRNA-based (Pfizer–BioNTech and Moderna) COVID-19 vaccines,26, 27 and is consistent with studies of other coronaviruses.³⁷ Reassuringly, early phase 1 data show that a third booster dose of the current BNT162b2 vaccine given 6 months after the second dose elicited neutralising antibody titres against the original SARS-CoV-2 wild-type strain, beta, and delta, which were several times higher than after two primary doses.34, 35 Modelling studies have predicted that these increases in neutralising antibody titres will restore high amounts of vaccine effectiveness.³⁶ Moreover, early unpublished data from an Israeli health maintenance organisation (Maccabi Health Services) suggest that a third booster dose is highly effective in a setting in which the delta variant accounts for nearly all cases.38, 39 These findings suggest that boosting with the current BNT162b2 vaccine rather than a delta-specific construct might be effective. Considerations of booster doses should also account for COVID-19 supply, as priority populations in some countries or subnational settings have not yet received a primary vaccination series.⁴⁰

Our study has potential limitations. We were unable to establish causal relationships between vaccination and COVID-19 outcomes in this observational study. Further, it is difficult to achieve a perfect balance of testing patterns and other characteristics between vaccinated and unvaccinated patients in this real-world observational study design. We attempt to address this issue by adjusting for proxies for general health-care seeking behaviour (visits across health-care settings before baseline), prior vaccination behaviour, demographics, comorbidities, and neighbourhood-level socioeconomic status. However, we did not have data for adherence to masking guidelines, social interactions, and occupation, which are likely to also affect likelihood of testing for SARS-CoV-2 either when experiencing symptoms or routinely as a preventive measure. KPSC maintained several drive-through testing clinics, did not have resource limitations on COVID-19 testing, and provided free testing to all members during the study period. We compared vaccinated and unvaccinated individuals at the same point in time, which balances the availability of testing, infection rates, and other secular inputs that might affect testing behaviours between vaccinated and unvaccinated patients to the extent possible in observational research. Effectiveness was lowest for PCR-positive specimens for which a sequence could not be determined. These specimens had higher Ct values than other PCR-positive specimens, which probably corresponded to milder or asymptomatic infections. Thus, our vaccine effectiveness estimates against SARS-CoV-2 infections and hospital admissions could be muted by mild or asymptomatic infections and are not directly comparable to estimates of effectiveness against symptomatic disease. Sequencing was more likely to fail in samples from vaccinated individuals due to lower viral loads, which could lead to an overestimate of variant-specific effectiveness. Finally, although the KPSC electronic health records might miss some vaccinations administered outside of the health system, our data capture through the California Immunization Registry minimised this effect.

Our results show high effectiveness of BNT162b2 against hospital admissions up until 6 months after being fully vaccinated in a large, diverse cohort under real-world vaccination conditions, even in the face of widespread dissemination of the delta variant. These findings underscore the importance of continuing to prioritise improving COVID-19 vaccination rates, including in hard-to-reach communities. Effectiveness against infections was high soon after full vaccination, both for delta and other variants of concern, but waned over the study period. Although waning effectiveness against hospital admissions was not observed in our study population to date, this possibility should be carefully monitored.³¹ Our findings underscore the importance of monitoring vaccine effectiveness over time and suggest that booster doses might eventually be needed to restore the high levels of protection observed early in the vaccination programme. These factors are especially important to help control heightened transmission of the delta variant as we enter the upcoming autumn and winter viral respiratory season.

Data sharing

Individual-level testing and clinical outcomes data reported in this study are not publicly shared. Individuals wishing to access disaggregated data, including data reported in this study, should submit requests for access to the corresponding author (sara.y.tartof@kp.org). De-identified data (including, as applicable, participant data and relevant data dictionaries) will be shared upon approval of analysis proposals with signed data-access agreements in place.

Declaration of interests

JMZ, SG, KP, FJA, LJ, SRV, and JMM are employees of and hold stock and stock options in Pfizer. TBF holds shares of Pfizer stock. SYT, JMS, HF, VH, BKA, ONR, TBF, and OAO received research support from Pfizer during the conduct of this study that was paid directly to KPSC. For work unrelated to this project, SYT received research funding from Gilead, GlaxoSmithKline, and Genentech; BKA received research funding from GlaxoSmithKline, Novavax, Dynavax, Genentech, Novartis, Seqirus, and Moderna; JMS received research funding from Novavax, Dynavax, and ALK; and HF received research funding from Genentech. All other authors declare no competing interests.