Covid-19 Vaccine Effectiveness in New York State

Abstract

Background

Population-based data from the United States on the effectiveness of the three coronavirus disease 2019 (Covid-19) vaccines currently authorized by the Food and Drug Administration are limited. Whether declines in effectiveness are due to waning immunity, the B.1.617.2 (delta) variant of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), or other causes is unknown.

Methods

We used data for 8,690,825 adults in New York State to assess the effectiveness of the BNT162b2, mRNA-1273, and Ad26.COV2.S vaccines against laboratory-confirmed Covid-19 and hospitalization with Covid-19 (i.e., Covid-19 diagnosed at or after admission). We compared cohorts defined according to vaccine product received, age, and month of full vaccination with age-specific unvaccinated cohorts by linking statewide testing, hospital, and vaccine registry databases. We assessed vaccine effectiveness against Covid-19 from May 1 through September 3, 2021, and against hospitalization with Covid-19 from May 1 through August 31, 2021.

Results

There were 150,865 cases of Covid-19 and 14,477 hospitalizations with Covid-19. During the week of May 1, 2021, when the delta variant made up 1.8% of the circulating variants, the median vaccine effectiveness against Covid-19 was 91.3% (range, 84.1 to 97.0) for BNT162b2, 96.9% (range, 93.7 to 98.0) for mRNA-1273, and 86.6% (range, 77.8 to 89.7) for Ad26.COV2.S. Subsequently, effectiveness declined contemporaneously in all cohorts, from a median of 93.4% (range, 77.8 to 98.0) during the week of May 1 to a nadir of 73.5% (range, 13.8 to 90.0) around July 10, when the prevalence of the delta variant was 85.3%. By the week of August 28, when the prevalence of the delta variant was 99.6%, the effectiveness was 74.2% (range, 63.4 to 86.8). Effectiveness against hospitalization with Covid-19 among adults 18 to 64 years of age remained almost exclusively greater than 86%, with no apparent time trend. Effectiveness declined from May through August among persons 65 years of age or older who had received BNT162b2 (from 94.8 to 88.6%) or mRNA-1273 (from 97.1 to 93.7%). The effectiveness of Ad26.COV2.S was lower than that of the other vaccines, with no trend observed over time (range, 80.0 to 90.6%).

Conclusions

The effectiveness of the three vaccines against Covid-19 declined after the delta variant became predominant. The effectiveness against hospitalization remained high, with modest declines limited to BNT162b2 and mRNA-1273 recipients 65 years of age or older.

As of September 29, 2021, more than 2.4 million people in New York State have been diagnosed with coronavirus disease 2019 (Covid-19), and more than 56,000 have died.¹ Covid-19 vaccines are a critical prevention tool. The three products currently approved or authorized by the Food and Drug Administration (FDA) were originally shown in trials to be highly efficacious against moderate-to-severe disease among adults, with an efficacy of 95% for the BNT162b2 (Pfizer–BioNTech) vaccine, an efficacy of 94% for the mRNA-1273 (Moderna) vaccine, and an efficacy of 72% for the Ad26.COV2.S (Johnson & Johnson–Janssen) vaccine.^2-4 The end points of those trials were evaluated at a relatively short follow-up of 14 to 28 days after series completion and during a period when the circulating strains were less transmissible than the currently predominant B.1.617.2 (delta) variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).⁵

Extended trial follow-up and real-world effectiveness studies have begun to document declines in vaccine effectiveness for some outcomes and in some populations during 2021. Studies in Israel have shown greater declines in vaccine effectiveness against infection and severe disease among recipients of BNT162b2 than have studies in the United States, which may be the result of earlier vaccination in Israel, differences in definitions of outcomes, or other methodological differences.^6-9 Population-based effectiveness studies in New York State and sentinel populations have documented declines in vaccine effectiveness, particularly in persons 65 years of age or older, during the period after public health mitigation policy measures were relaxed and during which the delta variant became predominant.^10-14 The limitations of open-cohort surveillance studies and limited numbers of events in more controlled studies make it difficult to isolate and quantify the extent to which declines in vaccine effectiveness can be attributed to waning immunity, the delta variant, behavioral changes among persons, or other causes, particularly across subgroups defined according to age, vaccine product, and risks of infection or severe illness.

Data are needed to understand the magnitude and sources of changes in vaccine effectiveness across outcomes, products, and population subgroups to inform public health policy and vaccine recommendations. After authorization was granted by the FDA, the Centers for Disease Control and Prevention (CDC) recommended booster doses of BNT162b2 for persons 65 years of age or older, for persons 18 to 64 years of age who have underlying conditions, and for persons 18 to 64 years of age who work or live in high-exposure settings.^15,16 The FDA has now recommended expanded eligibility for booster doses of the mRNA-1273 and Ad26.COV2.S vaccines. These recommendations were made despite gaps in data from the United States regarding vaccine effectiveness in groups according to age, time of vaccination, and product.^17,18 To address these gaps, we conducted a statewide, surveillance-based, prospective cohort study to assess vaccine effectiveness among adults in New York State. We defined closed cohorts according to vaccine product, age of recipient, and time of vaccination, from the time of the emergence of the delta variant to its predominance.

Methods

Data Sources

Four databases were linked to construct a surveillance-based cohort of adults 18 years of age or older residing in New York State.¹⁰ The Citywide Immunization Registry (CIR) collects and stores all data on Covid-19 vaccine administration for persons residing in New York City, and the New York State Immunization Information System (NYSIIS) collects data for the rest of the state (excluding data that are reported directly to the federal system, such as data for veterans and military personnel and data from the American Indian Health Program). The Electronic Clinical Laboratory Reporting System (ECLRS) collects all reportable Covid-19 test results (nucleic acid amplification testing or antigen testing) in New York State.¹⁹ The Health Electronic Response Data System (HERDS) includes a statewide, daily electronic survey of all inpatient facilities in New York State, which collects data on all new admissions of persons with a laboratory-confirmed Covid-19 diagnosis. The Covid-19 data from the NYSIIS and the CIR were combined and deduplicated on the basis of first name, last name, date of birth, and ZIP Code. The data were then matched to the ECLRS with the use of a deterministic algorithm on the basis of first name, last name, and date of birth and matched to HERDS on the basis of initials, sex, date of birth, and ZIP Code.

Study Cohorts

Administration of BNT162b2 in New York State began on December 14, 2020, mRNA-1273 on December 18, 2020, and Ad26.COV2.S on March 4, 2021. Starting in December 2020, vaccine eligibility was sequentially expanded to priority groups, including persons in long-term care facilities, health care workers, persons 65 years of age or older, frontline and essential workers, school staff, and persons with coexisting conditions (Table S1 in the Supplementary Appendix, available with the full text of this article at NEJM.org).

Closed cohorts included persons who were fully vaccinated (≥14 days after receipt of the final dose) as of May 1, 2021, and were defined according to age group (18 to 49 years, 50 to 64 years, or ≥65 years), vaccine product received (BNT162b2, mRNA-1273, or Ad26.COV2.S), and time of full vaccination (January or February [BNT162b2 or mRNA-1273 recipients only], March, or April 2021).²⁰ Whereas vaccinated cohorts were directly observed in the NYSIIS and CIR databases, three age-specific unvaccinated comparison cohorts were defined as the census population minus persons partially or fully vaccinated by September 23; persons with Covid-19 and persons who were hospitalized with Covid-19 (i.e., had Covid-19 diagnosed at or after admission) were classified as unvaccinated if they did not have a matching Covid-19 vaccine record according to the NYSIIS and CIR databases (see the Supplementary Methods section in the Supplementary Appendix).

Sensitivity Analyses

Sensitivity analyses assessed the effect of variations in the cohort definitions, stratification according to urbanicity of residence, and unmeasured confounding on estimates of vaccine effectiveness. Details are provided in the Supplementary Methods section in the Supplementary Appendix.

Statistical Analysis

In the person-level analysis of Covid-19 cases, the time to the first new positive SARS-CoV-2 nucleic acid amplification test or antigen test result (based on collection date) was assessed with the use of the life-table method at 7-day time intervals from the week of May 1, 2021, to the week of August 28 (ending on September 3). For each product and age group, weekly hazard rates and 95% confidence intervals were estimated overall and according to time cohort. We calculated hazard ratios and 95% confidence intervals for comparisons of the vaccinated cohorts with the corresponding age-specific unvaccinated cohorts.²¹ The vaccine effectiveness was estimated as 1 minus the hazard ratio.

Hospitalization with Covid-19 was defined as a new admission with a positive, laboratory-confirmed Covid-19 result during the period from May through August 2021. Because the unit of observation was an admission rather than an individual patient (approximately 9% of admissions were estimated to not represent unique patients), an aggregate-rates approach was used, as has been used elsewhere.^10,11 For each product and age cohort, the overall and time cohort–specific incidence rates of new admissions and 95% exact Poisson confidence intervals were estimated monthly according to time cohort because data were sparse at the week level. Incidence rate ratios and exact binomial 95% confidence intervals in each vaccinated cohort were compared with those in the respective age-specific unvaccinated cohort, with the vaccine effectiveness estimated as 1 minus the incidence rate ratio.

Trends in vaccine effectiveness against Covid-19 were compared with the percentage of delta-variant specimens from the CDC national SARS-CoV-2 genomic surveillance program for the Department of Health and Human Services region containing New York State.⁵ The association between vaccine effectiveness and delta-variant prevalence was summarized each week with the use of the Pearson correlation coefficient (r), after application of the logit transformation.

Results

Cohorts

Cohorts included in the analysis and outcomes are summarized in Table 1. Among 8,690,825 adults in the analysis, 5,638,142 persons (64.9%) were fully vaccinated: 48.5% had received BNT162b2, 41.5% mRNA-1273, and 10.0% Ad26.COV2.S. During follow-up, 38,419 cases of Covid-19 and 2354 hospitalizations with Covid-19 occurred among fully vaccinated persons and 112,446 cases of Covid-19 and 12,123 hospitalizations with Covid-19 occurred among unvaccinated persons.

Table 1. New Laboratory-Confirmed Covid-19 Cases and Hospitalizations with Covid-19 during Follow-up among Adults in New York State.^*.

Cohort	Persons	Cases	Hospitalizations
	number
Overall	8,690,825	150,865	14,477
Age 18–49 yr
BNT162b2	940,439	10,604	95
January–February	217,159	2,924	24
March	151,525	1,822	12
April	571,755	5,858	59
mRNA-1273	726,594	5,597	60
January–February	245,546	2,183	14
March	174,833	1,430	15
April	306,215	1,984	31
Ad26.COV2.S	268,056	3,268	38
March	50,350	818	14
April	217,706	2,450	24
Unvaccinated	2,074,191	83,092	4,576
Age 50–64 yr
BNT162b2	825,500	5,557	227
January–February	131,153	1,107	43
March	135,138	1,068	40
April	559,209	3,382	144
mRNA-1273	608,375	2,705	103
January–February	149,357	900	19
March	120,666	656	10
April	338,352	1,149	74
Ad26.COV2.S	180,687	1,399	92
March	54,060	489	27
April	126,627	910	65
Unvaccinated	607,737	19,428	3,271
Age ≥65 yr
BNT162b2	968,198	5,270	968
January–February	196,823	1,411	248
March	343,396	1,961	329
April	427,979	1,898	391
mRNA-1273	1,006,002	3,272	543
January–February	141,769	644	89
March	426,802	1,439	216
April	437,431	1,189	238
Ad26.COV2.S	114,291	747	228
March	49,109	342	87
April	65,182	405	141
Unvaccinated	370,755	9,926	4,276

^*Vaccinated cohorts were defined according to age group, vaccine product received, and month of full vaccination (i.e., ≥14 days after the final dose). Unvaccinated comparison cohorts were defined as the census population minus persons partially or fully vaccinated by September 23, 2021. Cases of coronavirus disease 2019 (Covid-19) were assessed from the week of May 1 to the week of August 28, 2021. Hospitalizations with Covid-19 were assessed from May through August 2021.

Incidence of Covid-19

During the week of May 1, 2021, the median number of incident cases of Covid-19 in the vaccinated cohorts was 2.4 cases per 100,000 person-days (range, 0.7 to 6.8), as compared with 34.6 cases per 100,000 person-days (range, 30.6 to 35.8) in the unvaccinated cohorts (Fig. S1 and Table S2). Rates decreased through late June, then increased after the delta variant became the most prevalent circulating variant (Table S3). By the week of August 28, the median incidence of Covid-19 was 16.4 cases per 100,000 person-days (range, 8.3 to 27.9) among vaccinated persons and 64.9 cases per 100,000 person-days (range, 54.4 to 76.4) among unvaccinated persons.

The estimated vaccine effectiveness against laboratory-confirmed Covid-19 declined contemporaneously across age, product, and time cohorts, with the highest effectiveness observed during the week of May 1 (median, 93.4%; range, 77.8 to 98.0) (when the prevalence of the delta variant was 1.8%), and the lowest effectiveness (median, 73.5%; range, 13.8 to 90.0) observed around July 10 (when the prevalence of the delta variant was 85.3%). A modest change in vaccine effectiveness occurred between July 10 and the week of August 28 (to a median of 74.2%; range, 63.4 to 86.8), when the prevalence of the delta variant was 99.6% (Figure 1 and Table 2). Weekly vaccine effectiveness was highly correlated with delta-variant prevalence, particularly among recipients of BNT162b2 and mRNA-1273 (Table S4).

Figure 1. Estimated Vaccine Effectiveness against Laboratory-Confirmed Coronavirus Disease 2019 (Covid-19) According to Vaccine Product, Age of Recipient, and Time of Full Vaccination.

The time of full vaccination was defined as at least 14 days after the final dose. Vaccine effectiveness was calculated as 1 minus the hazard ratio. The shaded areas indicate 95% confidence intervals.

Table 2. Estimated Vaccine Effectiveness against Laboratory-Confirmed Covid-19.^*.

Cohort	Vaccine Effectiveness (95% CI)
	Week of May 1, 2021	Week of July 10, 2021	Week of August 28, 2021
	percent
Age 18–49 yr
BNT162b2†	93.3 (92.2 to 94.4)	63.7 (59.7 to 67.6)	66.8 (65.0 to 68.5)
January–February	89.7 (87.0 to 92.4)	61.7 (53.7 to 69.7)	64.0 (60.5 to 67.5)
March	90.3 (87.1 to 93.4)	62.4 (53.0 to 71.8)	63.7 (59.5 to 67.9)
April	95.5 (94.4 to 96.6)	64.8 (59.9 to 69.6)	68.6 (66.5 to 70.7)
mRNA-1273†	96.3 (95.4 to 97.2)	76.0 (72.5 to 79.5)	77.0 (75.4 to 78.5)
January–February	93.7 (91.7 to 95.7)	79.0 (73.5 to 84.5)	70.1 (67.1 to 73.1)
March	97.7 (96.3 to 99.1)	73.6 (66.3 to 81.0)	77.8 (74.8 to 80.9)
April	97.5 (96.4 to 98.6)	74.9 (69.5 to 80.4)	82.0 (79.9 to 84.1)
Ad26.COV2.S†	89.0 (86.5 to 91.5)	48.7 (40.3 to 57.1)	68.7 (65.7 to 71.7)
March	89.7 (84.1 to 95.3)	13.8 (−10.9 to 38.4)	63.4 (56.1 to 70.7)
April	88.8 (86.0 to 91.6)	56.8 (48.3 to 65.3)	69.9 (66.7 to 73.1)
Age 50–64 yr
BNT162b2†	95.0 (94.0 to 96.0)	69.2 (63.8 to 74.6)	74.7 (72.8 to 76.6)
January–February	88.7 (84.9 to 92.4)	67.2 (55.0 to 79.3)	71.1 (66.6 to 75.6)
March	93.0 (90.1 to 95.9)	69.2 (57.6 to 80.8)	72.3 (68.0 to 76.6)
April	97.0 (96.0 to 97.9)	69.7 (63.6 to 75.9)	76.1 (74.0 to 78.2)
mRNA-1273†	97.3 (96.4 to 98.1)	84.7 (80.7 to 88.7)	81.8 (80.0 to 83.5)
January–February	95.9 (93.7 to 98.0)	76.0 (66.3 to 85.7)	74.7 (70.8 to 78.6)
March	96.9 (94.9 to 98.9)	82.2 (73.0 to 91.4)	76.5 (72.4 to 80.7)
April	98.0 (97.1 to 99.0)	89.4 (85.1 to 93.7)	86.8 (84.9 to 88.6)
Ad26.COV2.S†	86.1 (82.5 to 89.6)	70.6 (60.7 to 80.5)	74.7 (71.2 to 78.3)
March	87.8 (81.8 to 93.8)	73.4 (56.8 to 90.1)	68.8 (61.7 to 75.9)
April	85.3 (81.0 to 89.7)	69.4 (57.5 to 81.3)	77.3 (73.3 to 81.3)
Age ≥65 yr
BNT162b2†	91.4 (90.0 to 92.8)	77.5 (72.4 to 82.6)	76.0 (74.0 to 78.0)
January–February	84.1 (80.2 to 88.1)	70.2 (59.1 to 81.3)	73.8 (69.9 to 77.7)
March	91.3 (89.1 to 93.5)	79.2 (72.1 to 86.3)	75.4 (72.4 to 78.4)
April	94.9 (93.4 to 96.4)	79.5 (73.0 to 85.9)	77.5 (74.9 to 80.1)
mRNA-1273†	96.0 (95.1 to 96.9)	86.2 (82.5 to 89.8)	83.1 (81.5 to 84.7)
January–February	97.0 (95.1 to 99.0)	78.1 (67.2 to 88.9)	77.5 (73.3 to 81.7)
March	95.8 (94.5 to 97.2)	85.0 (79.6 to 90.4)	83.3 (81.1 to 85.5)
April	95.7 (94.4 to 97.1)	90.0 (85.7 to 94.2)	84.8 (82.7 to 86.8)
Ad26.COV2.S†	80.8 (75.2 to 86.5)	77.6 (65.4 to 89.7)	68.8 (63.3 to 74.3)
March	84.8 (77.3 to 92.3)	85.1 (70.3 to 99.8)	64.5 (55.7 to 73.3)
April	77.8 (69.8 to 85.8)	71.9 (54.1 to 89.7)	72.0 (65.3 to 78.8)

^*Vaccinated cohorts were defined according to age group, vaccine product received, and month of full vaccination (i.e., ≥14 days after the final dose). Vaccine effectiveness was calculated as 1 minus the hazard ratio.

^†The values in the row are derived from overall comparisons between vaccinated persons of the given age group who received the product and unvaccinated persons.

Among recipients of BNT162b2, the median vaccine effectiveness for the week of May 1 was 91.3% (range, 84.1 to 97), and by the week of August 28 it was 72.3% (range, 63.7 to 77.5) (Figure 1 and Table 2). For the week of May 1, the median vaccine effectiveness was 96.9% (range, 93.7 to 98.0) among recipients of mRNA-1273 and 86.6% (range, 77.8 to 89.7) among recipients of Ad26.COV2.S. By the week of August 28, the median vaccine effectiveness was 77.8% (range, 70.1 to 86.8) among recipients of mRNA-1273 and 69.4% (range, 63.4 to 77.3) among recipients of Ad26.COV2.S. Within-cohort declines in effectiveness between these weeks were similar for all three products: BNT162b2 effectiveness declined by a median of 20.7 percentage points (range, 10.3 to 26.9), mRNA-1273 effectiveness declined by a median of 19.5 percentage points (range, 10.9 to 23.6), and Ad26.COV2.S effectiveness declined by a median of 19.0 percentage points (range, 5.8 to 26.3). Although differences among age cohorts were limited, the decline in effectiveness among persons 18 to 49 years of age from May 1 to August 28 (median decline, 24.7 percentage points; range, 15.5 to 26.9) was greater than that among persons 50 to 64 years of age (median decline, 19.7 percentage points; range, 8.0 to 21.2) and among persons 65 years of age or older (median decline, 14.2 percentage points; range, 5.8 to 20.3).

For each combination of product and age group, the differences between time cohorts during the week of August 28 were smaller than the differences over calendar time. Among recipients of BNT162b2, the changes in vaccine effectiveness varied by 4.9 percentage points among recipients 18 to 49 years of age, by 5.0 percentage points among recipients 50 to 64 years of age, and by 3.7 percentage points among recipients 65 years of age or older. A similar trend was observed among Ad26.COV2.S recipients: effectiveness varied by 6.5 percentage points among persons 18 to 49 years of age, by 8.5 percentage points among persons 50 to 64 years of age, and by 7.5 percentage points among persons 65 years of age or older. The effectiveness range was numerically larger among mRNA-1273 recipients than among recipients of the other two vaccines: effectiveness varied by 11.9 percentage points, 12.1 percentage points, and 7.3 percentage points in the three age groups, respectively.

Incidence of Hospitalizations with Covid-19

The incidence of hospitalizations with Covid-19 generally declined in all cohorts from May through June 2021, then increased through August, with rates highest among unvaccinated persons and among persons 65 years of age or older (Fig. S2 and Table S5). Among persons 18 to 49 years of age and 50 to 64 years of age who received BNT162b2 or mRNA-1273, vaccine effectiveness against hospitalization was more than 90%, except in June among persons 18 to 49 years of age who had received mRNA-1273 and were fully vaccinated in April (vaccine effectiveness, 86.4%) (Figure 2 and Table 3). No clear time trend was observed. Among recipients of Ad26.COV2.S, vaccine effectiveness against hospitalization was more than 90% among persons 18 to 49 years of age, except in the month of June among those who were fully vaccinated in March (vaccine effectiveness, 50.5%) and among persons 50 to 64 years of age (vaccine effectiveness range, 85.7 to 92.7%).

Figure 2. Estimated Vaccine Effectiveness against Hospitalization with Laboratory-Confirmed Covid-19 According to Vaccine Product, Age of Recipient, and Time of Full Vaccination.

The time of full vaccination was defined as at least 14 days after the final dose. Vaccine effectiveness was calculated as 1 minus the incidence rate ratio. 𝙸 bars indicate 95% confidence intervals.

Table 3. Estimated Vaccine Effectiveness against Hospitalization with Laboratory-Confirmed Covid-19.^*.

Cohort	Vaccine Effectiveness (95% CI)
	May 2021	June 2021	July 2021	August 2021
	percent
Age 18–49 yr
BNT162b2†	96.1 (94.1 to 97.6)	94.3 (90.1 to 97.0)	95.5 (92.5 to 97.6)	95.2 (93.6 to 96.5)
January–February	97.7 (93.3 to 99.5)	98.1 (89.3 to 100.0)	95.9 (87.9 to 99.1)	92.4 (87.8 to 95.6)
March	98.9 (93.9 to 100.0)	97.3 (84.6 to 99.9)	96.0 (85.7 to 99.5)	94.9 (89.9 to 97.8)
April	94.8 (91.7 to 96.9)	92.0 (85.6 to 96.0)	95.3 (91.0 to 97.9)	96.4 (94.5 to 97.8)
mRNA-1273†	96.6 (94.3 to 98.1)	91.4 (85.7 to 95.2)	95.9 (92.4 to 98.0)	97.3 (95.9 to 98.4)
January–February	96.6 (92.1 to 98.9)	98.3 (90.5 to 100.0)	98.8 (93.2 to 100.0)	97.2 (94.3 to 98.9)
March	98.1 (93.1 to 99.8)	90.5 (75.5 to 97.4)	91.4 (79.9 to 97.2)	97.8 (94.3 to 99.4)
April	95.7 (91.5 to 98.1)	86.4 (74.8 to 93.5)	96.1 (89.9 to 98.9)	97.1 (94.6 to 98.7)
Ad26.COV2.S†	95.7 (91.1 to 98.3)	87.6 (75.4 to 94.7)	94.4 (86.9 to 98.2)	93.5 (89.6 to 96.1)
March	90.1 (71.1 to 98.0)	50.5 (−8.6 to 81.9)	94.1 (66.8 to 99.8)	92.3 (80.2 to 97.9)
April	97.0 (92.2 to 99.2)	96.2 (86.1 to 99.5)	94.5 (85.9 to 98.5)	93.7 (89.5 to 96.6)
Age 50–64 yr
BNT162b2†	95.6 (94.2 to 96.7)	95.0 (92.2 to 96.9)	93.4 (90.7 to 95.5)	94.9 (93.8 to 95.8)
January–February	92.4 (87.4 to 95.8)	92.8 (83.0 to 97.7)	96.5 (89.6 to 99.3)	94.4 (91.3 to 96.6)
March	93.6 (89.0 to 96.6)	95.8 (87.6 to 99.1)	90.8 (81.8 to 96.1)	95.6 (92.9 to 97.5)
April	96.8 (95.3 to 97.9)	95.3 (91.9 to 97.4)	93.4 (90.0 to 95.8)	94.8 (93.5 to 95.9)
mRNA-1273†	97.3 (95.9 to 98.2)	96.9 (94.2 to 98.5)	96.4 (94.0 to 98.1)	96.7 (95.7 to 97.5)
January–February	98.2 (95.4 to 99.5)	100.0 (95.3 to 100.0)	97.9 (92.5 to 99.8)	96.8 (94.5 to 98.3)
March	100.0 (98.0 to 100.0)	98.4 (91.2 to 100.0)	97.4 (90.7 to 99.7)	97.9 (95.6 to 99.1)
April	95.9 (93.6 to 97.5)	95.0 (90.3 to 97.7)	95.4 (91.5 to 97.8)	96.3 (94.8 to 97.4)
Ad26.COV2.S†	87.5 (82.4 to 91.4)	90.6 (81.9 to 95.7)	89.7 (81.9 to 94.7)	92.4 (89.5 to 94.7)
March	90.1 (80.5 to 95.8)	93.0 (74.5 to 99.2)	85.7 (66.3 to 95.4)	91.8 (85.6 to 95.8)
April	86.3 (79.8 to 91.1)	89.5 (78.2 to 95.8)	91.5 (82.2 to 96.6)	92.7 (89.2 to 95.3)
Age ≥65 yr
BNT162b2†	94.8 (94.0 to 95.5)	93.6 (92.0 to 95.0)	88.9 (86.6 to 90.8)	88.6 (87.4 to 89.6)
January–February	93.0 (90.8 to 94.7)	92.5 (88.4 to 95.4)	88.3 (83.1 to 92.1)	85.2 (82.4 to 87.6)
March	95.6 (94.3 to 96.7)	93.9 (91.1 to 95.9)	88.1 (84.2 to 91.1)	88.9 (87.1 to 90.5)
April	95.0 (93.8 to 96.1)	93.9 (91.5 to 95.8)	89.7 (86.5 to 92.3)	89.8 (88.3 to 91.2)
mRNA-1273†	97.1 (96.5 to 97.6)	96.3 (95.1 to 97.3)	95.0 (93.5 to 96.2)	93.7 (92.9 to 94.4)
January–February	97.3 (95.5 to 98.5)	96.0 (92.1 to 98.3)	91.6 (86.2 to 95.2)	92.8 (90.4 to 94.6)
March	97.4 (96.4 to 98.1)	97.4 (95.7 to 98.5)	95.5 (93.3 to 97.1)	93.7 (92.5 to 94.8)
April	96.8 (95.8 to 97.6)	95.3 (93.2 to 96.9)	95.6 (93.4 to 97.1)	93.9 (92.8 to 95.0)
Ad26.COV2.S†	85.2 (81.1 to 88.6)	80.4 (71.9 to 86.7)	81.1 (72.6 to 87.5)	81.9 (77.8 to 85.3)
March	90.6 (85.1 to 94.4)	80.0 (66.1 to 89.1)	81.8 (67.9 to 90.7)	82.0 (75.7 to 87.0)
April	81.1 (74.8 to 86.1)	80.6 (69.1 to 88.6)	80.6 (68.6 to 88.8)	81.7 (76.3 to 86.2)

^*Vaccinated cohorts were defined according to age group, vaccine product received, and month of full vaccination (i.e., ≥14 days after the final dose). Vaccine effectiveness was calculated as 1 minus the incidence rate ratio.

^†The values in the row are derived from overall comparisons between vaccinated persons of the given age group who received the product and unvaccinated persons.

Among persons 65 years of age or older, estimates of vaccine effectiveness against hospitalization declined among BNT162b2 recipients from May to August (from 93.0% among those vaccinated in January or February, 95.6% among those vaccinated in March, and 95.0% among those vaccinated in April to 85.2%, 88.9%, and 89.8% in the three cohorts, respectively). Smaller declines were observed among mRNA-1273 recipients from May to August (from 97.3% among those vaccinated in January or February, 97.4% among those vaccinated in March, and 96.8% among those vaccinated in April to 92.8%, 93.7%, and 93.9%, respectively). Estimates were lower among recipients of Ad26.COV2.S in both cohorts, ranging from 80.0 to 90.6%, with no clear time trend.

Sensitivity Analyses

Across the vaccinated cohorts, estimates of effectiveness against Covid-19 changed by a median of −4.4 percentage points when the estimated age-specific distribution of the 2020 census count was used and by −1.7 percentage points when persons whose Covid-19 was diagnosed within 90 days before May 1 without a vaccination registry match were included in the unvaccinated population (Tables S6 and S7). In all cohorts, estimates of effectiveness against hospitalization changed by a median of −1.1 percentage points and −0.4 percentage points, respectively, when these adjustments were made and by a median of +2.5 percentage points when the analysis was limited to hospitalizations specifically coded as “for Covid-19,” and by a median of −0.4 percentage points when persons whose Covid-19 was diagnosed within 90 days before May 1 were included (Tables S8 through S11).

Within strata of urbanicity of residence, the temporal patterns of vaccine effectiveness against Covid-19 were similar in shape to those of the primary analysis, although effectiveness values were lower in less urban counties. In the most urban counties, effectiveness against hospitalization was similar to that in the primary analysis (Figs. S3 through S6).

We also performed analyses that assessed unmeasured confounding across scenarios. For confounders that reduce the observed vaccine effectiveness, the median difference between the observed and the actual effectiveness was −3.2 percentage points (range, −14.1 to −0.4). For confounders that inflate the observed vaccine effectiveness, the median difference was +3.1 percentage points (range, +1.0 to +8.3) (Table S12).

Discussion

By analyzing large cohorts of New York State residents, we observed declines in vaccine effectiveness against Covid-19 from May through August 2021. These trends were inversely correlated with increasing delta-variant prevalence and plateaued among persons 18 to 64 years of age during the period in which the prevalence of the delta variant exceeded 85%. These changes occurred simultaneously across age, product, and time cohorts, with the largest declines seen among BNT162b2 recipients. Combined with the fact that synchronous declines in effectiveness were observed among time cohorts, these data suggest that waning of immunity may not have been a primary driver of those declines. In contrast, effectiveness against hospitalization with Covid-19 remained high, with lower effectiveness observed among persons 65 years of age or older and among those who received Ad26.COV2.S. Modest declines over time (within −6 percentage points) were observed among BNT162b2 and mRNA-1273 recipients 65 years of age or older.

Declines in effectiveness against Covid-19 during July 2021, followed by a plateau in late summer, have been reported recently by multiple jurisdictions using overall rate comparisons among open cohorts.^22-24 These observations could also reflect time-dependent changes in vaccinated and unvaccinated populations. Other recent studies have shown changes in effectiveness between times before delta was circulating and after the emergence of the variant in more well-controlled designs, but with less temporal resolution and fewer outcomes observed.^13,14

A strength of this study is the use of closed cohorts to control for challenges posed by previous surveillance-based approaches. As compared with more targeted studies, this study more precisely aligns patterns in time with population-level changes in the prevalence of the delta variant. The mechanism underlying lower vaccine effectiveness against the delta variant is unclear, but it may be related to increased transmissibility attributed to higher viral loads.^25,26

Time-dependent changes in prevention policies and behaviors according to vaccination status could contribute to the observed trends. On May 19, 2021, New York State adopted revised CDC guidance for fully vaccinated persons, followed in June by the end of the state of emergency in New York State, which reduced mask wearing, distancing, and other prevention practices; this may have increased exposure among vaccinated persons and lowered vaccine effectiveness.²⁷ On July 27, 2021, the CDC recommended mask usage for fully vaccinated persons on the basis of transmission levels.²⁸ The modest increases in effectiveness observed for some groups in August may reflect increased masking, and observed differences among regions may reflect variations in protective practices.

Although not supported as a primary driver of changes in effectiveness, we observed limited signals of waning immunity. For most groups, there were gradients in August according to time cohort, whereby higher effectiveness against Covid-19 was seen among more recently vaccinated persons, particularly among mRNA-1273 recipients; continued declines during August were observed among persons 65 years of age or older. Declines in effectiveness against hospitalization of less than 10 percentage points were seen among persons 65 years of age or older, particularly among those who received BNT162b2 and those who were vaccinated in January or February 2021. These findings are consistent with recent findings of reduced effectiveness against severe disease in older populations, particularly among persons living in long-term care facilities and those who received BNT162b2.^8,12,29 Because the federal vaccine-distribution program in long-term care facilities primarily distributed BNT162b2 in New York State, there is an overrepresentation of persons from long-term care facilities in the population of persons who were vaccinated with BNT162b2 in January or February, potentially contributing to the decline observed in that group.

Irrespective of the cause or propensity for continued declines in effectiveness, our findings have important implications for national vaccine policy. BNT162b2 booster doses have been shown to be safe and to increase short-term protection against the delta variant.³⁰ Our findings align with the CDC recommendation for BNT162b2 boosters in persons 65 years of age or older.¹⁶ The recommendation for boosters among persons 18 to 64 years of age who work or live in high-exposure settings was adopted by the CDC, after the Advisory Committee on Immunization Practices voted against the recommendation, with members citing limited supportive effectiveness data for persons younger than 65 years of age.

This study addressed key data gaps in the United States, showing population-wide declines in effectiveness against infection to less than 85% and smaller changes in effectiveness against severe disease that were limited to persons 65 years of age or older who received BNT162b2 or mRNA-1273. Our results suggest that a focus on booster shots for those 65 years of age or older is warranted at this time, with only limited need for booster expansion beyond that age group for the purposes of reducing severe Covid-19.

Our estimates of effectiveness may be influenced by unmeasured confounding resulting from differences in behavior, test seeking, or exposure between fully vaccinated and unvaccinated adults or among persons who have been vaccinated at different times. Our sensitivity analysis suggests considerable robustness to unmeasured confounding. Strengths of our study include large sample sizes, inclusion of all three vaccines authorized by the FDA, and outcome numbers that exceed those of studies conducted in other nations and those of studies with smaller, more controlled designs.^6,7,13,14 Our findings are further strengthened by broad age representation across cohorts and the consistency of changes in effectiveness over calendar time.

In this study, we did not account for indirect effects (e.g., herd immunity) between groups. On the basis of definitions of Covid-19 breakthrough cases, persons with previous diagnoses within 90 days before May 1 were excluded from this study. Such persons accounted for less than 1% of New Yorkers who were fully vaccinated by April 30, 2021, and were excluded initially and throughout the analysis because of the closed-cohort design; the weekly calculated conditional-hazard function was thus negligibly affected. In addition, an estimated 0.03% of persons vaccinated through April 30, 2021, received vaccines that had not been authorized by the FDA, and these persons were analytically classified as unvaccinated. Finally, the extent to which earlier SARS-CoV-2 infection modifies vaccine effectiveness is unclear; future research may explore effect modification according to previous diagnostic history.³¹

We observed consistent but moderate declines in vaccine effectiveness against Covid-19, contemporaneous with an increased prevalence of the delta variant and with changes in prevention policies, and the evidence for waning immunity was modest. Effectiveness against hospitalization remained high, with modest declines limited to persons 65 years of age or older who had received BNT162b2 or mRNA-1273. These results support the continued effectiveness of vaccines, supplemented by behavioral prevention strategies, for reducing morbidity from Covid-19.

This article was published on December 1, 2021, at NEJM.org.