Abstract
Background
The messenger RNA (mRNA)–based vaccines BNT162b2 and mRNA-1273 are more than 90% effective against coronavirus disease 2019 (Covid-19). However, their comparative effectiveness for a range of outcomes across diverse populations is unknown.
Methods
We emulated a target trial using the electronic health records of U.S. veterans who received a first dose of the BNT162b2 or mRNA-1273 vaccine between January 4 and May 14, 2021, during a period marked by predominance of the SARS-CoV-2 B.1.1.7 (alpha) variant. We matched recipients of each vaccine in a 1:1 ratio according to their risk factors. Outcomes included documented severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, symptomatic Covid-19, hospitalization for Covid-19, admission to an intensive care unit (ICU) for Covid-19, and death from Covid-19. We estimated risks using the Kaplan–Meier estimator. To assess the influence of the B.1.617.2 (delta) variant, we emulated a second target trial that involved veterans vaccinated between July 1 and September 20, 2021.
Results
Each vaccine group included 219,842 persons. Over 24 weeks of follow-up in a period marked by alpha-variant predominance, the estimated risk of documented infection was 5.75 events per 1000 persons (95% confidence interval [CI], 5.39 to 6.23) in the BNT162b2 group and 4.52 events per 1000 persons (95% CI, 4.17 to 4.84) in the mRNA-1273 group. The excess number of events per 1000 persons for BNT162b2 as compared with mRNA-1273 was 1.23 (95% CI, 0.72 to 1.81) for documented infection, 0.44 (95% CI, 0.25 to 0.70) for symptomatic Covid-19, 0.55 (95% CI, 0.36 to 0.83) for hospitalization for Covid-19, 0.10 (95% CI, 0.00 to 0.26) for ICU admission for Covid-19, and 0.02 (95% CI, −0.06 to 0.12) for death from Covid-19. The corresponding excess risk (BNT162b2 vs. mRNA-1273) of documented infection over 12 weeks of follow-up in a period marked by delta-variant predominance was 6.54 events per 1000 persons (95% CI, −2.58 to 11.82).
Conclusions
The 24-week risk of Covid-19 outcomes was low after vaccination with mRNA-1273 or BNT162b2, although risks were lower with mRNA-1273 than with BNT162b2. This pattern was consistent across periods marked by alpha- and delta-variant predominance. (Funded by the Department of Veterans Affairs and others.)
Early randomized trials showed strong effectiveness of messenger RNA (mRNA)–based vaccines for the prevention of symptomatic coronavirus disease 2019 (Covid-19), with effectiveness of 95% for BNT162b2 (Pfizer–BioNTech) and 94% for mRNA-1273 (Moderna).1,2 Observational studies then confirmed similar levels of effectiveness during real-world vaccination campaigns.3-5 However, head-to-head comparisons of these vaccines have been lacking, leaving open the question of which vaccine is more effective.
Both mRNA-based vaccines encode the prefusion stabilized full-length spike protein of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), but they differ in the mRNA content (100 μg for mRNA-1273 vs. 30 μg for BNT162b2), the interval between priming and boosting doses (4 weeks for mRNA-1273 vs. 3 weeks for BNT162b2), and the lipid composition of the nanoparticles used for packaging the mRNA content.1,2,6 These differences might explain emerging evidence for a higher antibody response among recipients of the mRNA-1273 vaccine than among recipients of the BNT162b2 vaccine.7,8 It has been suggested that this observed difference in antibody levels translates into a difference in the risk of Covid-19 outcomes.9,10 However, there has been a need for studies that compare the vaccines head-to-head, are large enough to provide precise risk estimates for severe Covid-19 outcomes, have adequate adjustment for confounding and sound study design, include racially diverse groups, and separately address periods that have predominance of different SARS-CoV-2 variants.
In this study, we used data from the national health care databases of the Department of Veterans Affairs (VA), the largest integrated health care system in the United States, to compare the effectiveness of the BNT162b2 vaccine and the mRNA-1273 vaccine with respect to five Covid-19 outcomes (documented SARS-CoV-2 infection, symptomatic Covid-19, hospitalization, admission to an intensive care unit [ICU], and death), both overall and in subgroups defined according to age and race. To evaluate the influence of SARS-CoV-2 variants on the comparative effectiveness of these vaccines, we conducted separate analyses in periods marked by predominance of the B.1.1.7 (alpha) variant and of the B.1.617.2 (delta) variant.
Methods
Specification of the Target Trials
We designed this observational analysis to emulate a target trial (i.e., a hypothetical pragmatic trial that would have answered the causal question of interest) of BNT162b2 as compared with mRNA-1273 for the prevention of Covid-19 outcomes in the VA health care system. The key components of the protocol are summarized in Table S1 in the Supplementary Appendix, available with the full text of this article at NEJM.org.
Eligibility criteria included veteran status, an age of at least 18 years between January 4 and May 14, 2021, no previously documented SARS-CoV-2 infection, no previous Covid-19 vaccination, and a known residential address outside of a long-term care facility, as well as known smoking status and body-mass index recorded within the previous year. Participants needed to have used the VA health care system during the previous year (defined as receiving care at a station eligible to administer the vaccines under study and having at least one primary care visit); however, they had to have had no interactions with the health care system in the previous 3 days (which may have indicated the start of symptomatic disease and precluded vaccination).
The interventions of interest were vaccination with either the BNT162b2 vaccine or the mRNA-1273 vaccine, with a second dose scheduled 21 days later for the BNT162b2 vaccine and 28 days later for the mRNA-1273 vaccine. To ensure balance of important characteristics across groups, eligible veterans in the target trial would be randomly assigned to one of these two vaccine groups within strata defined according to calendar date (5-day bins), age (5-year bins), sex (male or female), race (White, Black, other, or unknown), urbanicity of residence (urban or not urban), and geographic location (coded as one of 19 categories of the Veterans Integrated Services Network).
The five outcomes of interest were documented SARS-CoV-2 infection, documented symptomatic Covid-19, hospital admission for Covid-19, ICU admission for Covid-19, and death from Covid-19. For each eligible participant, follow-up started on the day the first dose of vaccine was received (baseline) and ended on the day of the outcome of interest, death, 168 days (24 weeks) after baseline, or the end of the study period (July 1, 2021), whichever occurred first.
This target trial was designed to evaluate the comparative effectiveness of the vaccines in a period during which the SARS-CoV-2 alpha variant was predominant. However, the alpha variant had decreased to a share of 26% of circulating variants in the United States as of June 26, 2021, as it was quickly displaced by the delta variant, which rose from a 68% share as of July 3, 2021, to 99% as of September 18, 2021.11 To evaluate the comparative effectiveness of the vaccines in a period with delta-variant predominance, we considered a second target trial that was identical to the first trial except that the recruitment period was July 1 to September 20, 2021, and the only outcome of interest was documented SARS-CoV-2 infection (because the period was too short to accumulate a sufficient number of rarer outcomes, such as hospitalization and death).
Emulation of the Target Trials
We emulated the above pragmatic target trials using the VA health care databases, which are described in the Supplementary Methods 1 section in the Supplementary Appendix. Table S2 provides detailed definitions of all study variables. Vaccination was identified with the use of records in the Immunization domain and procedures recorded in the Outpatient or Inpatient domain of the database. SARS-CoV-2 infections were identified with the use of the VA Covid-19 National Surveillance Tool,12 which integrates data on polymerase-chain-reaction (PCR) laboratory tests with natural language processing of clinical notes to capture diagnoses inside and outside the VA health care system. Symptomatic Covid-19 was defined as at least one of the following symptoms documented within the VA health care system within 4 days before or after documentation of SARS-CoV-2 infection: fever, chills, cough, shortness of breath or difficulty breathing, sore throat, loss of taste or smell, headache, myalgia, diarrhea, and vomiting. Symptoms were ascertained with the use of records in the Outpatient, Inpatient, Vital Signs, Health Factors, and Fee domains in the database. Hospitalization for Covid-19 was defined as a hospitalization within 21 days after documentation of SARS-CoV-2 infection (ascertained with the Inpatient domain), ICU admission for Covid-19 was defined as an ICU admission during hospitalization for Covid-19 (ascertained with the Inpatient domain and specialty transfer codes), and death from Covid-19 was defined as a death within 30 days after documentation of SARS-CoV-2 infection (ascertained using the Patient domain).
To mimic the stratified randomization of the target trial, we matched eligible persons who were vaccinated with BNT162b2 in a 1:1 ratio to eligible persons who were vaccinated with mRNA-1273. The matching factors (calendar date, age, sex, race, urbanicity of residence, and geographic location) are associated with the probability of receiving a particular vaccine, as well as with the risk of SARS-CoV-2 infection or severe Covid-19. (Additional details on the matching algorithm are provided in the Supplementary Methods 2 section in the Supplementary Appendix.)
To explore the possibility of residual confounding (e.g., by underlying health status or health care–seeking behavior), we used two negative outcome controls that are not directly affected by vaccination but for which the effect of vaccination might be similarly confounded.13 First, we evaluated the risk of symptomatic Covid-19 in the first 10 days after the first vaccine dose, during which no difference in risk between the vaccines is expected.1,2 Second, we evaluated the risk of death from causes other than Covid-19 during the follow-up period.
Statistical Analysis
Covariate balance after matching was evaluated by plotting the mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered to be acceptable.14 Cumulative incidence (risk) curves for the vaccination groups were estimated with the Kaplan–Meier estimator.15 We considered the period from the day of the first dose of vaccine until the end of follow-up. We used the Kaplan–Meier estimator with daily outcome events to compute the probability (risk) of the outcome during the period. We then calculated 24-week risk differences and risk ratios between the vaccination groups. We conducted subgroup analyses according to age (<70 or ≥70 years) and race (Black or White). Nonparametric bootstrapping with 500 samples was used to calculate percentile-based 95% confidence intervals for all estimates.
Analyses were performed with R software, version 3.6.0 (R Foundation for Statistical Computing), and SAS software, version 8.2 (SAS Institute). Information on authors’ contributions to the study is provided in the Supplementary Methods 3 section in the Supplementary Appendix. The first and last authors vouch for the accuracy and completeness of the data presented in this report.
Results
Study Population and Follow-up
Among the 3,103,470 veterans who received their first dose of any Covid-19 vaccine between January 4 and May 14, 2021, a total of 367,113 recipients of the BNT162b2 vaccine and 397,690 recipients of the mRNA-1273 vaccine were eligible for the study (Figure 1). Among these vaccine recipients, 219,842 of those who received the BNT162b2 vaccine were matched to 219,842 of those who received the mRNA-1273 vaccine. The matched population was similar to the eligible population with respect to baseline demographic and clinical characteristics (Table S3). The baseline characteristics of the matched population are shown in Table 1. All measured variables were well-balanced between the two vaccine groups (Fig. S1).
Figure 1. Selection of Persons for the Emulation of a Target Trial Evaluating the Comparative Effectiveness of the BNT162b2 and mRNA-1273 Vaccines during a Period Marked by SARS-CoV-2 Alpha-Variant Predominance (January 4–July 1, 2021).
There were 43,418 recipients of the Ad26.COV2.S (Johnson & Johnson–Janssen) vaccine, which was not studied here. SARS-CoV-2 denotes severe acute respiratory syndrome coronavirus 2, and VA Department of Veterans Affairs.
Table 1. Baseline Characteristics of Matched Persons in the Target-Trial Emulation Evaluating the Comparative Effectiveness of the BNT162b2 and mRNA-1273 Vaccines during a Period Marked by Alpha-Variant Predominance (January 4–July 1, 2021).*.
| Characteristic | BNT162b2 Recipients (N=219,842) |
mRNA-1273 Recipients (N=219,842) |
|---|---|---|
| Median age (IQR) — yr | 69 (60–74) | 69 (60–74) |
| Age group — no. (%) | ||
| 18–39 yr | 9,097 (4.1) | 9,097 (4.1) |
| 40–49 yr | 12,222 (5.6) | 12,222 (5.6) |
| 50–59 yr | 31,055 (14.1) | 31,055 (14.1) |
| 60–69 yr | 60,256 (27.4) | 60,256 (27.4) |
| 70–79 yr | 84,459 (38.4) | 84,459 (38.4) |
| ≥80 yr | 22,753 (10.3) | 22,753 (10.3) |
| Sex — no. (%) | ||
| Male | 203,726 (92.7) | 203,726 (92.7) |
| Female | 16,116 (7.3) | 16,116 (7.3) |
| Race — no. (%)† | ||
| White | 163,759 (74.5) | 163,759 (74.5) |
| Black | 44,967 (20.5) | 44,967 (20.5) |
| Other | 4,380 (2.0) | 4,380 (2.0) |
| Unknown | 6,736 (3.1) | 6,736 (3.1) |
| Ethnic group — no. (%)† | ||
| Not Hispanic | 198,649 (90.4) | 193,108 (87.8) |
| Hispanic | 14,939 (6.8) | 20,493 (9.3) |
| Unknown | 6,254 (2.8) | 6,241 (2.8) |
| Urban residence — no. (%) | 161,023 (73.2) | 161,023 (73.2) |
| Smoking status — no. (%) | ||
| Never | 75,341 (34.3) | 73,711 (33.5) |
| Former | 70,347 (32.0) | 66,141 (30.1) |
| Current | 74,154 (33.7) | 79,990 (36.4) |
| Coexisting conditions — no. (%) | ||
| Chronic lung disease‡ | 36,793 (16.7) | 40,166 (18.3) |
| Cardiovascular disease§ | 60,311 (27.4) | 60,423 (27.5) |
| Hypertension | 139,451 (63.4) | 142,733 (64.9) |
| Diabetes | 73,884 (33.6) | 80,061 (36.4) |
| Chronic kidney disease | 21,100 (9.6) | 22,186 (10.1) |
| Liver disease | 859 (0.4) | 784 (0.4) |
| Cancer¶ | 30,870 (14.0) | 29,151 (13.3) |
| Immunocompromised state‖ | 17,872 (8.1) | 17,537 (8.0) |
| Obesity** | 101,740 (46.3) | 102,280 (46.5) |
| No. of primary care visits in the past 5 yr — no. (%) | ||
| 1–9 | 30,788 (14.0) | 25,271 (11.5) |
| 10–19 | 72,752 (33.1) | 70,081 (31.9) |
| 20–29 | 53,420 (24.3) | 55,660 (25.3) |
| ≥30 | 62,882 (28.6) | 68,830 (31.3) |
| No. of influenza vaccinations in the past 5 yr — no. (%) | ||
| 0 | 29,515 (13.4) | 29,355 (13.4) |
| 1 or 2 | 39,765 (18.1) | 39,085 (17.8) |
| 3 or 4 | 70,086 (31.9) | 70,313 (32.0) |
| ≥5 | 80,476 (36.6) | 81,089 (36.9) |
Persons included in this target-trial emulation received a first dose of BNT162b2 or mRNA-1273 between January 4 and May 14, 2021. Percentages may not total 100 because of rounding. IQR denotes interquartile range.
Race and ethnic group were reported by each person in the database.
Chronic lung disease included asthma, bronchitis, and chronic obstructive pulmonary disease.
Cardiovascular disease included acute myocardial infarction, cardiomyopathy, cerebrovascular disease, coronary heart disease, heart failure, and peripheral vascular disease.
Not included here are nonmelanoma skin cancer, benign neoplasms, cancers in situ, and neoplasms of uncertain behavior.
Immunocompromised states included human immunodeficiency virus infection, organ or tissue transplantation, bone marrow biopsy, or the use of any of the following medications (prescribed two or more times during the previous year): systemic glucocorticoids, antiinflammatory or antirheumatic agents in combination with glucocorticoids, and immunosuppressants.
Obesity was defined as a body-mass index (the weight in kilograms divided by the square of the height in meters) of 30 or greater.
The median follow-up period was 126 days (interquartile range, 107 to 147). Over a 24-week follow-up period, 2016 SARS-CoV-2 infections were documented, of which 559 (28%) were detected as symptomatic Covid-19 within the VA health care system, 411 led to hospitalization, 125 led to ICU admission, and 81 resulted in death. Adherence to vaccine-deployment protocols was strict in this population: among persons who received a dose of the BNT162b2 vaccine and had at least 21 days of follow-up, 99% received a second dose of the vaccine (of whom 93% received it before day 24 and 97% received it before day 28). Among persons who received a dose of the mRNA-1273 vaccine and had at least 28 days of follow-up, 98% received a second dose of the vaccine (of whom 92% received it before day 31 and 97% received it before day 35).
Corresponding information for the matched population of veterans vaccinated between July 1 and September 20, 2021, in the analysis of the period marked by delta-variant predominance is provided in the Supplemental Results 1 section in the Supplementary Appendix.
Vaccine Effectiveness
Figure 2 shows the cumulative incidence curves for the study outcomes in each vaccine group during the period marked by alpha-variant predominance. The absolute risks of the outcomes were low in both groups (Table 2). For example, over a 24-week period, the estimated risk of documented infection was 5.75 events per 1000 persons (95% confidence interval [CI], 5.39 to 6.23) for the BNT162b2 vaccine and 4.52 events per 1000 persons (95% CI, 4.17 to 4.84) for the mRNA-1273 vaccine. As expected, we found a nearly identical risk pattern in the two vaccine groups in the evaluations of symptomatic Covid-19 in the first 10 days after the first vaccine dose (Fig. S2) and non–Covid-19–related death during follow-up (Fig. S3).
Figure 2. Cumulative Incidence of Covid-19 Outcomes during a Period Marked by SARS-CoV-2 Alpha-Variant Predominance (January 4–July 1, 2021).
Shaded areas represent pointwise 95% confidence intervals. ICU denotes intensive care unit.
Table 2. Estimated Comparative Effectiveness of the BNT162b2 and mRNA-1273 Vaccines during a Period Marked by Alpha-Variant Predominance (January 4–July 1, 2021).*.
| Covid-19 Outcome | No. of Events | 24-Wk Risk (95% CI) | Risk Difference (95% CI) | Risk Ratio (95% CI) | ||
|---|---|---|---|---|---|---|
| BNT162b2 | mRNA-1273 | BNT162b2 | mRNA-1273 | |||
| events/1000 persons | events/1000 persons | |||||
| Documented infection | 1135 | 881 | 5.75 (5.39 to 6.23) | 4.52 (4.17 to 4.84) | 1.23 (0.72 to 1.81) | 1.27 (1.15 to 1.42) |
| Symptomatic Covid-19 | 327 | 232 | 1.57 (1.42 to 1.76) | 1.13 (0.96 to 1.27) | 0.44 (0.25 to 0.70) | 1.39 (1.21 to 1.70) |
| Hospitalization | 258 | 153 | 1.33 (1.16 to 1.57) | 0.78 (0.64 to 0.91) | 0.55 (0.36 to 0.83) | 1.70 (1.42 to 2.24) |
| ICU admission | 77 | 48 | 0.36 (0.30 to 0.47) | 0.26 (0.17 to 0.36) | 0.10 (0.00 to 0.26) | 1.38 (1.01 to 2.42) |
| Death | 43 | 38 | 0.22 (0.15 to 0.27) | 0.20 (0.12 to 0.26) | 0.02 (−0.06 to 0.12) | 1.11 (0.69 to 1.91) |
Persons newly vaccinated with BNT162b2 were matched in a 1:1 ratio to persons newly vaccinated with mRNA-1273 according to the following variables: calendar date, age, sex, race, urbanicity of residence, and geographic location coded as categories of Veterans Integrated Services Network. CI denotes confidence interval, Covid-19 coronavirus disease 2019, and ICU intensive care unit.
The 24-week risk ratios for recipients of the BNT162b2 vaccine as compared with recipients of the mRNA-1273 vaccine were 1.27 (95% CI, 1.15 to 1.42) for documented SARS-CoV-2 infection, 1.39 (95% CI, 1.21 to 1.70) for symptomatic Covid-19, 1.70 (95% CI, 1.42 to 2.24) for hospitalization for Covid-19, 1.38 (95% CI, 1.01 to 2.42) for ICU admission for Covid-19, and 1.11 (95% CI, 0.69 to 1.91) for death from Covid-19 (Table 2). The risk differences (BNT162b2 minus mRNA-1273), expressed as events over 24 weeks per 1000 persons, were 1.23 (95% CI, 0.72 to 1.81) for documented infection, 0.44 (95% CI, 0.25 to 0.70) for symptomatic Covid-19, 0.55 (95% CI, 0.36 to 0.83) for hospitalization for Covid-19, 0.10 (95% CI, 0.00 to 0.26) for ICU admission for Covid-19, and 0.02 (95% CI, −0.06 to 0.12) for death from Covid-19. On the basis of these estimates, the estimated number needed to vaccinate with mRNA-1273 instead of BNT162b2 during the study period would be 813 (95% CI, 552 to 1389) to prevent one case of documented infection, 2273 (95% CI, 1429 to 4000) to prevent one case of symptomatic Covid-19, and 1818 (95% CI, 1205 to 2778) to prevent one case of Covid-19 hospitalization. Estimates were similar across subgroups defined at baseline on the basis of age and race (Table 3).
Table 3. Estimated Comparative Effectiveness of the BNT162b2 and mRNA-1273 Vaccines in Subgroups Based on Age and Race during a Period Marked by Alpha-Variant Predominance (January 4–July 1, 2021).*.
| Subgroup and Covid-19 Outcome | No. of Events† | 24-Wk Risk (95% CI) | Risk Difference (95% CI) | Risk Ratio (95% CI) | ||
|---|---|---|---|---|---|---|
| BNT162b2 | mRNA-1273 | BNT162b2 | mRNA-1273 | |||
| events/1000 persons | events/1000 persons | |||||
| Age, <70 yr | ||||||
| Documented infection | 516 | 409 | 5.02 (4.46 to 5.39) | 4.56 (3.70 to 5.55) | 0.46 (−0.65 to 1.43) | 1.10 (0.88 to 1.39) |
| Symptomatic Covid-19 | 142 | 91 | 1.30 (1.14 to 1.58) | 0.91 (0.76 to 1.27) | 0.38 (0.00 to 0.68) | 1.42 (1.00 to 1.84) |
| Hospitalization | 102 | 50 | 1.03 (0.76 to 1.23) | 0.48 (0.42 to 0.72) | 0.54 (0.16 to 0.70) | 2.12 (1.23 to 2.48) |
| ICU admission | 32 | 20 | 0.29 (0.17 to 0.40) | 0.19 (0.12 to 0.31) | 0.10 (−0.08 to 0.21) | 1.49 (0.73 to 2.43) |
| Death | — | — | — | — | — | — |
| Age, ≥70 yr | ||||||
| Documented infection | 608 | 497 | 6.31 (5.81 to 6.84) | 4.95 (4.23 to 5.17) | 1.36 (0.93 to 2.31) | 1.27 (1.19 to 1.53) |
| Symptomatic Covid-19 | 179 | 137 | 1.72 (1.49 to 2.01) | 1.33 (1.01 to 1.46) | 0.39 (0.15 to 0.89) | 1.30 (1.11 to 1.82) |
| Hospitalization | 150 | 109 | 1.54 (1.35 to 1.93) | 1.07 (0.73 to 1.13) | 0.47 (0.31 to 1.06) | 1.43 (1.28 to 2.36) |
| ICU admission | 48 | 29 | 0.46 (0.36 to 0.61) | 0.31 (0.17 to 0.39) | 0.15 (0.03 to 0.38) | 1.47 (1.07 to 3.16) |
| Death | 29 | 22 | 0.29 (0.21 to 0.44) | 0.21 (0.14 to 0.31) | 0.08 (−0.04 to 0.24) | 1.37 (0.87 to 2.42) |
| Race, White | ||||||
| Documented infection | 837 | 660 | 5.60 (5.33 to 6.25) | 4.69 (4.06 to 4.86) | 0.90 (0.75 to 1.97) | 1.19 (1.16 to 1.48) |
| Symptomatic Covid-19 | 246 | 178 | 1.56 (1.39 to 1.83) | 1.14 (0.98 to 1.34) | 0.42 (0.16 to 0.76) | 1.37 (1.13 to 1.74) |
| Hospitalization | 190 | 118 | 1.33 (1.11 to 1.65) | 0.80 (0.62 to 0.93) | 0.52 (0.30 to 0.90) | 1.65 (1.35 to 2.36) |
| ICU admission | 54 | 42 | 0.34 (0.27 to 0.46) | 0.30 (0.18 to 0.38) | 0.04 (−0.06 to 0.24) | 1.13 (0.85 to 2.22) |
| Death | 34 | 28 | 0.24 (0.17 to 0.33) | 0.18 (0.11 to 0.25) | 0.05 (−0.04 to 0.19) | 1.30 (0.83 to 2.60) |
| Race, Black | ||||||
| Documented infection | 219 | 201 | 5.95 (4.93 to 7.12) | 5.01 (4.06 to 5.72) | 0.94 (−0.20 to 2.52) | 1.19 (0.96 to 1.57) |
| Symptomatic Covid-19 | 58 | 42 | 1.37 (1.12 to 1.82) | 1.04 (0.82 to 1.58) | 0.33 (−0.21 to 0.84) | 1.32 (0.86 to 1.95) |
| Hospitalization | 49 | 36 | 1.22 (0.91 to 1.71) | 0.86 (0.55 to 1.16) | 0.36 (−0.01 to 0.93) | 1.43 (0.99 to 2.56) |
| ICU admission | — | — | — | — | — | — |
| Death | — | — | — | — | — | — |
Persons newly vaccinated with BNT162b2 were matched in a 1:1 ratio to persons newly vaccinated with mRNA-1273 according to the following variables: calendar date, age, sex, race (except for subgroup analyses restricted to race), urbanicity of residence, and geographic location coded as categories of Veterans Integrated Services Network. Estimates were calculated only in analyses in which there were more than 10 outcome events in the vaccine groups under comparison.
The sum of events across subgroups may not equal the sum of events in the overall population because the entire analysis (including matching) was repeated after stratification of the population according to baseline characteristics.
The 12-week risk of documented SARS-CoV-2 infection in a period marked by delta-variant predominance was also higher with the BNT162b2 vaccine than with the mRNA-1273 vaccine: the risk ratio was 1.58 (95% CI, 0.85 to 2.33), and the risk difference was 6.54 events per 1000 persons (95% CI, −2.58 to 11.82) (see the Supplemental Results 1 section in the Supplementary Appendix). As compared with the matched population during the period marked by alpha-variant predominance, the matched population during the period marked by delta-variant predominance was, on average, younger; included higher percentages of Black persons, current smokers, and persons who had received no influenza vaccinations in the previous 5 years at a VA facility; and included a lower percentage of persons with coexisting conditions (Table S3).
Discussion
We quantified the comparative effectiveness of the BNT162b2 and mRNA-1273 vaccines for the prevention of Covid-19 outcomes in the largest integrated health care system in the United States. The risks of outcomes were low, regardless of the vaccine received. Recipients of the BNT162b2 vaccine had a 27% higher risk of documented SARS-CoV-2 infection and a 70% higher risk of hospitalization for Covid-19 than recipients of the mRNA-1273 vaccine over 24 weeks of follow-up in a period marked by alpha-variant predominance. We also found a higher risk of documented infection among recipients of BNT162b2 than among recipients of mRNA-1273 over 12 weeks of follow-up in a period marked by delta-variant predominance, although the estimate was less precise because of the smaller number of eligible persons.
Our findings are consistent with those of studies that have reported a higher SARS-CoV-2–binding antibody response among recipients of the mRNA-1273 vaccine than among recipients of the BNT162b2 vaccine.7,8 Although these studies did not measure neutralizing antibodies, a recent unpublished report showed a lower risk of Covid-19 outcomes among recipients of mRNA-1273 than among recipients of BNT162b2 in the Mayo Clinic Health System (a study population that was approximately 95% White).9 However, the interpretation of this finding is not straightforward because the analyses were conditional on postbaseline factors (the study population was restricted to persons who underwent at least one SARS-CoV-2 PCR test during follow-up) and included few events for severe Covid-19 outcomes.9 In another report, effectiveness against Covid-19 hospitalization was evaluated separately for each mRNA vaccine as compared with no vaccination in two case–control studies.10 An indirect comparison of the results from the two case–control studies suggests that mRNA-1273 was more effective than BNT162b2 if the controls from both studies had similar characteristics, but the case–control design precluded estimation of absolute risk.
A difference in effectiveness between the BNT162b2 and mRNA-1273 vaccines might be the result of the different mRNA content of the vaccines (100 μg for mRNA-1273 vs. 30 μg for BNT162b2), the different interval between the priming and boosting doses (4 weeks for mRNA-1273 vs. 3 weeks for BNT162b2), or other factors, such as the lipid composition of the nanoparticles used for packaging the mRNA content.1,2,6
Our study has several strengths. First, the VA health care databases capture rich data on demographic factors, medical records, laboratory test results (for the outcome of documented infection), and health care encounters in both outpatient settings (for the outcome of symptomatic Covid-19) and inpatient settings (for outcomes related to severe Covid-19) for millions of persons nationwide, with nightly updates that allow for nearly real-time analyses. The richness of these data allowed us to characterize recipients of each vaccine type with high resolution and to closely match them according to key confounders. Second, the large size of the study population allowed us to evaluate the comparative effectiveness of mRNA-based vaccines with respect to less common Covid-19 outcomes (including 411 hospitalizations, 125 ICU admissions, and 81 deaths). Third, the demographic composition of the U.S. veteran population allowed us to provide evidence for a diverse cohort (21% Black and 8% Hispanic), as well as to conduct subgroup analyses among older persons (≥70 years of age) and Black persons, who have been disproportionately affected by Covid-19 yet are often underrepresented in biomedical research.16
Our study also has several potential limitations. First, as in any observational analysis, assignment to a particular vaccine was not randomized. If the two vaccine groups had different distributions of risk factors, the effect estimates would be confounded. However, we rigorously matched recipients of each vaccine type; the vaccine groups were similar with respect to their demographic characteristics, medical history, and markers of health care utilization (e.g., number of primary care visits and number of influenza vaccinations in the previous 5 years); and much less confounding is expected when comparing recipients of different vaccines than when comparing vaccinated and unvaccinated persons. In addition, our two analyses involving negative outcome controls13 suggested little confounding. Second, the possibility of outcome misclassification cannot be ruled out if veterans obtained care outside the VA health care system. However, our use of the VA Covid-19 National Surveillance Tool allowed us to integrate data on laboratory tests with natural language processing of clinical notes to capture infections documented inside and outside the VA health care system. Furthermore, our eligibility criteria were designed to select regular VA users with a known residential address to improve outcome ascertainment. Even in the presence of residual misclassification, we would expect this to be nondifferential between the vaccination groups under comparison; nondifferential misclassification would have minimal influence on the relative measures of effect, although the absolute risks may have been slightly underestimated. Finally, our study population was mostly made up of men (93%) and older persons (90% were >50 years of age), which may limit the generalizability of our findings.
Although this study provides evidence of potentially different effectiveness of the BNT162b2 and mRNA-1273 vaccines, any choice between two vaccines must also take their comparative safety into consideration, and safety was not studied here. Head-to-head comparisons of the BNT162b2 and mRNA-1273 vaccines for safety outcomes are lacking, but early randomized trials identified only transient local and systemic reactions (e.g., pain at the injection site and headache) that are common among other viral vaccines,1,2 and observational studies and surveillance efforts have confirmed the safety of these vaccines for the population overall.17,18 In fact, even with respect to events for which the risk is increased after vaccination, the risk is even greater after natural infection with SARS-CoV-2 than after vaccination.17 Given the high effectiveness and the safety profile of both mRNA vaccines, either vaccine is strongly recommended.1-3,17
In summary, although the absolute risks of each studied outcome were low in both vaccine groups, this study involving a nationwide cohort of U.S. veterans provides evidence of a lower 24-week risk of Covid-19–related outcomes among recipients of the mRNA-1273 vaccine than among recipients of the BNT162b2 vaccine. This pattern was consistent across periods marked by alpha-variant and delta-variant predominance. Further evaluation of the comparative effectiveness and safety of these vaccines is needed.
Acknowledgments
We thank Roger W. Logan for advice on the analysis, Daniel C. Posner and Yuk-Lam (Anne) Ho for insights on Covid-19 data extraction and phenotype definitions, Jessica K. Wise for assistance executing data extraction queries, Constance A. Hoag for management of the administrative and regulatory aspects of the project, the VA Covid-19 Shared Data Resource team for their contributions and support, and the VA health care providers, employees, and volunteers for their dedication to caring for our veterans through this pandemic.
Supplementary Appendix
Disclosure Forms
The contents of this article do not represent the views of the U.S. Department of Veterans Affairs or the U.S. Government. The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Department of Health and Human Services and its agencies, including the Biomedical Advanced Research and Development Authority and the Food and Drug Administration, as well as any other agency of the U.S. Government. Assumptions made within and interpretations from the analysis do not necessarily reflect the position of any U.S. Government entity.
This article was published on December 1, 2021, at NEJM.org.
Footnotes
Supported by the Department of Veterans Affairs (VA) Office of Research and Development Cooperative Studies Program (CSP) Epidemiology Center at the VA Boston Healthcare System through CSP 2032, by resources and the use of facilities at the VA Boston Healthcare System and VA Informatics and Computing Infrastructure (VA HSR RES 13-457), and by the use of data from the VA Covid-19 Shared Data Resource. Dr. Dickerman is supported by a grant (K99 CA248335) from the National Institutes of Health. Dr. Gerlovin and Mr. Ferolito are supported by a grant (MVP000) from the VA Million Veteran Program Data Core. Mr. Figueroa Muñiz is supported by a grant (T32 GM140972) from the National Institute of General Medical Sciences Interdisciplinary Training Program for Biostatisticians.
Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.
References
- 1.Polack FP, Thomas SJ, Kitchin N, et al. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020;383:2603-2615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med 2021;384:403-416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dagan N, Barda N, Kepten E, et al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med 2021;384:1412-1423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hall VJ, Foulkes S, Saei A, et al. COVID-19 vaccine coverage in health-care workers in England and effectiveness of BNT162b2 mRNA vaccine against infection (SIREN): a prospective, multicentre, cohort study. Lancet 2021;397:1725-1735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Thompson MG, Burgess JL, Naleway AL, et al. Interim estimates of vaccine effectiveness of BNT162b2 and mRNA-1273 COVID-19 vaccines in preventing SARS-CoV-2 infection among health care personnel, first responders, and other essential and frontline workers — eight U.S. locations, December 2020–March 2021. MMWR Morb Mortal Wkly Rep 2021;70:495-500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Schoenmaker L, Witzigmann D, Kulkarni JA, et al. mRNA-lipid nanoparticle COVID-19 vaccines: structure and stability. Int J Pharm 2021;601:120586-120586. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Steensels D, Pierlet N, Penders J, Mesotten D, Heylen L. Comparison of SARS-CoV-2 antibody response following vaccination with BNT162b2 and mRNA-1273. JAMA 2021;326:1533-1535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Richards NE, Keshavarz B, Workman LJ, Nelson MR, Platts-Mills TAE, Wilson JM. Comparison of SARS-CoV-2 antibody response by age among recipients of the BNT162b2 vs the mRNA-1273 vaccine. JAMA Netw Open 2021;4(9):e2124331-e2124331. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Puranik A, Lenehan PJ, Silvert E, et al. Comparison of two highly-effective mRNA vaccines for COVID-19 during periods of alpha and delta variant prevalence. August 21, 2021. (https://www.medrxiv.org/content/10.1101/2021.08.06.21261707v3). preprint.
- 10.Self WH, Tenforde MW, Rhoads JP, et al. Comparative effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) vaccines in preventing COVID-19 hospitalizations among adults without immunocompromising conditions — United States, March–August 2021. MMWR Morb Mortal Wkly Rep 2021;70:1337-1343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Centers for Disease Control and Prevention. COVID data tracker: variant proportions (https://covid.cdc.gov/covid-data-tracker/#variant-proportions).
- 12.Chapman A, Peterson K, Turano A, Box T, Wallace K, Jones M. A natural language processing system for national COVID-19 surveillance in the US Department of Veterans Affairs. In: Proceedings and abstracts of the 58th Annual Meeting of the Association for Computational Linguistics, July 6–10, 2020. Stroudsburg, PA: Association for Computational Linguistics, 2020. (https://www.aclweb.org/anthology/2020.nlpcovid19-acl.10). [Google Scholar]
- 13.Lipsitch M, Tchetgen Tchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology 2010;21:383-388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Harder VS, Stuart EA, Anthony JC. Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychol Methods 2010;15:234-249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc 1958;53:457-481. [Google Scholar]
- 16.Centers for Disease Control and Prevention. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity. 2021. (https://www.cdc.gov/coronavirus/2019-ncov/covid-data/investigations-discovery/hospitalization-death-by-race-ethnicity.html).
- 17.Barda N, Dagan N, Ben-Shlomo Y, et al. Safety of the BNT162b2 mRNA Covid-19 vaccine in a nationwide setting. N Engl J Med 2021;385:1078-1090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Klein NP, Lewis N, Goddard K, et al. Surveillance for adverse events after COVID-19 mRNA vaccination. JAMA 2021;326:1390-1399. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.