This cohort study uses data from the national health care databases of the US Department of Veterans Affairs to compare the risks of adverse events associated with BNT162b2 and mRNA-1273 vaccines reported among US veterans.
Key Points
Question
How do risks of adverse events compare after vaccination with BNT162b2 (Pfizer-BioNTech) vs mRNA-1273 (Moderna Inc) vaccines?
Findings
In this cohort study of 433 672 US veterans during 38 weeks of follow-up, recipients of the BNT162b2 vaccine, compared with recipients of the mRNA-1273 vaccine, had an excess per 10 000 persons of 10.9 ischemic stroke events, 14.8 myocardial infarction events, 11.3 other thromboembolic events, and 17.1 kidney injury events. Small-magnitude differences between the 2 vaccines were seen within 42 days of the first dose, and few differences were seen within 14 days of the first dose.
Meaning
This study’s findings suggest that there were few differences in risk of adverse events within 14 days of the first dose of either the BNT162b2 or the mRNA-1273 vaccine and small-magnitude differences within 42 days and 38 weeks of the first dose.
Abstract
Importance
The risk of adverse events has been found to be low for participants receiving the BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna Inc) vaccines in randomized trials. However, a head-to-head comparison of their safety for a broader range of potential adverse events over longer follow-up and in larger and more diverse populations is lacking, to our knowledge.
Objective
To compare the head-to-head safety in terms of risk of adverse events of the BNT162b2 and mRNA-1273 vaccines in the national health care databases of the US Department of Veterans Affairs, the largest integrated health care system in the US.
Design, Setting, and Participants
In this cohort study, the electronic health records of US veterans who received a first dose of the BNT162b2 or mRNA-1273 vaccine between January 4 and September 20, 2021, were used. Recipients of each vaccine were matched in a 1:1 ratio according to their risk factors.
Exposures
Vaccination with either the BNT162b2 vaccine, with a second dose scheduled 21 days later, or the mRNA-1273 vaccine, with a second dose scheduled 28 days later.
Main Outcomes and Measures
A large panel of potential adverse events was evaluated; the panel included neurologic events, hematologic events, hemorrhagic stroke, ischemic stroke, myocardial infarction, other thromboembolic events, myocarditis or pericarditis, arrhythmia, kidney injury, appendicitis, autoimmune events, herpes zoster or simplex, arthritis or arthropathy, and pneumonia. Risks over 38 weeks were estimated using the Kaplan-Meier estimator.
Results
Among 433 672 persons included in the matched vaccine groups, the median age was 69 years (IQR, 60-74 years), 93% of individuals were male, and 20% were Black. Estimated 38-week risks of adverse events were generally low after administration of either the BNT162b2 or the mRNA-1273 vaccine. Compared with the mRNA-1273 group, the BNT162b2 group had an excess per 10 000 persons of 10.9 events (95% CI, 1.9-17.4 events) of ischemic stroke, 14.8 events (95% CI, 7.9-21.8 events) of myocardial infarction, 11.3 events (95% CI, 3.4-17.7 events) of other thromboembolic events, and 17.1 events (95% CI, 8.8-30.2 events) of kidney injury. Estimates were largely similar among subgroups defined by age (<40, 40-69, and ≥70 years) and race (Black, White), but there were higher magnitudes of risk differences of ischemic stroke among older persons and White persons, kidney injury among older persons, and other thromboembolic events among Black persons. Small-magnitude differences between the 2 vaccines were seen within 42 days of the first dose, and few differences were seen within 14 days of the first dose.
Conclusions and Relevance
The findings of this cohort study suggest that there were few differences in risk of adverse events within 14 days of the first dose of either the BNT162b2 or the mRNA-1273 vaccine and small-magnitude differences within 42 days of the first dose. The 38-week risks of adverse events were low in both vaccine groups, although risks were lower for recipients of the mRNA-1273 vaccine than for recipients of the BNT162b2 vaccine. Although the primary analysis was designed to detect safety events unrelated to SARS-CoV-2 infection, the possibility that these differences may partially be explained by a lower effectiveness of the BNT162b2 vaccine in preventing the sequelae of SARS-CoV-2 infection compared with the mRNA-1273 vaccine could not be ruled out. These findings may help inform decision-making in future vaccination campaigns.
Introduction
Early randomized trials of the 2 messenger RNA (mRNA)–based vaccines—BNT162b2 (Pfizer-BioNTech) and mRNA-1273 (Moderna Inc)—revealed a low incidence of adverse events, largely limited to transient reactogenicity common among other viral vaccines.1,2 However, the randomized trials were relatively small and short, and their participants did not reflect the entire populations ultimately receiving the vaccine. Postapproval surveillance via observational studies is therefore critical for monitoring adverse events in vaccine campaigns.
Observational studies3,4 have provided evidence of the overall safety of these vaccines: for adverse events for which the risk appears to increase after vaccination,3,4 the risk is greater after infection with SARS-CoV-2 among unvaccinated persons.3 However, the 2 mRNA-based vaccines need to be compared head-to-head to inform public health decisions. Ideally, these comparative safety studies would be large enough to provide precise estimates for a broad panel of potential adverse events, have sufficiently long follow-up to evaluate adverse events for which recording may be delayed or that may unfold over time, and include vulnerable (eg, older) and racially and ethnically diverse groups.
A previous head-to-head comparison of the effectiveness of the BNT162b2 and mRNA-1273 vaccines5 showed a low risk of documented SARS-CoV-2 infection in both vaccine groups but a lower risk for the mRNA-1273 vaccine. Because of this difference in effectiveness, evaluating vaccine safety requires distinguishing adverse events that arise from the vaccines themselves from adverse events due to the infections they are intended to prevent.
In this study, we used data from the national health care databases of the US Department of Veterans Affairs (VA), the largest integrated health care system in the US, to compare the safety profiles of the BNT162b2 and mRNA-1273 vaccines during 38 weeks of follow-up overall and in subgroups defined by age and by race.
Methods
Specification of the Target Trials
We designed this cohort study to emulate target trials (ie, hypothetical pragmatic trials that would answer the causal questions of interest) of BNT162b2 as compared with mRNA-1273 and risk of potential adverse events in the VA health care system. eTable 1 in the Supplement summarizes the key protocol components. This work was approved by the VA Boston Healthcare System Research & Development Committee and Human Subjects Subcommittee, and an exemption of informed consent and Health Insurance Portability and Accountability Act (HIPAA) authorization was granted because it was determined that the research presented no more than minimal risk to privacy and could not be practically conducted without the waiver.
Eligibility criteria included veteran status; age of at least 18 years between January 4 and September 20, 2021; no previously documented COVID-19 vaccination; no previously documented SARS-CoV-2 infection; no history of the adverse event of interest; known residential address outside of a long-term care facility; known smoking status; and known body mass index within the previous year. Individuals had to have used the VA health care system during the previous year (defined as receiving care at a station eligible to administer both vaccines under study and having at least 1 primary care visit) but not within the previous 3 days (which might suggest the start of symptomatic disease that precluded vaccination).
The interventions of interest were vaccination with either the BNT162b2 vaccine with a second dose scheduled 21 days later or the mRNA-1273 vaccine with a second dose scheduled 28 days later. To ensure balance of important characteristics across groups, eligible veterans in the target trial would be randomly assigned to 1 of these 2 vaccines within strata defined by calendar date (5-day bins), age (5-year bins), sex (male, female), race (Black, White, other, unknown), urbanicity of residence (urban, not urban), and geographic location (coded as 19 categories of Veterans Integrated Services Network). Race and ethnicity were reported by each person.
The prespecified adverse events of interest were classified as neurologic events, hematologic events, hemorrhagic stroke, ischemic stroke, myocardial infarction, other thromboembolic events, myocarditis or pericarditis, arrhythmia, kidney injury, appendicitis, autoimmune events, herpes zoster or simplex, arthritis or arthropathy, and pneumonia. We selected these categories based on their inclusion in prior vaccine safety studies,3,4 imbalances in results of phase 3 randomized trials,1,2 and potential signals from ongoing surveillance and case reports.4,6,7,8,9,10,11 Mild and transient local and systemic reactions (eg, pain at the injection site, fatigue, headache), which have been commonly reported after receipt of mRNA-based vaccines12 but are unlikely to require medical attention, were not studied.
For each eligible individual, follow-up started on the day the first dose of vaccine was received (baseline) and ended on the day of the outcome of interest, 38 weeks after baseline or October 5, 2021, whichever occurred first. Follow-up of a matched pair also ended when either member developed a documented SARS-CoV-2 infection for evaluation of potential complications arising from the vaccines themselves and not from the infections they were intended to prevent.
Emulation of the Target Trials
We emulated the aforementioned target trials using the VA health care databases as described previously.5 Vaccination was identified using records in the immunization domain and procedures recorded in the outpatient or inpatient domain of the database. The adverse events were defined using diagnosis codes recorded in inpatient and outpatient domains as well as fee domains (claims for out-of-network care) to capture diagnoses documented both inside and outside the VA health care system. For diagnoses that may be sequelae of SARS-CoV-2 infection (ie, myocarditis or pericarditis, kidney injury, thromboembolic events), we verified that they could not be attributed to infection based on manual review of clinical notes available from the electronic health records for a sample of individuals. SARS-CoV-2 infections were identified with the VA COVID-19 National Surveillance Tool,13 which integrates data on polymerase chain reaction laboratory tests with natural language processing of clinical notes to capture diagnoses inside and outside the VA health care system. Detailed definitions of all study variables are provided in eTable 2 in the Supplement.
In the emulation of the target trials, vaccine type was not randomly assigned within the strata defined previously. In some instances, individuals may have received the first dose of whichever vaccine was readily available to them; in others, individuals may have sought out a particular vaccine type. We attempted to mimic the stratified randomization of the target trial by matching eligible persons who were vaccinated with BNT162b2 to eligible persons vaccinated with mRNA-1273, using the same matching algorithm described in a previous study.5 The matching factors (calendar date, age, sex, race, urbanicity of residence, and geographic location) are associated with the probability of receiving a particular vaccine and risk of the outcomes.
Statistical Analysis
Covariate balance after matching was assessed by evaluating mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered acceptable.14 Cumulative incidence (risk) curves for the vaccination groups were estimated using the Kaplan-Meier estimator.15 We then reported the 38-week risks of each outcome and compared them between vaccination groups via differences and ratios.
SARS-CoV-2 infections may cause some adverse events (eg, myocarditis) that might have been prevented by vaccination. Then, even if both vaccines had identical safety profiles in the absence of SARS-CoV-2 infection, the more effective vaccine would appear to be safer during a COVID-19 outbreak. To estimate the risk of adverse events attributable to a harmful effect of the vaccine rather than to lower effectiveness, we truncated the follow-up of a matched pair when either member had a documented SARS-CoV-2 infection, as described in the Specification of the Target Trials subsection. Because some undocumented asymptomatic and mild infections might cause adverse events, we also studied the outcome “pneumonia,” which was not expected to be a complication of vaccination, as a marker of undocumented SARS-CoV-2 infection. In a secondary analysis, individuals who had a documented SARS-CoV-2 infection during the follow-up period were not censored to estimate risks that capture complications arising both from the vaccines themselves and from the incident infections that they were intended to prevent (see also the eMethods in the Supplement).
We also conducted secondary analyses evaluating 14-day and 42-day risk to distinguish adverse events that might be documented in the weeks after the first dose and after the second dose of the vaccine, respectively. We conducted subgroup analyses according to age (<40, 40-69, ≥70 years) and race (Black, White).
A nonparametric bootstrapping procedure with 500 iterations was used to calculate percentile-based 95% CIs for all estimates. The bootstrap populations were resampled from the entire eligible population, and the bootstrap procedure included all analyses beginning with matching to account for all contributors to the variance estimates.
Analyses were performed with R software, version 3.6.0 (R Foundation for Statistical Computing) and SAS software, version 8.2 (SAS Institute Inc).
Results
Study Population and Follow-up
Of the 369 846 recipients of the BNT162b2 vaccine and the 400 261 recipients of the mRNA-1273 vaccine who were eligible for the study (Figure), 216 836 BNT162b2 recipients were matched to 216 836 mRNA-1273 recipients before outcome-specific exclusion criteria were applied. Table 1 provides the baseline characteristics of this matched population, which were similar to those of the eligible population (eTable 3 in the Supplement). The median age was 69 years (IQR, 60-74 years), 93% of individuals were male, 20% were Black, and 8% were Hispanic. The 2 vaccine groups had similar distributions of all measured variables, including demographics, coexisting conditions, and markers of health care utilization. The median follow-up was 223 days (IQR, 203-246 days) for BNT162b2 and 223 days (IQR, 203-245 days) for mRNA-1273. The baseline characteristics of matched populations after application of outcome-specific exclusion criteria are provided in eTable 4 in the Supplement.
Figure. Selection of Persons for the Emulation of a Target Trial Evaluating the Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines (January 4 to October 5, 2021).
BMI indicates body mass index; VA, US Department of Veterans Affairs.
Table 1. Baseline Characteristics of Matched Persons in the Target Trial Emulation Evaluating the Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines, Veterans Health Administrationa.
| Characteristic | No. (%)b | |
|---|---|---|
| BNT162b2 recipients (n = 216 836) | mRNA-1273 recipients (n = 216 836) | |
| Age, median (IQR), y | 69.0 (60.0-74.0) | 69.0 (60.0-74.0) |
| Age group, y | ||
| 18-39 | 9335 (4) | 9335 (4) |
| 40-49 | 12 394 (6) | 12 394 (6) |
| 50-59 | 30 382 (14) | 30 382 (14) |
| 60-69 | 57 749 (27) | 57 749 (27) |
| 70-79 | 84 377 (39) | 84 377 (39) |
| ≥80 | 22 599 (10) | 22 599 (10) |
| Sex | ||
| Male | 200 908 (93) | 200 908 (93) |
| Female | 15 928 (7) | 15 928 (7) |
| Racec | ||
| Black | 43 452 (20) | 43 452 (20) |
| White | 162 603 (75) | 162 603 (75) |
| Otherd | 4202 (2) | 4202 (2) |
| Unknown | 6579 (3) | 6579 (3) |
| Ethnicityc | ||
| Hispanic | 14 791 (7) | 20 425 (9) |
| Not Hispanic | 195 911 (90) | 190 205 (88) |
| Unknown | 6134 (3) | 6206 (3) |
| Urban residence | 157 955 (73) | 157 955 (73) |
| Smoking status | ||
| Current | 71 148 (33) | 76 917 (35) |
| Former | 70 012 (32) | 65 829 (30) |
| Never | 75 676 (35) | 74 090 (34) |
| Coexisting conditions | ||
| Chronic lung diseasee | 35 616 (16) | 38 890 (18) |
| Cardiovascular diseasef | 59 153 (27) | 59 266 (27) |
| Hypertension | 137 265 (63) | 140 774 (65) |
| Diabetes | 72 895 (34) | 79 338 (37) |
| Chronic kidney disease | 20 844 (10) | 21 738 (10) |
| Liver disease | 821 (<1) | 767 (<1) |
| Cancerg | 30 490 (14) | 28 688 (13) |
| Immunocompromised stateh | 17 370 (8) | 16 764 (8) |
| Obesityi | 100 885 (47) | 101 207 (47) |
| No. of primary care visits in the past 5 y | ||
| 1-9 | 30 854 (14) | 25 291 (12) |
| 10-19 | 72 717 (34) | 70 252 (32) |
| 20-29 | 52 481 (24) | 55 013 (25) |
| ≥30 | 60 784 (28) | 66 280 (31) |
| No. of influenza vaccinations in the past 5 y | ||
| 0 | 29 863 (14) | 29 758 (14) |
| 1 or 2 | 39 133 (18) | 38 179 (18) |
| 3 or 4 | 68 123 (31) | 68 107 (31) |
| ≥5 | 79 717 (37) | 80 792 (37) |
Persons included in this target trial emulation received a first dose of BNT162b2 (Pfizer-BioNTech) or mRNA-1273 (Moderna Inc) between January 4 and September 20, 2021. (eTable 2 in the Supplement provides detailed definitions of all baseline characteristics.)
Percentages may not total 100% because of rounding.
Race and ethnicity were reported by each person in the database.
Other included Alaska Native or American Indian, Asian, and Native Hawaiian or other Pacific Islander.
Chronic lung disease included asthma, bronchitis, and chronic obstructive pulmonary disease.
Cardiovascular disease included acute myocardial infarction, cardiomyopathy, cerebrovascular disease, coronary heart disease, heart failure, and peripheral vascular disease.
Not included here are nonmelanoma skin cancer, benign neoplasms, cancers in situ, and neoplasms of uncertain behavior.
Immunocompromised state included HIV infection, organ or tissue transplant, bone marrow biopsy, and use of any of the following medications (prescribed ≥2 times over the past year): systemic glucocorticoids, anti-inflammatory or anti-rheumatic agents in combination with glucocorticoids, immunosuppressants.
Obesity was defined as a body mass index (weight in kilograms divided by height in meters squared) of 30 or greater.
Adherence to vaccine protocols was strict. Among persons who received a dose of the BNT162b2 vaccine and had at least 21 days of follow-up, 98% received a second dose of the vaccine (of whom 92% received it before day 24 and 96% received it before day 28). Among persons who received a dose of the mRNA-1273 vaccine and had at least 28 days of follow-up, 98% received a second dose of the vaccine (of whom 92% received it before day 31 and 97% received it before day 35).
Comparative Safety
The risks of adverse events ranged from approximately 3 to 347 events per 10 000 persons across 38 weeks, depending on the event (Table 2 and eFigure 1 in the Supplement). During those 38 weeks, recipients of the BNT162b2 vaccine, compared with recipients of the mRNA-1273 vaccine, had an excess per 10 000 persons of 10.9 events (95% CI, 1.9-17.4 events) of ischemic stroke, 14.8 events (95% CI, 7.9-21.8 events) of myocardial infarction, 11.3 events (95% CI, 3.4-17.7 events) of other thromboembolic events, and 17.1 events (95% CI, 8.8-30.2 events) of kidney injury (Table 2). The corresponding risk ratios (BNT162b2 vs mRNA-1273) were 1.17 (95% CI, 1.03-1.28) for ischemic stroke, 1.32 (95% CI, 1.16-1.49) for myocardial infarction, 1.20 (95% CI, 1.05-1.32) for other thromboembolic events, and 1.16 (95% CI, 1.08-1.29) for kidney injury. In analyses that did not attempt to exclude COVID-19-driven adverse events, estimated risk differences were only slightly greater except for the estimated risk difference for pneumonia, which increased from 5.9 (95% CI, −2.9 to 12.7) events per 10 000 persons to 20.9 (95% CI, 11.4-29.5) events per 10 000 persons (eTable 5, eFigure 2 in the Supplement).
Table 2. Estimated Comparative Safety of the BNT162b2 and mRNA-1273 Vaccines, Veterans Health Administration (January 4 to October 5, 2021)a.
| Event | No. of persons | No. of events | No. of events/10 000 persons (95% CI) | Risk ratio (95% CI) | |||
|---|---|---|---|---|---|---|---|
| BNT162b2 | mRNA-1273 | 38-wk Risk | Risk difference | ||||
| BNT162b2 | mRNA-1273 | ||||||
| Neurologicb | 233 970 | 2978 | 2976 | 313.4 (303.5 to 335.5) | 308.2 (292.2 to 320.4) | 5.2 (−7.4 to 34.3) | 1.02 (0.98 to 1.11) |
| Hematologicc | 279 014 | 3648 | 3693 | 337.5 (322.5 to 355.9) | 347.1 (329.2 to 361.1) | −9.6 (−30.9 to 17.9) | 0.97 (0.91 to 1.05) |
| Hemorrhagic stroke | 431 508 | 82 | 68 | 4.4 (3.1 to 5.0) | 3.5 (3.0 to 5.0) | 0.9 (−1.4 to 1.5) | 1.25 (0.72 to 1.45) |
| Ischemic stroke | 392 262 | 1178 | 1002 | 74.8 (70.9 to 81.0) | 64.0 (60.5 to 72.0) | 10.9 (1.9 to 17.4) | 1.17 (1.03 to 1.28) |
| Myocardial infarction | 406 136 | 949 | 752 | 60.7 (57.0 to 68.3) | 45.9 (43.2 to 51.9) | 14.8 (7.9 to 21.8) | 1.32 (1.16 to 1.49) |
| Other thromboembolicd | 402 126 | 1140 | 901 | 68.7 (65.3 to 74.7) | 57.4 (54.1 to 64.2) | 11.3 (3.4 to 17.7) | 1.20 (1.05 to 1.32) |
| Myocarditis or pericarditis | 429 564 | 80 | 48 | 5.4 (3.4 to 6.6) | 2.7 (2.0 to 3.7) | 2.7 (0.1 to 4.0) | 2.01 (1.03 to 2.68) |
| Arrhythmia | 277 438 | 3000 | 2693 | 277.6 (256.2 to 284.7) | 251.4 (241.7 to 266.6) | 26.2 (−1.3 to 36.2) | 1.10 (1.00 to 1.15) |
| Kidney injury | 356 008 | 1811 | 1515 | 126.5 (120.4 to 136.6) | 109.4 (101.0 to 117.1) | 17.1 (8.8 to 30.2) | 1.16 (1.08 to 1.29) |
| Appendicitis | 428 856 | 80 | 81 | 4.5 (4.0 to 6.9) | 4.7 (3.1 to 5.3) | −0.2 (−0.6 to 3.2) | 0.95 (0.87 to 1.88) |
| Autoimmunee | 376 088 | 911 | 928 | 65.0 (58.3 to 72.8) | 63.4 (57.3 to 68.6) | 1.6 (−7.0 to 11.8) | 1.03 (0.89 to 1.19) |
| Herpes zoster or simplex | 387 056 | 699 | 657 | 43.2 (39.2 to 47.1) | 41.4 (37.7 to 44.9) | 1.9 (−3.9 to 6.6) | 1.05 (0.91 to 1.16) |
| Arthritis or arthropathy | 397 732 | 798 | 748 | 47.9 (44.5 to 52.2) | 45.7 (41.1 to 49.6) | 2.2 (−2.4 to 9.2) | 1.05 (0.95 to 1.22) |
| Pneumonia | 379 978 | 1255 | 1165 | 83.3 (78.1 to 90.1) | 77.5 (73.7 to 84.4) | 5.9 (−2.9 to 12.7) | 1.08 (0.96 to 1.17) |
Persons newly vaccinated with the BNT162b2 vaccine (Pfizer-BioNTech) were matched in a 1:1 ratio to persons newly vaccinated with the mRNA-1273 vaccine (Moderna Inc) according to the following variables: calendar date, age, sex, race, urbanicity of residence, and geographic location coded as categories of Veterans Integrated Services Network. Follow-up of a matched pair ended when either member developed a documented SARS-CoV-2 infection.
Neurologic events included Bell palsy, nonfacial paralysis, paresthesia, seizure, syncope, and vertigo.
Hematologic events included anemia, lymphopenia, neutropenia, and thrombocytopenia.
Other thromboembolic events included cerebral venous sinus thrombosis, deep vein thrombosis or thrombophlebitis, pulmonary embolism, and other thrombosis (arterial embolism and thrombosis, vascular insufficiency of the intestine).
Autoimmune events included Guillain-Barré syndrome, inflammatory bowel disease, multiple sclerosis, psoriasis, rheumatoid arthritis, systemic lupus erythematosus, and uveitis.
During the first 14 days after receipt of the first vaccine dose, the estimated risks of the evaluated safety outcomes were low and similar in both vaccine groups (eTable 6 in the Supplement). The estimated risks were also low during the first 42 days after receipt of the first vaccine dose, but recipients of the BNT162b2 vaccine compared with recipients of the mRNA-1273 vaccine had an excess per 10 000 persons of other thromboembolic events (1.8; 95% CI, 0.2-4.2 events), arrhythmia (7.5; 95% CI, 1.9-11.5 events), and kidney injury (3.8; 95% CI, 2.1-8.0 events) (eTable 7 in the Supplement).
Estimates were largely similar among subgroups defined according to age and race, with several exceptions. For example, the estimated 38-week risk difference (BNT162b2 minus mRNA-1273) was higher for ischemic stroke among persons 70 years or older (15.5; 95% CI, 3.2-23.2) than among persons aged 40 to 69 years (5.7; 95% CI, −1.2 to 19.3) and younger than 40 years (3.4; 95% CI, −3.8 to 10.4); higher for kidney injury among persons 70 years or older (24.4; 95% CI, 11.7-42.6) than among persons aged 40 to 69 years (2.4; 95% CI, −8.8 to 34.2) and younger than 40 years (5.7; 95% CI, −0.7 to 21.5); higher for ischemic stroke among White persons (10.5; 95% CI, 2.4-20.4) than among Black persons (−1.2; 95% CI, −20.5 to 13.2); and higher for other thromboembolic events among Black persons (38.7; 95% CI, 15.8-49.0) than among White persons (10.0; 95% CI, −0.7 to 13.9) (eTables 8-12 in the Supplement).
Discussion
In this cohort study, we evaluated the comparative safety of the BNT162b2 and mRNA-1273 vaccines in the largest integrated health care system in the US over a relatively long follow-up, which allowed us to quantify the risk of adverse events for which reporting may be delayed or that may emerge over time. The 38-week risks of adverse events were low in both vaccine groups, but recipients of BNT162b2 had an excess risk of ischemic stroke, myocardial infarction, other thromboembolic events, and kidney injury compared with recipients of mRNA-1273. Small-magnitude differences between the 2 vaccines were seen within 42 days of the first dose, and few differences were seen within 14 days of the first dose.
When the safety of vaccines with varying effectiveness is compared,5 an important consideration is whether to target risk that includes adverse events due to SARS-CoV-2 infection in addition to adverse events of the vaccines themselves. If no attempt is made to distinguish adverse events arising from infections vs those due to vaccines, then the less effective vaccine may appear to have more adverse events during a COVID-19 outbreak, even if the 2 vaccines had an identical safety profile in the absence of SARS-CoV-2 infection. In our primary analysis, we censored matched pairs when either member had a documented SARS-CoV-2 infection in an attempt to target the risk of adverse events due to the vaccines themselves. In our secondary analysis, we did not perform this censoring to capture the risk of adverse events due to the vaccines or COVID-19. As expected, the estimated risk difference for pneumonia (a known sequela of SARS-CoV-2 infection) was lower in the primary analysis (5.9 events per 10 000 persons) than it was in the secondary analysis (20.9 events per 10 000 persons), suggesting that our censoring in the primary analysis appropriately excluded most infection-driven events. Incomplete or lagged information on SARS-CoV-2 infection may have led to the inadvertent inclusion of some infection-driven events in our primary analysis. However, incomplete or lagged information on SARS-CoV-2 infection was expected to be nondifferential between the 2 vaccine groups. In addition, the similarity in the risk curves for pneumonia between the vaccine groups suggests that incomplete or lagged information on infection would not entirely explain the divergence of risk curves observed for other outcomes.
Head-to-head comparisons of the safety of the mRNA-based COVID-19 vaccines, as in the present analysis, have been lacking. The safety of these vaccines has been assessed separately in studies that compared (1) the short-term risk of adverse events among vaccine recipients vs unvaccinated persons (eg, a study of approximately 900 000 individuals in each group in Israel3); (2) the risk during an interval shortly after receipt of the vaccine with the risk during a later interval among the same individuals (eg, analyses of surveillance data from Vaccine Safety Datalink4); or (3) the incidence of adverse events after vaccination with the incidence during a period before vaccination (eg, a report from 40 US hospitals9). Findings from these studies have suggested that the risk of adverse events after vaccination with an mRNA-based vaccine was low. Compared with no vaccination, the risk for some events (eg, myocarditis) was higher after vaccination but not as high as the risk after SARS-CoV-2 infection.3 Case reports (ie, without any comparator) have described myocarditis diagnoses within 4 days after receipt of an mRNA-based vaccine, particularly after the second dose.6,7,8 Likewise, findings for thromboembolism and kidney injury after receipt of mRNA-based vaccines have been largely limited to case reports.10,11,16,17,18
Both mRNA-based vaccines encode the prefusion stabilized full-length spike protein of SARS-CoV-2, but each uses a slightly different system for intracellular delivery of the mRNA. They also differ with respect to the total dose of mRNA (30 μg for BNT162b2 vs 100 μg for mRNA-1273) and dosing schedule (3 weeks between first and second doses for BNT162b2 vs 4 weeks for mRNA-1273). However, specific mechanisms underlying any observed difference in safety profiles are not understood.
Strengths and Limitations
This study has several strengths. First, the VA health care databases contain detailed information on demographic characteristics and medical history, which allowed us to carefully characterize and match recipients of each vaccine type according to potential confounders. Second, the large size of the study population allowed us to evaluate the comparative safety of mRNA-based vaccines for a broad panel of potentially serious and rare adverse events. Third, the relatively long follow-up (38 weeks) allowed us to examine the risk of adverse events for which reporting may be delayed or that may emerge over time. Fourth, the demographic composition of the US veteran population allowed us to provide evidence for a diverse cohort in terms of racial and ethnic groups (20% Black, 8% Hispanic) and to conduct subgroup analyses of various age groups (<40, 40-69, ≥70 years) and among Black persons.
This study also has several limitations. First, as in any analysis of nonrandomized data, a key challenge was ensuring that the vaccine groups under study were comparable with respect to risk factors for the outcomes. However, after careful matching, the 2 vaccine groups were similar with respect to demographic characteristics, coexisting conditions, and markers of health care–seeking behavior. Furthermore, much less confounding is expected when unintended effects (ie, adverse events) are evaluated than when intended effects (ie, COVID-19 outcomes) are evaluated and when recipients of different vaccines are compared than when vaccinated and unvaccinated persons are compared.
Second, incomplete outcome ascertainment is possible. To reduce this problem, the analysis captured adverse events recorded in both inpatient and outpatient encounters and was restricted to patients who regularly used VA health care services, had a known residential address, and had routinely collected information (body mass index, smoking status) documented in the database. We expected residual incomplete ascertainment to be nondifferential between the 2 vaccine groups, which may have slightly biased the absolute risks but had few repercussions for the relative effect measures. As with any analysis of electronic health records, a lag between the occurrence of an adverse event and its reporting cannot be ruled out. This potential lag was expected to be nondifferential but may have contributed to a delayed divergence of the cumulative incidence curves between the 2 groups.
Third, the estimated absolute risks and risk differences were dependent on the baseline risk, which may vary across populations. However, we also presented risk ratios, which are less dependent on baseline risk. Fourth, our study population was restricted to individuals with no history of the adverse event of interest in outcome-specific analyses (to evaluate incident events), comprised regular users of the VA health care system (to maximize capture of these events), and was mostly made up of men (93%) and older individuals (90% were >50 years of age), which may limit the generalizability of the findings.
Decisions about vaccination campaigns are complex and must incorporate considerations that extend beyond the scope of the present study. Therefore, our findings need to be combined with those from previous studies to inform future decisions about choice of vaccine in coordinated public health responses. Research has shown that the benefit of any available vaccine (vs no vaccine) on an individual’s risk related to SARS-CoV-2 infection clearly outweighs the risk of rare vaccination-related adverse effects.
Conclusions
This cohort study found few differences in risk of adverse events within 14 days of the first dose of either the BNT162b2 or the mRNA-1273 vaccine and small-magnitude differences within 42 days of the first dose. The 38-week risks of adverse events were low in both vaccine groups, although risks were lower for recipients of the mRNA-1273 vaccine than the recipients of the BNT162b2 vaccine. Although our primary analysis was designed to detect safety events unrelated to SARS-CoV-2 infection, we are unable to exclude the possibility that these differences may be partially explained by a lower effectiveness of the BNT162b2 vaccine in preventing the sequelae of SARS-CoV-2 infection compared with the mRNA-1273 vaccine.
eMethods. Handling Incident SARS-CoV-2 Infections
eFigure 1. Cumulative Incidence of Potential Adverse Events (January 4–October 5, 2021)
eFigure 2. Secondary Analysis Without Censoring Incident SARS-CoV-2 Infections During the Follow-up: Cumulative Incidence Plots
eTable 1. Target Trial Specification and Emulation
eTable 2. Study Variables
eTable 3. Baseline Characteristics of Eligible vs. Matched Vaccinated Persons
eTable 4. Baseline Characteristics of Matched Vaccinated Persons for Each Outcome-Specific Analysis
eTable 5. Secondary Analysis Without Censoring Incident SARS-CoV-2 Infections During the Follow-up: Estimated Risks of Potential Adverse Events
eTable 6. Secondary Analysis: Evaluating 14-Day Risk
eTable 7. Secondary Analysis: Evaluating 42-Day Risk
eTable 8. Subgroup Analysis: Among Persons <40 Years of Age
eTable 9. Subgroup Analysis: Among Persons 40-69 Years of Age
eTable 10. Subgroup Analysis: Among Persons ≥70 Years of Age
eTable 11. Subgroup Analysis: Among Persons of Black Race
eTable 12. Subgroup Analysis: Among Persons of White Race
eReference
References
- 1.Polack FP, Thomas SJ, Kitchin N, et al. ; C4591001 Clinical Trial Group . Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine. N Engl J Med. 2020;383(27):2603-2615. doi: 10.1056/NEJMoa2034577 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Baden LR, El Sahly HM, Essink B, et al. ; COVE Study Group . Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384(5):403-416. doi: 10.1056/NEJMoa2035389 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Barda N, Dagan N, Ben-Shlomo Y, et al. Safety of the BNT162b2 mRNA COVID-19 vaccine in a nationwide setting. N Engl J Med. 2021;385(12):1078-1090. doi: 10.1056/NEJMoa2110475 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Klein NP, Lewis N, Goddard K, et al. Surveillance for adverse events after COVID-19 mRNA vaccination. JAMA. 2021;326(14):1390-1399. doi: 10.1001/jama.2021.15072 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in US veterans. N Engl J Med. 2022;386(2):105-115. doi: 10.1056/NEJMoa2115463 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Montgomery J, Ryan M, Engler R, et al. Myocarditis following immunization with mRNA COVID-19 vaccines in members of the US military. JAMA Cardiol. 2021;6(10):1202-1206. doi: 10.1001/jamacardio.2021.2833 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Marshall M, Ferguson ID, Lewis P, et al. Symptomatic acute myocarditis in 7 adolescents after Pfizer-BioNTech COVID-19 vaccination. Pediatrics. 2021;148(3):e2021052478. doi: 10.1542/peds.2021-052478 [DOI] [PubMed] [Google Scholar]
- 8.Muthukumar A, Narasimhan M, Li QZ, et al. In-depth evaluation of a case of presumed myocarditis after the second dose of COVID-19 mRNA vaccine. Circulation. 2021;144(6):487-498. doi: 10.1161/CIRCULATIONAHA.121.056038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Diaz GA, Parsons GT, Gering SK, Meier AR, Hutchinson IV, Robicsek A. Myocarditis and pericarditis after vaccination for COVID-19. JAMA. 2021;326(12):1210-1212. doi: 10.1001/jama.2021.13443 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Carli G, Nichele I, Ruggeri M, Barra S, Tosetto A. Deep vein thrombosis (DVT) occurring shortly after the second dose of mRNA SARS-CoV-2 vaccine. Intern Emerg Med. 2021;16(3):803-804. doi: 10.1007/s11739-021-02685-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dias L, Soares-Dos-Reis R, Meira J, et al. Cerebral venous thrombosis after BNT162b2 mRNA SARS-CoV-2 vaccine. J Stroke Cerebrovasc Dis. 2021;30(8):105906. doi: 10.1016/j.jstrokecerebrovasdis.2021.105906 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chapin-Bardales J, Gee J, Myers T. Reactogenicity following receipt of mRNA-based COVID-19 vaccines. JAMA. 2021;325(21):2201-2202. doi: 10.1001/jama.2021.5374 [DOI] [PubMed] [Google Scholar]
- 13.Chapman A, Peterson K, Turano A, Box T, Wallace K, Jones M. A natural language processing system for national COVID-19 surveillance in the US Department of Veterans Affairs. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. July 2020. https://www.aclweb.org/anthology/2020.nlpcovid19-acl.10
- 14.Harder VS, Stuart EA, Anthony JC. Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychol Methods. 2010;15(3):234-249. doi: 10.1037/a0019623 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53:457-481. doi: 10.1080/01621459.1958.10501452 [DOI] [Google Scholar]
- 16.Lebedev L, Sapojnikov M, Wechsler A, et al. Minimal change disease following the Pfizer-BioNTech COVID-19 vaccine. Am J Kidney Dis. 2021;78(1):142-145. doi: 10.1053/j.ajkd.2021.03.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.D’Agati VD, Kudose S, Bomback AS, Adamidis A, Tartini A. Minimal change disease and acute kidney injury following the Pfizer-BioNTech COVID-19 vaccine. Kidney Int. 2021;100(2):461-463. doi: 10.1016/j.kint.2021.04.035 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Holzworth A, Couchot P, Cruz-Knight W, Brucculeri M. Minimal change disease following the Moderna mRNA-1273 SARS-CoV-2 vaccine. Kidney Int. 2021;100(2):463-464. doi: 10.1016/j.kint.2021.05.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eMethods. Handling Incident SARS-CoV-2 Infections
eFigure 1. Cumulative Incidence of Potential Adverse Events (January 4–October 5, 2021)
eFigure 2. Secondary Analysis Without Censoring Incident SARS-CoV-2 Infections During the Follow-up: Cumulative Incidence Plots
eTable 1. Target Trial Specification and Emulation
eTable 2. Study Variables
eTable 3. Baseline Characteristics of Eligible vs. Matched Vaccinated Persons
eTable 4. Baseline Characteristics of Matched Vaccinated Persons for Each Outcome-Specific Analysis
eTable 5. Secondary Analysis Without Censoring Incident SARS-CoV-2 Infections During the Follow-up: Estimated Risks of Potential Adverse Events
eTable 6. Secondary Analysis: Evaluating 14-Day Risk
eTable 7. Secondary Analysis: Evaluating 42-Day Risk
eTable 8. Subgroup Analysis: Among Persons <40 Years of Age
eTable 9. Subgroup Analysis: Among Persons 40-69 Years of Age
eTable 10. Subgroup Analysis: Among Persons ≥70 Years of Age
eTable 11. Subgroup Analysis: Among Persons of Black Race
eTable 12. Subgroup Analysis: Among Persons of White Race
eReference