The Importance of Context in Covid-19 Vaccine Safety

Vaccine safety is critical for the successful implementation of any vaccination program, especially during a pandemic. In February 1976, the Centers for Disease Control and Prevention confirmed a cluster of cases of severe influenza-like illness among Army recruits at Fort Dix, New Jersey.¹ A swine influenza A strain that resembled the 1918 pandemic influenza strain was identified,² and a vaccination program was subsequently initiated for the entire U.S. population. After more than 40 million persons were vaccinated, a small excess risk of Guillain–Barré syndrome was noted, with an attributable risk of approximately 1 case per 100,000 doses administered. Given these concerns and because the pandemic did not materialize, the vaccination program was halted in December 1976 so that the issue could be explored further. This experience shed light on the need for real-time vaccine safety surveillance and the importance of context in decision making during a pandemic.

In a study now reported in the Journal by Barda et al., the investigators simultaneously evaluated the risk of adverse events among persons (≥16 years of age) who had received the BNT162b2 vaccine (Pfizer–BioNTech) and the risk of the same events after severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection.³ The authors used data from the largest integrated payer–provider health care organization in Israel, in conjunction with data on SARS-CoV-2 polymerase-chain-reaction tests and data on coronavirus disease 2019 (Covid-19) vaccine administration from the Israeli Ministry of Health.

This use of multiple data sets highlights the importance of investment in digital capabilities and meaningful integration across systems in order to provide real-time answers to key public health questions. The design of rigorous postauthorization vaccine safety studies during the Covid-19 pandemic has been a challenge because the pandemic itself has caused changes in health care utilization, the rollout of Covid-19 vaccines has occurred in phases because of initial supply limitations, and there have been disparities in access to vaccines. Barda et al. broadly addressed many of these challenges by emulating a trial that matched eligible vaccinees to unvaccinated controls according to sociodemographic characteristics, the number of preexisting chronic health conditions, previous health care utilization, and pregnancy status.

In the vaccination analysis, the study included 42 days of follow-up (i.e., 21 days after the first dose and 21 days after the second dose). This analysis accounted for seasonal and secular trends by matching on the day of vaccination, rather than relying on historical risk estimates that may not have been comparable in the pandemic setting. In the SARS-CoV-2 analysis, a similar approach was used to match persons with a newly diagnosed infection to uninfected persons.

Although the risk estimates in the vaccination and the SARS-CoV-2 analyses were not directly comparable because of differences in the populations (i.e., events were evaluated per 100,000 vaccinated persons and per 100,000 infected persons, respectively), these risks were placed in context. The most salient example is myocarditis, which has received much attention recently given the preponderance of reported cases after vaccination among adolescents and young adults and the incidence of myocarditis observed after SARS-CoV-2 infection.^4-6 In the population-based cohort in the study conducted by Barda and colleagues, the risk ratios for myocarditis were 3.24 (95% confidence interval [CI], 1.55 to 12.44) after vaccination and 18.28 (95% CI, 3.95 to 25.12) after SARS-CoV-2 infection, with risk differences of 2.7 events per 100,000 persons (95% CI, 1.0 to 4.6) and 11.0 events per 100,000 persons (95% CI, 5.6 to 15.8), respectively. What is even more compelling about these data is the substantial protective effect of vaccines with respect to adverse events such as acute kidney injury, intracranial hemorrhage, and anemia, probably because infection was prevented. Furthermore, the persons with SARS-CoV-2 infection appeared to be at substantially higher risk for arrhythmia, myocardial infarction, deep-vein thrombosis, pulmonary embolism, pericarditis, intracerebral hemorrhage, and thrombocytopenia than those who received the BNT162b2 vaccine.

National discussions about benefit–risk balance often focus on the benefits of preventing symptomatic disease, hospitalization, or death due to Covid-19 and the risks of serious adverse events after vaccination.^7,8 As specific adverse events such as myocarditis are highlighted, however, the lack of corresponding specificity about benefits can hamper efforts to communicate effectively with patients. Messenger RNA (mRNA) vaccines may be associated with myocarditis, but they can also prevent cases of myocarditis, acute kidney injury, arrhythmia, and thromboembolic disease. The key to comparing these risks depends on the risk of SARS-CoV-2 infection to an individual person, and that risk can vary according to place and over time. Given the current state of the global pandemic, however, the risk of exposure to SARS-CoV-2 appears to be inevitable.

One major limitation of this study is the lack of risk estimates according to age group and sex. For example, thrombosis with thrombocytopenia syndrome occurs predominantly in young adult women who have received adenoviral vector vaccines against SARS-CoV-2, whereas myocarditis predominantly occurs in male teens and young men who have received mRNA vaccines.^5,9,10 Age- and sex-stratified comparisons that reflect local epidemiologic factors might support public understanding of different approaches to vaccine use in different countries, such as Israel, the United Kingdom, and the United States. Other limitations of the study include the paucity of data regarding younger teens and children, the conservative assumption that vaccines have no effect on transmission, and the absence of medical record review to validate computable phenotypes (i.e., algorithms used to identify a cohort on the basis of patient records).

As new knowledge of the safety and benefits of vaccines continues to evolve, studies like this one may help to support decision making about the use of Covid-19 vaccines. The benefit–risk balance should be reassessed, refined, and communicated as the disease burden changes, new variants and safety signals emerge, and vaccine effectiveness begins to wane. Context matters, which means that we as a country need to be ready for continual learning and change.

This editorial was published on August 25, 2021, at NEJM.org.