General determination of causation between Covid-19 vaccines and possible adverse events

1. Introduction

The availability of multiple Covid-19 vaccines following clinical trial results indicating high efficacy in preventing Covid-19 are encouraging for prospects for controlling the Covid-19 pandemic. However, though powered to assess the vaccines’ efficacy and detect common adverse events, these and similar phase three randomized controlled trials cannot identify causal associations for serious adverse events that are very rare or much more likely among populations not included in the clinical trials. As illustrated by reports of a risk for anaphylaxis following vaccination with a Covid-19 mRNA vaccine, particularly among individuals with a prior history of serious allergic reactions [1], potential safety problems may be identified only after widespread use (Table 1 ) [2], [3], [4], [5], [6]. Failure to undertake rapid, credible assessments of such potential safety problems risks missing a true safety concern or, conversely, use of a vaccine being restricted unnecessarily, as well as loss of trust of the public or health professionals, leading to underuse of vaccines and continued spread of the virus.

Table 1.

Pre-Covid-19 Vaccine-Adverse Event Associations Identified Through Post-Licensure Safety Surveillance.

Vaccine	Date of Introduction of General Vaccination	Number Vaccinated Before Problem Identified	Adverse Event	Date When Identified (Time since Introduction)	Additional Risk Attributable to Vaccine	Fate of the Vaccine
1976 Swine Flu	October 1976	40 to 45 million	Guillain Barre Syndrome (GBS)	December 1976 (2 months)	1 case per 100,000 vaccinees	Vaccine withdrawn from market
Rotashield	October 1998	600,000 to 1.2 million	Intussusception	May 1999 (7 months)	1 case per 10,000 vaccinees	Vaccine withdrawn from market
Nasalflu	October 2000	>90,000	Bell’s Palsy	March 2001 (6 months)	13 cases per 10,000 vaccinees	Vaccine withdrawn from market
Pandemrix	October 2009	>5.6 million	Narcolepsy	August 2010 (10 months)	14 cases per 100,000 vaccinees	Vaccine market authorization lapsed

The novelty of, and the intense public interest in Covid-19 vaccines, as well as prior publicly expressed safety concerns add to the urgency and complexity of assessing possible post-licensure safety signals. Furthermore, given the need and plans to rapidly deploy Covid-19 vaccines across all countries, any signal in one country has immediate global implications for decisions about the continued use of the specific Covid-19 vaccine, liability and compensation for any injuries associated with it, and public communications regarding Covid-19 vaccination. Rapid introduction across many different countries also increases the potential for conflicting safety signals. Hence, clear approaches to assessing such signals globally will be essential.

Efforts to decide if a particular vaccine can cause a particular type of adverse event should consider multiple factors, including temporality, size of effect and uncertainty inherent in its measurement, coherence across multiple lines of evidence, and biological plausibility [7]. However, lack of a clear biological mechanism for the vaccine to cause the particular adverse event should not be used to preclude a causal association. Almost by definition, a new and unexpected adverse reaction will not have an immediately obvious mechanism, for example, Guillain-Barre syndrome and the 1976 swine flu vaccine [2]. Such assessments should also be open to new evidence over time, as late reported cases, new diagnostic testing, or other factors may lead to different conclusions from those originally reached.

Confirmation or refutation of causal associations requires that the strength of available data be taken into account [8]. The strongest evidence comes from: high-quality controlled clinical trials; carefully conducted self-controlled case series or case-crossover studies; demonstration of a live vaccine strain virus from a sterile body site (for example, cerebrospinal fluid in post-vaccine viral meningitis); and well-documented appearance of a non-relapsing syndrome or disease after each of two or more doses of a particular vaccine given to the same individual, particularly when documented in multiple individuals from different independent sources [9], [10]. In practice, evidence of laboratory isolation of the vaccine strain or repeated occurrence of a non-relapsing adverse event in the same individual are only rarely available. High-quality randomized controlled trials often offer the strongest evidence but can be logistically or ethically infeasible for assessing rare serious adverse events once a vaccine is in widespread use. The next strongest types of evidence come from cohort studies with either historical or contemporaneous controls and from case-control studies, especially ones which avoid recall and other biases. Statistical techniques such as propensity scores (for cohort studies) can help strengthen the evidence provided by such studies. This is especially so in dealing with rare reactions where ordinary regression methods are vulnerable when there are few outcome events. The weakest evidence comes from uncontrolled observational studies such as ecological studies, uncontrolled case series, data from passive vaccine adverse event surveillance systems, case reports, and editorial articles. These weakest types of evidence are generally useful only for hypothesis generation.

A key element of initial assessments of a signal will often be comparison of adverse event rates between vaccine recipients and a comparison group. Background rate data are critical for rapid assessments of claims of safety issues, as has been observed with multiple vaccines, including human papillomavirus vaccines, influenza vaccines, and measles, mumps, rubella vaccines [11]. However, background rates can vary considerably between populations in different areas due to differences in disease exposure and other factors, so assessments of safety signals should generally rely on background rates for the same population from which the signals arise. Similarly, analyses of adverse events among subpopulations such as pregnant women and the elderly require background rate data reflecting factors such as health care utilization and underlying health conditions of those subpopulations. Ideally, background rates should be from a time when health care utilization was similar as at the time of administration of the vaccine in that population. In comparisons between adverse event rates among vaccine recipients and background rates, a moderate increase, for example, an at-least two-fold increase (with lower 95% confidence limit above 1.5), should generally be considered a relevant increase in risk for objective outcomes such as mortality or myocardial infarctions. For outcomes with less objective definitions, such as some neurological or autoimmune disorders, a larger increase in effect size may be required for credibility.

Given the important differences between various Covid-19 vaccines, each will need separate assessment, though similar vaccines may share adverse effects. However, if a vaccine is confirmed to cause a specific adverse event in one population, the assumption should be that it can cause that adverse event in all populations, unless data of at least equivalent quality show the contrary in a different population. For example, despite evidence of a low-level association of the rotavirus vaccine Rotarix with intussusception in North and South America, no such evidence has been detected in Africa in very strong self-controlled case series [12].

There will be considerable variation in the type and quality of data from different regions. Besides data from randomized controlled trials conducted for licensure or authorization, strong evidence from self-controlled case series and cohort studies is likely to be available from existing population registries and databases on health care utilization, particularly in multiple high-income countries. Important examples are the Vaccine Safety Datalink in the United States and the European Union Pharmacoepidemiology and Pharmacovigilance Research Network’s ACCESS project [13], [14]. Multinational groups such as the Global Vaccine Data Network can help coordinate large studies across such high performing systems [15]. However, in many low- and middle-income countries, the most readily available evidence will be from the weakest category, being sufficient only to generate signals of possible safety problems but probably inadequate to assess whether those signals are real. In such settings, even case-control studies to evaluate signals will often be impractical due to challenges such as relevant medical data on individuals not being collected before or at the time of vaccination or during the course of the adverse event. Even when such medical data are collected, accessing and analyzing them retrospectively can be extremely difficult due to variations in how they are recorded and stored. Limitations in evaluation of safety signals may be particularly problematic for any vaccines that are used in low- and middle-income countries but not in high income countries, or for evaluating safety signals that appear in low- and middle-income countries but not in high income countries for widely used vaccines.

Given the need to assess Covid-19 safety signals rapidly and credibly, it is critical that country governments and vaccine manufacturers share relevant evidence as quickly as possible with the World Health Organization, regulators, and other stakeholders. In addition, reliable systems to collect evidence relevant for assessing such safety signals need to be set up in all relevant populations, including through organization and funding of prospective high quality controlled observational studies in some low- and middle-income countries and use of standard protocols for investigating serious adverse events following immunization with Covid-19 vaccines as described in the World Health Organization Covid-19 Vaccines Safety Surveillance Manual [16]. The availability of such data worldwide will help ensure that potential risks of Covid-19 vaccines are identified quickly and acted on appropriately worldwide to allow equitable access to the benefits from Covid-19 vaccination.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.