“Prepandemic” Immunization for Novel Influenza Viruses, “Swine Flu” Vaccine, Guillain-Barré Syndrome, and the Detection of Rare Severe Adverse Events

Abstract

The availability of immunogenic, licensed H5N1 vaccines and the anticipated development of vaccines against “swine” influenza A(H1N1) have stimulated debate about the possible use of these vaccines for protection of those exposed to potential pandemic influenza viruses and for immunization or “priming” of populations in the so-called “prepandemic” (interpandemic) era. However, the safety of such vaccines is a critical issue in policy development for wide-scale application of vaccines in the interpandemic period. For example, wide-scale interpandemic use of H5N1 vaccines could lead to millions of persons receiving vaccines of uncertain efficacy potentially associated with rare severe adverse events and against a virus that may not cause a pandemic. Here, we first review aspects of the 1976 National Influenza Immunization Programme against “swine flu” and its well-documented association with Guillain-Barré syndrome as a case study illustration of a suspected vaccine-associated severe adverse event in a mass interpandemic immunization setting. This case study is especially timely, given the recent spread of a novel influenza A(H1N1) virus in humans in Mexico and beyond. Following this, we examine available safety data from clinical trials of H5N1 vaccines and briefly discuss how vaccine safety could be monitored in a postmarketing surveillance setting.

The pandemic threats from highly pathogenic avian influenza H5N1 [1] and the recently emergent novel “swine” influenza A(H1N1) virus [2] have stimulated considerable effort in the development of effective pandemic counter-measures. Immunization is widely thought to provide the most effective tool against a pandemic virus, although concerns remain around manufacture and supply of a vaccines in a pandemic situation [1]. Now that immunogenic H5N1 vaccines have been approved by regulatory authorities in many countries and influenza A(H1N1) vaccines are under development, one must consider when and how best to use them. One approach, currently under consideration for H5N1 vaccination, would be to start vaccinating individuals now, in the so-called “prepandemic” (interpandemic) era, because immunized individuals might benefit from “priming” and produce a stronger and faster immune response upon receiving a booster dose matched to the actual pandemic virus as soon as possible after the start of a pandemic [1]. However, both safety and effectiveness remain critical issues in policy development for broad application of such vaccines in the interpandemic period. For example, in the worst case, interpandemic use of H5N1 vaccines could lead to millions of persons receiving vaccines potentially associated with rare but severe adverse events and against a virus subtype that may not cause a pandemic.

Making policy recommendations on interpandemic use of H5N1 or analogous vaccines requires careful scrutiny of safety data. Decisions must consider the multiple vaccine formulations (eg, whole virion, split virion, and subunit) and adjuvants (eg, oil-in-water and alum) that have been studied and also identify surveillance protocols to monitor severe adverse events (SAEs) once the immunization campaign commences. The previous experience of a rare SAE associated with influenza immunization, Guillain-Barré syndrome (GBS), following the 1976 program against swine influenza in the United States, serves as a valuable case study and is especially timely given the recent spread of a novel influenza A(H1N1) in humans in Mexico and beyond (currently designated pandemic H1N1 2009 virus) [2] and ongoing efforts to develop a specific vaccine for human use. GBS is a rare, acute, often postinfectious, immune-mediated disorder of the peripheral nervous system characterized by rapidly evolving, bilateral, ascending motor neuron paralysis, with variable sensory changes [3]. The mortality rate is low with intensive care support, but recovery can be protracted; the mean cost per patient with GBS has been estimated to be approximately $320,000 in the United States [4].

In this commentary, we first describe this case study before reviewing other data on the potential association of GBS with influenza immunization. We then evaluate available safety data from clinical trials of H5N1 vaccines and discuss how safety could be monitored to detect possible novel influenza virus vaccine-associated SAEs such as GBS.

GBS AND INFLUENZA IMMUNIZATION

The 1976 US National Influenza Immunization Program

The National Influenza Immunization Program in the United States involved the immunization of ~45 million persons over a 10-week period against an A(H1N1) influenza virus of swine origin that had initially infected soldiers at Fort Dix, New Jersey. Vaccine field trials were at first conducted with >7000 volunteers [5] before the program was rolled out nationwide. The coverage level was below expectations, and the program was stopped prematurely. Several factors were identified as initial critical blows to the National Influenza Immunization Program, including the public perception that indemnification legislation demanded by vaccine manufacturers indicated an unsafe product; the appearance of legionnaires disease and initial speculation that the government was capitalizing on the threat to stimulate the National Influenza Immunization Program; and the unrelated deaths of 3 elderly patients who received the vaccine in the same clinic [6]. However, the recognition that no additional swine influenza infections were occurring and the appearance of GBS in vaccinees ultimately led to the cessation of the National Influenza Immunization Program. The consensus at the time was that the number of GBS cases was in excess of background incidence, and although background data on GBS incidence were not firm, a political decision was taken to end swine influenza vaccination [7].

The original retrospective study found that the vaccine-attributable risk averaged over all age groups in the 6 weeks after vaccination was ~8.8 cases per million vaccinees, a relative risk of 7.6, with the vaccine-attributable risk in the 18–24-year age group significantly lower [8]. The incidence of GBS in the unvaccinated populations averaged 0.97 cases per million people per month across all age groups. However, subsequent studies indicated that risk estimates for GBS attributable to swine flu vaccination in the 6 weeks after immunization ranged from 4.9 to 11.7 cases per 1 million adult vaccinees [9]. In comparison, the incidence of GBS in the general population has been estimated to be 0.6–4.0 cases per 100,000 per year, with differences by study, sex, and age group [3], so that GBS may be expected to occur at a background rate of 0.07–0.46 cases per 100,000 vaccinees within 6 weeks of any vaccination [10].

GBS and influenza vaccination after 1976

Studies that have analyzed seasonal influenza vaccination after 1976 have failed to demonstrate a consistent causal relation between influenza vaccination and GBS. One US study found a borderline significant increased risk of GBS of ~1 excess case per million adult vaccinees for 2 combined influenza seasons during 1992–1994 [11]. Another study appeared to confirm this association during the period 1991–1999, when influenza vaccine recipients were compared with adult tetanus-diphtheria vaccine control groups [12]. This study used passive surveillance data from the US Vaccine Adverse Event Reporting System (VAERS), a postmarketing reporting system for vaccine adverse events initiated in 1990. Data from VAERS have some limitations; in particular, they do not permit accurate calculation of population-based incidences of adverse events, making conclusions difficult to draw [13]. An additional study using VAERS data suggested that reporting rates of GBS after influenza vaccination decreased from 1990 to 2003, despite overall reporting of adverse events related to influenza vaccination increasing during this period [14]. Most recently, a study of 15 years of VAERS data demonstrated that reporting of adverse events associated with trivalent inactivated seasonal influenza vaccines has remained reasonably constant, with GBS the most frequently reported SAE, at 0.70 reports per million vaccinations in adults. GBS was reported nearly twice as often among persons aged 50–64 years than among either elderly persons (age, ≥65 years) or younger adults (age, 18–49 years), although the study does not comment on how this compares to the background incidence [15]. A Canadian study suggested that influenza vaccination is associated with a “small but significantly increased risk for hospitalization because of GBS” [16, p. 2217]. However, at the population level, this study found no significant increase in hospital admissions due to GBS after the introduction of universal influenza immunization in 2000.

Multiple other studies have failed to demonstrate a significant link between GBS and influenza vaccination [17-21]. In addition to the lack of vaccine-associated risk of GBS found in a study of the 1978–1981 influenza vaccine season using VAERS surveillance data [17], a retrospective study of ~5.6 million US Army influenza vaccine recipients during the period 1980–1988 detected no temporally related increase in GBS following the mass vaccination program [18], although the rigor of this study has been questioned [22]. A more recent study using the UK General Practice Research Database over the period 1990–2005 also found no increased GBS risk within 90 days of receiving seasonal influenza vaccination [19]. Some studies have even demonstrated a potential decreased risk of acquiring GBS after influenza vaccination [17, 20]. A link between GBS and live, intranasal influenza vaccine formulation has also not been demonstrated since licensure in the United States in 2003 [21].

Thus, the link between influenza vaccines and GBS remains inconclusive, in part because of the difficulties in establishing definitive background rates of GBS in the general population. Furthermore, several studies have demonstrated a link between GBS and influenza infection itself [19, 23]. This would suggest that any risk of GBS that might result from vaccination may be more than offset by a risk of GBS following natural virus infection.

Possible mechanisms of GBS association with influenza vaccine

The pathogenesis of GBS appears to vary by subtype of the disorder [3]. Most cases of GBS are sporadic, and the cause in most remains unidentified [23]. However, a number of antecedent events to GBS have been described, ranging from infections, most commonly Campylobacter jejuni gastroenteritis or cytomegalovirus infection, to being struck by lightning. Some subsets of patients with GBS develop anti-ganglioside antibodies that are implicated in the pathogenesis of the disease [3]. Antibodies to a number of different complex gangliosides can be detected in patients with GBS and may arise as a consequence of molecular mimicry with antigens on infecting pathogens. Ganglioside GM1, a monosialylated glycosphingolipid, is one of several gangliosides considered a target antigen in the pathogenesis of GBS [9]. C. jejuni expresses ganglioside-like structures that can induce anti-ganglioside antibodies, such as anti-GM1. Whether host factors are important is uncertain, and there does not appear to be any particular Human Leukocyte Antigen associations with GBS [24].

The risk of GBS may depend on vaccine formulation, which could explain why the 1976 swine flu vaccine was associated to a greater degree with GBS. Several hypotheses have been put forward to explain such an association. One is that the biological mechanism causing GBS following influenza vaccination may involve the synergistic effects of endotoxin and vaccine-induced autoimmunity, with the concentration of the endotoxin contaminant in vaccine formulations influencing risk of GBS [12]. Salmonella contamination of the embryonated chicken eggs used to produce influenza vaccines might influence the concentration of endotoxin [12]. One study hypothesized that C. jejuni contamination of vaccines may be associated with increased incidence of autoantibody formation, although no C. jejuni contamination was found in the vaccine preparations by direct polymerase chain reaction testing or by measurement of anti-C. jejuni antibodies in immunized mice [9]. However, this study found that mice immunized with influenza vaccines from 1976, 1991–1992, and 2004–2005 did make anti-GM1 antibodies concurrently with anti-hemagglutinin (HA) antibodies. The authors hypothesized that influenza HA may be involved in eliciting anti-GM1 antibodies in mice, and that the differential risk of GBS in these vaccinees might be explained by viral neuraminidase levels. Under this theory, the 1976 swine flu vaccine might have allowed formation of sialic acid–HA complexes that mimic GM1 ganglioside in susceptible hosts due to low levels of viral neuraminidase in the vaccine preparation [9]. No direct evidence supporting this hypothesis was provided in the report. Two recombinant HA proteins derived from the H5N1 viruses A/HK/156/97 and A/Vietnam/1203/04 also induced anti-GM1 antibodies in mice [9].

Studies examining the possible induction of such anti-ganglioside antibodies in humans after receipt of influenza vaccine have not been reported, to our knowledge. However, testing of anti-ganglioside antibody titer increases between acute- and convalescent-phase serum samples by commercially available assays [3] in recipients of both novel and seasonal influenza vaccines might prove informative.

Other adverse events associated with influenza immunization

A recent study of VAERS data examining adverse events following seasonal trivalent inactivated influenza vaccines found an overall adverse event reporting rate of 24.4 cases per 1 million vaccinations over a 15-year period; 14% of the events were classified as serious [15]. Although GBS was the most common SAE reported, paresthesia was the most reported neurological event (rate of 1.80 cases per 1 million vaccinations). Other neurological events included myelitis (0.12 cases per 1 million vaccinations), ataxia (0.16 cases per 1 million vaccinations), and optic neuritis (0.04 cases per 1 million vaccinations). However, this report examines coding terms reported to VAERS, some of which are symptoms, rather than SAEs.

Other adverse events that have been investigated for an association with influenza vaccines include Bell palsy; demyelinating conditions, such as optic neuritis and incident multiple sclerosis; and oculorespiratory syndrome. In addition, as with all vaccines, immediate hypersensitivity reactions, such as acute anaphylaxis, may also rarely be associated with influenza vaccines [15]. However, GBS remains the SAE scrutinized most closely for a causative relationship with influenza vaccines.

Having considered the precedent of GBS association with influenza vaccination, any future use of H5N1 or another novel influenza virus vaccine will require rigorous surveillance to detect whether the appearance of an SAE is vaccine attributable. Different types of surveillance systems can be used to assess vaccine safety; these may monitor different populations or different types of events at different times, both before and after marketing.

PREMARKETING MONITORING OF H5N1 VACCINE SAFETY

It is possible to use data on individuals who have received vaccines to determine the frequency of SAEs potentially attributable to the vaccine that can be reasonably excluded. In this way, the likelihood of detecting high-frequency SAEs in studies of H5N1 or other vaccines can be estimated from the existing clinical trial data.

As of April 2009 (table 1), ~18,784 subjects had received H5N1 vaccines in clinical trials, with no SAEs related to immunization recorded [25]. The upper bound (ie, the most conservative estimate) of the risk rate of a SAE in vaccinated individuals that can be excluded is 1 of 6270 for type 1 error α = 5% (see Appendix A for technical details). This upper bound can also be computed using recipient subsets for inactivated whole virion (upper bound, 1 of 1095), inactivated split virion (1 of 3206), and inactivated subunit (1 of 1970) vaccines (table 1). Although these data suggest that we can safely exclude a very frequent association between H5N1 vaccination and SAEs, they are not informative in determining the potential occurrence of rarer SAEs (eg, of a few cases per 100,000 individuals).

Table 1.

Summary of Study Populations and Severe Adverse Events (SAEs) Observed in H5N1 Clinical Trials

Type of vaccine	Total study population				SAEs
	Children	Adults	Elderly persons	Total	Observed number	Upper bound for the risk of SAE
Inactivated whole virion	12	3077	190	3279	0	1 of 1095
Inactivated split virion	370	9038	196	9604	0	1 of 3206
Inactivated subunit	0	5728	173	5901	0	1 of 1970
Total	382	17,843	559	18,784	0	1 of 6270

NOTE. Data are based on information presented to the World Health Organization SAGE Group, April 2009 [25]. The upper bound for the risk of SAE is for type 1 error α = 5% (Appendix A).

Alternatively, it is possible to determine the number of SAE-free vaccinated subjects in a clinical trial required to reject the existence of a certain event risk at a range of different type 1 error probabilities (Appendix A). For example, for type 1 error α = 5%, ~300,000 SAE-free vaccinated subjects are required to exclude the risk level of 1 of 100,000 (table 2).

Table 2.

Number of Severe Adverse Event–Free Vaccinated Subjects Required in a Clinical Trial to Reject the Existence of a Certain Event Risk at a Range of Different Type 1 Error Probabilities

Level of risk to be rejected	Type 1 error α
	10%	5%	1%
1 of 100	2.30 × 102	3.00 × 102	4.61 × 102
1 of 1000	2.30 × 103	3.00 × 103	4.61 × 103
1 of 10,000	2.30 × 104	3.00 × 104	4.61 × 104
1 of 100,000	2.30 × 105	3.00 × 105	4.61 × 105
1 of 1,000,000	2.30 × 106	3.00 × 106	4.61 × 106
1 of 10,000,000	2.30× 107	3.00× 107	4.61× 107

NOTE. See Appendix A for technical details.

These approaches rely on a sample of individuals without the incidence of any SAEs recorded. However, SAEs observed in vaccinated subjects of clinical trials are not necessarily attributable to vaccination, because such events can also be observed in unvaccinated subjects. Table 3 summarizes the number of subjects required to demonstrate an increased risk of SAEs in vaccinees by factors from 1.5-fold to 100-fold relative to a range of background rates in the general population. For example, if the background rate is 1 in 100,000; 53,500 and 1,238,000 vaccinated subjects are needed to demonstrate an increase in the SAEs risk in vaccinees by 10-fold and 2-fold, respectively. In general, the number of subjects required increases logarithmically in proportion to 10-fold decreases in the background frequency of SAEs.

Table 3.

Number of Subjects Required to Test the Existence of an Increased Level of Risk Relative to Background Rate, with Type 1 Error α = 5% and Power of 90%

Background SAE rate in the general population	SAE rate in vaccinated populations
	×1.5	×2	×5	×10	×100
1 of 10,000	453,000	141,000	16,000	5500	500
1 of 100,000	4,530,000	1,238,000	160,000	53,500	2500
1 of 1,000,000	43,128,500	12,951,500	1,599,000	532,500	23,500
1 of 10,000,000	438,525,500	123,781,500	15,987,500	5,322,500	230,500
1 of 100,000,000	4,312,838,000	1,237,814,000	159,872,000	53,223,500	2,303,000

NOTE. See Appendix A for technical details. SAE, serious adverse event.

To illustrate this approach for a novel vaccine, we can assume that the risk of GBS associated with the vaccine is similar to that associated with the 1976 swine flu vaccine (risk of ~8.8 cases per 1 million and a GBS background rate of 0.7–4.6 cases per 1 million within 6 weeks after vaccination) [8]. Under these conditions, an indicative vaccinated population size of 409,000 to 970,000 would be required to demonstrate that the adverse event rate in vaccinated individuals is greater than the background rate (type 1 error α = 5%; power = 90%).

Even in the unlikely case that these sample size requirements could be satisfied using data from clinical trials and the SAEs rate in vaccinees was found to be significantly higher than the background rate, the strength of evidence would remain weak in the absence of randomization. Close monitoring of vaccine safety will be an important consideration if and when H5N1 or other novel virus vaccination begins to be used in large populations.

POSTMARKETING MONITORING OF VACCINE SAFETY

Postlicensure monitoring is essential to assess the safety of vaccines in populations much larger than those observed in clinical trials. Most postmarketing monitoring of vaccine safety analyzes information reported to passive surveillance systems, such as VAERS in the United States. Passive surveillance systems are, however, subject to delays between reporting of initial adverse events and the requirement for subsequent studies to evaluate whether observed SAEs are likely to have been vaccine related or not. Such systems are also prone to underreporting and reporting bias, and they do not provide insight into background rates of disease in unvaccinated populations [26]. Two alternative postmarketing protocols are briefly introduced below.

Real-time vaccine safety monitoring

This approach can be illustrated by maximized sequential probability ratio testing of data from surveillance studies, such as the Vaccine Safety Datalink project in the United States, which has been used to analyze an association between meningococcal conjugate vaccine and GBS [26]. The approach relies on an automated system to gather data on a weekly basis from various health management organizations in the United States on (1) exposure to vaccination, (2) preventive visit exposure, and (3) outcome of interest (eg, GBS). This gives a prospective cohort design to evaluate the link between vaccination and GBS in which preventive visits act as controls. A particularly interesting and important feature of the system is that vaccine safety assessment can be updated weekly as data become available, and there is no need to wait the end of the study period.

Evaluation of vaccine safety using case-only methods

A range of methods have been described using data on “cases” of adverse events to determine potential causality with vaccination without the need for “non-cases” or independent control groups [27]. These approaches often rely on the notion that cases act as their own controls, although the perspective can also be ecological (ie, undertaken at group level). For example, by comparing the number of GBS cases before and during a mass measles vaccination campaign and controlling for changes in vaccine coverage, one study found no association between measles vaccination and GBS [28]. Such analyses may prove attractive, because data from readily available databases, such as hospital admission records, can be analyzed much more rapidly than data obtained by a traditional cohort or case-controlled approach. However, because initial ascertainment of cases must be independent of vaccination status, data from postvaccination surveillance systems, such as VAERS or Vaccine Safety Datalink, cannot be used for case-only analyses.

COMMENTS

When considering policy concerning the widespread use of novel influenza vaccines in the interpandemic setting, vaccine safety is paramount. Although the risk of any SAE should be assessed, the possible relationship between GBS and influenza vaccination is the primary and most enduring association among epidemiologists and policymakers. Therefore, we used GBS as a case study of an SAE associated with influenza immunization. Although it is difficult to establish the definitive risk association of vaccination-related GBS, the 1976 National Influenza Immunization Program provides a useful precedent for considering policy around interpandemic novel influenza vaccination, given that both situations consider mass vaccination against an unknown threat. A report on the National Influenza Immunization Program was commissioned and prepared in 1978 to reveal lessons that could be applied to subsequent influenza threats. More recently, one of its authors has revisited the report to suggest seven lessons for avian influenza preparedness [29]. The author urges policymakers to “beware of overconfidence in models drawn from meagre evidence” (p. S16); “invest in a balanced portfolio of research and contemporary preparedness” (p. S16)—including new vaccine technologies; and “refrain from overstatement of objectives and misrepresentation of risk” (p S17). It is hoped that policymakers will be able to use the precedent of swine flu to develop an effective response to the unknown but potentially catastrophic threat of an influenza pandemic.

To date, H5N1 vaccine formulations have been administered to ~19,000 subjects, with no severe adverse events related to immunization. A possible risk of GBS or other SAEs from receipt of H5N1 vaccine cannot be assessed adequately because of the small numbers of vaccines and multiplicity of vaccine formulations and adjuvants used. Measurement of anti-ganglioside antibody responses in vaccine recipients would be of interest but has not yet been proven to predict the likelihood of GBS [3]. Any interpandemic vaccination policy will require a robust surveillance system to monitor occurrence of rare SAEs in populations receiving an H5N1 or other novel vaccine, such as a vaccine for influenza A(H1N1) virus.

A lack of definitive data on background rates of SAEs in the general population limits analysis of SAE data derived from established surveillance mechanisms such as VAERS. A greater knowledge of background data on immune-mediated and other SAEs for potential target populations would allow for more robust analysis. Background rates of diseases, such as Bell palsy and various autoimmune disorders, have recently been determined in adolescent and young women in to anticipate surveillance of SAEs in the context of human papillomavirus vaccination [30]. In addition, the recent publication of 15 years of VAERS data on seasonal inactivated influenza vaccine provides a useful resource to compare the rates of adverse events associated with novel influenza virus vaccines [15]. Monitoring of potential SAEs during wider-scale use of H5N1 vaccines or a new H1N1 virus vaccine in populations in developing countries is especially difficult, in part due to the lack of background data on illnesses such as GBS. In children, the worldwide incidence of acute flaccid paralysis (AFP) not attributable to poliomyelitis can provide an estimate of the background incidence of GBS for children aged <15 years, since GBS is a major cause of AFP. These AFP data are routinely collected by the World Health Organization in the context of global polio surveillance [31]. In one study in Latin America which analyzed AFP data, the average annual incidence of GBS was calculated at 0.62 cases per 100,000 in children aged 9 months to 15 years [28].

In summary, the risk of SAEs will remain important considerations in developing immunization policies for interpandemic use of novel influenza vaccines and implementing mass immunization programs. These issues are particularly challenging when the risks of severe illness or of a future pandemic are uncertain and, therefore, safety concerns more acute, as is the case in interpandemic vaccination. However, even if an association between SAEs and interpandemic vaccination could be discounted with confidence, the reality is that public perception of a link has the potential to undermine a mass vaccination strategy over and above the scientific evidence, as clearly evidenced in the 1976 experience.

APPENDIX A

BRIEF NOTE ON STATISTICAL PROCESSES

NOTATIONS AND THEORY

• N: number of subjects.

• n: number of adverse events.

• λ: risk of an adverse event.

The number of severe adverse events n follows a Binomial distribution:n~ Bin(N, λ). The distribution can be approximated by a Poisson distribution, because λ is small and N is large: n~Pois(N λ).

The probability to observe n = 0 adverse events out of N is therefore P{n = 0} = exp(−N λ).

UPPER BOUND FOR THE RISK OF ADVERSE EVENTS ESTIMATED FROM A SAMPLE OF SAE-FREE VACCINATED INDIVIDUALS

The idea is to reject levels of risk which are “inconsistent” with the observation that there are n = 0 adverse event out of N subjects. “Inconsistent” levels of risk are those for which the occurrence of n = 0 adverse event out of N subjects would be probabilistically unlikely, with a probability smaller than α (type 1 error; in general, α = 5%).

So, for a specified type 1 error α, the upper bound λ_α for the risk of adverse events is given by the equation

$$\alpha =P\{n=0\}=\mathit{exp}(-N{\lambda }_{\alpha }).$$

The solution is

$${\lambda }_{\alpha }=-\frac{\mathit{log}\left(\alpha \right)}{N}.$$

In practice, it means that for a specified type 1 error α, “0 adverse event out of N subjects” is consistent with risks smaller or equal to λ_a = −log(α)/N.

NUMBER OF SAE-FREE VACCINATED INDIVIDUALS NEEDED TO REJECT A SPECIFIC LEVEL OF RISK

For type 1 error α, the same principles can be used to determine the number of SAE-free vaccinated subjects that are needed to reject a level of risk λ*. One needs to have at least the following number of subjects:

$$N^{\ast }=-\frac{\mathit{log}\left(\alpha \right)}{{\lambda }^{\ast }}.$$

In practice, it means that we need to find 0 adverse events out of N* = − log(α)/λ* subjects to conclude that the level of adverse event is smaller or equal to λ* with type 1 error α.

SAMPLE SIZE CALCULATION TO DETERMINE WHETHER THE SAE RATE AMONG VACCINATED INDIVIDUALS IS LARGER THAN THE SAE BACKGROUND RATE FROM A SAMPLE OF VACCINATED INDIVIDUALS

Assume that the SAE background rate p₀ is known and denote p₁ the SAE rate in the vaccinated population. Here, we determine the number n of vaccinated individuals needed to detect a statistically significant positive difference ∈ between the rates (∈ = p₁ − p₀ > 0), with type 1 error α ≤ 5% and type 2 error β ≤ 10% (power is 1 − β ≥ 90%). This is implemented via a one-sided test of the null hypothesis H0: “the SAE rate p₁ in vaccinated individuals is equal to the SAE background rate p₀” against the alternative hypothesis H1: “p₁ is larger than p₀.”

If the type of events under study is common (that is, if p₀>>0 and p₁>>0), standard formulas on sample size can be found in the literature on clinical trials. Those formulas rely on a Normal approximation. Under this approximation, for type 1 error α, and type 2 error β, sample size n is n = (z_α + z_β)².p₁(1 − p₁)/(p₁ − p₀)² where z_u is the upper u-th quantile of the standard normal distribution. For α = 5% and β = 10%, z_α = 1.64 and z_β = 1.28 [32].

However, the normal approximation breaks down for p₀ and p₁ close to 0. We have, therefore, determined sample sizes on the basis of exact methods (that is, that does not rely on the Normal approximation), still controlling for type 1 error α ≤ 5% and type 2 error β ≤ 10% [32].

Using notations defined earlier in the Appendix, the number n of SAE in a vaccinated population of size N is Poisson distributed n~Pois(Nλ), and we want to test the null hypothesis H0: λ = p₀ against H1: λ > p₀. We denote P_Nλ(n < x) the probability that a number n drawn from Pois(Nλ) is smaller than x.

Under the null hypothesis H0: λ= p₀, for a sample size N, we can determine the smaller number n_N that satisfies the condition PNp0(n > n_N) ≤ 5% (ie, under H0 and for a sample size N, there is a probability smaller than 5% that the number of SAE n is larger than n_N). We will therefore reject H0 with a type 1 error α ≤ 5% if the number of adverse events satisfies n > n_N.

Under the alternative hypothesis λ = p₁, for a sample size N, we can compute the probability β_N that hypothesis H0 is not rejected β_N = P_Np1(n ≤ n_N). β_N is the type 2 error.

The sample size is the smallest value of N such that β_N ≤ 10%. Note that this procedure ensures that type 1 error α ≤ 5% and type 2 error β ≤ 10%. Results were unchanged when the number of SAE is modelled by a Binomial distribution rather than by a Poisson distribution. Example: we need a sample size N = 532,500 to detect a difference between a SAE background rate p₀ = 1,000,000 and a rate in vaccinated p₁ = 100,000.

For N = 532,500 and under the null hypothesis H0: λ = p₀ = 1,000,000, there is 1.7% chance that n>2 (but 10.0% chance that n>1). So, we will reject H0 if n > 2 with type 1 error α = 1.7% (<5%).

ForN = 532,500 and under the null hypothesis H1: λ = p₁ = 100,000, the probability that n ≤ 2 (that is, H0 is not rejected) is 9.98%. This is the type 2 error β which is smaller than 10%.