Importance of background rates of disease in assessment of vaccine safety during mass immunisation with pandemic H1N1 influenza vaccines

Abstract

Because of the advent of a new influenza A H1N1 strain, many countries have begun mass immunisation programmes. Awareness of the background rates of possible adverse events will be a crucial part of assessment of possible vaccine safety concerns and will help to separate legitimate safety concerns from events that are temporally associated with but not caused by vaccination. We identified background rates of selected medical events for several countries. Rates of disease events varied by age, sex, method of ascertainment, and geography. Highly visible health conditions, such as Guillain-Barré syndrome, spontaneous abortion, or even death, will occur in coincident temporal association with novel influenza vaccination. On the basis of the reviewed data, if a cohort of 10 million individuals was vaccinated in the UK, 21·5 cases of Guillain-Barré syndrome and 5·75 cases of sudden death would be expected to occur within 6 weeks of vaccination as coincident background cases. In female vaccinees in the USA, 86·3 cases of optic neuritis per 10 million population would be expected within 6 weeks of vaccination. 397 per 1 million vaccinated pregnant women would be predicted to have a spontaneous abortion within 1 day of vaccination.

Introduction

On June 11, 2009, WHO raised the worldwide pandemic influenza alert to its highest level in response to the global spread of a novel influenza A H1N1 virus (pandemic H1N1). The rapid progression of this outbreak and broad population susceptibility make it likely that a substantial part of the global population will be affected over the next 2 years.

Mass vaccination campaigns against this pandemic H1N1 strain are now being considered or have begun in many countries. Different vaccine products, some containing a novel adjuvant, are likely to be available for use from several manufacturers in late 2009. Such mass campaigns will be unprecedented on a global scale and pose many challenges to assessment of vaccine safety. New questions about vaccine safety will undoubtedly arise. Public health officials will need to monitor for previously unrecognised serious adverse events that might be related to the new vaccines. Additionally, the public will need frequent reassurance of vaccine safety when events that are temporally associated with vaccination are identified, even when these events have other causes and occur at the expected background rate. Although scientists know that a temporal association between vaccine exposure and a subsequent adverse event does not prove that the vaccine caused the event, identification of such temporally associated events could nonetheless raise public concern. Adverse event reporting systems such as the Vaccine Adverse Event Reporting System (VAERS) in the USA or the yellow card system in the UK are designed to detect signals of concern but not assess or prove causality.¹

Unfortunately, the availability of the internet together with an increased public concern and engagement in interpretation of vaccine adverse event data have increasingly allowed for spurious associations to be promoted as fact. Widespread beliefs that such false associations are true can and do disrupt immunisation programmes, often to the detriment of public health. For example, when an association between the measles, mumps, and rubella (MMR) vaccine and risk of autism was made,² it had a negative effect on public uptake of measles prevention programmes in the UK and elsewhere, with a consequent rise in morbidity and mortality due to measles.³ In Nigeria, opposition of religious chiefs to oral polio immunisation campaigns because of unfounded concerns resulted in government officials refusing to allow such campaigns and a substantial rise in polio cases.⁴ In Austria shortly after introduction of the human papillomavirus vaccine, there were calls for withdrawal of the vaccine because of the death of one teenage athlete.⁵ However, data on background rates of sudden death in adolescents were available and the vaccine programme continued.

Another relevant example is the interruption of the 2006 seasonal influenza vaccine campaign in Israel after four deaths occurred within 24 h of immunisation. These four patients were at high risk of sudden death because of their age and underlying disorders, and their clinical presentation was consistent with a cardiac cause of death. Post-event analyses suggested that death occurs at a rate greater than 1 per 1000 per week in this population of patients, such that 20 coincidental deaths might be expected by chance alone within 24 h of immunisation. Nevertheless, this clustering of deaths in time and location resulted in global news coverage and interference with the vaccination campaign.⁶ Factors that contributed to the quick acceptance of several of these false assumptions of causal association were a lack of understanding of how passive reports of adverse events can be interpreted^1,7 and poor understanding of the scientific methods used to assess causality.⁸

Inappropriate assessment of vaccine safety data could severely undermine the effectiveness of mass campaigns against pandemic H1N1 2009 influenza. Guillain-Barré syndrome is a good example to consider. Since the 1976–77 swine influenza vaccination campaign was associated with an increased number of cases of Guillain-Barré syndrome, assessment of such cases after vaccination will be a high priority. Therefore, it is important to know the background rates of this syndrome and how this rate might vary with regard to population demographics. The background rate of the syndrome in the USA is about 1–2 cases per 1 million person-months of observation.⁹ During a pandemic H1N1 vaccine campaign in the USA, 100 million individuals could be vaccinated. For a 6-week follow-up period for each dose, this corresponds to 150 million person-months of observation time during which a predicted 200 or more new cases of Guillain-Barré syndrome would occur as background coincident cases. The reporting of even a fraction of such a large number of cases as adverse events after immunisation, with attendant media coverage, would probably give rise to intense public concern, even though the occurrence of such cases was completely predictable and would have happened in the absence of a mass campaign.

Since recent experience with mass immunisation campaigns in developed countries is limited, we have undertaken a review of considerations that we believe should be made in assessment of vaccine safety during pandemic H1N1 vaccine use. This report is not designed to be a comprehensive review or meta-analysis; rather, we provide examples of background rates of selected events that are likely to raise concerns for pandemic vaccination programmes. We report geographical variations of the incidence of such events to inform discussion of inevitable temporal associations between vaccination and adverse health outcomes in the event of a mass vaccination campaign.

Data collection and analysis

A list of possible outcomes that might need to be assessed after receipt of a pandemic H1N1 influenza vaccine was developed by the US Food and Drug Administration (unpublished). We selected disease events from this list on the basis of availability of several published studies that could provide background rates and our ability to develop background rates from automated data sources. Input on the contents of our list was provided by the US Centers for Disease Control and Prevention, FDA, and the UK Health Protection Agency. The selected events and background rates included in this report are not comprehensive but rather are meant to serve as examples to emphasise the importance of being able to develop and access such rates.

For the literature review, authors were each assigned specific disease outcomes to review. The diagnosis specified was entered as a key word in PubMed and articles that included incidence rates were identified. We gave priority to recent articles and those that included sex-specific and age-specific incidence rates. In addition to the literature review, we developed incidence rates for several outcomes to show the difference between such rates and rates from registries or other studies in which case validation through chart review had been done. We searched hospital admission databases in the UK, Finland, and the USA for a defined population to calculate incidence rates of selected disease outcomes. Table 1 shows the International Classification of Diseases (ICD)-9 (USA, Canada, and UK) and ICD-10 codes (Finland) used to identify disease outcomes. For Bell’s palsy in the USA, both outpatient and hospital databases at Kaiser Permanente Northern California (CA, USA) were searched with the specified ICD-9 code for Bell’s palsy, whereas in the UK, READ and Oxford Medical Information Systems (OXMIS) coding 350FA, F310.00, 350F, 350FN, and 350R were used. Chart review was not done for these assessments, and cases identified through these data searches were accepted without further review as might happen in a rapid real-life situation. Where necessary, a 2-year washout period was used to identify individuals who had a condition before the start of the observation period to separate incident events from repeat episodes of the same diagnosis. Finally, we used background rates of selected events at selected sites to calculate rates of events likely to occur within 1 day, 1 week, and 6 weeks after receipt of a hypothetical dose of vaccine by multiplying the number of hypothetically vaccinated people by the background rates and the risk window.

Table 1.

ICD-9 and ICD-10 codes used to identify disease outccomes

	ICD-9 code	ICD-10 code
Acute infectious and postinfectious polyneuritis*	357.0	G61.0
Acute transverse myelitis	341.2	G36.0, G37.0, G37.8, G37.9
Optic neuritis	377.3	H46
Bell’s palsy	351	G51.0
Anaphylaxis	995.0	T78.2
Seizure	780.39	R56.0, R56.8
Multiple sclerosis	340	G35
Spontaneous abortion	634	003.1–9
Autoimmune thrombocytopenia	287.31	D69.4
Sudden death <24 h after onset of symptoms, cause unknown	798.2	R96.1
Any death	798	R95, R96.0, R96.1, R98
Preterm labour	644.21	060

ICD=I international Classification of Diseases.

^*Includes Guillain-Barré syndrome.

Background incidence of disease

Table 2 shows background incidence of selected disease events in several countries, with age-specific incidence reported when available. Incidence of Guillain-Barré syndrome and related syndromes was highest in older individuals and in people in Finland, and was in some places about two times higher in men than in women. In Finland, incidence varied from 0·18 per 100 000 person-years in boys aged 0–17 years to 10·13 per 100 000 person-years in men aged 65 years or older. In the UK, incidence was higher in older individuals but substantially lower than it was in Finland, with 4·57 cases per 100 000 person-years in men aged 65 years or older. Incidence of the syndrome in the USA was similar to that in the UK. Brazil had the lowest incidence of the disease. The data from the USA, UK, and Finland are from computerised outcomes databases, whereas data from Brazil are from a disease registry in which case validation criteria might be more stringent. Hospital accessibility might also be lower in Brazil.

Table 2.

Incidence of disease events that might be temporally associated with vaccine, by country and age group

	Incidence (per 100 000 person-years)
Acute infectious and postinfectious polyneuritis*
Brazil10
0–15 years	0.46
0–4 years	0·56
5–9 years	0.47
10–15 years	0·37
Finland†
0–17 years	Female 1·68, male 0·18
18–44 years	Female 1·24, male 3·02
45–64 years	Female 3·95, male 7·15
≥65 years	Female 6.18, male 10·13
UK‡
0–17 years	Female 0.79, male 0·70
18–44 years	Female 1·57, male 1·63
45–65 years	Female 2·07, male 2·50
>65 years	Female 2·52, male 4·57
USA11
10–17 years	Female 1·8, male 2·1
18–25 years	Female 0·4, male 0·8
26–62 years	Female 2·3, male 3·3
New-onset multiple sclerosis
Canada, Alberta12
All ages	19·6 to 25·0§
Canada, Nova Scotia13
All ages	10·81
Optic neuritis
Finland†
0–17 years	Female 0·37, male 0·36
18–44 years	Female 6·76, male 1·83
45–64 years	Female 4·08, male 0·93
≥65 years	Female 1·35, male 1·13
Singapore14
All ages	0·83
Sweden15
0–9 years	0
10–19 years	Female 6·0, male 0·5
20–39 years	Female 17·5, male 5·5
40–59 years	Female 10·8, male 1·5
USA16
All ages	Female 7·5, male 2·6
Acute transverse myelitis
Australia17
<15 years	0·32
Canada18
<18 years	0·2
Finland†
0–17 years	7·27
18–44 years	4·06
45–64 years	5·39
≥65 years	9·04
Israel19
0–9 years	0·40
10–19 years	1·93
20–29 years	1·42
30–39 years	0·89
40–49 years	1·51
50–59 years	1·97
60–69 years	1·77
≥70 years	3·0
USA20
All ages	4·6
Bell’s palsy
UK‡
0–17 years	11·98
18–44 years	28·92
45–65 years	36·28
≥65 years	44·91
USA¶
0–17 years	24
Anaphylaxis‖
Australia21
All ages	0·02–2·6**
Finland†
0–17 years	4·47
18–44 years	1·69
45–64 years	2·50
≥65 years	2·69
USA22
0–18 years	0·65††
Seizure
Finland†
0–17 years	106·61
18–44 years	23·44
45–64 years	39.72
≥65 years	54·64
France23
>18 years	71·3
Sweden24
>18 years	76
Switzerland23
>18 years	70·8
Switzerland24
0–4 years	460
USA11
>18 years	100
Autoimmune thrombocytopenia
Finland†
0–17 years	0·19
18–44 years	0·23
45–64 years	0
≥65 years	0·38
Nordic countries25
<15 years	4·8
UK26
<18 years	Female 37, male 47
18–64 years	Female 3·8, male 2·0
>65 years	Female 7·1, male 7·8
USA11
10–17 years	Female 1·5, male 0·6
18–25 years	Female 3·3, male 1·2
26–62 years	Female 3·1, male 34
Sudden death (within 1 h of onset of symptoms)
Italy27
12–35 years	0·06
UK28
16–64 years	0·5
USA29
All ages	Female 54, male 44

Data from publications and medical databases.

^*Includes Guillain-Barré syndrome.

^†Hospital discharge data for Finland (2007) developed for this report by JE.

^‡Analysis of General Practice Research Database (2008) undertaken by JS for this report.

^§Varying by year.

^¶Analysis of Kaiser Permanente Northern California (CA, USA) database (2004) undertaken by Jerome Klein (Boston Children’s Hospital, Boston, MA, USA) for this report

^‖Anaphylaxis rates from Australia and the USA are specific to vaccine whereas those from Finland are overall rates.

^**Depending on vaccine.

^††Per million doses of vaccine.

Optic neuritis is often the first presentation of multiple sclerosis. Incidence of optic neuritis varies by sex; the risk in girls aged 10–19 years in Sweden is 12 times higher than the risk in boys in the same age group. Incidence of optic neuritis also varies by country with 17·5 cases per 100 000 person-years in 20–39-year-old women in Sweden, 7·5 cases per 100 000 person-years in females (all ages) in the USA, and only 0·83 cases per 100 000 person-years for people of all ages in Singapore. Incidence of transverse myelitis and autoimmune thrombocytopenia also varied by country. Low rates of transverse myelitis were seen in Canada, Australia, and Israel, and higher rates were seen in the USA and Finland. Incidence of autoimmune thrombocytopenia was higher in the UK and Nordic countries than in the USA and Finland. Two sites in Canada had different rates of multiple sclerosis, showing the high level of geographical variability in incidence of this disease, even within one country.

In the UK, incidence of Bell’s palsy in younger individuals was less than half that recorded in adults, which emphasises the importance of age adjustment for background rates.

Table 3 shows mortality in two age groups in selected countries (WHO mortality statistics). Although these estimates span a long period and are not directly relevant to mortality within the short period after vaccination, the variability in these rates highlights the need for locally relevant data for mortality. Even in developed countries with low overall mortality, the mortality rate is not inconsequential and deaths will occur in all age groups. An important subset of overall mortality is sudden death. Table 2 shows rates of sudden death within 1 h of onset of symptoms. These rates vary substantially by country but also by occupation: the rate of sudden death in individuals in the US military²⁹ is much higher than the rate in the general population, despite the military population being younger.

Table 3.

Mortality by country, age, and sex (2006)³⁰

	Adult mortality* (per 1000 population)			Under-5 mortality† (per 1000 livebirths)
	Total	Female	Male	Total	Female	Male
Argentina	124	86	162	17	15	18
Australia	65	47	82	6	5	6
Brazil	176	121	230	20	18	22
Canada	72	55	89	6	5	6
China	116	87	143	24	27	21
Finland	96	57	132	3	3	4
India	241	203	276	76	81	72
Philippines	219	157	277	32	26	37
UK	80	61	98	6	5	6
USA	109	80	137	8	7	8
Vietnam	155	116	194	17	16	17

^*Ages 15–60 years.

^†Death by 5 years of age.

The current pandemic H1N1 influenza virus causes increased morbidity and mortality in pregnant women; this population is therefore a high priority for vaccination.³¹ Table 4 and table 5 show the rates of spontaneous abortion and premature labour in several countries. Rates vary by age and country, but the proportion of pregnancies that result in either spontaneous abortion or premature labour are high in all of the countries reported.

Table 4.

Rates of spontaneous abortion by country, surveillance year, and age group

	Rate (%)
Australia (1998–2009)32
18–23 years	3·5%
22–27 years	6·2%
25–30 years	9·8%
28–33 years	14·5%
Finland (1994)33
18–24 years	21·2%
25–29 years	12·1%
30–34 years	11·9%
35–39 years	13·1%
40–44 years	13.7%
All ages	13·2%
UK (1995–2005)34
All ages	12·0%
USA (1960–80)35
All ages	15·8%
USA (1980–90)36
≤24 years	10·4%
25–29 years	13·6%
30–34 years	22·3%
≥35 years	22·4%
All ages	14·5%

Table 5.

Selected rates of preterm labour or delivery (<37 weeks gestation), by country and year

	Rate (%)
Finland (2007)*	5.7%
France (1998)37	6.27%
Sweden (2004)38	5·8%
USA (2008)39	10·4–11·5%

^*Hospital discharge data for Finland (2007) developed for this report by JE.

Expected number of background disease events after vaccination

Table 6 shows the number of coincident events that might be expected as background rate events within 1 day, 1 week, and 6 weeks after receipt of a hypothetical vaccine. Even within the short 1-week period, a substantial number of events can be expected for rare events. For example, approximately four cases of Guillain-Barré syndrome per 10 million vaccinated people are predicted to occur within 1 week of vaccination as background coincident cases. The exact number would depend on the demographics of the vaccinated population. For some of the other events, such as spontaneous abortion or death, the numbers of expected events are quite large. 397 per 1 million vaccinated pregnant women are predicted to have a spontaneous abortion within 1 day of vaccination. However, rates of vaccination are not uniform throughout pregnancy. For this reason and others, our predicted rate probably overestimates the number of events that would be seen after vaccination in an actual campaign. In fact, the predicted value would have to be adjusted according to the proportion of women vaccinated in each trimester and, if possible, when in that trimester vaccination occurred. For sudden death within 1 h of onset of symptoms, 5·75 such events within 6 weeks of vaccination would be expected as background coincident cases per 10 million vaccinated people.

Table 6.

Predicted numbers of coincident, temporally associated events after a single dose of a hypothetical vaccine, based upon background incidence rates

	Number of coincident events since a vaccine dose			Baseline rate used for estimate
	Within 1 day	Within 7 days	Within 6 weeks
Guillain-Barré syndrome (per 10 million vaccinated people)	0·51	3·58	21·50	1·87 per 100 000 person-years (all ages; UK Health Protection Agency data)
Optic neuritis (per 10 million female vaccinees)	2·05	1440	86·30	7·5 per 100 000 person-years in US females (table 2)16
Spontaneous abortions (per 1 million vaccinated pregnant women)	397	2780	16 684	Based on data from the UK (12% of pregnancies)34
Sudden death within 1 h of onset of any symptoms (per 10 million vaccinated people)	014	0·98	575	Based upon UK background rate of 0·5 per 100 000 person-years (table 2)28

Discussion

Safety should be monitored to detect previously unrecognised serious adverse events that might be related to new vaccines. A timely and thorough analysis of safety concerns will need to account for the likelihood that large numbers of disease events—which might be misinterpreted as causally related to vaccines—can be expected to occur in large pandemic H1N1 influenza vaccination campaigns. In the past, the occurrence of such events has threatened or stopped large vaccination campaigns. Assessment of causality for events associated with vaccines will be aided by knowledge of their background incidence rates. Additionally, it is possible to look for temporal or geographical clustering when assessing causality. However, one should expect that rates of adverse health outcomes that are temporally associated with vaccination, such as spontaneous abortion, might also be clustered geographically and within clinical practices just by chance alone. Even random events can appear to have patterns. The chance occurrence of geographical clustering of rare cancers has been noted repeatedly. For example, 55% of California census tracts will have at least one type of cancer statistically raised by chance (p≤0·01) owing to the multiple hypothesis testing of these data.⁴⁰

With regard to possible outcomes after vaccination, if the practice-level rates of spontaneous abortion after vaccination follow the normal distribution, there will be a small number of practices (eg, about 2%) with a seemingly high rate (>2 SDs above the mean) of spontaneous abortion. Patients—and maybe even the practitioners in these practices—might view this cluster as being higher than background rates and consequently suspect an association with vaccination or even with a specific manufacturer’s vaccine. Clustering of adverse events geographically and within health-care practices after vaccination should be expected, and not interpreted as an indication of a causal relation with vaccination unless supported by more careful study.

Many countries have developed vaccine safety assessment plans. Many of these plans rely on identification of possible adverse events or “signal detection”, but few have the ability to rapidly analyse any signals that are identified. Passive reporting systems such as VAERS in the USA or the yellow card system in the UK⁴¹ identify possible signals through review of the number of reported cases of adverse events. Analysis of data from such systems is complex. Additionally, to identify possible signals, it is not appropriate to rely on a review of the number of cases reported to these systems. In a pandemic vaccination programme, this approach might not be straightforward because heightened public awareness could lead to increased reporting of events after identification of a possible vaccine safety concern. Furthermore, if such signals are compared with events reported after other vaccines or seasonal influenza vaccines from previous years, the signal might seem to be stronger than expected because passive reporting is usually biased towards under-reporting unless there is heightened public awareness. In addition to the systems in use to detect potential adverse events, the USA, the UK, and other countries have been developing more robust and comprehensive vaccine safety systems to study potential causal associations; however, it is beyond the scope of this report to describe these in detail.

During a rapid-paced immunisation campaign, an adverse event reported on day 1 might result in a spate of similar events being reported over the following days, leading to a media reporting bias. Similarly, active or prompted telephone reporting systems that rely on individuals to call in and report adverse events can also identify a higher than usual number of events in close temporal association with vaccination. A key problem with passive reporting systems is that they provide a number of events (or numerator) but do not allow calculation of a rate or an attributable risk because the number of people vaccinated (or denominator) is not known. Because of this drawback, such systems might contribute to concerns about a false vaccine safety association because they can identify possible signals but cannot analyse causality.

In the UK, the USA, and Denmark, the availability of large databases that link medical outcomes with vaccine data provides a means of assessing signals identified passively and can provide estimates of a true incidence of medically attended events after vaccination. However, these systems can be affected by relatively small denominators (compared with the rarity of the event) and a time lag in the availability of data. Even though some of these systems track millions of people, for very rare events such as Guillain-Barré syndrome or for outcomes affecting a subset of the population such as pregnant women, they might still not have sufficient power to assess a possible safety concern. Additionally, the possible misclassification of events due to miscoding⁴² usually requires time-consuming chart review for accurate case ascertainment. The rates of Guillain-Barré syndrome reported from automated data in Finland were higher than those reported elsewhere in the published work, perhaps because of the lack of case validation. This discrepancy highlights the need for cautious interpretation of such results.

One approach to the analysis of events that occur after vaccination is to compare observed rates with expected rates. Background rates can provide the media, the public, public-health officials, and politicians with important information about the expected number of events that can occur in the absence of any vaccination programme. Additionally, they can be used to estimate the number of such events that will occur after immunisation of any number of individuals. In New Zealand, during a mass campaign against meningococcus type B, background rates were used to calculate observed versus expected ratios for adverse events as the vaccine campaign progressed.⁴³ With this approach, scientists and the public could be assured that the number of events observed was not higher than expected.

When calculating background rates and observed versus expected ratios, one must be aware of the geographical, seasonal, ethnic, and age differences in such rates and their dependence on the method used to develop these rates. Such rates are point estimates and, especially for rare events, the uncertainty in such estimates should be taken into account when comparing rates by use of 95% CIs around the estimate or rate ratio. Registries for given diseases such as multiple sclerosis, where all cases are reviewed and validated, will have lower rates of disease incidence than rates calculated from unconfirmed cases identified in large automated databases. For example, during a post-hoc assessment of a possible association between Guillain-Barré syndrome and influenza vaccination in 1992–94, a review of automated hospital data showed that of the cases identified in such databases, only 14% were confirmed as definite cases after chart review, 35% were probable cases, 20% were possible cases, and 31% were not judged to be cases of Guillain-Barré syndrome.⁴⁴ There were also large differences in incidence by sex, age, and geographical location. Expected background rates of events need to represent as far as possible the age, sex, ethnic, and geographical characteristics of the population being vaccinated. Use of US data to assess the risk of Guillain-Barré syndrome in the UK or Brazil could lead to inappropriate conclusions. Similarly, since the pandemic H1N1 influenza vaccination programmes are likely to target priority groups whose age or sex distribution might differ from that of the general population, it will be important to take these differences into account when assessing the risk of any possible vaccine adverse events.

The prospect of large mass immunisation campaigns against pandemic H1N1 influenza in several countries poses unique challenges to the appropriate assessment of vaccine safety. Such assessment needs to detect and analyse vaccine safety signals and take appropriate action to investigate possible unexpected adverse events. However, it is very likely that concerns about disease events that would have occurred even in the absence of vaccination will raise public concern. Uncommon events such as Guillain-Barré syndrome will occur in close proximity to vaccination in substantial numbers if large populations are vaccinated. Additionally, temporal and geographical clustering of such events can occur by chance alone. Misinterpretation of adverse health outcomes that are only temporally related to vaccination will not only threaten the success of the pandemic H1N1 influenza vaccine programme, but also potentially hinder the development of newer vaccines. Therefore, careful interpretation of vaccine safety signals is crucial to detect real reactions to vaccine and to ensure that temporally related events not caused by vaccination do not unjustly affect public opinion of the vaccine. Development and availability of data banks that can provide locally relevant background rates of disease incidence are important to aid assessment of vaccine safety concerns.