Influenza vaccination: policy versus evidence

Short abstract

Each year enormous effort goes into producing influenza vaccines for that specific year and delivering them to appropriate sections of the population. Is this effort justified?

Viral infections of the respiratory tract impose a high burden on society. In the last half of the 20th century, efforts to prevent or minimise their impact centred on the use of influenza vaccines. Each year enormous effort goes into producing that year's vaccine and delivering it to appropriate sections of the population. Here, I will discuss policies on the use of inactivated vaccines for seasonal influenza; the evidence for their efficacy, effectiveness, and safety (“effects”); and possible reasons for the gap between policy and evidence.

Policies

Every vaccination campaign has stated aims against which its effects must be measured. The US Advisory Committee on Immunisation Practices produces a regularly updated rationale for vaccination against influenza.¹ The current version identifies 11 categories of patients at high risk of complications from influenza (box).

The rationale rests on the heavy burden that influenza imposes on the population and the benefits of vaccination. For example, reductions in cases, admissions to hospital, mortality of elderly people in families with children, contacts with healthcare professionals, antibiotic prescriptions, and absenteeism for children and household contacts are the main arguments for extending vaccination to healthy children aged 6-23 months in the United States.² Canada introduced a similar policy in 2004.³ Less comprehensive policies recommending vaccination for all people aged 60 or 65 and over are in place in 40 of 51 developed or rapidly developing countries.⁴ On the basis of single studies, the World Health Organization estimates that “vaccination of the elderly reduces the risk of serious complications or of death by 70-85%.”⁵ Given the global nature of these recommendations, what type of evidence should we expect to support them and what does available evidence tell us?⁴

Which evidence?

When considering the best evidence for vaccination we must take into account the unique epidemiological features of influenza viruses and the rationale for immunisation. The incidence and circulation of seasonal influenza and other respiratory viruses vary greatly each year, each season, and even in each setting. A systematic review of the incidence of influenza in people up to 19 years' old reported a seasonal variability of 0-46%; during a five year period the average incidence was 4.6% in this age group. During a period of 25 years the incidence was 9.5% in children under 5.⁶ Because of this variability and lack of carryover protection from one year's vaccine to the next,⁷ especially if the virus changes its antigenic configuration, single studies reporting data from one or two seasons are difficult to interpret. Single studies are also not reliable sources for generalising and forecasting the effects of vaccines, especially when numbers are small. They introduce further instability into already problematic forecasting. Additional limitations to our forecasting ability are imposed by our use (and misuse) of studies assessing the effects of influenza vaccines. Although the effect assessed depends on the aims of the particular campaign, most concentrate on serious effects (such as pneumonia or death) and person to person transmission (table 1). Field efficacy studies are only relevant when viral circulation is high, but no one can forecast with precision the impact on next year's influenza.

Table 1.

Effects of inactivated influenza vaccines and preferred designs of primary studies to assess them

Effect	Definition	Preferred study design	Relevance for public health
Efficacy	Capacity of the vaccine to induce antibody responses (immunogenicity) to influenza viruses	Placebo controlled RCT	Important for the yearly registration of new vaccines containing the forthcoming “season's” viral antigens. Immunogenicity is the only way of testing the likely efficacy of the candidate vaccine in the absence of viral circulation
Field efficacy	Capacity of the vaccine to prevent influenza A or B virus and its complications	Placebo controlled RCT	High, if viral circulation is high (as in an epidemic or a pandemic). Studies assessing field efficacy are usually well resourced with reliable and quick virological feedback and cases of influenza are recognised as such. Estimates of efficacy cannot be generalised to seasons with low circulation of the influenza virus even if other respiratory viruses have a higher circulation
Effectiveness	Capacity of the vaccines to prevent influenza-like illness and its consequences	Placebo controlled RCT	High in conditions of good match between vaccine and viral antigen and high viral circulation. Higher if effects on major outcomes are reported
Harms	A harmful event potentially associated with exposure to influenza vaccines	Placebo controlled RCT or non-randomised, comparative study	Depends on incidence, latency, and type of harm

RCT, randomised controlled trial.

Figure 1.

Flu vaccination: is the effort justified?

Credit: SATURN STILLS/SPL

Studies of the effects on influenza-like illness and its complications most closely replicate real life conditions because no one knows what agent (if any) causes this disease. Influenza-like illness is an acute respiratory disease caused by many different viruses (including influenza A and B), which presents with symptoms and signs that cannot be distinguished from those of influenza. Influenza-like illness does not have documented laboratory isolation of the causative agent and is the syndrome that most commonly presents to doctors (“the flu”).

In general the most powerful and reliable studies are those that “average” out several years and perform subanalyses by setting, population, viral circulation, and viral-vaccine antigenic match—variables that affect interpretation of the effects of a vaccine. Systematic reviews are the best way to perform such analyses, and provide powerful evidence weighted by the methodological quality of the studies involved. Large datasets containing several decades of observations help us to assess the performance of vaccines more accurately.

People for whom vaccination is recommended in the United States¹

People aged 65 or more

Patients in institutions who have chronic medical conditions

Adults and children with chronic disorders of the cardiovascular and respiratory systems (including asthma but excluding hypertension)

Adults and children who have been treated in hospital in the preceding 12 months for a range of conditions (for example, diabetes or haemoglobinopathy) Adults and children with conditions that compromise respiratory function or handling of infected secretions Children aged 6 months to 18 years being treated with aspirin

Women who are pregnant during the influenza “season”

Children aged 6-59 months

Adults aged 50-64 years

Carers and household contacts (including children) of those in the above risk categories and of children aged 0-59 months

Healthcare workers

The evidence

I searched for relevant systematic reviews when updating and expanding the Clinical Evidence chapter on influenza (see bmj.com)—evidence was plentiful. The examples in table 2 show the strength of the evidence and the contradictions in relation to the stated aims of the vaccination campaign. Whenever possible, I chose evidence gathered in the optimal circumstances (for inactivated vaccines)—high viral circulation and a good match between the viral antigen and the vaccine.

Table 2.

Examples of evidence from systematic reviews comparing inactivated influenza vaccines with placebo or no intervention

Population	Study design included in review	Outcome	No of participants	Vaccine field efficacy or effectiveness*
Children aged up to 23 months8	RCT†	Influenza	786	0.55 (0.18 to 1.69)
		Influenza-like illness	—	No data
Children 6 years or more8	RCT†	Influenza	710	69%; 0.31 (0.22 to 0.45)
		Influenza-like illness	18 912	28%; 0.72 (0.66 to 0.78)
Children up to 16 years8	RCT†	Transmission	123	1.68 (0.56 to 4.99)
		School absence	254	0.46 (0.17 to 1.22)
		Lower respiratory tract infection	136	0.30 (0.01 to 6.17)
		Admission to hospital	765	1.41 (0.62 to 3.24)
		Death	—	No data
Healthy adults9	RCT†	Influenza	2411	67%; 0.33 (0.22 to 0.49)
		Influenza-like illness	5579	22%; 0.78 (0.67 to 0.91)
		Admission to hospital	5261	Relative risk fixed effects model 0.65 (0.34 to 1.22)
		Working days lost	5572	Weighted mean difference random effects model −0.12 (−0.24 to 0.00)
Healthcare workers (to protect elderly patients in their care)10	Cluster RCT and cohort	Influenza	752	0.87 (0.46 to 1.63)
		Lower respiratory tract infection	1059	0.70 (0.41 to 1.20)
		Death from pneumonia	1059	39%; 0.61 (0.38 to 0.98)
		All-cause mortality	2496	40%; 0.60 (0.50 to 0.73)
Elderly people in the community; high circulation of influenza virus and good vaccine-antigen matching11	Cohort	Influenza-like illness	—	No data
		Influenza	427	95%; 0.05 (0.01 to 0.37)
		Pneumonia	—	No data
		Death from influenza and pneumonia	163 391	0.87 (0.70 to 1.09)
		All-cause mortality (not adjusted for confounding)	300 332	41%; 0.59 (0.50 to 0.70)
		All-cause mortality (adjusted for confounding)	742 575	47%; odds ratio random effects model 0.53 (0.46 to 0.61)
Elderly people in institutions; high circulation of influenza virus and good vaccine-antigen matching11	Cohort	Influenza-like illness	5963	23%; 0.77 (0.64 to 0.94)
		Influenza	658	1.04 (0.43 to 2.51)
		Pneumonia	4482	46%; 0.54 (0.42 to 0.70)
		Death from influenza and pneumonia	6127	42%; 0.58 (0.41 to 0.83)
		All-cause mortality	305	60%; 0.40 (0.21 to 0.77)
Patients with asthma12	RCT†	Influenza related exacerbation (early)	696	Risk difference fixed effects model 0.01 (−0.02 to 0.04)
Patients with chronic obstructive pulmonary disease13	RCT†	Exacerbations (total number)	180	Weighted mean difference −0.37 (−0.64 to −0.11), P=0.006

RCT, randomised controlled trial.

^*Values are vaccine field efficacy or effectiveness (where available); relative risk random effects model (95% confidence intervals) unless stated otherwise. Relative risk reported when difference is not significant.

^†Placebo controlled comparison.

Three problems are immediately apparent. The first is heavy reliance on non-randomised studies (chiefly cohort studies), especially in the elderly. This makes assessment of methodological quality an important part of data interpretation. For example, of 40 datasets assessing the effects of influenza vaccines in elderly people in institutions, only 26 reported data on viral types in circulation and only 21 gave information on vaccine content. Insufficient data were available in 11 of 17 retrospective studies of elderly people in institutions to allow reviewers to assess the authors' claim of “high” or “epidemic” viral circulation.¹¹,¹⁴ A metaanalysis of inactivated vaccines in elderly people showed a gradient from no effect against influenza or influenza-like illness to a large effect (up to 60%) in preventing all-cause mortality. These findings are both counterintuitive and implausible, as other causes of death are far more prevalent in elderly people even in the winter months.¹⁵,¹⁶ It is impossible for a vaccine that does not prevent influenza to prevent its complications, including admission to hospital.

A more likely explanation for such a finding is selection bias, where one half of the study population (hemi-cohort) systematically differs from the other in one or more key characteristics.^14-16 In this case, the vaccinated hemi-cohort may have been more mobile, healthy, and wealthy than the control hemi-cohort, thus explaining the differences in all-cause mortality.¹¹,¹⁴ The same effect is seen in stronger study designs (such as cluster randomised trials) that are badly executed, which introduces bias.¹⁰ Its presence seems to be a marker of confounders that persist even after adjusting for known ones, and it makes accurate interpretation of the data difficult. Caution in interpretation should thus be the rule, not the exception. This problem (in the opposite direction—with frailer people more likely to be vaccinated) has been identified before but not heeded.¹⁷ The only way that all known and unknown confounders can be adequately controlled for is by randomisation.

The influence of poor study quality is also seen in the outcome of a review of evidence supporting the vaccination of all children to minimise transmission to family contacts.¹⁸ Five randomised studies and five non-randomised studies were reviewed, but although data were suggestive of protection, its extent was impossible to measure because of the weak methods used in the primary studies.¹⁸

The second problem is either the absence of evidence or the absence of convincing evidence on most of the effects at the centre of campaign objectives (table 2). In children under 2 years inactivated vaccines had the same field efficacy as placebo,⁸ and in healthy people under 65 vaccination did not affect hospital stay, time off work, or death from influenza and its complications.⁹ Reviews found no evidence of an effect in patients with asthma or cystic fibrosis, but inactivated vaccines reduced the incidence of exacerbations after three to four weeks by 39% in those with chronic obstructive pulmonary disease.¹²,¹³,¹⁹ All reviewers reported small data sets (such as 180 people with chronic obstructive pulmonary disease¹³), which may explain the lack of demonstrable effect.

The third problem is the small and heterogeneous dataset on the safety of inactivated vaccines, which is surprising given their longstanding and widespread use. A Cochrane Database Systematic Review found only one old trial with data from 35 participants aged 12-28 months.⁸ In the general population of elderly people, despite a dataset of several million observations, safety was only reported in five randomised controlled trials (2963 observations in total) on local and systemic adverse events seen within a week of giving parenteral inactivated vaccine.¹¹ Although there appears to be no evidence that annual revaccination is harmful, such a lack of knowledge is surprising.

Gap between policy and evidence

The large gap between policy and what the data tell us (when rigorously assembled and evaluated) is surprising. The reasons for this situation are not clear and may be complex. The starting point is the potential confusion between influenza and influenza-like illness, when any case of illness resembling influenza is seen as real influenza, especially during peak periods of activity. Some surveillance systems report cases of influenza-like illness as influenza without further explanation. This confusion leads to a gross overestimation of the impact of influenza, unrealistic expectations of the performance of vaccines, and spurious certainty of our ability to predict viral circulation and impact. The consequences are seen in the impractical advice given by public bodies on thresholds of the incidence of influenza-like illness at which influenza specific interventions (antivirals) should be used.²⁰

The confusion between influenza and influenza-like illness is compounded by the lack of accurate and fast surveillance systems that can tell what viruses are circulating in a setting or community within a short time frame, and after the “season” is finished give an accurate picture of what went on to enable better forecasting of future trends.²¹ Accurate surveillance must be based on a properly worked out sampling system for cases of influenza-like illness that meet set criteria, with accurate and quick feedback of a presumptive microbiological diagnosis. Without this, we cannot generalise from random sampling.

Another reason may be “availability creep.” In their efforts to deal with, or be seen to deal with, policy makers favour intervention with what is available—registered influenza vaccines. A similar philosophy is the “we have to make decisions and cannot wait to have perfect data” approach. This attitude may have an altruistic basis but has two important consequences. Firstly, it uses up resources that could be invested in a proper evaluation of influenza vaccines or on other health interventions of proven effectiveness. Secondly, the inception of a vaccination campaign seems to preclude the assessment of a vaccine through placebo controlled randomised trials on ethical grounds. Far from being unethical, however, such trials are desperately needed and we should invest in them without delay. A further consequence is reliance on non-randomised studies once the campaign is under way. It is debatable whether these can contribute to our understanding of the effectiveness of vaccines. Ultimately non-randomised designs cannot answer questions on the effects of influenza vaccines.

Summary points

Public policy worldwide recommends the use of inactivated influenza vaccines to prevent seasonal outbreaks

Because viral circulation and antigenic match vary each year and non-randomised studies predominate, systematic reviews of large datasets from several decades provide the best information on vaccine performance

Evidence from systematic reviews shows that inactivated vaccines have little or no effect on the effects measured

Most studies are of poor methodological quality and the impact of confounders is high

Little comparative evidence exists on the safety of these vaccines

Reasons for the current gap between policy and evidence are unclear, but given the huge resources involved, a re-evaluation should be urgently undertaken

The optimistic and confident tone of some predictions of viral circulation and of the impact of inactivated vaccines, which are at odds with the evidence, is striking. The reasons are probably complex and may involve “a messy blend of truth conflicts and conflicts of interest making it difficult to separate factual disputes from value disputes”²² or a manifestation of optimism bias (an unwarranted belief in the efficacy of interventions).²³

Whatever the reasons, it is a sobering thought that Archie Cochrane's 1972 statement that we should use what has been tested and found to reach its objectives is as revolutionary now as it was then.