Abstract
Please cite this paper as: Kelly et al. Moderate influenza vaccine effectiveness with variable effectiveness by match between circulating and vaccine strains in Australian adults aged 20–64 years, 2007–2011. Influenza and Other Respiratory Viruses DOI:10.1111/irv.12018.
Background Influenza vaccines are licensed annually based on immunogenicity studies. We used five sequential years of data to estimate influenza vaccine effectiveness (VE), the critical outcome in the field.
Methods Between 2007 and 2011, we performed annual prospective test‐negative design case–control studies among adults aged 20–64 years recruited from sentinel general practices in the Australian state of Victoria. We used PCR‐confirmed influenza as the endpoint to estimate influenza VE for all years. We compared annual VE estimates with the match between circulating and vaccine strains, determined by haemagglutination inhibition assays.
Results The adjusted VE estimate for all years (excluding 2009) was 62% (95% CI 43, 75). By type and subtype, the point estimates of VE by year ranged between 31% for seasonal influenza A(H1N1) and 88% for influenza A(H1N1)pdm09. In 2007, when circulating strains were assessed as incompletely matched, the point estimate of the adjusted VE against all influenza was 58%. The point estimate was 59% in 2011 when all strains were assessed as well matched.
Conclusion Trivalent inactivated vaccines provided moderate protection against laboratory‐confirmed influenza in adults of working age, although VE estimates were sensitive to the model used. VE estimates correlated poorly with circulating strain match, as assessed by haemagglutination inhibition assays, suggesting a need for VE studies that incorporate antigenic characterization data.
Keywords: influenza, influenza vaccine, influenza‐like illness, vaccine effectiveness
Introduction
Trivalent influenza vaccines are licensed annually based on limited immunogenicity studies, most often among healthy adults. 1 Given extensive past experience with influenza vaccines among adults, this process is widely accepted. It is also the only process that is feasible, given the current vaccine production and regulation processes. Each year influenza vaccines include selected strains of influenza A(H3N2), A(H1N1) and B viruses. Because the vaccine strains may need to change, depending on the drift of the circulating viruses, there is insufficient time for large‐scale vaccine efficacy and safety studies prior to vaccine licensing each year. Immunogenicity is, therefore, used as a proxy for vaccine efficacy.
Immunogenicity assesses the antibody response to the specific vaccine antigens, while vaccine efficacy estimates the proportion of influenza infections prevented by vaccination in a randomized controlled trial. Vaccine effectiveness (VE) is the same measure from an observational study. 2 Immunogenicity is not precisely correlated with VE, although effectiveness would normally be regarded as the ultimate test of a vaccine, as it assesses how well the vaccine protects against disease when delivered in routine practice. 3 In recent years, a number of investigators from Europe, 4 United States, 5 Canada 6 and Australia 7 have conducted observational studies using similar designs to monitor influenza VE.
Using methodological insights gained from these previous studies, we have studied patients recruited from an existing network of sentinel general practitioners (GP) in Victoria, Australia, to estimate influenza VE. Victoria has a temperate southern hemisphere climate and a population of approximately 5·5 million. The influenza season usually occurs between May and September. In a previous feasibility study, we suggested that the sentinel surveillance system is best suited to estimating influenza VE in adults aged 20–64 years, a group often characterized as working‐age adults. 8 Moreover, this age group is most often used in vaccine trials. Confining our analysis to this group allows a comparison of results from this observational study with published trial results. This study provides summary estimates of influenza VE by type and subtype over 4 years from 2007 to 2011, years during which there were significant antigenic changes in all three types/subtypes included in the vaccine. We compare the annual VE with the match between circulating and vaccine strains.
Methods
Study design
We used the prospective test‐negative variant of the case–control study 9 to estimate VE against laboratory‐confirmed influenza among patients presenting to a sentinel GP in Victoria between 2007 and 2011. In this study design, patients suspected of having influenza are recruited by the GP and swabbed at recruitment. Cases are patients who subsequently test positive for influenza, and controls are those who test negative. Control selection leads to the description of this study design as ‘test negative’. 9 In the prospective form of the test‐negative design, patients are recruited before their case status is known, that is, before the result of their swab is available. This study design is, therefore, not strictly a case–control design in which cases and controls are recruited based on known case status. We confined our analysis to adults aged 20–64 years as younger and older patients were under‐represented.
GP sentinel network
Over the 5 years of the study, sentinel GPs were recruited from metropolitan Melbourne and regional Victoria. GPs were rewarded for their participation with continuing education points from the Colleges of General Practice and Rural and Remote Medicine. GPs also received a weekly influenza surveillance report 10 and provided annual feedback by a brief survey. GP participation increased over the years from 65 in 2007 to 97 in 2011. Our GP survey data show that an average of 94·8% of GPs assessed the scheme as useful or very useful in this period.
GPs were asked to recruit patients with an ILI, defined as a combination of fever (measured or reported), cough and fatigue. 11 At the discretion of the GP, patients had a combined nose and throat swab, which was tested for influenza virus RNA at the Victorian Infectious Diseases Reference Laboratory (VIDRL) using a range of in‐house reverse transcriptase and real‐time PCR assays as previously reported. 7 , 12 , 13 , 14 The laboratory is designated as a National Influenza Centre by the World Health Organization. The sensitivity of an early in‐house assay, which is dependent on time from symptom onset until swabbing, was estimated as 90%, while specificity was estimated as 100%. 15 It has previously been shown that perfect specificity in the presence of imperfect, non‐differential sensitivity will provide unbiased point estimates of VE from a TND study when compared with the estimate from a cohort study. 9 , 16
In addition to symptoms, GPs collected data on the age and sex of patients and the date of influenza vaccination. In 2011, data on influenza vaccination in the previous year and the presence of comorbidities for which influenza vaccination is funded by the National Immunisation Program were also collected. Comorbidities were recorded as yes/no and included all those conditions that are indicated for influenza vaccination in Australia, such as immunosuppression, pre‐existing respiratory disease and pre‐existing cardiovascular disease. 17 Data in this study were collected, used and reported under the legislative authority of the Public Health and Wellbeing Act 2008 and the Public Health and Wellbeing Regulations 2009 and did not require approval from a Human Research Ethics Committee. Nonetheless, patients provided written informed consent for their swab to be collected, with an understanding that anonymous results may be used for surveillance purposes.
Estimating influenza VE
Vaccine status was recorded by the GP, based on GP records or patient report. As a proxy validation for accurate vaccine status, we required the GPs to provide the precise date of vaccination. In a case series in 2009, we found good concordance between GP and patient reports of vaccination, even when influenza vaccine had been administered outside the practice. 18 Patients were administered trivalent inactivated vaccines provided by a variety of manufacturers that changed by year. Vaccines from six manufacturers were licensed in Australia during the study period. 17 We did not collect data on vaccine manufacturer and assumed all vaccines were equally effective. Vaccines were analysed as potentially effective if administered at least 14 days prior to symptom onset. Patients whose vaccination occurred <14 days prior to symptom onset were excluded from the primary analysis. We also excluded any patient who had been vaccinated with only monovalent pandemic vaccine in 2009 or 2010 or those whose vaccination status was unknown.
Differences between those who tested positive or negative for influenza, and between the vaccinated and unvaccinated, were compared by Fisher’s exact test for categorical variables and t‐test for continuous variables. In the primary analysis, laboratory‐confirmed influenza was the outcome of interest and influenza vaccination the exposure. We estimated a crude odds ratio (OR) for vaccination comparing cases and controls for each year and each influenza type/subtype. Multivariable models were also fitted to adjust for potential confounders, including age, month of swab and time between symptom onset and swab. It is generally assumed that immunocompetency does not vary significantly in adults between the ages of 20–64 years. Age was, therefore, included as a continuous variable within this age group and recentred so that 0 represented age 20 and rescaled to decades, so that 40 years became 2 [(40–20)/10 = 2]. This allowed for variation of VE by age within the age group. To ensure valid comparisons, the same model was used for all years, but a sensitivity analysis was performed for 2011, including the extra covariates on comorbidities and previous influenza vaccination. This was the only year these covariates were collected. VE was calculated as 1‐OR and reported as a percentage with a 95% confidence interval. In the model combining data for the years 2007–2011, we included year as a covariate. In this estimation, we omitted 2009 when pandemic influenza was the predominant viral strain detected, and the vaccine was completely mismatched. 19
Our primary analysis included all patients for whom we had complete data, without censoring any variables. However, we conducted a number of sensitivity analyses on reduced data sets. When influenza infection is present, volunteer studies have shown that it is more likely to be detected within the first 4 days of infection, presumably because of decreased viral load as the infection resolves. 20 In the sensitivity analyses, we, therefore, examined the effect of excluding any patients who presented more than 4 days after symptom onset, compared with including the length of time from onset of symptoms to swabbing as a continuous variable. We also confined our analysis to the influenza season each year, with the season identified by two consecutive weeks in which one or more detections of influenza were made from sentinel patients with ILI. 7 , 13 , 14
All analyses were conducted in Stata version 11 (StataCorp. 2009. Stata Statistical Software: Release 11; College Station, TX, USA).
Comparison of circulating and vaccine strains
The composition of the influenza vaccine for each year was extracted from the website of the WHO Collaborating Centre for Research and Surveillance of Influenza in Melbourne. 21 The circulating strains were identified by the WHO Collaborating Centre based on specimens referred to the Centre from Victorian laboratories. Circulating and vaccine strains were compared based on the degree of cross‐reaction between strains and were conventionally assessed as being incompletely matched if there was ≥8‐fold difference in haemagglutination inhibition titres between the vaccine antigen and ferret‐derived antibodies to the circulating strain. 1 We accepted a match as incomplete when the vaccine and predominant circulating strains differed.
Results
Sentinel patients
There were 3136 sentinel patients with laboratory results from the 5 years of the study, of whom 2099 (67%) were aged 20–64 years. One case of influenza C was excluded from further analysis, and two patients had no laboratory results. The vaccination status was unknown or unspecified for 64 patients, 11 were vaccinated <14 days prior to the onset of ILI symptoms and 18 were vaccinated with the monovalent H1N1 vaccine. After excluding these patients, the final sample size was 2003.
The proportion of patients with an unknown vaccination status was low, but varied by year, with 1·7% unknown in 2007, 0·7% in 2008, 4·6% in 2009, 2·8% in 2010 and 3·6% in 2011 (P = 0·008). There was no difference by case status (P = 0·6). In the 5 years combined, 368 (18%) patients were recorded as having been vaccinated, with a tendency for higher vaccine coverage (22%) in 2009, the year of the influenza A(H1N1) pandemic.
In all, 655 (33%) patients tested positive for influenza of any type or subtype (Table 1). There were 96 cases of influenza B and 559 cases of influenza A, including 36 seasonal H1N1, 313 pandemic H1N1, 160 H3N2, 1 mixed H1N1/H3N2, and 49 were not subtyped. The proportion of cases and controls ascertained by month differed by year (Figure 1).
Table 1.
Characteristics of 2003 adults aged 20–64 with influenza‐like illness included in the analysis, by year and vaccination status. Values are n (%) unless otherwise indicated
| Characteristic | Year | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 2007 | 2008 | 2009 | 2010 | 2011 | ||||||
| Vaccinated | Unvaccinated | Vaccinated | Unvaccinated | Vaccinated | Unvaccinated | Vaccinated | Unvaccinated | Vaccinated | Unvaccinated | |
| Age in years [mean (SD)] | 45.8 (11.0) | 35.2 (11.6) | 42.0 (12.5) | 36.6 (11.0) | 41.6 (12.5) | 34.3 (11.7) | 43.1 (12.0) | 37.1 (11.2) | 43.2 (12.5) | 37.1 (11.2) |
| Gender | ||||||||||
| F | 34 (52) | 124 (44) | 23 (51) | 117 (48) | 74 (52) | 249 (49) | 28 (57) | 108 (46) | 33 (53) | 158 (47) |
| M | 32 (48) | 158 (56) | 22 (49) | 126 (52) | 68 (48) | 264 (51) | 21 (43) | 129 (54) | 29 (47) | 176 (53) |
| Influenza PCR | ||||||||||
| Negative | 49 (74) | 146 (52) | 36 (80) | 177 (73) | 93 (65) | 337 (65) | 48 (92) | 159 (62) | 53 (85) | 250 (74) |
| Positive | 17 (26) | 136 (48) | 9 (20) | 66 (27) | 50 (35) | 180 (35) | 4 (8) | 98 (38) | 9 (15) | 86 (26) |
| Influenza type/subtype* | ||||||||||
| A (not subtyped) | 2 (12) | 4 (3) | 0 (0) | 5 (8) | 3 (6) | 20 (11) | 0 (0) | 8 (8) | 2 (22) | 5 (6) |
| A (H1N1) | 4 (24) | 26 (19) | 1 (11) | 3 (5) | 0 (0) | 2 (1) | 0 (0) | 0 (0) | 0 (0) | 0 (0) |
| A (H3N2) | 9 (53) | 81 (60) | 6 (67) | 24 (36) | 0 (0) | 3 (2) | 0 (0) | 3 (3) | 5 (56) | 29 (34) |
| A (H1N1)pdm09 | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 47 (94) | 155 (86) | 4 (100) | 85 (87) | 0 (0) | 22 (26) |
| A (H1|H3) | 0 (0) | 1 (1) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) |
| B | 2 (12) | 24 (18) | 2 (22) | 34 (52) | 0 (0) | 0 (0) | 0 (0) | 2 (2) | 2 (22) | 30 (35) |
| Comorbid condition | ||||||||||
| No | Data not collected in these years | 37 (69) | 256 (89) | |||||||
| Yes | 17 (31) | 32 (11) | ||||||||
| Previously vaccinated | ||||||||||
| No | 15 (27) | 263 (83) | ||||||||
| Yes | 41 (73) | 53 (17) | ||||||||
*Percents may not add to 100 due to rounding errors.
Figure 1.
Cases and controls by season1, Victorian sentinel patients 2007–2011; 1Autumn: March–May; Winter: June–August; Spring: September–November; Summer: December–February.
In 2011, the only year that data on comorbidities and previous vaccination were collected, 12% of 398 patients were recorded as having a comorbidity that increased their risk of an adverse outcome to infection. While more men than women recorded a comorbidity (19% versus 9%, P = 0·005), there was marginal difference by case status (8% cases versus 16% controls, P = 0·08). As expected, persons with a comorbidity were more likely to be vaccinated (33% versus 13%, P < 0·001). Patients who had been vaccinated in 2011 were more likely to have been vaccinated in the previous year (71% versus 17%, P < 0·001).
Influenza vaccine effectiveness
Overall, cases (patients with influenza) were less likely than controls (patients without influenza) to have been vaccinated (OR = 0·40), corresponding to a crude VE = 60%, 95% CI 43, 72). This was the case for 2007 and 2010 but, based on a crude analysis, cases were not significantly more likely than controls to have been vaccinated in 2008, 2009 or 2011 (Table 2).
Table 2.
Influenza vaccine effective estimates by year and influenza type and subtype, Victorian sentinel patients aged 20–64 years
| Year | Influenza type and subtype* | Cases n** | Vaccinated cases n (%) | Controls n | Vaccinated controls n (%) | Crude VE% (95% CI) | Adjusted VE%*** (95% CI) |
|---|---|---|---|---|---|---|---|
| 2007 | All | 153 | 17 (11) | 195 | 49 (25) | 63 (32, 80) | 58 (17, 79) |
| A(H1N1) | 31 | 4 (13) | 56 (−32, 85) | 49 (−70, 85) | |||
| A(H3N2) | 91 | 9 (10) | 67 (30, 85) | 63 (15, 84) | |||
| B | 26 | 2 (8) | 75 (−9, 94) | 78 (−16, 96) | |||
| 2008 | All | 75 | 9 (12) | 213 | 36 (17) | 33 (−47, 69) | 29 (−71, 71) |
| A(H1N1) | 4 | 1 (25) | −64 (−1521, 83) | −75 (−2065, 86) | |||
| A(H3N2) | 30 | 6 (20) | −23 (−222, 53) | −8 (−238, 65) | |||
| B | 36 | 2 (6) | 71 (−26, 93) | 57 (−95, 91) | |||
| 2009 | All | 230 | 50 (22) | 430 | 93 (22) | −1 (−48, 32) | −32 (−116, 19) |
| A(H1N1)pdm09† | 202 | 47 (23) | −10 (−64, 26) | −41 (−135, 16) | |||
| 2010 | All | 102 | 4 (4) | 207 | 48 (23) | 86 (61, 95) | 87 (61, 96) |
| A(H1N1)pdm09 | 89 | 4 (4) | 84 (55, 95) | 85 (56, 95) | |||
| 2011‡ | All | 95 | 9 (9) | 303 | 53 (17) | 51 (−4, 77) | 59 (4, 82) |
| A(H3N2) | 34 | 5 (15) | 19 (−120, 70) | 54 (−49, 86) | |||
| B | 32 | 2 (6) | 69 (−36, 93) | 64 (−61, 92) | |||
| Overall§ | All | 425 | 39 (9) | 918 | 186 (20) | 60 (43, 72) | 62 (43, 75) |
| A(H1N1) | 35 | 5 (14) | 34 (−71, 75) | 31 (−107, 77) | |||
| A(H3N2) | 158 | 20 (13) | 43 (6, 65) | 52 (14, 73) | |||
| A(H1N1)pdm09 | 111 | 4 (4) | 85 (60, 95) | 88 (64, 96) | |||
| B | 96 | 6 (6) | 74 (39, 89) | 65 (17, 86) |
*VE is reported for subtypes where at least one vaccinated case was detected.
**Cases by type and subtype will not add to total cases because typing/subtyping was not available for all cases.
***Adjusted for delay between symptom onset and swab, age and month of presentation. Model for all years, 2007–2011 also adjusted for year.
†Influenza A(H1N1)pdm09 is the pandemic strain of influenza.
‡None of the 22 cases of A(H1N1)pdm09 detected in 2011 were vaccinated.
§2009 was not included in the overall VE estimate.
VE was calculated for each year for all influenza cases and by influenza type and subtype (Table 2). With the exception of 2009, the adjusted VE estimates were largely similar to the crude estimates in all years when the outcome was all influenza detections. Against all influenza types and subtypes, the adjusted VE showed a statistically significant protective effect in 2007 (VE = 58%, 95% CI 17, 79), 2010 (VE = 87%, 95% CI 61, 96) and 2011 (VE = 59%, 95% CI 4, 82) and a non‐significant protective effect in 2008 (VE = 29%, 95% CI −71, 71). In 2009, the year of the pandemic, the point estimate for VE was non‐protective (VE = −32%, 95% CI −116, 19), but this was not statistically significant. Although crude and adjusted VE estimates were mostly similar for VE against influenza types and subtypes, estimates were variable and often not significant, likely owing to the small numbers of vaccinated cases in these categories by year (Table 2).
The adjusted VE estimate for the 4 years excluding 2009 was 62% (95% CI 43, 75). Age was not a significant predictor [OR = 0·93 (95% CI 0·83, 1·04)]. When analysed by type and subtype, the point estimates of VE ranged between 31% for seasonal influenza A(H1N1) and 88% for influenza A(H1N1)pdm09 (Table 2).
The sensitivity of the estimates was assessed when the model was modified in three ways. First, for 2011, the only year for which comorbidity and previous vaccination status were available, the adjusted VE including these variables in the model gave an estimate of 48% (95% CI −41, 81), lower than the adjusted estimate when these variables were not included (VE = 59%, 95% CI 4, 82) (Table 3). Second, among patients with information on the time between symptom onset and the collection of a nasopharyngeal swab, 151 of 1270 (12%) samples were collected after 4 days onset, but 6·4% of cases compared with 15% of controls had swabs collected after 4 days (P < 0·001). When these patients presenting late were excluded from the analysis, the overall, adjusted estimate of vaccine effectiveness improved to 66% (95% CI 48, 78; 2009 omitted). Finally, when only patients presenting during the influenza season were considered (n = 1230), the adjusted VE reduced slightly to 60% (95% CI 40, 73).
Table 3.
Sensitivity of the VE estimates under different models
| Model | N (n*) | VE% (95% CI) |
|---|---|---|
| Adjusted model including comorbidity status and previous vaccination status, 2011 only | 398 (274) | 48 (−41, 81) |
| Adjusted model excluding patients who presented >4 day after symptom onset, 2007–2011 (2009 omitted) | 1270 (1107) | 66 (48, 78) |
| Adjusted model excluding patients presenting outside the season, 2007–2011 (2009 omitted) | 1230 (1227) | 60 (40, 73) |
*Numbers in parentheses are the number included in the regression model (complete case analysis).
Adjusted VE estimates by type and subtype were compared with assessments of the match between circulating and vaccine strains (Table 4). In 2007, when the majority of circulating strains were assessed as incompletely matched by the haemagglutination inhibition assay, the point estimate of the adjusted VE against all influenza was 58%. The point estimate of the VE was 87% in 2010 when vaccine and circulating strains were matched, but was 59% in 2011 when all strains were again assessed as well matched.
Table 4.
Influenza vaccine effectiveness estimates. The VE for each year is provided alongside the strains included in that year’s vaccine as well as the predominantly circulating strain in Victoria that year*. Strains in bold indicate an incompletely matched vaccine strain
| Year | VE, adjusted (95% CI) | Type/subtype | N* | Vaccine | Predominant strain |
|---|---|---|---|---|---|
| 2007 | 58 (17, 79) | A/H1 | 65 | A/New Caledonia/20/99 (H1N1) | 83% A/Solomon Islands/3/2006‐like |
| A/H3 | 74 | A/Wisconsin/67/2005 (H3N2) | 64% A/Brisbane/10/2007‐like | ||
| B | 18 | B/Malaysia/2506/67/2004 (Victoria lineage) | 58% B/Florida/4/2006‐like | ||
| 2008 | 29 (−71, 71) | A/H1 | 1 | A/Solomon Islands/3/2006 (H1N1) | 100% A/Brisbane/59/2007‐like |
| A/H3 | 39 | A/Brisbane/10/2007 (H3N2) | 81% A/Brisbane/10/2007‐like | ||
| B | 19 | B/Florida/4/2006 (Yamagata lineage) | 38% B/Florida/4/2006‐like | ||
| 2009 | −32 (−116, 19) | A/H1 | 99 | A/Brisbane/59/2007 (H1N1) | 91% A/California/7/2009‐like |
| A/H3 | 17 | A/Brisbane/10/2007 (H3N2) | 59% A/Perth/16/2009‐like | ||
| B | B/Florida/4/2006 (Yamagata lineage) | (No samples received from Victoria) | |||
| 2010 | 87 (61, 96) | A/H1 | 233 | A/California/7/2009 (H1N1)‐like virus | 98% A/California/7/2009‐like |
| A/H3 | 23 | A/Perth/16/2009 (H3N2)‐like virus | 96% A/Perth/16/2009‐like | ||
| B | 9 | B/Brisbane/60/2008‐like virus (Victoria lineage) | 90% B/Brisbane/60/2008‐like | ||
| 2011 | 59 (4, 82) | A/H1 | 79 | A/California/7/2009 (H1N1)‐like virus | 89% A/California/7/2009‐like |
| A/H3 | 135 | A/Perth/16/2009 (H3N2)‐like virus | 98% A/Perth/16/2009‐like | ||
| B | 128 | B/Brisbane/60/2008‐like virus (Victoria lineage) | 95% B/Brisbane/60/2008‐like |
*Circulating strains are determined for a sample of viruses from Victoria by the WHO Collaborating Centre for Reference and Research on Influenza and may not be representative of the strains circulating in the community.
Discussion
Based on a prospective test‐negative design variant of a case–control study, we estimated influenza VE against laboratory‐confirmed influenza for adults aged 20–64 years attending a Victorian sentinel general practice in 2007–2011 as 62% (95% CI 43, 75), excluding the pandemic year of 2009. Using data for 4 years resulted in a sample size exceeding 1300 even after exclusion of 2009 when influenza A(H1N1)pdm09 was the dominant circulating strain. PCR‐confirmed influenza defined the study endpoint. Relative to PCR, viral culture will miss cases, and serology will overestimate VE for trivalent inactivated vaccines. 22 For studies of inactivated influenza vaccines, such as this study, PCR is the laboratory test of choice.
Differences in VE estimates from this study and those from our previous publications resulted from restriction of our analysis to the 20‐ to 64‐year‐old age group, analysing age as a continuous variable within the group and the inclusion of the delay between symptom onset and swabbing as a continuous covariate in this analysis rather than censoring data at 4 days delay. However, comparison with previously reported results and the sensitivity analyses in this study showed the differences in approach made only marginal differences to the VE estimates by year, except for the pandemic year of 2009. 7 , 12 , 13 , 14 We did not include that year in our summary VE estimate, and an exploration of possible reasons for the differences in VE estimates will be reported separately.
In addition to limitations common to observational studies, the test‐negative design has its own methodological limitations, not all of which have been completely explored. Our study was limited by the fact that we did not collect comorbidity and previous vaccination status until 2011. We had tried to keep the system as simple as possible to facilitate GP involvement, but the collection of the extra data in 2011 did not appear to burden GPs. We allow GPs discretion in determining which patients to swab, whereas other surveillance schemes use a systematic approach to swabbing, to try to limit bias. 4 All observational studies are limited by the lack of randomization of vaccination, a potential source of bias. For example, patients with comorbidities should be more likely to be vaccinated (exposure by indication) but additionally may be more likely to be tested.
Given these potential limitations, we acknowledge that the VE estimates from this study may be biased. It is, therefore, instructive to compare our results with those from contemporary studies using the same endpoint of PCR‐confirmed influenza in patient groups of similar ages. The gold standard comparator is the randomized controlled trial. Results from a large randomized controlled trial conducted in Australia and New Zealand in 2008–2009 found an efficacy of 60% (95% CI 44, 72) for matched strains and 42% (95% CI 30, 52) for all strains, which included A(H1N1)pdm 2009. 23 A meta‐analysis of vaccines licensed for use in the USA estimated a pooled vaccine efficacy of 59% (95% CI 51, 67) from published trials. 24 A recent pooled test‐negative design of eight studies from Europe estimated adjusted VE for all influenza in 15‐ to 59‐year‐olds as 41% (95% CI: −3, 66) in 2010–2011. 25 There are acknowledged potential biases in the test‐negative design, but when comparisons from this design are limited to influenza laboratory–detected by PCR among adults of working age, efficacy (trial results) and effectiveness (observational study results) estimates are similar (Table 5).
Table 5.
Comparison of contemporary vaccine effect measures from community‐based studies using a PCR endpoint in working‐age adults
| Study and setting | Design | Years | Age group | Vaccine effect measure | Participants | VE (95% CI) | ||
|---|---|---|---|---|---|---|---|---|
| Total | With influenza | Without influenza | ||||||
| Observational study, Australia (this study) | Test‐negative design | 2007–2008 and 2010–20011 | 20–64 years | Effectiveness of all strains | 1343 | 425 | 918 | 62 (43, 75) |
| Pooled observational study, Europe [25] | Test‐negative design from eight countries | 2010–2011 | 15–59 years | Effectiveness of all strains | 2511 | 1117 | 1394 | 41 (−3, 66) |
| Systematic review, vaccines licensed in the USA [24] | Mantel–Haenszelrandom effects model meta‐analysis | Searched for eligible studies 1967–2011 | 18–64 years | Efficacy of all strains | 32 470 | 578 | 31 892 | 59 (51, 67) |
| Vaccine licensure study, Australia and New Zealand [23] | Randomized controlled trial | 2008–2009 | 18–64 years | Efficacy‐matched strains Efficacy of non‐matched strains | 14 859 | 277 | 14 582 | 60 (44, 72) 42 (30, 52) |
However, our study also suggests that VE results are not directly related to the proportion of circulating strains that are matched to the vaccine. This observation may result from under‐representation of viruses received by the WHO Collaborating Centre in Melbourne. The Centre receives about 15% of laboratory‐confirmed influenza viruses reported by the state of Victoria each year, but it is difficult to know whether those viruses submitted represent equal proportions of the circulating strains. Even with perfect representativeness, haemagglutination inhibition assays are a blunt tool for the assessment of VE for inactivated vaccines. 22 It has also been suggested that these assays may be suboptimal for the determination of strain match, especially for more recently circulating H3N2 strains for which problems with agglutination of chicken and turkey red blood cells have been documented and assay results give sometimes discrepant results depending on whether the isolate was grown in eggs or cell culture. 1 Other options for the assessment of vaccine match have their own limitations; microneutralization is labour‐ and time‐intensive and has limited accuracy, 26 and phylogenetic analysis does not reliably correlate with antigenic drift. 27
A study from Taiwan that modelled excess seasonal pneumonia and influenza mortality in older persons showed a lower mortality when vaccine and circulating strains were matched. However, there was also a trend towards lower mortality with mismatched vaccines during the post‐SARS period. 28 In an analysis from the 2007–2008 influenza season in the USA, VE was estimated as 37% with a suboptimal match for both the H3N2 and B strains. 29 The authors concluded that, in any season, assessment of the clinical effectiveness of influenza vaccines cannot be determined solely by laboratory evaluation of the degree of antigenic match between vaccine and circulation strains. This was confirmed in the 2010–2011 influenza season in Canada, when an incompletely matched H3N2 strain was identified by both reduced subtype VE estimation and phylogenetic analysis, but not by haemagglutination inhibition assay. 27
We conclude that the trivalent influenza vaccine provides only moderate protection, of the order of 60%, against medically attended ILI due to laboratory‐confirmed influenza in working‐age adults. Other VE estimates for the 2010–2011 northern hemisphere season and the 2010 and 2011 southern hemisphere seasons are consistent with this conclusion. 14 , 27 , 30 , 31 , 32 In future seasons, we plan to continue to collect data on important confounders, such as comorbidity status and incorporate antigenic characterization data to estimate VE by strain. While it must be stressed that current influenza vaccines are proven to be effective in both trials and observational studies, it is our view that reliance on vaccines of moderate effectiveness should not be allowed to delay the development of new potentially improved vaccines.
Established and evolving observational study designs to estimate influenza vaccine effectiveness should continue to be improved. Such improvements could involve standardizing study designs internationally, as has already been done in Europe. 25 Increasing sample sizes could increase the precision of VE estimates, especially by influenza type and subtype. Improved study designs would facilitate reliable field effectiveness estimates of new‐generation vaccines as they become available.
Conflict of interest statement
We the authors state that none of us has a commercial or other association that might pose a conflict of interest.
Heath Kelly – no conflict
Sheena Sullivan – no conflict
Kristina Grant – no conflict.
James Fielding – no conflict.
Sources of financial support
The General Practitioner Sentinel Surveillance system is partly funded by the Victorian Government Department of Health.
Previous presentation of information
None of the data in this manuscript have been presented elsewhere.
Acknowledgements
For thoughtful comments on the manuscript, we would like to thank Dr Nicholas Kelley and Dr Ian Barr. We gratefully acknowledge the participation of all general practitioners involved in sentinel surveillance in Victoria. We thank the laboratory staff of the WHO Collaborating Centre for Reference and Research on Influenza and the Viral Identification Laboratory, both at the Victorian Infectious Diseases Reference Laboratory, Melbourne, Australia. The Melbourne WHO Collaborating Centre for Reference and Research on Influenza is supported by the Australian Government Department of Health and Ageing. Sentinel surveillance in Victoria is supported by the Victorian Department of Health.
References
- 1. Barr IG, McCauley J, Cox N et al. Epidemiological, antigenic and genetic characteristics of seasonal influenza A(H1N1), A(H3N2) and B influenza viruses: basis for the WHO recommendation on the composition of influenza vaccines for use in the 2009–2010 Northern Hemisphere season. Vaccine 2009; 28:1156–1167. [DOI] [PubMed] [Google Scholar]
- 2. Last J, Porta M. A Dictionary of Epidemiology, 5th edn New York, USA: Oxford University Press, 2008. [Google Scholar]
- 3. Kelly H, Barr I. Large trials confirm immunogenicity of H1N1 vaccines. Lancet 2009; 375:6–9. [DOI] [PubMed] [Google Scholar]
- 4. Valenciano M, Kissling E, Cohen JM et al. Estimates of pandemic influenza vaccine effectiveness in Europe, 2009–2010: results of Influenza Monitoring Vaccine Effectiveness in Europe (I‐MOVE) multicentre case‐control study. PLoS Med 2010; 8:e1000388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Belongia EA, Kieke BA, Donahue JG et al. Effectiveness of inactivated influenza vaccines varied substantially with antigenic match from the 2004–2005 season to the 2006–2007 season. J Infect Dis 2009; 199:159–167. [DOI] [PubMed] [Google Scholar]
- 6. Skowronski DM, Masaro C, Kwindt TL et al. Estimating vaccine effectiveness against laboratory‐confirmed influenza using a sentinel physician network: results from the 2005–2006 season of dual A and B vaccine mismatch in Canada. Vaccine 2007; 25:2842–2851. [DOI] [PubMed] [Google Scholar]
- 7. Fielding JE, Grant KA, Papadakis G, Kelly HA. Estimation of type‐ and subtype‐specific influenza vaccine effectiveness in Victoria, Australia using a test negative case control method, 2007–2008. BMC Infect Dis 2011; 11:170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Kelly H, Carville K, Grant K, Jacoby P, Tran T, Barr I. Estimation of influenza vaccine effectiveness from routine surveillance data. PLoS ONE 2009; 4:e5079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Orenstein EW, De Serres G, Haber MJ et al. Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness. Int J Epidemiol 2007; 36:623–631. [DOI] [PubMed] [Google Scholar]
- 10. Victorian Infectious Diseases References Laboratory . The Victorian influenza vaccine effectiveness audit. Available at: https://www.victorianflusurveillance.com.au/ (Accessed 27 September 2012).
- 11. Thursky K, Cordova SP, Smith D, Kelly H. Working towards a simple case definition for influenza surveillance. J Clin Virol 2003; 27:170–179. [DOI] [PubMed] [Google Scholar]
- 12. Kelly HA, Grant KA, Fielding JE et al. Pandemic influenza H1N1 2009 infection in Victoria, Australia: No evidence for harm or benefit following receipt of seasonal influenza vaccine in 2009. Vaccine 2011; 29:6419–6426. [DOI] [PubMed] [Google Scholar]
- 13. Fielding JE, Grant KA, Garcia K, Kelly HA. Effectiveness of seasonal influenza vaccine against pandemic (H1N1) 2009 virus, Australia, 2010. Emerg Infect Dis 2011; 17:1181–1187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Fielding J, Grant K, Tran T, Kelly H. Moderate influenza vaccine effectiveness in Victoria, Australia, 2011. Euro Surveill 2012; 17:pii=20115. [PubMed] [Google Scholar]
- 15. Druce JD, Tran T, Kelly H et al. Laboratory diagnosis and surveillance of human respiratory viruses by PCR in Victoria, Australia, 2002–2003. J Med Virol 2005; 75:122–129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Greenland S, Lash TL. Bias anaylsis; in Rothman KJ, Greenland S, Lash TL. (eds): Modern Epidemiology. 3rd edn Philadelphia: Lippincott, Williams & Wilkins, 2008;345–380. [Google Scholar]
- 17. Australian Government Department of Health and Ageing . Immunise Australia Program. Available at: http://www.immunise.health.gov.au/internet/immunise/publishing.nsf/Content/immunise‐influenza (Accessed 27 September 2012).
- 18. Looker C, Carville K, Grant K, Kelly H. Influenza A (H1N1) in Victoria, Australia: a community case series and analysis of household transmission. PLoS ONE 2010; 5:e13702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Kelly H, Grant K. Interim analysis of pandemic influenza (H1N1) 2009 in Australia: surveillance trends, age of infection and effectiveness of seasonal vaccination. Euro Surveill 2009; 14:pii=19288. [DOI] [PubMed] [Google Scholar]
- 20. Carrat F, Vergu E, Ferguson NM et al. Time lines of infection and disease in human influenza: a review of volunteer challenge studies. Am J Epidemiol 2008; 167:775–785. [DOI] [PubMed] [Google Scholar]
- 21. WHO Collaborating Centre for Reference and Research on Influenza . Current Vaccine Recommendations. Available at: http://www.influenzacentre.org/centre_vaccines.htm (Accessed 16 February 2012).
- 22. Petrie JG, Ohmit SE, Johnson E, Cross RT, Monto AS. Efficacy studies of influenza vaccines: effect of end points used and characteristics of vaccine failures. J Infect Dis 2011; 203:1309–1315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. CSL Limited . A Study of the Efficacy, Safety and Tolerability Profile of CSL Limited’s Influenza Virus Vaccine (CSL’s IVV) Administered Intramuscularly in Healthy Adults. Available at: http://clinicaltrials.gov/ct2/show/NCT00562484?term=A+Study+of+the+Efficacy%2C+Safety+and+Tolerability+Profile+of+CSL+Limited%27s+Influenza+Virus+Vaccine&rank=1 (Accessed 27 September 2012).
- 24. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta‐analysis. Lancet Infect Dis 2012; 12:36–44. [DOI] [PubMed] [Google Scholar]
- 25. Kissling E, Valenciano M, Cohen JM et al. I‐MOVE multi‐centre case control study 2010–11: overall and stratified estimates of influenza vaccine effectiveness in Europe. PLoS ONE 2011; 6:e27622. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Ansaldi F, Bacilieri S, Amicizia D et al. Antigenic characterisation of influenza B virus with a new microneutralisation assay: comparison to haemagglutination and sequence analysis. J Med Virol 2004; 74:141–146. [DOI] [PubMed] [Google Scholar]
- 27. Skowronski DM, Janjua NZ, De Serres G et al. A sentinel platform to evaluate influenza vaccine effectiveness and new variant circulation, Canada 2010‐11 Season. Clin Infect Dis 2012; 55:332–342. [DOI] [PubMed] [Google Scholar]
- 28. Chan TC, Hsiao CK, Lee CC et al. The impact of matching vaccine strains and post‐SARS public health efforts on reducing influenza‐associated mortality among the elderly. PLoS ONE 2010; 5:e11317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Belongia EA, Kieke BA, Donahue JG et al. Influenza vaccine effectiveness in Wisconsin during the 2007–2008 season: comparison of interim and final results. Vaccine 2011; 29:6558–6563. [DOI] [PubMed] [Google Scholar]
- 30. Cheng AC, Kotsimbos T, Kelly HA et al. Effectiveness of H1N1/09 monovalent and trivalent influenza vaccines against hospitalization with laboratory‐confirmed H1N1/09 influenza in Australia: a test‐negative case control study. Vaccine 2011; 29:7320–7325. [DOI] [PubMed] [Google Scholar]
- 31. Castilla J, Martinez‐Artola V, Salcedo E et al. Vaccine effectiveness in preventing influenza hospitalizations in Navarre, Spain, 2010–2011: cohort and case‐control study. Vaccine 2012; 30:195–200. [DOI] [PubMed] [Google Scholar]
- 32. Jimenez‐Jorge S, Savulescu C, Pozo F et al. Effectiveness of the 2010–2011 seasonal trivalent influenza vaccine in Spain: cycEVA study. Vaccine 2012; 30:24. [DOI] [PubMed] [Google Scholar]