Abstract
Background
The observational test-negative study design is used to estimate vaccine effectiveness against influenza virus infection. An important assumption of the test-negative design is that vaccination does not affect the risk of infection with another virus. If such virus interference occurred, detection of other respiratory viruses would be more common among influenza vaccine recipients and vaccine effectiveness estimates could differ. We evaluated the potential for virus interference using data from the Influenza Incidence Surveillance Project.
Methods
From 2010 to 2013, outpatients presenting to clinics in 13 US jurisdictions with acute respiratory infections were tested for influenza and other respiratory viruses. We investigated whether virus interference might affect vaccine effectiveness estimates by first evaluating the sensitivity of estimates using alternative control groups that include or exclude patients with other respiratory virus detections by age group and early/middle/late stage of influenza seasons. Second, we evaluated the association between influenza vaccination receipt and other respiratory virus detection among influenza test negative patients.
Results
Influenza was detected in 3,743/10,650 patients (35%), and overall vaccine effectiveness was 47% (95% CI: 42%, 52%). Estimates using each control group were consistent overall or when stratified by age groups, and there were no differences among early, middle, or late phase during influenza season. We found no associations between detection of other respiratory viruses and receipt of influenza vaccination.
Conclusions
In this 3-year test-negative design study in an outpatient setting in the United States, we found no evidence of virus interference or impact on influenza vaccine effectiveness estimation.
Introduction
The test-negative study design has been applied for estimating influenza vaccine effectiveness in many countries since 2005 and monitoring changes in vaccine effectiveness from year to year.4-11 In this variant of the case control study, patients presenting for ambulatory care with acute respiratory infection are tested for influenza virus. Cases are defined as patients testing positive for influenza virus, while those testing negative form the control group. Similar to a traditional case control study, vaccine effectiveness in a test-negative design is estimated as one minus the odds ratio of vaccination coverage among test-positive patients versus test-negative patients, adjusted for potential confounding factors, such as age, to permit a causal interpretation of the adjusted association as a measure of vaccine effectiveness.12-14
Influenza viruses co-circulate with other respiratory viruses during the winters in temperate locations. Virus interference describes an ecologic phenomenon that is occasionally reported, where an epidemic of one virus appears to ‘interfere’ with the occurrence of another epidemic, either by delaying its start, or by advancing its end if it has already started.15,16 Studies have reported apparent interference between epidemics of influenza and epidemics of other respiratory viruses, most notably rhinoviruses, which are the most frequently detected viruses in many studies of patients with acute respiratory infection.16-20 Some studies have suggested that virus interference may occur in some seasons but not others, or may occur after epidemics of some influenza types/subtypes but not others.16,21 This phenomenon is still controversial and is not consistently observed. The biological mechanism causing the ecological phenomenon of virus interference could be temporary non-specific immunity following an episode of acute respiratory infection, due to stimulation of the innate immune response during and for a short time after infection.22,23 Consistent with that mechanism, one small randomized trial of influenza vaccination in Hong Kong reported a significantly increased risk of other respiratory viruses among recipients of influenza vaccination.24
To permit valid estimation of vaccine effectiveness, one assumption of the test-negative design is that the probability of infection with another respiratory virus is not affected by receipt of influenza vaccination.26,27 Whether virus interference could affect vaccine effectiveness estimation in the test-negative design remains unclear, although a simulation study showed that only a small bias would be introduced during typical influenza epidemics because the cumulative incidence of influenza virus infection is thought to be relatively low in a typical epidemic and therefore any consequent interference effect would be limited.27-29 A study examining the potential for virus interference to affect vaccine effectiveness by comparing estimates from test-negative design-type analyses verses cohort-type analyses of data from randomized controlled trials of live attenuated influenza vaccines also found little difference between estimates generated by the two types of study.30 However, when the incidence of influenza infection is relatively high compared to those of other respiratory viruses, or when non-specific immunity following infection could last for a longer period, vaccine effectiveness estimates obtained from the test-negative design could be biased.28,31
To examine the potential for virus interference in a test-negative study in the United States, we used data from the Influenza Incidence Surveillance Project (IISP), which was established by the US Centers for Disease Control and Prevention (CDC).32-35 Data on patients' self-reported vaccination history and influenza and other respiratory virus testing results were collected for the 2010-11, 2011-12, and 2012-13 influenza seasons. Vaccine strains were generally well matched to the prevalent strains each season.36-38 The objectives of the present study were to evaluate the potential for virus interference to affect vaccine effectiveness estimates during the entire influenza season or different phases in each influenza season by (1) determining the sensitivity of vaccine effectiveness estimates to the choice of control group and; (2) examining the associations between influenza vaccination and detection of other respiratory viruses in patients who tested negative for influenza. If virus interference were to occur, people who received influenza vaccination could have a lower risk of influenza virus infection and a higher risk of infection with other respiratory viruses. Patients who tested positive for other respiratory viruses might be more likely to be vaccinated than the general population.24 Thus estimates by different control groups could be different, and association between influenza vaccination and detection of other respiratory viruses could be away from null.
Methods
Subjects
The IISP comprised 101 clinics covering 13 U.S. jurisdictions (sites) each year including Florida, Iowa, Minnesota, North Dakota, Oregon, Wisconsin, New York City, New Jersey, Virginia, Los Angeles County, and Philadelphia for all seasons (2010-2013), Utah for 2010-11, and Texas for 2011-12 and 2012-13. Clinical staff identified patients with acute respiratory illnesses within 7 days from onset and collected a specimen for laboratory testing. Acute respiratory illness was defined as any two of the following reported symptoms: fever, cough, sore throat, rhinorrhea, and congestion. In 2011-12 and 2012-13, seven and ten sites, respectively, narrowed case criteria to require fever plus cough or sore throat. A nasopharyngeal, nasal aspirate, oropharyngeal or nasal swab was collected from the first 10 patients consulted each week. Self-reported vaccination status, demographic data, and clinical data were collected by general practitioners at each study site. All specimens were tested for influenza and, unless depleted, other respiratory viruses, including respiratory syncytial virus (RSV), parainfluenza viruses (PIV) 1-3, metapneumovirus (MPV), rhinoviruses and adenovirus by real-time reverse transcriptase polymerase chain reaction platforms which have been described previously.32 The IISP uses routinely collected specimens and public health surveillance data, and was therefore determined by CDC not to be subject to institutional review board approval for human research protections.
Statistical analysis
We restricted analysis to the three influenza seasons covered by the study period. The start and end of each season were defined by the first and last week in which at least 10 patients tested positive for influenza virus from all participating clinics combined. We defined the peak of each influenza season as the week during each season with the highest proportion of patients testing positive for influenza, and we defined the 5-week period with 2 weeks on each side of the peak week, as well as the peak week, as the “middle phase” of that season. The “early phase” of each season was defined as the weeks from the start of the season prior to the “middle phase”, and similarly the “late phase” was the period after the “middle phase” until the end of that season.
To make the most of available data, we used multiple imputations for missing data on age, sex and vaccination history with 20 imputed datasets within each of the three influenza seasons. 39 Variables including age, sex, state, calendar week of visit, vaccination history and laboratory results for influenza virus were considered in an additive regression model for the multiple imputation. In our previous study there was no difference between vaccine effectiveness estimates in complete case analyses versus analyses with multiple imputation of missing values.33
We used three approaches to determine whether there was any evidence that virus interference affected estimation of vaccine effectiveness in this study. First, we examined the potential for virus interference to affect vaccine effectiveness by determining the sensitivity of estimates to the choice of control groups. We estimated and compared vaccine effectiveness with three different control groups: patients testing negative for influenza virus, patients testing negative for influenza virus but positive for at least one other respiratory virus, and patients testing pan-negative (those who tested negative for both influenza and other respiratory viruses) . The null hypothesis is that there is no difference in vaccine effectiveness against influenza infection using these three control groups. If virus interference were to occur, patients who tested positive for other respiratory viruses might be more likely to be vaccinated, resulting in a higher value of vaccine effectiveness compared to estimates using controls testing negative for influenza or controls testing pan-negative26 We used conditional logistic regression models to estimate vaccine effectiveness, adjusting for age group and sex and conditioning by week of clinic visit to account for changes in vaccine coverage over calendar time.33,40 Estimates of vaccine effectiveness were also stratified by age group, influenza season, and influenza type/subtype.
Second, we examined whether any differences between vaccine effectiveness estimates using the three control groups occurred during the middle and late phase compared to the early phase within each influenza season. If non-specific immunity were to occur after an influenza virus infection and last for a few weeks after infection, the proportion of the population experiencing temporary non-specific immunity would be highest around the peak of an influenza season or shortly afterwards.23 Therefore, an increase in the risk of other respiratory virus infections among vaccinated persons, and thus higher vaccine effectiveness estimates when using controls testing positive for another respiratory virus compared to estimates when using controls testing negative for influenza or testing pan-negative, would more likely to be observed during or after the influenza peak rather than the start of an influenza epidemic.24
Third, we examined the association between influenza vaccination and detection of other respiratory viruses during different phases of each influenza season among patients that tested negative for influenza virus. The null hypothesis of no association between vaccination and detection of other respiratory viruses would correspond to an odds ratio of 1. Based on the purported mechanism of temporary non-specific immunity, if influenza vaccination were to increase the risk of infection with other respiratory virus during or after the peak of the influenza season, the odds ratio could be greater than 1, particularly during middle or late phases of the influenza season. We first compared vaccination coverage between pan-negative controls versus controls that tested positive for other respiratory viruses, and then used conditional logistic regression where the outcome variable was detection of another respiratory virus and the exposure of interest was vaccination history, conditioning by week of visit, and adjusting for age group and sex. Because the risk of respiratory virus infections tends to be highest in children, we repeated these analyses by age group with the hypothesis that any differences due to virus interference might be greatest or most obvious in children. We repeated these analyses to examine the associations between receipt of influenza vaccination and detection of each specific other respiratory viruses. In addition, we conducted sensitivity analysis, restricting analyses to subjects who presented with influenza-like illness (defined as fever plus cough or sore throat).
Assuming that the ratio of patients testing positive for other respiratory viruses compared to pan-negative is 1:1, in order to have 80% power at a 5% significance level to detect an odds ratio for vaccination among those positive for other respiratory viruses compared with pan-negative controls of 1.2, we estimated that a sample size of 1900-2800 patients would be needed for vaccination coverage ranging from 30%-50%. We therefore ensured that our sample size would be large enough to include at least 1800-2200 patients in most age strata, for stratified analyses. All analyses were conducted using R version 3.1.1 (R Foundation for Statistical Computing, Vienna, Austria).
Results
From August 2, 2010 to December 31, 2013, 17652 patient specimens were collected. We excluded patients who were not tested for influenza virus (n=572), who were less than 6 months of age (n=397), or presented outside of influenza seasons (n=6033). At least 10 test-positive patients per week were identified for 20, 17, and 23 consecutive weeks in the 2010-11, 2011-12, and 2012-13 seasons respectively, and the weekly proportions of patients testing positive for influenza peaked in late February 2011, in mid-March 2012, and in early January 2013, respectively (Figure 1). The influenza season in 2010-11 was a mixed season with co-circulation of three influenza types/subtypes, while the season in 2011-12 and 2012-13 was dominated by A(H3N2). A total of 4208, 2164, and 4278 patients were included in each of these three influenza seasons respectively, and a total of 10650/17652 (60%) patients were included in the main analyses. Among 10650 patients tested for influenza, 10614 (99.7%) patients were further tested for rhinoviruses, RSV, PIV 1-3, MPV and adenovirus.
Figure 1.
(A) Detection of influenza viruses among patients presenting for outpatient consultation with acute respiratory infections through three influenza seasons in the Influenza Incidence Surveillance Project. (B) Patterns in detection of other respiratory viruses through the same period.
In both influenza-positive cases and the pan-negative control group (those who tested negative for both influenza and other respiratory viruses), the majority of patients were school-age children 6 to 17 years of age and adults 18 to 49 years of age (Table 1). In the other respiratory virus positive control group 1330/2835 (47%) patients were young children aged 6 months to 5 years, while in the pan-negative control group there were only 703/4036 (17%) young children aged 6 months to 5 years. Slightly more than half of the patients were female among influenza-positive cases, other virus positive controls and pan-negative controls while a larger proportion of females was observed in the pan-negative control group (58%). Influenza positive patients less frequently reported receipt of influenza vaccination compared to other control groups, and approximately 20% of all groups were missing vaccination status (Table 1). The most commonly detected other respiratory virus was rhinovirus (n=1101), followed by RSV (n=783) (eTable 1). Children aged 6 months to 5 years comprised a large proportion of the cases infected with rhinovirus (n=442) and RSV (n=496) (eTable 1).
Table 1.
Comparison of the characteristics of patients testing positive for any influenza virus, patients testing positive for other respiratory virus, and patients testing negative for all viruses.
| Characteristic | Influenza-positive (n=3743) N (%) | Other respiratory virus positive (n=2835) N (%) | Pan-negative a (n=4036) N (%) |
|---|---|---|---|
| Age group | |||
| 6 months- 5 years | 685 (18.3%) | 1330 (46.9%) | 703 (17.4%) |
| 6-17 years | 1345 (35.9%) | 638 (22.5%) | 1132 (28.0%) |
| 18-49 years | 1372 (36.7%) | 663 (23.4%) | 1745 (43.2%) |
| ≥50 years | 338 (9.0%) | 203 (7.2%) | 454 (11.2%) |
| Unknown | 3 (0.1%) | 1 (0%) | 2 (0%) |
| Sex | |||
| Male | 1797 (48.0%) | 1369 (48.3%) | 1669 (41.4%) |
| Female | 1914 (51.1%) | 1445 (51.0%) | 2329 (57.7%) |
| Unknown | 32 (0.9%) | 21 (0.7%) | 38 (0.9%) |
| Influenza vaccination history >2 weeks prior to illness onset | |||
| Yes | 696 (18.6%) | 934 (32.9%) | 1155 (28.6%) |
| No | 2204 (58.9%) | 1322 (46.6%) | 2026 (50.2%) |
| Reported as unknown | 843 (22.5%) | 579 (20.4%) | 855 (21.2%) |
| Influenza season | |||
| 2010-11 | 1424 (38.0%) | 1176 (41.5%) | 1591 (39.4%) |
| 2011-12 | 472 (12.6%) | 701 (24.7%) | 984 (24.4%) |
| 2012-13 | 1847 (49.3%) | 958 (33.8%) | 1461 (36.2%) |
Pan-negative patients tested negative for influenza virus, respiratory syncytial virus (RSV), parainfluenza viruses (PIV) 1-3, metapneumovirus (MPV), rhinovirus and adenovirus.
We compared influenza vaccination coverage between influenza negative controls, other respiratory virus positive controls and pan-negative controls, stratified by influenza season and age group after multiple imputation (Table 2). There was no difference in vaccine coverage within each age group and influenza season in complete data analysis verses multiple imputation (results not shown). Vaccination coverage was higher among younger children aged 6 months to 5 years and adults aged >50 years, and lowest among young adults aged 18 to 49 years across three influenza seasons. We found no differences in vaccination coverage for each age group by influenza season except among children aged 6 months to 5 years among which vaccination coverage was higher in pan-negative controls (48%) during the 2010-11 season and in other virus positive controls (58%) during 2011-12. We also examined influenza vaccination coverage by other respiratory virus tested and found no association between influenza vaccination and detection of RSV, rhinoviruses, PIV 1-3, MPV, and adenovirus for each age group compared with pan-negative controls (eTable 1).
Table 2.
Comparison of the influenza vaccination coverage among patients testing negative for any influenza virus, patients testing positive for other respiratory virus and patients testing negative for all viruses after multiple imputation, by season and age group. a
| Influenza season and age group | All influenza negative | Other respiratory virus positive | Pan-negative b |
|---|---|---|---|
| 2010-11 Season | 957/2784 (34%) | 422/1176 (36%) | 528/1591 (33%) |
| 6 months- 5 years | 400/911 (44%) | 247/596 (41%) | 151/313 (48%) |
| 6-17 years | 254/735 (35%) | 91/261 (35%) | 161/471 (34%) |
| 18-49 years | 233/972 (24%) | 62/270 (23%) | 168/692 (24%) |
| ≥50 years | 71/165 (43%) | 22/49 (45%) | 48/115 (42%) |
| 2011-12 Season | 721/1692 (43%) | 343/701 (49%) | 375/984 (38%) |
| 6 months- 5 years | 252/463 (54%) | 174/302 (58%) | 78/160 (49%) |
| 6-17 years | 211/494 (43%) | 90/193 (47%) | 120/300 (40%) |
| 18-49 years | 160/555 (29%) | 46/144 (32%) | 112/406 (28%) |
| ≥50 years | 99/180 (55%) | 34/62 (55%) | 65/118 (55%) |
| 2012-13 Season | 907/2431 (37%) | 379/958 (40%) | 526/1461 (36%) |
| 6 months- 5 years | 312/666 (47%) | 203/432 (47%) | 108/230 (47%) |
| 6-17 years | 181/551 (33%) | 58/184 (32%) | 122/362 (34%) |
| 18-49 years | 246/901 (27%) | 70/250 (28%) | 176/648 (27%) |
| ≥50 years | 169/313 (54%) | 49/92 (53%) | 120/221 (54%) |
Missing data on age (n=7), sex (n=91) and vaccination history (n=2286) were imputed 20 times. A total of 10,650 subjects were included.
Pan-negative patients tested negative for influenza virus, respiratory syncytial virus (RSV), parainfluenza viruses (PIV) 1-3, metapneumovirus (MPV), rhinovirus and adenovirus.
For our first analysis, we estimated vaccine effectiveness using conditional logistic regression matching by calendar week. The overall vaccine effectiveness for those testing influenza negative was 47% (95% confidence interval, CI: 42%, 52%) (Table 3). Consistent between each of the control groups, vaccine effectiveness tended to decrease with age. Using each of the three control groups, vaccine effectiveness estimates against influenza A(H3N2) across three seasons fell in the range 50%-60% for children aged 6 months to 5 years while point estimates were generally lower for the other three age groups (Figure 2). The vaccine effectiveness estimates by influenza type/subtype indicated a higher effectiveness against influenza A(H1N1) compared to A(H3N2) and B. Although estimation for adults older than 50 years was not possible, vaccine effectiveness estimates against influenza (H1N1) were all larger than 45% for other age groups. Estimates of vaccine effectiveness using controls testing positive for another respiratory virus were slightly higher than estimates using controls that were influenza negative or pan-negative, but estimates by all control groups had widely overlapping confidence intervals, and differences between estimates were mostly less than 10 percentage points when stratified by season or influenza virus type/subtype (Figure 2). The estimates by three control groups were consistent and no differences by control group were found when stratified by age groups, influenza season, and/or type/subtype (Figure 2, eFigure 1, Table 3).
Table 3.
Estimates of influenza vaccine effectiveness overall, and by age group and influenza type/subtype, 2010-13. a
| Vaccine effectiveness (95% confidence interval) b | |||
|---|---|---|---|
|
| |||
| Test-positive versus all test-negative cases (n=10650) | Test-positive versus other respiratory virus positive cases (n=6578) | Test-positive versus pan-negative cases (n=7779) | |
| Overall | 47% (42%, 52%) | 51% (44%, 57%) | 44% (38%, 50%) |
|
| |||
| Age group | |||
| 6 months- 5 years | 58% (48%, 66%) | 57% (46%, 66%) | 61% (49%, 70%) |
| 6-17 years | 45% (33%, 54%) | 45% (29%, 57%) | 45% (32%, 55%) |
| 18-49 years | 36% (22%, 48%) | 42% (24%, 55%) | 35% (20%, 47%) |
| ≥50 years | 35% (11%, 52%) | 35% (0%, 58%) | 35% (9%, 53%) |
| Influenza season | |||
| 2010-2011 | 40% (29%, 49%) | 39% (24%, 50%) | 41% (29%, 52%) |
| 2011-2012 | 50% (36%, 61%) | 59% (46%, 69%) | 43% (26%, 57%) |
| 2012-2013 | 51% (43%, 58%) | 57% (48%, 64%) | 47% (37%, 55%) |
| Influenza virus type/subtype | |||
| H1N1 | 63% (51%, 72%) | 65% (53%, 74%) | 63% (50%, 73%) |
| H3N2 | 39% (31%, 47%) | 44% (34%, 52%) | 35% (26%, 44%) |
| B | 50% (42%, 58%) | 52% (43%, 60%) | 50% (40%, 57%) |
Missing data on age (n=7), sex (n=91) and vaccination history (n=2286) were imputed (n=10650) 20 times. 73/10650 (0.7%) patients with unsubtypeable influenza A virus infections were excluded from the estimation of VE for A(H1N1) and A(H3N2) but included in the overall estimates.
We adjusted in each regression model for age and sex, and conditioned by calendar week.
Figure 2.
(A) Estimates of influenza vaccine effectiveness by age group and season using three different control groups. (B) Estimates of influenza vaccine effectiveness by age group and influenza type/subtype using three different control groups. VEANY- : control group includes all patients that tested negative for influenza virus; VEORV+: control group includes patients that tested negative for influenza virus and positive for another respiratory virus; VEPAN- : control group includes patients that tested negative for influenza virus and other respiratory viruses. Estimates with confidence intervals wider than 250 percentage points are not shown.
In our second analysis, we further examined the association of influenza vaccine effectiveness by age group and with detection of other respiratory viruses in the early, middle and late phases of each season as defined above. We found no major differences between the three vaccine effectiveness estimates in each phase of each influenza season (eFigure 2). In our third analysis, we found no clear evidence of an association between other respiratory virus detection and receipt of influenza vaccination (Figure 3). Likewise, we found no association between influenza vaccination and other respiratory virus detection when further stratified by age group (Figure 4) or individual respiratory virus detection (eFigure 3). When analyses were restricted to patients meeting the influenza-like illness case definition only, 6792 patients in three influenza seasons were included in statistical analysis, and the findings were very similar and with wider confidence intervals on point estimates (data not shown).
Figure 3.
Estimates of the association between influenza vaccination and detection of other respiratory viruses, quantified by the odds of vaccination in patients testing positive for a respiratory virus divided by the odds of vaccination in patients testing negative for a respiratory virus, in the subset of patients testing negative for influenza virus. Analyses were done overall, and stratified by the early, middle, and late phases of each influenza season, with adjustment for age group and sex and matched by calendar week.
Figure 4.
Estimates of the association between influenza vaccination and detection of other respiratory viruses, quantified by the odds of vaccination in patients testing positive for a respiratory virus divided by the odds of vaccination in patients testing negative for a respiratory virus, in the subset of patients testing negative for influenza virus. Analyses were done overall, and stratified by four age groups, in the early, middle, and late phases of each influenza season, and overall, with adjustment for age group and sex and matched by calendar week.
Discussion
Based on 10650 patients 6 months to 99 years of age, we found no difference in estimates of vaccine effectiveness against any influenza virus when using influenza test negative controls, other respiratory virus positive controls, or pan-negative controls (those who tested negative for both influenza and other respiratory viruses) over a period of three seasons in the United States. Previously, a randomized controlled trial for children aged 6 to 17 years in Hong Kong reported that vaccine effectiveness estimates may differ depending on choice of control groups, and thus needed further research.24 Using surveillance data from the United States in the present study, we addressed the impact of control groups and observed no substantial differences across the three control groups in overall vaccine effectiveness estimates and estimates stratified by age, type/subtype, or season (Table 3, Figure 2), which is consistent with previous theoretical considerations of the study design.26,31 Although comparatively larger differences were observed between vaccine effectiveness estimates based on the three control groups during the 2011-12 season (Figure 2, eFigure 1, eFigure 2), this could be explained by the smaller influenza peak and thus smaller sample size during that season (Figure 1, Table 3). Moreover, no difference in effectiveness estimates based on the three control groups were found in this season overall or when stratified by age group, influenza type/subtype, and early/middle/late phase.
The results of the present study are consistent with other studies that compared alternative control groups when estimating influenza vaccine effectiveness in different countries and settings.7,29,41-44 Four studies examined virus interference from pooled estimates over more than one influenza season: one of these studies recruited children in hospitals in Australia,42 one recruited children from hospitals in Hong Kong,7 one recruited hospitalized pneumonia patients of all ages in the United States43 and one included children and older adults seeking ambulatory care or admitted to hospitals in the US.41 Consistently, no effect of virus interference was detected according to pooled estimates across several influenza seasons in these studies. Moreover, two other studies from Australia and New Zealand examined vaccine effectiveness estimates for a single influenza season by alternative control groups, and very similar estimates were derived using influenza negative controls and other respiratory viruses positive controls.29,44 Three other studies that recruited relatively smaller numbers of patients in single influenza seasons found larger differences in vaccine effectiveness point estimates by alternative control groups with overlapping confidence intervals.21,31,45 Although virus interference was given as a possible explanation for differences in vaccine effectiveness estimates by control group, those studies may have been underpowered to identify real effects. Previous studies have suggested that choosing a control group testing positive for another respiratory virus might be preferred to one based only on testing negative for influenza because detection of other viruses confirms adequate swab quality, especially for older children or adults, who can have lower levels of virus shedding compared to young children.42,45 In our study, we did find that other respiratory viruses were more likely to be detected among young children compared to other age groups, but there were no major differences between vaccine effectiveness estimates using the control group testing positive for other respiratory viruses and the control group testing negative for influenza- (Table 3, Figure 2).
We found consistent detection of other respiratory viruses among vaccinated and unvaccinated patients during three phases of each influenza season (Figure 3), providing further evidence that virus interference did not have any substantial effect on vaccine effectiveness estimates in this study. Based on the hypothesis that unvaccinated persons might have a higher risk of influenza virus infection, but then a higher proportion of unvaccinated persons would have temporary protection around and after the peak in influenza activity due to short-term stimulation of innate non-specific immunity22,46 or reduced exposure during convalescence,25 we examined the association between influenza vaccination and detection of other respiratory virus during the early, middle, and late phases of each influenza epidemic among influenza-negative patients. However, no differential association was found among any age group in each season (Figure 4). This evidence, across three seasons, indicates that virus interference did not appear to have any substantial effect on estimates of vaccine effectiveness using the test-negative design.
While vaccine effectiveness estimates varied by age group, vaccination coverage was quite similar among control groups, with the exception of children aged 6 months to 5 years in the 2010-11 and 2011-12 influenza seasons. We found no association between influenza vaccination and detection of any other respiratory virus by age group, and consistently, our finding suggested no age-group specific association between influenza vaccination and detection with each of the other respiratory viruses tested including RSV, rhinoviruses, PIV 1-3, MPV and adenovirus (eTable 1). Furthermore, there was no association between influenza vaccination and detection of other respiratory viruses after adjusting for age group and sex and conditioning on calendar week (eFigure 3). We found that the risk of other respiratory virus detection was not associated with influenza vaccination for each age group or overall, with point estimates around unity and confidence intervals all across unity (Figure 4). Similarly, the 2013 study by Sundaram et al. examined the association between influenza vaccination and detection of RSV, adenovirus, MPV, PIV 1-4, rhinoviruses, or coronaviruses among children aged 6 months to 5 years and adults elder than 50 years using univariate analysis.41 No association was identified except for PIV 1-4, and the association was reversed between children and adults. Furthermore, a hospital-based study conducted in Spain also examined vaccine effectiveness against RSV-related hospitalization and reported no association between receipt of influenza vaccination and detection of RSV.47
A strength of this study is that the IISP platform incorporates routine molecular testing for influenza and other respiratory viruses, allowing comparison of vaccine effectiveness based on different control groups and estimation of potential of virus interference. A second advantage of this study is the large sample size, which permitted precise estimation of season-specific vaccine effectiveness and vaccine effectiveness by early/middle and late phases of each influenza season in most age groups, which has not been reported in previous studies. We were also able to examine the impact of choices of alternative test-negative groups on a seasonal basis for most age groups (eFigure 1. eFigure 2), and thus provide robust evidence on the potential for virus interference to affect VE estimates. Nevertheless, the difference in point estimates was 7 percentage points between the vaccine effectiveness estimate using the control group testing positive for other respiratory viruses and the estimate using the pan-negative control group (Table 3). While the confidence interval was across 0, and despite our large sample size, we could not rule out small effects of virus interference on vaccine effectiveness. Our study is limited by self-reported data on vaccination status, which precludes evaluation by type of vaccine received and might introduce recall bias. If the bias was non-differential, vaccine effectiveness could be underestimated across three alternative control groups. In addition, we could not adjust for underlying medical conditions, which is a potential confounding factor. Another limitation is that around 20% of patients had missing vaccination records. We conducted multiple imputations by generalized additive model to minimize potential bias and make the most of all available data. The IISP is a large surveillance program; however, within year, age group and specific influenza virus types, we were insufficiently powered to calculate vaccine effectiveness for some subgroups of patients. Finally, patients testing pan-negative may have had false negative results due to low levels of virus shedding or imperfect swab collection, and that would lead to misclassification of outcome status and reduce the power of our study to identify differences between the various control groups.
In conclusion, we did not find any evidence that virus interference affected vaccine effectiveness estimation in a test-negative study using all influenza negative, other virus positive, and all virus negative control groups over three consecutive seasons in the United States. The dynamic interactions of respiratory viruses and their circulation among humans are not well understood, and virus interference remains a controversial phenomenon. Simulation studies have indicated that unless there is very high incidence of influenza virus infections in the community, and prolonged non-specific immunity following one infection, interference is unlikely to cause any major bias in vaccine effectiveness estimates from the test-negative design.28 Our empirical findings also support the use of the test-negative control study design for the assessment of vaccine effectiveness in protecting individuals of all ages against influenza virus infection.
Supplementary Material
eFigure 1. Estimates of influenza vaccine effectiveness using three different control groups, by influenza type/subtype. VEANY-, VEORV+ and VEPAN- are defined in the Figure 2 legend and in the text. Point estimate lower than -50% or confidence interval wider than 250% is not shown.
eFigure 2. Estimates of influenza vaccine effectiveness using three different control groups in the early, middle and late phases of each influenza season as defined in the text, and overall. VEANY- , VEORV+ and VEPAN- are defined in the Figure 2 legend and in the text. Point estimate lower than -50% or confidence interval wider than 250% is not shown.
eFigure 3. Estimates of the association between influenza vaccination and detection of rhinovirus, RSV, PIV 1-3, MPV or adenovirus by age group defined in the text, and overall.
Acknowledgments
The authors would like to thank the IISP Working Group members for their leadership in coordinating the clinical surveillance and laboratory testing that make IISP a successful program, including University of Wisconsin School of Medicine and Public Health: J. Temte, S. Barlow, E. Reisdorf; New York City Department of Health and Mental Hygiene: S. Di Lonardo, J. Fu; New Jersey Department of Health: L. McHugh, P. Bryant; Minnesota Department of Health: R. Lynfield, K. Martin, D. Boxrud; Florida Department of Health: H. Rubino, V. Mock, L. Heberlein-Larson; North Dakota Department of Health: J. Baber, M. Feist; Texas Department of Health: J. Ledbetter, L. Brannan; Los Angeles County Department of Public Health: C. Selzer, N. Green; Philadelphia Department of Public Health: J. Lojo; Iowa Department of Public Health: O. Oni; Virginia Department of Health: K. Kurkjian; Oregon Public Health Division: A. Thomas; Utah Department of Health: R. Boulton; Centers for Disease Control and Prevention: S. Lindstrom, D. Erdman, and B. Whittaker. The authors thank Simon Cauchemez for helpful comments.
Funding: This work was supported by the Council of State and Territorial Epidemiologists (cooperative agreement 5U38HM000414- 04 from the Centers for Disease Control and Prevention). BJC is supported by the Harvard Center for Communicable Disease Dynamics from the National Institute of General Medical Sciences (grant no. U54 GM088558), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. T11-705/14N), and a commissioned grant from the Health and Medical Research Fund, Food and Health Bureau, Government of the Hong Kong Special Administrative Region. The funding bodies had no role in study design, data collection and analysis, preparation of the manuscript, or the decision to publish.
BJC has received research funding from Sanofi Pasteur and consults for Crucell NV.
Footnotes
Potential Conflicts of Interest: The authors report no other potential conflicts of interest.
Disclaimer: The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention (CDC). The use of trade names is for identification only and does not imply endorsement by the Department of Health and Human Services or the CDC.
References
- 1.Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect Dis. 2012;12(1):36–44. doi: 10.1016/S1473-3099(11)70295-X. [DOI] [PubMed] [Google Scholar]
- 2.Jefferson T, Smith S, Demicheli V, Harnden A, Rivetti A, Di Pietrantonj C. Assessment of the efficacy and effectiveness of influenza vaccines in healthy children: systematic review. The Lancet. 2005;365(9461):773–780. doi: 10.1016/S0140-6736(05)17984-7. [DOI] [PubMed] [Google Scholar]
- 3.Tricco AC, Chit A, Soobiah C, Hallett D, Meier G, Chen MH, Tashkandi M, Bauch CT, Loeb M. Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis. BMC Med. 2013;11:153. doi: 10.1186/1741-7015-11-153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kissling E, Valenciano M, Larrauri A, Oroszi B, Cohen J, Nunes B, Pitigoi D, Rizzo C, Rebolledo J, Paradowska-Stankiewicz I. Low and decreasing vaccine effectiveness against influenza A (H3) in 2011/12 among vaccination target groups in Europe: results from the I-MOVE multicentre case-control study. Euro Surveill. 2013 doi: 10.2807/ese.18.05.20390-en. [DOI] [PubMed] [Google Scholar]
- 5.Levy A, Sullivan SG, Tempone SS, Wong KL, Regan AK, Dowse GK, Effler PV, Smith DW. Influenza vaccine effectiveness estimates for Western Australia during a period of vaccine and virus strain stability, 2010 to 2012. Vaccine. 2014;32(47):6312–6318. doi: 10.1016/j.vaccine.2014.08.066. [DOI] [PubMed] [Google Scholar]
- 6.Skowronski D, Masaro C, Kwindt T, Mak A, Petric M, Li Y, Sebastian R, Chong M, Tam T, De Serres G. Estimating vaccine effectiveness against laboratory-confirmed influenza using a sentinel physician network: results from the 2005–2006 season of dual A and B vaccine mismatch in Canada. Vaccine. 2007;25(15):2842–2851. doi: 10.1016/j.vaccine.2006.10.002. [DOI] [PubMed] [Google Scholar]
- 7.Cowling BJ, Chan KH, Feng S, Chan EL, Lo JY, Peiris JM, Chiu SS. The effectiveness of influenza vaccination in preventing hospitalizations in children in Hong Kong, 2009–2013. Vaccine. 2014;32(41):5278–5284. doi: 10.1016/j.vaccine.2014.07.084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Flannery B, Clippard J, Zimmerman RK, Nowalk MP, Jackson ML, Jackson LA, Monto AS, Petrie JG, McLean HQ, Belongia EA. Early estimates of seasonal influenza vaccine effectiveness-United States, January 2015. MMWR Morb Mortal Wkly Rep. 2015;64(1):10–5. [PMC free article] [PubMed] [Google Scholar]
- 9.Sullivan SG, Feng S, Cowling BJ. Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review. Expert Rev Vaccines. 2014;13(12):1571–1591. doi: 10.1586/14760584.2014.966695. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ohmit SE, Thompson MG, Petrie JG, Thaker SN, Jackson ML, Belongia EA, Zimmerman RK, Gaglani M, Lamerato L, Spencer SM. Influenza vaccine effectiveness in the 2011–2012 season: protection against each circulating virus and the effect of prior vaccination on estimates. Clin Infect Dis. 2014;58(3):319–327. doi: 10.1093/cid/cit736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Pebody R, Warburton F, Andrews N, Ellis J, von Wissmann B, Robertson C, Yonova I, Cottrell S, Gallagher N, Green H. Effectiveness of seasonal influenza vaccine in preventing laboratory-confirmed influenza in primary care in the United Kingdom: 2014/15 end of season results. Euro Surveill. 2015;20(5) [PubMed] [Google Scholar]
- 12.Sullivan SG, Cowling BJ. “Crude Vaccine Effectiveness” Is a Misleading Term in Test-negative Studies of Influenza Vaccine Effectiveness. Epidemiology. 2015;26(5):e60. doi: 10.1097/EDE.0000000000000343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jackson ML, Nelson JC. The test-negative design for estimating influenza vaccine effectiveness. Vaccine. 2013;31(17):2165–2168. doi: 10.1016/j.vaccine.2013.02.053. [DOI] [PubMed] [Google Scholar]
- 14.Sullivan SG, Tchetgen Tchetgen EJ, Cowling BJ. Theoretical Basis of the Test-Negative Study Design for Assessment of Influenza Vaccine Effectiveness. Am J Epidemiol. 2016;184(5):345–353. doi: 10.1093/aje/kww064. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Anestad G. Interference between outbreaks of respiratory syncytial virus and influenza virus infection. Lancet. 1982;1(8270):502. doi: 10.1016/s0140-6736(82)91466-0. [DOI] [PubMed] [Google Scholar]
- 16.van Asten L, Bijkerk P, Fanoy E, van Ginkel A, Suijkerbuijk A, van der Hoek W, Meijer A, Vennema H. Early occurrence of Influenza A epidemics coincided with changes in occurrence of other respiratory virus infections. Influenza Other Respi Viruses. 2015 doi: 10.1111/irv.12348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rhedin S, Hamrin J, Naucler P, Bennet R, Rotzén-Östlund M, Färnert A, Eriksson M. Respiratory viruses in hospitalized children with influenza-like illness during the H1N1 2009 pandemic in Sweden. PLoS One. 2012 doi: 10.1371/journal.pone.0051491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Greer R, McErlean P, Arden K, Faux C, Nitsche A, Lambert S, Nissen M, Sloots T, Mackay I. Do rhinoviruses reduce the probability of viral co-detection during acute respiratory tract infections? J Clin Virol. 2009;45(1):10–15. doi: 10.1016/j.jcv.2009.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Casalegno J, Ottmann M, Bouscambert Duchamp M, Escuret V, Billaud G, Frobert E, Morfin F, Lina B. Rhinoviruses delayed the circulation of the pandemic influenza A (H1N1) 2009 virus in France. Clin Microbiol Infect. 2010;16(4):326–329. doi: 10.1111/j.1469-0691.2010.03167.x. [DOI] [PubMed] [Google Scholar]
- 20.Ånestad G, Nordbø SA. Virus interference. Did rhinoviruses activity hamper the progress of the 2009 influenza A (H1N1) pandemic in Norway? Med Hypotheses. 2011;77(6):1132–1134. doi: 10.1016/j.mehy.2011.09.021. [DOI] [PubMed] [Google Scholar]
- 21.Nunes B, Machado A, Guiomar R, Pechirra P, Conde P, Cristovao P, Falcao I. Estimates of 2012/13 influenza vaccine effectiveness using the case test-negative control design with different influenza negative control groups. Vaccine. 2014;32(35):4443–9. doi: 10.1016/j.vaccine.2014.06.053. [DOI] [PubMed] [Google Scholar]
- 22.Khaitov M, Laza‐Stanca V, Edwards M, Walton R, Rohde G, Contoli M, Papi A, Stanciu L, Kotenko S, Johnston S. Respiratory virus induction of alpha‐, beta‐and lambda‐interferons in bronchial epithelial cells and peripheral blood mononuclear cells. Allergy. 2009;64(3):375–386. doi: 10.1111/j.1398-9995.2008.01826.x. [DOI] [PubMed] [Google Scholar]
- 23.Kelly H, Barry S, Laurie K, Mercer G. Seasonal influenza vaccination and the risk of infection with pandemic influenza: a possible illustration of non-specific temporary immunity following infection. Euro Surveill. 2010;15(47) doi: 10.2807/ese.15.47.19722-en. [DOI] [PubMed] [Google Scholar]
- 24.Cowling BJ, Fang VJ, Nishiura H, Chan KH, Ng S, Ip DK, Chiu SS, Leung GM, Peiris JM. Increased risk of noninfluenza respiratory virus infections associated with receipt of inactivated influenza vaccine. Clin Infect Dis. 2012;54(12):1778–1783. doi: 10.1093/cid/cis307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Rohani P, Green C, Mantilla-Beniers N, Grenfell B. Ecological interference between fatal diseases. Nature. 2003;422(6934):885–888. doi: 10.1038/nature01542. [DOI] [PubMed] [Google Scholar]
- 26.Cowling BJ, Nishiura H. Virus interference and estimates of influenza vaccine effectiveness from test-negative studies. Epidemiology. 2012;23(6):930–931. doi: 10.1097/EDE.0b013e31826b300e. [DOI] [PubMed] [Google Scholar]
- 27.Foppa IM, Ferdinands JM, Chaves SS, Haber MJ, Reynolds SB, Flannery B, Fry AM. The case test-negative design for studies of the effectiveness of influenza vaccine in inpatient settings. Int J Epidemiol. 2016:dyw022. doi: 10.1093/ije/dyw022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Foppa IM, Haber M, Ferdinands JM, Shay DK. The case test-negative design for studies of the effectiveness of influenza vaccine. Vaccine. 2013;31(30):3104–3109. doi: 10.1016/j.vaccine.2013.04.026. [DOI] [PubMed] [Google Scholar]
- 29.Pierse N, Kelly H, Thompson MG, Bissielo A, Radke S, Huang QS, Baker MG, Turner N. Influenza vaccine effectiveness for hospital and community patients using control groups with and without non-influenza respiratory viruses detected, Auckland, New Zealand 2014. Vaccine. 2016;34(4):503–509. doi: 10.1016/j.vaccine.2015.11.073. [DOI] [PubMed] [Google Scholar]
- 30.De Serres G, Skowronski D, Wu X, Ambrose C. The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials. Euro Surveill. 2013;18(37):20585. doi: 10.2807/1560-7917.es2013.18.37.20585. [DOI] [PubMed] [Google Scholar]
- 31.Suzuki M, Camacho A, Ariyoshi K. Potential effect of virus interference on influenza vaccine effectiveness estimates in test-negative designs. Epidemiol Infect. 2014;142(12):2642–2646. doi: 10.1017/S0950268814000107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Fowlkes A, Giorgi A, Erdman D, Temte J, Goodin K, Di Lonardo S, Sun Y, Martin K, Feist M, Linz R. Viruses associated with acute respiratory infections and influenza-like illness among outpatients from the Influenza Incidence Surveillance Project, 2010–2011. J Infect Dis. 2014;209(11):1715–1725. doi: 10.1093/infdis/jit806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Cowling BJ, Feng S, Finelli L, Steffens A, Fowlkes A. Assessment of influenza vaccine effectiveness in a sentinel surveillance network 2010-13, United States. Vaccine. 2016;34(1):61–66. doi: 10.1016/j.vaccine.2015.11.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Fowlkes A, Dasgupta S, Chao E, Lemmings J, Goodin K, Harris M, Martin K, Feist M, Wu W, Boulton R. Estimating influenza incidence and rates of influenza‐like illness in the outpatient setting. Influenza Other Respi Viruses. 2013;7(5):694–700. doi: 10.1111/irv.12014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Fowlkes A, Steffens A, Temte J, Di Lonardo S, McHugh L, Martin K, Rubino H, Feist M, Davis C, Selzer C. Incidence of medically attended influenza during pandemic and post-pandemic seasons through the Influenza Incidence Surveillance Project, 2009–13. Lancet Respir Med. 2015 doi: 10.1016/S2213-2600(15)00278-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.MMWR Morb Mortal Wkly Rep. Vol. 62. Centers for Disease Control and Prevention; 2013. Influenza activity--United States, 2012-13 season and composition of the 2013-14 influenza vaccine; p. 473. [PMC free article] [PubMed] [Google Scholar]
- 37.MMWR Morb Mortal Wkly Rep. Vol. 60. Centers for Disease Control and Prevention; 2011. Update: influenza activity--United States, 2010-11 season, and composition of the 2011-12 influenza vaccine; p. 705. [PubMed] [Google Scholar]
- 38.MMWR Morb Mortal Wkly Rep. Vol. 61. Centers for Disease Control and Prevention; 2012. Update: influenza activity-United States, 2011-12 season and composition of the 2012-13 influenza vaccine; p. 414. [PubMed] [Google Scholar]
- 39.Schafer JL. Multiple imputation: a primer. Stat Methods Med Res. 1999;8(1):3–15. doi: 10.1177/096228029900800102. [DOI] [PubMed] [Google Scholar]
- 40.Bond HS, Sullivan SG, Cowling BJ. Regression approaches in the test-negative study design for assessment of influenza vaccine effectiveness. Epidemiol Infect. 2016;144(8):1601–11. doi: 10.1017/S095026881500309X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Sundaram M, McClure D, VanWormer J, Friedrich T, Meece J, Belongia E. Influenza vaccination is not associated with detection of noninfluenza respiratory viruses in seasonal studies of influenza vaccine effectiveness. Clin Infect Dis. 2013;57(6):789. doi: 10.1093/cid/cit379. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Blyth CC, Jacoby P, Effler PV, Kelly H, Smith DW, Robins C, Willis GA, Levy A, Keil AD, Richmond PC. Effectiveness of trivalent flu vaccine in healthy young children. Pediatrics. 2014;133(5):e1218–e1225. doi: 10.1542/peds.2013-3707. [DOI] [PubMed] [Google Scholar]
- 43.Grijalva CG, Zhu Y, Williams DJ, Self WH, Ampofo K, Pavia AT, Stockmann CR, McCullers J, Arnold SR, Wunderink RG. Association between hospitalization with community-acquired laboratory-confirmed influenza pneumonia and prior receipt of influenza vaccination. JAMA. 2015:1488–1497. doi: 10.1001/jama.2015.12160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Sullivan SG, Chilver M, Higgins G, Cheng AC, Stocks NP. Influenza vaccine effectiveness in Australia: results from the Australian Sentinel Practices Research Network. The Medical journal of Australia. 2014;201(2):109–111. doi: 10.5694/mja14.00106. [DOI] [PubMed] [Google Scholar]
- 45.Kelly H, Jacoby P, Dixon GA, Carcione D, Williams S, Moore HC, Smith DW, Keil AD, Van Buynder P, Richmond PC. Vaccine effectiveness against laboratory-confirmed influenza in healthy young children: a case–control study. Pediatr Infect Dis J. 2011;30(2):107–111. doi: 10.1097/INF.0b013e318201811c. [DOI] [PubMed] [Google Scholar]
- 46.Eggo RM, Scott JG, Galvani AP, Meyers LA. Respiratory virus transmission dynamics determine timing of asthma exacerbation peaks: Evidence from a population-level model. Proceedings of the National Academy of Sciences. 2016:201518677. doi: 10.1073/pnas.1518677113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Puig-Barbera J, Diez-Domingo J, Arnedo-Pena A, Ruiz-Garcia M, Perez-Vilar S, Mico-Esparza J, Belenguer-Varea A, Carratalá-Munuera C, Gil-Guillén V, Schwarz-Chavarri H. Effectiveness of the 2010–2011 seasonal influenza vaccine in preventing confirmed influenza hospitalizations in adults: a case–case comparison, case-control study. Vaccine. 2012;30(39):5714–5720. doi: 10.1016/j.vaccine.2012.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eFigure 1. Estimates of influenza vaccine effectiveness using three different control groups, by influenza type/subtype. VEANY-, VEORV+ and VEPAN- are defined in the Figure 2 legend and in the text. Point estimate lower than -50% or confidence interval wider than 250% is not shown.
eFigure 2. Estimates of influenza vaccine effectiveness using three different control groups in the early, middle and late phases of each influenza season as defined in the text, and overall. VEANY- , VEORV+ and VEPAN- are defined in the Figure 2 legend and in the text. Point estimate lower than -50% or confidence interval wider than 250% is not shown.
eFigure 3. Estimates of the association between influenza vaccination and detection of rhinovirus, RSV, PIV 1-3, MPV or adenovirus by age group defined in the text, and overall.