Detectable Risks in Studies of the Fetal Benefits of Maternal Influenza Vaccination

Abstract

Maternal influenza vaccination prevents influenza illness in both mothers and newborns. Results from some recent studies have suggested that influenza vaccination might also prevent adverse pregnancy outcomes, such as preterm birth. However, it is challenging to conduct epidemiologic studies to evaluate the benefits to the fetus of maternal influenza vaccination because the causal benefit of vaccination is likely only experienced by the small fraction of the cohort in whom influenza illness is prevented by vaccination. The plausibility of detecting true differences in risks between groups under such conditions is rarely discussed. We aimed to inform the interpretation of studies in which the fetal benefits of maternal influenza vaccination are evaluated by estimating detectable risk ratios and necessary sample sizes for different study scenarios. Estimates of rates of influenza illness, vaccine effectiveness, vaccine uptake, and preterm birth and of the association of influenza illness with preterm birth were identified from the published literature. We calculated detectable risk ratios for preterm birth in vaccinated versus unvaccinated women and the associated sample size requirements. Our results demonstrated that under most scenarios, plausible differences between groups will be extremely challenging to detect (risk ratios for preterm birth of 0.9 to 1.0) and will require sample sizes infeasible for prospective epidemiologic research. This suggests that the large fetal benefits from influenza vaccination observed in epidemiologic studies are unlikely to be causal.

Influenza vaccination during pregnancy is recommended by numerous public health agencies because of its effectiveness in preventing influenza illness in both mothers and newborn infants (1–4). Maternal influenza vaccination causes no apparent harm to the developing fetus (5), and results from recent studies even suggest that it can improve birth outcomes by reducing the rates of stillbirth, growth restriction, and preterm birth (6–15). The idea of a reduction in adverse fetal outcomes after maternal influenza vaccination is appealing because it would increase support for influenza vaccination during pregnancy, particularly in low-resource settings in which uptake has traditionally been low (1).

However, special considerations are required when conducting studies to evaluate the fetal benefits of maternal influenza vaccination by comparing pregnancy outcomes in vaccinated and unvaccinated women. In studies to measure the harm caused by vaccination, all vaccinated women could plausibly experience an adverse outcome attributable to their vaccine exposure; that is, all women in the exposed (vaccinated) group can be assumed to be at risk of the adverse effect of vaccination (7). In contrast, fetal benefit from vaccination is unlikely to result directly from vaccination per se; rather, it will likely come from the vaccine's ability to prevent maternal influenza illness (7). As shown in Figure 1, the fraction of a cohort whose influenza illness status will be prevented (or reduced in severity) by vaccination is small. Most women do not contract influenza in a given influenza season (the United States Centers for Disease Control and Prevention estimates annual attack rates between 5% and 20% in the general adult population (16)), and not all cases of influenza illness are averted by vaccination (vaccine effectiveness in adults is typically 50%–60% (16)). Investigators seeking to evaluate the fetal benefit from maternal immunization must therefore assume that fetal benefits can only occur among the small fraction of the vaccinated cohort for whom maternal influenza illness was averted because of the vaccination (2.5%–12% of pregnant women vaccinated before the influenza season, based on the rates above (influenza attack rate × vaccine effectiveness)).

Figure 1.

Schematic of influenza illness in women who were and were not vaccinated against influenza. Because the baseline seasonal attack rate for influenza illness is typically 5%–20% (16), most women will not contract influenza illness, irrespective of vaccination status. This fraction of the cohort is denoted as the “never influenza illness” group. Among those who receive the influenza vaccine, not all cases of influenza illness will be averted because the effectiveness of the influenza vaccine is not 100%. This fraction of the cohort is denoted as the “always influenza illness” group. The causal effect of vaccination will only be observed in the fraction of vaccinated women who did not contract influenza illness as a result of the vaccine (but who would have contracted influenza illness if they had not been vaccinated), denoted as the “causal effect of vaccine” group. The fraction of the vaccinated cohort for whom maternal influenza illness was averted because of vaccination is 2.5%–12% of women, assuming the above parameters.

The feasibility of detecting true reductions in risk under this scenario is rarely discussed. Our goal is to inform the design and interpretation of studies in which the fetal benefits of maternal influenza vaccination are evaluated by providing estimates of detectable risks and associated sample size requirements under a range of realistic disease incidence and immunization scenarios.

METHODS

Estimates of plausible rates of influenza illness, vaccine effectiveness, vaccine uptake (i.e., receipt of vaccine), and preterm birth, as well as of the association between maternal influenza illness and fetal outcome, were taken from the literature. The range of and justification for rates used in our calculations are summarized in Web Table 1 (available at http://aje.oxfordjournals.org/); they are intended to cover scenarios for both seasonal and pandemic (e.g., H1N1) influenza. For simplicity, we focused our analyses on preterm birth. However, because small-for-gestational-age birth occurs at a comparable rate (10%, by definition), our results can also be used to inform studies of that outcome. Stillbirth occurs less frequently (<1% in high-income settings, 1.9% globally (17)), which would result in considerably higher sample size requirements.

Detectable risks for the association between maternal influenza vaccine exposure and preterm birth were estimated using the following steps. First, we estimated the fraction of women whose influenza illness status would be altered by receipt of vaccination (i.e., the size of the “causal effect” group in Figure 1) by multiplying the influenza attack rate by the vaccine effectiveness rate. For example, a baseline attack rate of 20% and a vaccine effectiveness of 60% would result in 12% of the vaccinated cohort having influenza illness averted by vaccination (0.2 × 0.6 = 0.12).

Second, we calculated the risks of preterm birth in the vaccinated and unvaccinated cohorts based on women's influenza illness status and the extent to which influenza illness increases the risk of preterm birth. For example, with a 20% influenza attack rate, 60% vaccine effectiveness, 10% baseline risk of preterm birth among women without influenza illness, and risk ratio for preterm birth of 1.5 in women with influenza illness in pregnancy compared with women without influenza illness, the risks can be calculated as follows. In the unvaccinated cohort: 10% preterm births in the 80% of the cohort without influenza illness + 15% preterm births in the 20% of the cohort with influenza illness = 11.0% preterm births in the unvaccinated cohort (i.e., 0.1 × 0.8 + 0.15 × 0.2). In the vaccinated cohort: 10% preterm births in the 80% of the cohort who would not have contracted influenza illness (irrespective of vaccination status) + 10% preterm births in the 12% of the cohort who did not contract influenza illness because they were vaccinated + 15% preterm births in the 8% of the cohort with influenza (despite vaccination) = 10.4% preterm births in the vaccinated cohort (i.e., 0.1 × 0.8 + 0.15 × 0.12 + 0.15 × 0.8).

Next, we calculated the expected risk ratio for preterm birth associated with vaccination by dividing the risk of preterm birth in the vaccinated group by the risk in the unvaccinated group. For the scenario outlined above, the risks of 10.4/100 and 11.0/100 in the vaccinated and unvaccinated cohorts, respectively, corresponded to a risk ratio for preterm birth of 0.95.

Finally, we calculated the sample sizes needed to detect differences between groups given the estimated risk ratios using the Stata, version 13 command sampsi (StataCorp LP, College Station, Texas) for a 2-sample comparison of proportions (based on Pearson's χ² test), with 80% power and an α of 0.05, at various rates of vaccine uptake.

RESULTS

The expected risk ratios for preterm birth in vaccinated versus unvaccinated women under a range of assumptions regarding vaccine effectiveness, influenza attack rate, and risk of preterm birth due to influenza infection ranged from 0.62 to 1.00 (Table 1). For example, under the plausible scenario of the globally estimated 11.1% baseline risk of preterm birth among women without influenza illness, 5% influenza attack rate, 50% vaccine effectiveness and with a moderate association between preterm birth and influenza illness (risk ratio = 1.5), the overall expected risk ratio in vaccinated versus unvaccinated women is 0.99. In general, expected risk ratios were below 0.9 only under the assumption that influenza illness increases the risk of preterm birth by 2-fold or more. The majority of plausible combinations of influenza attack rate, vaccine effectiveness, vaccine uptake, and risk of preterm birth due to influenza illness produced small effect sizes (e.g., risk ratios for preterm birth associated with vaccination between 0.8 and 1.0) that would be challenging to convincingly distinguish from the contributions of residual confounding, measurement error, or selection bias. In Web Table 2, these measures of effect are presented in an alternative format, as the number needed to treat (vaccinate) to prevent 1 preterm birth.

Table 1.

Expected Risks Ratios for Preterm Birth in Vaccinated Versus Unvaccinated Women in Studies of Fetal Benefits of Maternal Influenza Vaccinations

Influenza Attack Rate and Vaccine Effectiveness, %	Percent of Vaccinated Women With Influenza Illness Status Altered by Vaccinationa	Risk Ratio for Preterm Birth Associated With Influenza Illnessb
		1.2	1.5	2.0	4.0
Attack rate of 5%
30	1.5	1.00	0.99	0.99	0.96
40	2	1.00	0.99	0.98	0.95
50	2.5c	1.00	0.99	0.98	0.94
60	3	0.99	0.99	0.97	0.92
70	3.5	0.99	0.98	0.97	0.91
Attack rate of 10%
30	3	0.99	0.99	0.97	0.93
40	4	0.99	0.98	0.96	0.91
50	5	0.99	0.98	0.96	0.89
60	6	0.99	0.97	0.95	0.86
70	7d	0.99	0.97	0.94	0.84
Attack rate of 20%
30	6	0.99	0.97	0.95	0.89
40	8	0.99	0.96	0.93	0.85
50	10	0.98	0.96	0.92	0.81
60	12	0.98	0.95	0.90	0.78
70	14	0.97	0.94	0.88	0.74
Attack rate of 30%
30	9	0.98	0.96	0.93	0.86
40	12	0.98	0.95	0.91	0.81
50	15	0.97	0.94	0.89	0.76
60	18	0.97	0.92	0.86	0.72
70	21	0.96	0.91	0.84	0.67
Attack rate of 40%
30	12	0.98	0.95	0.91	0.84
40	16	0.97	0.93	0.89	0.78
50	20	0.96	0.92	0.86	0.73
60	24	0.96	0.90	0.83	0.67
70	28	0.95	0.88	0.80	0.62

^a Calculated as the product of influenza attack rate and vaccine effectiveness.

^b Expected risks ratios for preterm birth in vaccinated versus unvaccinated women.

^c Corresponds to the percent of women whose influenza illness status was altered by vaccination in the only placebo-controlled, randomized clinical trial of maternal influenza immunization among women not infected with the human immunodeficiency virus (28).

^d Estimated percent of women whose influenza illness status was altered by vaccination in the only other randomized clinical trial of maternal influenza immunization (vs. pneumococcal polysaccharide vaccine control) (27, 29), assuming all febrile respiratory illnesses reported among women who received the comparator vaccine were due to influenza virus infection (7.5%) and assuming that the 63% vaccine efficacy against confirmed influenza illness seen in infants was the same in pregnant women.

The sample sizes needed to detect the Table 1 risk ratios are shown in Figure 2, under the scenarios of 20%, 35%, and 50% vaccine uptake. For example, detecting a significant difference between groups with a risk ratio of 0.99 (which is the detectable risk of preterm birth assuming a 5% influenza attack rate, 50% vaccine effectiveness, and a risk ratio for preterm birth of 1.5) would require a sample size of approximately 2.5 million women (dashed gray line in Figure 2). Alternatively, a fixed sample size of 10,000 pregnancies would be powered to detect a risk ratio of 0.86, which would require the unlikely scenario of a 30% or greater influenza illness attack rate plus a 2-fold higher risk of preterm birth caused by influenza illness (dashed black line in Figure 2).

Figure 2.

Sample sizes needed to detect various risk ratios for preterm birth associated with influenza vaccination at 80% power and α of 0.05, assuming an 11.1% baseline risk of preterm birth among women without influenza illness in pregnancy. Vertical dashed lines illustrate examples of a plausible scenario of a 5% influenza illness attack rate, 50% vaccine effectiveness, and a 1.5-fold increased risk of preterm birth associated with influenza illness (expected risk ratio of 0.99; gray dashed line) and the detectable risk ratio (0.86) under the scenario of a fixed sample size of 10,000 women with 50% vaccine uptake (or equal allocation to immunization and control in a randomized trial; black dashed line).

In a sensitivity analysis in which we assumed an extreme 20% baseline risk of preterm birth (based on the upper limit of the 95% confidence interval for the highest observed regional estimate for preterm birth rate; Web Table 1), the expected risk ratios were similar to those shown in Table 1 (to the second decimal place), whereas the sample sizes needed to detect statistically significant differences between vaccinated and unvaccinated cohorts decreased (Web Figure 1, Web Table 3). For example, a 5% influenza attack rate, 50% vaccine effectiveness, and 1.5-fold higher risk of preterm birth caused by influenza illness would require a sample size of approximately 1 million rather than 2.5 million women, whereas a fixed sample size of 10,000 women could be used to detect a risk ratio of approximately 0.9 rather than 0.86.

DISCUSSION

It has been suggested in several recent studies that maternal influenza vaccination may protect against adverse fetal outcomes such as preterm birth, growth restriction, and stillbirth (6, 8–15, 18). Risk ratios as extreme as 0.63 (95% confidence interval: 0.47, 0.84) and 0.44 (95% confidence interval: 0.21, 0.94) have been reported for preterm birth (14) and stillbirth (12), respectively. Yet, as outlined in the present study, the fraction of women whose influenza illness status is altered through vaccination is small, resulting in extremely small overall differences in fetal risks between vaccinated and unvaccinated women. The small magnitude of detectable effect sizes casts doubt on the plausibility of many previously observed fetal benefits attributable to averted maternal influenza illness.

Results from our analyses suggest that a strong association (≥2-fold) between influenza illness and adverse fetal outcomes is needed to yield risk ratios that are meaningfully different from the null. In a recent systematic review to evaluate the risk of preterm birth associated with influenza illness during pregnancy, investigators found little support for an association of this magnitude (19, 20). Although a risk ratio as high as 4 has been reported, that result reflected severe maternal influenza disease, which would impact only a very small proportion of pregnant women. Moreover, the magnitude of the reported risk ratio was probably inflated by diagnostic ascertainment bias—a concern in epidemiologic studies on this topic (20). For perspective, even a 2-fold higher risk would make influenza illness one of the strongest known exogenous risk factors for preterm birth (21). For example, maternal smoking increases the risk of preterm birth by 1.27-fold (95% confidence interval: 1.21, 1.33) (22), whereas bacterial vaginosis increases the risk by 2-fold (23). Strong links between maternal influenza illness and other fetal outcomes have likewise not been convincingly established (19).

In the absence of a strong causal link between influenza illness and adverse fetal outcomes, findings of fetal benefit from maternal influenza vaccination seem questionable. Confounding due to higher vaccine uptake among women with a more favorable risk profile and/or immortal time bias are more likely explanations for reported protective effects of vaccination in observational studies (7, 24, 25). True benefit can only operate on a small fraction of the population, whereas confounding affects the whole population. It is therefore much easier to detect a spurious effect due to uncontrolled confounding than a true vaccine effect. Implausible effects of influenza vaccination have previously been highlighted in elderly populations in which failure to adequately control for differences in frailty status was shown to produce spurious protective effects from influenza vaccination (26).

Our findings highlight the critical importance of well-designed randomized trials to build the evidence base for influenza vaccination, particularly in low-resource settings in which the burden of influenza illness is likely to be high and in which access to health care may be limited. Although not adequately powered to detect fetal outcomes, 2 randomized trials of maternal influenza vaccination from Bangladesh and South Africa found no significant overall differences in risks of preterm birth in mothers with and without influenza vaccine receipt (27–29). In a post hoc subgroup analysis of births by influenza season in one of the trials, however, researchers observed a large protective effect of vaccination on small-for-gestational-age birth (adjusted odds ratio = 0.44, 95% confidence interval: 0.19, 0.99) (27). Our estimates suggest that this difference is unlikely to result solely from prevention of maternal influenza illness and should prompt consideration of alternative causal mechanisms if the finding is replicated.

Although we present results that cover a broad range of realistic scenarios for disease and immunization rates, our conclusions may not hold outside these ranges. There is the remote possibility that influenza vaccine exerts a protective effect on fetal health through some unknown mechanism that is entirely separate from protection against maternal influenza illness. In studies carried out during periods without circulating influenza virus, investigators have not observed reduced adverse birth outcomes in vaccinated women compared with unvaccinated women, providing some evidence against nonspecific vaccine effects. Nevertheless, the wider confidence intervals observed in the preinfluenza activity period precludes firm conclusions on effects (or lack thereof) during this period (11).

Despite enthusiasm for establishing a protective effect of influenza vaccination on birth outcomes (18), our results suggest that any such effect will be extremely challenging to detect in overall comparisons of vaccinated versus unvaccinated pregnant women. Weak effects between maternal influenza immunization and improved fetal outcomes may exist, but they are likely beyond the resolving power of most epidemiologic research designs. Conclusions on the causality of observed vaccine effects in existing and future studies should be interpreted in the context of our estimates. Finally, there is good evidence that maternal and postnatal newborn immunity benefits from maternal influenza vaccination (28, 29). Thus, any lack of fetal benefit from vaccination should in no way undermine the current recommendations that pregnant women be targeted for influenza vaccination.