Ecological Fallacy, Nonspecific Outcomes, and the Attribution of Disproportionate Vaccine Benefits

To the Editor—In their recent publication, Luca et al [1] used an ecological design to compare all-cause pneumonia hospitalizations and associated costs in Ontario before (1992–2001) and after infant pneumococcal conjugate vaccine (PCV) availability—initially as 7-valent PCV (PCV7) for private purchase (2001–2004), then as publicly funded programs of PCV7 (2005–2009), 10-valent PCV (2009–2010), and 13-valent PCV (2010–2014). The authors attribute large reductions in pneumonia hospitalizations to the direct and indirect effects of infant PCV—reaching a 45% reduction among infants and a comparable reduction of 40% in elderly adults.

However, virtually all of the reduction in pneumonia hospitalizations that Luca et al report among elderly adults accrued during the period when infant PCV7 was only available for private purchase [1, figure 1]. During that period, vaccine coverage was too low to attribute such reduction to the effects of herd immunity. A 40% indirect reduction in elderly adults is also improbable given that 14%–23% of pneumonias among hospitalized adults in Canada are still due to Streptococcus pneumoniae (two-thirds of which were 13-valent PCV serovars) [2]—requiring the unlikely precondition that >50% of all pneumonia cases before 2001 had that cause. Other disproportionate findings reported by the authors include an 80% reduction in pneumonia hospitalizations among children 5-17 years old—particularly puzzling given no reported benefit in children 2-4 years old who were curiously omitted from their Table 3. The latter should have benefitted most directly from a complete infant PCV series and would also be among the most likely to experience indirect benefits from vaccinated siblings or peers.

As shown by Luca et al [1], nonpneumonia hospitalizations gradually and substantially decreased in Ontario between 1992 and 2014 [1, figure 1]. The reasons for that decrease are unexplored, but the authors seem to assume the same endogenous factors equally explain temporal trends in pneumonia and nonpneumonia hospitalizations, and that these are therefore addressed by their difference-of-differences analysis. However, pneumonia hospitalizations showed distinct seasonal variability and, before 2001, did not parallel the steady decline in nonpneumonia hospitalizations in elderly adults. Given interactions between influenza and secondary bacterial pneumonia [3–5], with further modification in influenza risk by age and subtype [6, 7], temporal trends in seasonal influenza epidemics should also have been examined.

Influenza A(H3N2) viruses are associated with more severe epidemics—particularly affecting elderly adults—and were more predominant during the 1990s, whereas A(H1N1) viruses have made greater contribution since 2000 [6–8]. The number of influenza-attributable hospitalizations estimated in Canada was about 80% higher during the 4 seasons 1997–1998 to 2000–2001, compared with the 4 seasons 2001–2002 to 2004–2005, which coincided with private-purchase availability of infant PCV7 [6]. The dramatic reduction in pneumonia hospitalizations that Luca et al attribute to infant PCV may be confounded by these endogenous differences in influenza activity before and after 2000. We note that the same design was previously applied, also in the province of Ontario, to estimate benefits before (1997–2000) and after (2000–2004) the Universal Influenza Immunization Program (UIIP) [9]. Similarly overlooking the natural variations in influenza activity, Kwong et al also reported disproportionate UIIP benefits, including 75% reduction in hospitalizations with just 20% increase in influenza vaccine coverage, and 32% relative reduction compared with provinces without UIIP, despite just a 10% difference in their vaccine coverage rates.

In summary, such ecological studies based on nonspecific outcomes suffer too many flaws to infer causality or to provide accurate quantification of vaccine-attributable benefits or their cost savings.

Note

Potential conflicts of interest. G. D. S. has received grants for investigator-initiated studies unrelated to pneumococcal vaccine from GSK and Pfizer and provided paid expert testimony for the Ontario Nurses Association, the Quebec Ministry of Justice and GSK. D. M. S. has no conflict of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.