The controversy over the efficacy of pneumococcal vaccine

In this issue of CMAJ, Huss and colleagues¹ report the results of their meta-analysis of the efficacy of pneumococcal polysaccharide vaccine in adults. By our reckoning, this is the twelfth of such meta-analyses, including the current Cochrane systematic review,² of which we are coauthors. Each has re-analyzed essentially the same clinical trials but with varied findings depending on which trials were included. Huss and colleagues make a valuable contribution by analyzing the data according to the methodologic quality of the trials. They highlight the importance of high-quality clinical trials and note the difficulty of assessing the methodologic quality of trials that were completed before standard reporting requirements were in place. Their conclusions, however, go well beyond their evidence.

Huss and colleagues suggest that countries with a childhood immunization program that includes the pneumococcal conjugate vaccine reconsider the recommended use of the pneumococcal polysaccharide vaccine in adults. The implication that the polysaccharide vaccine should be withdrawn in such settings exceeds the scope of their evidence. The confidence intervals (CIs) were extremely wide for efficacy against both outcomes related to invasive pneumococcal disease: definitive pneumococcal pneumonia (95% CI 0.05–8.61) and bacteremia (95% CI 0.46–1.77). These were rare events in the evaluated clinical trials, with only 7 cases of definitive pneumococcal pneumonia from 2 studies (n = 794) and 44 cases of bacteremia from 6 studies (n = 32 770).

Unlike our Cochrane review,² Huss and colleagues excluded the 1947 trial by Kaufman³ and the Papua New Guinea trial by Riley and colleagues.⁴ They excluded Kaufman's trial as not being truly randomized because a substantial number of individuals were said to have volunteered to be vaccinated. However, there is no information in the published report to this effect; indeed, Kaufman states that people were “selected at random” each year in the pneumonia season.³ As an unblinded open study, the Kaufman study was not of high quality, but that was not a basis for its exclusion by Huss and colleagues. The study by Honkanen and colleagues⁵ should have been excluded rather than the Kaufman study, because it allowed subjects to choose which vaccine they received although it was “randomized” by year of birth.⁵

The basis for excluding the study by Riley and colleagues⁴ is also questionable. One reason given was that confirmation of pneumococcal infection by lung aspirate had been included as a proxy for invasive pneumococcal disease. Although susceptible to contamination (as is blood culture), lung tissue is considered usually a sterile site; therefore, pneumococcal infection in lung tissue meets the definition of invasive pneumococcal disease used by Huss and colleagues. Even so, data about blood culture could have been readily extracted to meet this definition, since these data were provided separately. The other reason for exclusion was that access to care and diagnostic procedures was limited. One wonders whether this was a post-hoc decision, since it was not specified in the methods. With the study being a randomized controlled double-blind trial, both the intervention and control groups would be expected to have had equal access to care and to diagnostic procedures. As such, these factors should not have introduced bias in the study design.

Whereas the authors excluded infection of lung tissue from their classification of invasive pneumococcal disease, their criteria were much less specific for the diagnosis of pneumonia or presumptive pneumococcal pneumonia. The specificity of radiologically confirmed pneumonia is questionable, but their undefined classification of “typical clinical” pneumonia seems even more unreliable. Huss and colleagues rightly note concerns about the specificity of presumptive pneumococcal pneumonia, but the same could be said for all-cause pneumonia. Analysis of high-quality trials for these 2 outcomes produced inconclusive results. The authors comment that their meta-analysis “did not exclude a detrimental effect of the pneumococcal polysaccharide vaccine on presumptive pneumonia or all-cause pneumonia.” However, this inappropriately suggests that the vaccine increases the risk of these diseases when the evidence does not support this.

We do not dispute the potential of childhood immunization to provide indirect benefit to adults arising from herd immunity. This conclusion, however, is beyond the scope of Huss and colleagues' meta-analysis. The evidence cited to support this claim of indirect benefit arises from observational studies.⁶ However, the authors did not consider other data from observational studies on the direct impact of pneumococcal polysaccharide vaccine on invasive pneumococcal disease. It seems unlikely that childhood vaccination alone will be adequate to prevent invasive pneumococcal disease in adults. The epidemiology of the disease varies by age. Also, there is the emerging possibility of serotype replacement, as reported recently in the Yukon Delta,⁷ where rates of invasive pneumococcal disease among children now exceed prevaccination levels following the emergence of nonvaccine serotypes.

A need exists for new trials to contribute more data on preventing pneumococcal disease in adults. Repeat analyses of the same data are likely to be of limited benefit. When data are available, meta-analyses and systematic reviews provide the best available evidence from a policy-maker's perspective. Meta-analyses need to consider the strength of the evidence and, as Huss and colleagues have shown, the quality of a trial and its reporting. Equally, the quality of the methods of a meta-analysis must be considered.

On balance, the study by Huss and colleagues does not justify ceasing pneumococcal polysaccharide vaccination of adults. The findings of their study were noted in the current World Health Organization position paper on the 23-valent vaccine, but recommendations for the use of the vaccine remained virtually unchanged.⁸ The position of the World Health Organization is that randomized trials, meta-analyses of randomized trials, and most observational studies are consistent with a protective effect against invasive pneumococcal disease among healthy adults and, to a lesser extent, among individuals aged 65 years or more. In the absence of any new data to the contrary, we support that position.

@@ See related research paper by Huss and colleagues, page 48

Key points.

• Meta-analyses of the efficacy of pneumococcal polysaccharide vaccine in adults have produced varied results depending on which trials were included.

• The current meta-analysis by Huss and colleagues is a valuable contribution to the evidence base because it analyzes the data according to the methodologic quality of the trials.

• The authors provide a balanced assessment of their study's strengths and weakness, but their conclusions go beyond the evidence presented.

• The evidence does not support a change in policy for pneumococcal polysaccharide vaccination to prevent invasive pneumococcal disease in adults.