Vaccines for preventing pneumococcal infection in adults

Abstract

Background

Diseases caused by Streptococcus pneumoniae (S. pneumoniae) continue to cause substantial morbidity and mortality globally. Whilst pneumococcal polysaccharide vaccines (PPVs) have the potential to prevent disease and death, the degree of protection afforded against various clinical endpoints and within different populations is uncertain.

Objectives

To assess the efficacy and effectiveness of PPVs in preventing pneumococcal disease or death in adults. We did not assess adverse events.

Search methods

We searched CENTRAL 2012, Issue 6, MEDLINE (January 1966 to June Week 2, 2012) and EMBASE (1974 to June 2012).

Selection criteria

We considered randomised controlled trials (RCTs) in adults, provided the study outcome met the definition of the outcome considered in the review. We also considered non‐RCTs in adults, where the study assessed PPV effectiveness against culture‐confirmed invasive pneumococcal disease (IPD), provided the study controlled for important confounding factors.

Data collection and analysis

Two review authors assessed trial quality of RCTs and three review authors extracted the data. We estimated odds ratios (ORs) and 95% confidence intervals (CIs) using a random‐effects model. Two review authors assessed study quality and extracted data for non‐RCTs. We calculated ORs and 95% CIs using a random‐effects model following the conversion of each study outcome to a log OR and standard error (SE).

Main results

Twenty‐five studies met our inclusion criteria (18 RCTs involving 64,852 participants and seven non‐RCTs involving 62,294 participants). Meta‐analysis of the RCTs found strong evidence of PPV efficacy against IPD with no statistical heterogeneity (OR 0.26, 95% CI 0.14 to 0.45; random‐effects model, I² statistic = 0%). There was efficacy against all‐cause pneumonia in low‐income (OR 0.54, 95% CI 0.43 to 0.67, I² statistic = 19%) but not high‐income countries in either the general population (OR 0.71, 95% CI 0.45 to 1.12, I² statistic = 93%) or in adults with chronic illness (OR 0.93, 95% CI 0.73 to 1.19, I² statistic = 10%). PPV was not associated with substantial reductions in all‐cause mortality (OR 0.90, 95% CI 0.74 to 1.09; random‐effects model, I² statistic = 69%). Vaccine efficacy against primary outcomes appeared poorer in adults with chronic illness. Non‐RCTs provided evidence for protection against IPD in populations for whom the vaccine is currently utilised (OR 0.48, 95% CI 0.37 to 0.61; random‐effects model, I² statistic = 31%). This review did not consider adverse events as it was outside the scope of the review.

Authors' conclusions

This meta‐analysis provides evidence supporting the recommendation for PPV to prevent IPD in adults. The evidence from RCTs is less clear with respect to adults with chronic illness. This might be because of lack of effect or lack of power in the studies. The meta‐analysis does not provide evidence to support the routine use of PPV to prevent all‐cause pneumonia or mortality.

Plain language summary

Vaccination for preventing pneumococcal infection in adults

Streptococcus pneumoniae (S. pneumoniae) is responsible for a considerable burden of illness and death in adults worldwide, usually from pneumonia and less often from invasive pneumococcal disease. Vaccination (using pneumococcal polysaccharide vaccines) might reduce such illness and death. This updated review included 18 randomised controlled trials involving 64,901 participants and seven non‐randomised controlled trials involving 62,294 participants (the latter contributing outcomes for culture‐confirmed invasive pneumococcal disease only).

We found consistently strong evidence that the vaccine is effective in preventing the rarer outcome of invasive pneumococcal disease. Evidence from the included studies indicates vaccination might not afford as much protection in adults with chronic illness as it does for healthy adults. The available evidence does not demonstrate that pneumococcal polysaccharide vaccines prevent pneumonia (of all causes) or mortality in adults. This review did not consider adverse events as it was outside the scope of the study.

Background

Description of the condition

Pneumococcal pneumonia and other diseases caused by Streptococcus pneumoniae (S. pneumoniae) continue to cause substantial morbidity and mortality throughout the world. Pneumonia is the most common presentation of pneumococcal disease in adults. Bacteraemic pneumonia is the most common cause of invasive pneumococcal disease (IPD), accounting for 90% of all cases (Fedson 2004). Mortality associated with invasive pneumococcal pneumonia in adults has remained unchanged at about 20% over the past 60 years (Austrian 1964; Harboe 2009; Rello 2010).

Studies on the epidemiology of pneumococcal disease in high‐income countries indicate that the highest incidence of disease is among the very young and the elderly (Butler 2004). However, some populations experience an increased disease incidence at a young adult age, such as Australian Indigenous persons aged 15 to 49 years, where the incidence of IPD is 10 times greater than the rate in non‐Indigenous population (Menzies 2004). The epidemiology of pneumococcal disease in adults in low‐income countries has not been well described but the global burden of pneumonia in adults is likely to be significantly underestimated (Mulholland 1999).

The continuing burden of pneumococcal disease is worsened by increasing numbers of people with chronic disease or HIV infection and an ageing population in many high‐income countries. Antibiotic resistance continues to present a major threat to the successful treatment of infections (Reacher 2000; Tomasz 1995). In low‐income countries large numbers of people lack access to basic curative health care but might be reached by vaccination programmes.

Several population groups are at elevated risk of pneumococcal disease. Individuals with chronic disease (chronic lung disease, sickle cell anaemia, asplenic patients) or other conditions associated with a compromised immune status have been shown to have increased susceptibility to disease (Butler 2004). Other populations are at elevated risk due to environmental conditions, including overcrowding, exposure to air pollutants such as smoke and differences in serotype distribution (Butler 2004). It is unknown whether the higher rates of IPD in older adults in high‐income countries is due to changes in the immune system, or the presence of underlying disease. A recent Cochrane Review (Walters 2010) failed to show any protective efficacy of pneumococcal polysaccharide vaccine (PPV) in patients with chronic obstructive pulmonary disease, with seven trials included in the meta‐analysis.

Description of the intervention

The search for a vaccine to protect against pneumococcal disease began with the first whole cell vaccine trial involving miners in South Africa, in 1911 (Wright 1914). Following MacLeod's trial of a four‐valent PPV on military recruits in 1944 to 1945 (MacLeod 1945), a six‐valent PPV became available for a short period, from 1946 to 1948 but was withdrawn following the increased use of penicillin. The continued high burden and severity of pneumococcal disease (despite the availability of antibiotics), has led to renewed calls for vaccine development and use. Trials of a six‐valent PPV and a 13‐valent PPV among South African gold miners (Austrian 1976a) and a 14‐valent PPV trial in Papua New Guinean highlanders (Riley 1977), showed strong vaccine efficacy against bacteraemic pneumonia. Where specified, these vaccines contained 50 µG of each purified capsular polysaccharide. A 14‐valent PPV was licensed for use in the United States in 1977; in 1983 this was replaced by a 23‐valent formulation, containing a reduced 25 µG of each purified capsular polysaccharide, without additional pre‐licensure trials.

Nasopharyngeal pneumococcal colonisation plays an essential role in the disease process. It is also recognised for its value in measuring the potential benefits (or harms) of vaccination, including direct effects on immunised individuals as well as indirect effects via transmission between immunised and non‐immunised individuals (herd effects). Changes in nasopharyngeal carriage in response to pneumococcal conjugate vaccine (PCV) indicate a reduction in carriage of vaccine serotypes in immunised individuals and, importantly, a reduction in transmission to the non‐immunised population. These herd effects are reflected in significant reductions in rates of vaccine serotype invasive disease in the non‐immunised population (Whitney 2003). Conversely, replacement of vaccine serotypes by carriage with serotypes not included in the vaccine has been documented in both vaccinated and non‐vaccinated individuals and has been variously reflected in rates of disease caused by non‐vaccine serotypes (Whitney 2003). In contrast to PCV, herd effects of PPV have been poorly described, possibly because the required threshold of vaccine coverage has not been achieved in any studied population. There is a general consensus that PPV does not confer protection against carriage (Makela 2004). In view of the herd effects of PCV and in order to establish any evidence for this assumed consensus, we added this additional outcome to the meta‐analysis when the protocol was amended for the previous Cochrane Review (Moberley 2008).

How the intervention might work

Prior to the discovery of antibiotics, anti‐sera to the polysaccharide capsule was the most effective therapy for pneumococcal pneumonia, reducing mortality to 5% if administered within 24 hours of the onset of symptoms (Casadevall 1994). The discovery of differences in the structure of the polysaccharide capsule (serotypes) and the need for anti‐pneumococcal sera to be specific to each capsular serotype, led to the belief that the most effective protection against pneumococcal disease was opsonisation of the polysaccharide coat with antibody. There are over 90 different serotypes of S. pneumoniae; some are highly invasive, whereas others rarely cause disease. In addition, there is variation in the serotype distribution between age groups and across different geographical populations (Fedson 2004).

Although there is no known immunological correlate of protection against pneumococcal disease, several groups (with some chronic illnesses, immunocompromised conditions and older adults) have been identified as either poor responders to all or some of the pneumococcal polysaccharide serotypes contained within the vaccine. Differences in the immunogenicity of 23‐valent PPV have also been noted across population groups (McMahon 1993) with genetic factors also thought to influence antibody responses to the pneumococcal capsular polysaccharides (Pandey 2000).

Why it is important to do this review

The 23‐valent PPV has been utilised internationally to varying extents but mainly limited to older adults and adults with medical risk factors for IPD in high‐income countries (Fedson 1998). This review updates the previous Cochrane Review (Moberley 2008) and addresses whether PPV is effective in all adult populations or whether only some groups benefit.

Objectives

To assess the efficacy and effectiveness of PPVs in preventing pneumococcal disease or death in adults. We did not assess adverse events. Specifically, the primary aims of this review were to assess:

• the efficacy and effectiveness of PPV in preventing IPD;

• the efficacy of PPV against all‐cause pneumonia; and

• the efficacy against all‐cause mortality in adults.

Secondary aims of this review were to assess vaccine efficacy in preventing:

• definitive and presumptive pneumococcal pneumonia;

• death due to pneumonia or pneumococcal disease; and

• pneumococcal nasopharyngeal colonisation.

Methods

Criteria for considering studies for this review

Types of studies

1. Prospective, randomised controlled trials (RCTs) or quasi‐RCTs that compared PPV with placebo, control vaccines or no intervention.

2. Non‐RCTs that assessed pneumococcal vaccine effectiveness against sterile site, culture confirmed IPD where the trial design allowed for the control of important confounding factors (case‐control and cohort studies). We excluded studies reporting outcomes according to International Classification of Diseases codes.

Types of participants

Adults of either sex, aged 16 years and above. We excluded studies limited to HIV‐positive participants as these are the subject of a Cochrane Protocol (Louie 2000).

Types of interventions

Vaccination with any PPV. We included studies making the following comparisons:

• vaccine compared with placebo;

• vaccine compared with no intervention; and

• a combination of pneumococcal vaccine with a non‐pneumococcal vaccine compared with the other vaccine given alone.

Where reported, we limited disease outcomes to those occurring 14 days or more after vaccination.

Types of outcome measures

Primary outcomes

A. RCTs

• A1. IPD: a pneumococcal infection with S. pneumoniae isolated from a usually sterile body fluid.

• A2. Pneumonia (all‐cause): clinically and radiographically confirmed pneumonia, independent of the cause of pneumonia.

• A3. Mortality (all‐cause).

Secondary outcomes

• A4. IPD (as defined in A1): of a pneumococcal serotype included in the vaccine administered.

• A5. Definitive pneumococcal pneumonia: clinically and radiographically confirmed pneumonia with S. pneumoniae isolated from a usually sterile site.

• A6. Definitive pneumococcal pneumonia (as defined in A5): of a pneumococcal serotype included in the vaccine administered.

• A7. Presumptive pneumococcal pneumonia: clinically and radiographically confirmed pneumonia with S. pneumoniae isolated from a culture of sputum or a nasal swab.

• A8. Presumptive pneumococcal pneumonia (as defined in A7): of a pneumococcal serotype included in the vaccine administered.

• A9. Mortality due to pneumonia (pneumonia as defined in A2).

• A10. Mortality due to pneumococcal infection (as defined in A1).

• A11. Pneumococcal nasopharyngeal colonisation: defined as the detection of S. pneumoniae isolated from a culture from a nose or nasopharyngeal swab.

B. Non‐RCTs

• B1. IPD: a pneumococcal infection with S. pneumoniae isolated from a usually sterile site.

• B2. IPD (as defined in B1): of a pneumococcal serotype included in the vaccine administered.

Search methods for identification of studies

Electronic searches

For this review update we searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2012, Issue 6, part of The Cochrane Library, www.thecochranelibrary.com (accessed 22 June 2012), which contains the Acute Respiratory Infections (ARI) Group's Specialised Register, MEDLINE (from 01 April 2007 to June Week 2, 2012) and EMBASE (from 01 April 2007 to June 2012).

We used the search strategy detailed in Appendix 1 to search CENTRAL and MEDLINE for randomised trials. We combined the MEDLINE search strategy with the Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE: sensitivity‐ and precision‐maximising version (2008 revision); Ovid format (Lefebvre 2011). We adapted the search strategy to search EMBASE (Appendix 2).

To identify non‐randomised studies we adapted the search strategy devised by Fraser (Fraser 2006) to search MEDLINE (see Appendix 3) and EMBASE (Appendix 4). We did not use any language or publication restrictions.

Searching other resources

We did not search additional resources for this update. Please see Appendix 5 for details of the previous searches.

Data collection and analysis

Selection of studies

Two review authors independently reviewed the full text of studies retrieved by the electronic searches for both the RCTs and non‐RCTs (JH, DT and SM, RA, respectively) that appeared to meet the inclusion criteria from the abstract.

Data extraction and management

Two review authors independently extracted data from the RCTs and non‐RCTs (JH, DT and SM, RA, respectively) from the published reports using standardised data extraction forms. We collected study details and outcome data and outlined them in the Characteristics of included studies table.

Assessment of risk of bias in included studies

We assessed the risk of bias of the included studies according to the Cochrane 'Risk of bias' tool (Higgins 2011). Two review authors independently assessed the risk of bias of included RCTs and non‐RCTs (JH, DT and SM, RA, respectively). We discussed any inconsistency in the scores so as to agree on a final score. We scored each trial on the following criteria:

Sequence generation

• Low risk of bias, if the allocation sequence was generated by a computer, a random number table, or similar.

• Uncertain risk of bias, if the trial was described as randomised but the method used for the allocation sequence generation was not described.

• High risk of bias, if an inadequate system was used for the allocation of patients (such as a non‐randomised study).

Allocation concealment

• Low risk of bias, if the allocation of participants involved a central independent unit; on‐site locked computer; or consecutively numbered, sealed envelopes for a randomised study and if study investigators determining participant inclusion unlikely to be aware of vaccination status for non‐randomised studies.

• Uncertain risk of bias, if the trial was described as randomised but the method used to conceal the allocation was not described.

• High risk of bias, if the allocation sequence was known to the investigators who assigned participants.

 Blinding of participants and personnel

• Low risk of bias, if study participants and study personnel were blinded and the method of blinding was described.

• Uncertain risk of bias, if study participants and study personnel were blinded and the method of blinding was not described.

• High risk of bias, if the study participants or study personnel were not blinded.

 Blinding of outcome assessors

• Low risk of bias, if the outcome assessors were blinded and the method of blinding was described.

• Uncertain risk of bias, if the outcome assessors were blinded and the method of blinding was not described.

• High risk of bias, if the outcome assessors were not blinded.

 Incomplete data outcomes

• Low risk of bias, if any post‐randomisation drop‐outs or withdrawals, if they occurred, were clearly described and the reasons described.

• Uncertain risk of bias, if it was not clear whether there were any drop‐outs or withdrawals or if the reasons for these drop‐outs were not clear.

• High risk of bias, if the reasons for missing data were likely to be related to the outcomes, such as if ‘as‐treated’ analysis was performed; or patients with missing data were likely to induce clinically relevant bias in vaccination effect size.  

 Selective outcome reporting

• Low risk of bias, if all the predefined (primary and secondary) outcomes mentioned in the trials' protocol or in the design article were reported and the reporting had been done in the prespecified way.

• Uncertain risk of bias, if there was insufficient information to assess whether a risk of selective outcome reporting was present.

• High risk of bias, if not all the prespecified outcomes were reported, if the primary outcomes were changed, or if some of the important outcomes were incompletely reported.

Besides investigating each bias domain, we also evaluated the overall risk of bias for the study. When we judged sequence generation, allocation concealment and blinding criteria to be low risk, we classified the study as being at overall low risk of bias.

In addition to the above criteria, we assessed non‐RCTs for their control of confounding factors. We determined a pre‐determined list of important confounders (age, sex, smoking status, chronic illness, nursing home residence and influenza vaccination status). Judgement included consideration of the proportion of important confounding factors, whether the most important confounders were considered and balance of these between groups at baseline.

A study was assessed as:

• low risk of bias, if all important confounding factors were balanced at baseline or measured and controlled for in the analysis;

• uncertain risk of bias, if the control of confounding factors was not described (these studies would have been excluded); and

• high risk of bias, if the most important confounding factors were not adequately controlled for (these studies would have also have been excluded).

Measures of treatment effect

For the analysis of RCTs, we summarised vaccine efficacy as odds ratios (ORs) and 95% confidence intervals (CIs). We used random‐effects models. We examined publication bias using funnel plots. For the analysis of non‐RCTs, we summarised effectiveness following the conversion of each study outcome to a log OR and standard error (SE). We used random‐effects models. We conducted analyses using RevMan 2011. Where results were also presented as estimates of vaccine effectiveness, this has been calculated as 100 (1 ‐ pooled OR). This formula has also been used for the RCT component of the review, as an approximation for vaccine efficacy, as the risk of the included outcomes is so low. The rationale for using OR was based on the inclusion of non‐randomised studies (such as matched case‐control studies), where a simple comparison of the number of cases in vaccinated compared to unvaccinated is invalid.

Unit of analysis issues

We analyzed RCT and non‐RCTs separately. There were no special issues in the analysis of studies with non‐standard designs.

Dealing with missing data

Whenever we identified non‐reporting or partial reporting of data we attempted to contact the corresponding trial author and requested missing data.

Assessment of heterogeneity

We calculated the I² statistic for each pooled estimate to assess the effect of statistical heterogeneity.

Assessment of reporting biases

We assessed possible publication bias through visual inspection of funnel plots.

Data synthesis

We pooled studies regardless of which vaccine valency was utilised within the study. We used random‐effect models throughout to take account of the between‐study variance in our findings.

Subgroup analysis and investigation of heterogeneity

A. RCTs

We conducted a subgroup analysis of the RCTs according to pre‐specified characteristics of trials that were considered clinically relevant and would lead to recommendations for vaccination according to populations at risk of pneumococcal disease due to different factors. True pre‐specification was not possible due to authors' prior knowledge of trials but was conducted prior to determining the number of participants in each group. We conducted an analysis for the following three subgroups of participants.

1. Otherwise healthy adults in low‐income countries. This population group included otherwise healthy adults who were likely to be at greater risk of pneumococcal disease than their counterparts in high‐income countries due to environmental factors such as overcrowding and exposure to smoke, together with likely differences in serotype distribution.

2. Adults with chronic illness in high‐income countries. This population group was likely to be at elevated susceptibility to disease and potentially had suboptimal vaccine immunogenicity.

3. Otherwise healthy adults in high‐income countries. This population group included participants from high‐income countries who were not recruited on the basis of underlying disease. We refer to these participants as 'otherwise healthy adults' and expected that they may have a better immune response to vaccination than their counterparts in subgroup ii).

We performed this subgroup analysis for the primary outcomes that included the larger number of studies (at least 10). We investigated differences between subgroups according to the method described by Deeks 2001 whereby significance of difference between groups was measured by Chi² = Chi²_{IPD All} ‐ (Chi²_{IPD Grp1} + Chi²_{IPD Grp2} + Chi²_{IPD Grp3}).

B. Non‐RCTs

We conducted subgroup analysis of the non‐RCTs according to study type (case‐control or indirect cohort study compared to cohort study) using the methods described by Deeks 2001. Case‐control studies are thought to be more susceptible to bias and, as such, may give an elevated protective effect. We performed analyses on two subgroups of participants from non‐RCTs:

1. studies on immunocompetent participants; and

2. studies on immunocompetent older adults.

Study participants were considered immunocompetent if they were not severely immunocompromised. Unless they could be identified within the study, non‐RCTs that included immunocompromised participants (those with haematologic cancers, or receiving prednisolone) were excluded from this analysis. Study participants were considered to be immunocompetent older adults if they met the immunocompetent definition (above) and all study participants were above 55 years of age, or if adjusted analysis for this age group was reported.

Sensitivity analysis

We conducted sensitivity analysis on studies deemed to have an overall low risk of bias and studies not utilising influenza vaccine as a control vaccine (since this vaccine may potentially have had a beneficial effect on the outcomes of interest (Jefferson 2010) for the primary outcomes of the review (IPD, all‐cause pneumonia and all‐cause mortality).

Results

Description of studies

Results of the search

For this updated review, we screened a total of 1817 (299 RCTs and 1518 non‐RCTs) titles from the databases searched (Cochrane Central Register of Controlled Trials (CENTRAL) 2012, Issue 6, MEDLINE (January 1966 to June Week 2, 2012) and EMBASE (1974 to June 2012) (Appendix 1; Appendix 2; Appendix 3; Appendix 4). We reviewed 19 studies in full text (six RCTs and 13 non‐RCTs). Three additional RCTs met our inclusion criteria and have been considered in this updated review (Furomoto 2008; Kawakami 2010; Maruyama 2010), bringing the total number of included studies to 25 (18 RCTs involving 64,901 participants and seven non‐RCTs involving 62,294 participants). In addition, a publication by Klugman (Klugman 2011) reported additional data for a study that was already included in the review (Austrian 1976a). Further details on individual trials are provided in the Characteristics of included studies tables.

Included studies

A. RCTs

The RCTs included in the meta‐analysis were conducted among diverse population groups, which we have classified into three subgroups. Firstly, otherwise healthy adults from low‐income countries; this group consisted of African mine workers (Austrian 1976a; Smit 1977a; Smit 1977b) and community‐dwelling adults residing in the highlands of Papua New Guinea (Riley 1977), considered to have been at elevated risk of disease due to overcrowding and environmental factors. These studies were conducted with vaccines containing six to 14 pneumococcal polysaccharide serotypes.

The second group were those populations that were considered to be at elevated risk of pneumococcal disease due to underlying medical illnesses such as chronic obstructive pulmonary disease (Alfageme 2006; Davis 1987; Furomoto 2008; Kawakami 2010; Leech 1987) or patients with bronchogenic carcinoma (Klastersky 1986). Simberkoff 1986 recruited participants on the basis of age (> 55 years) and the presence of underlying chronic illness. Given the number of deaths in both participant groups, this group was considered very high risk. Collectively, these studies utilised either the 14‐valent or 23‐valent PPVs in developed country settings: Spain, USA, Canada, Belgium, Japan and Denmark.

The third population group of the included RCTs were participants in high‐income countries who were not recruited on the basis of underlying disease. These studies were conducted with vaccines containing two to 23‐valent pneumococcal polysaccharide serotypes. The studies included participants at elevated risk of pneumococcal disease due to their age and place of residence; institution‐based in New York (Kaufman 1947), hospices or retirements homes (Gaillat 1985; Maruyama 2010); older adults aged 50 to 85 years with previous hospital admission for community‐acquired pneumonia in Sweden (Ortqvist 1998); and community‐based older adults in Finland (Koivula 1997). The two other trials were conducted in the United States among participants whose ages were not specified: adult inpatients were recruited from a psychiatric hospital (Austrian 1980a); and adults members were recruited from the Kaiser Permanente Health Plan (Austrian 1980b).

The most commonly reported outcome from the RCTs was all‐cause radiologically confirmed pneumonia (16 studies) followed by all‐cause mortality (14 studies) and IPD (11 studies). Five studies reported on vaccine‐type IPD and four studies reported outcomes on vaccine‐type definitive pneumonia. Mortality due to pneumonia was reported in nine studies and pneumococcal‐specific mortality was reported in four studies. Pneumococcal nasopharyngeal colonisation was an outcome in two of the included studies but could not be included in this review due to incomplete reporting.

B. Non‐RCTs

Five case‐control studies and two large cohort studies were included in the meta‐analysis of non‐RCTs. They considered the 14‐valent and 23‐valent PPVs.

Three case‐control studies that were set in the United States included participants aged from 18 years with medical conditions that placed them at higher risk of pneumococcal disease, or participants who were above 65 years of age (Benin 2003; Shapiro 1984; Shapiro 1991). The other two case‐control studies related only to older adults: Sims 1988 recruited immunocompetent adults from 55 years of age (United States); and Dominguez 2005 included older adults from 65 years of age (Spain). The two cohort studies both contained large numbers of participants aged 65 years and above. One assessed 47,365 members of a Group Health Co‐operative in the United States, over a three‐year period (Jackson 2003); the other followed up 11,241 community‐dwelling Spanish residents for just over three years (Vila‐Corcoles 2006). Both of these studies considered the 23‐valent PPV.

Excluded studies

For this update, we excluded three RCTs and 13 non‐RCTs (please see Characteristics of excluded studies). Two RCTs were excluded as they were only available in abstract form, where a full assessment of outcome definitions or trial quality could not be determined (Teramoto 2007; Ya Tseimakh 2006). One RCT was excluded as we were unable to separate the role of steroids in the study (Steentoft 2007). All non‐RCTs were excluded due to not considering culture confirmed IPD as an outcome, including one study which used ICD codes to diagnose IPD.

Risk of bias in included studies

The risk of bias in included studies is described within the Characteristics of included studies tables and summarised graphically in Figure 1 and Figure 2.

1.

'Risk of bias' summary: review authors' judgements about each risk of bias item for each included randomised study.

2.

'Risk of bias' summary: review authors' judgements about each risk of bias item for each included non‐randomised study.

Most RCTs scored poorly across the factors assessed and only four trials were deemed to have an overall low risk of bias (Austrian 1980b; Maruyama 2010; Ortqvist 1998; Riley 1977).

Thirty‐nine per cent of trials described an adequate method of sequence generation and allocation concealment, 44% of trials reported adequate blinding of participants and personnel and 67% reported adequate blinding of outcome assessors, 22% scored low risk of bias for incomplete outcome data, and 17% scored low risk of bias on selective reporting.

The poor scores were more common in the earlier trials and were largely due to inadequate reporting rather than known inadequate methods.

Non‐RCTs

Non‐RCTs scored low risk of bias against allocation concealment (as adequate control of confounding factors was a study inclusion criterion). The predetermined important confounding factors were age, chronic illness, smoking, influenza vaccination and nursing home residency (Table 3). The five case‐control studies matched participants according to the presence of underlying disease (severity and number of conditions) and date of hospital admission. All studies apart from Sims 1988 reported matching according to age.

1. Risk of bias for non‐randomised studies.

Confounder	Jackson 2003	Vila‐Corcoles 2006	Benin 2003	Dominguez 2005	Shapiro 1991	Shapiro 1984
Age	Yes	Yes	Yes	Yes	Yes	Yes
Sex	Yes	Yes	Yes	Yes	No	No
Chronic Illness	Yes	Yes	Yes	Yes	Yes	Yes
Smoking	Yes	Yes	No	No	No	No
Influenza vaccination	Yes	Yes	Yes	No	No	No
Nursing home resident	Yes	Yes	No	No	No	No

Both cohort studies followed participants for three years and controlled for age, sex and underlying medical conditions; they included compromised immune status, smoking status and influenza vaccination status in the model of effectiveness of PPV. Vaccination was a time variable factor and participants were considered to be vaccinated 14 days following vaccine administration.

Effects of interventions

A. Randomised controlled trials (RCTs)

Outcome A1. Invasive pneumococcal disease (IPD)

We included 11 studies involving 36,489 participants for this outcome, with 15 events in the vaccinated group and 63 events in the control group. Pneumococcal polysaccharide vaccine (PPV) reduced the risk of all IPD with a pooled estimated odds ratio (OR) of 0.26 (95% confidence interval (CI) 0.14 to 0.45; random‐effects model), that is, a protective vaccine efficacy of 74% (95% CI 55% to 86%). Statistical heterogeneity was not present (I² = 0%, P = 0.56).

Subgroups: outcome A1

(i) Otherwise healthy adults in low‐income countries

Riley 1977 was the only study included in this subgroup analysis, involving 5373 participants with two events in the vaccinated group compared to 14 events in the control group. There was evidence of protective efficacy against IPD for this subgroup with OR 0.14 (95% CI 0.03 to 0.61).

(ii) Adults with chronic disease in high‐income countries

Five studies involving 3230 participants were included in this subgroup analysis with four events in the vaccinated group compared to two events in the control group. There was no evidence of protective efficacy (or any harm) from vaccination against IPD in this subgroup. However, as demonstrated by the large CIs, there was a lack of power to demonstrate a significant difference between the vaccinated group and the control groups (OR 1.56, 95% CI 0.35 to 6.94; random‐effects model, I² = 0%).

(iii) Otherwise healthy adults in high‐income countries

Five studies involving 27,886 participants were included in this subgroup analysis with nine events in the vaccinated group and 47 events in the control group. There was evidence of protective efficacy against IPD in this subgroup (OR 0.20, 95% CI 0.10 to 0.39; random‐effects model, I² = 0%).

The difference between the heterogeneity for all studies compared to subgroups i, ii and iii was statistically significant (Chi² = 6.7, 1 df, P < 0.01) indicating that the pooled estimate may not be representative for each population group.

Outcome A2. Pneumonia (all‐cause)

We included 16 studies involving 47,734 participants for this outcome, with 978 events in the vaccinated group and 1547 events in the control group. PPV was shown to be effective against all‐cause pneumonia with a pooled estimated OR of 0.72 (95% CI 0.56 to 0.93; random‐effects model). However, there was a high level of statistical heterogeneity present amongst the included studies (I² = 85%, P < 0.00001).

Subgroups: outcome A2

(i) Adults in low‐income countries

Four studies involving 14,562 participants were included in this subgroup analysis with 158 events in the vaccinated group compared to 548 events in the control group. Pooled estimates showed evidence of protective efficacy against all‐cause pneumonia in this population subgroup (OR 0.54, 95% CI 0.43 to 0.67; random‐effects model, I² = 19%).

(ii) Adults with chronic illness in high‐income countries

Six studies involving 4010 participants were included in this subgroup analysis with 170 events in the vaccinated group compared to 181 events in the control group. As with other outcomes in this population subgroup, wide CIs highlighted the lack of power to demonstrate protective efficacy (or lack thereof) against all‐cause pneumonia (OR 0.93, 95% CI 0.73 to 1.19; random‐effects model, I² = 10%).

(iii) Adults in high‐income countries

Six studies involving 29,186 participants were included in this subgroup analysis with 650 events in the vaccinated group and 818 events in the control group. There was no evidence of protective efficacy against all‐cause pneumonia for this subgroup, although it again should be noted that the CIs were wide (OR 0.71, 95% CI 0.45 to 1.12; random‐effects model). There was also a high level of statistical heterogeneity present (I² = 93%).

As with IPD, the difference between the heterogeneity for all studies compared to groups i, ii and iii was statistically significant (Chi² 19.23, 2 df, P < 0.001), indicating that the pooled estimate may not be representative for each population group.

Outcome A3. Mortality (all‐cause)

We included 14 studies involving 47,560 participants for this outcome, with 1018 events in the vaccinated group and 1039 in the control group. There was no evidence of protective efficacy against all‐cause mortality, with a pooled estimated OR of 0.90 (95% CI 0.74 to 1.09; random‐effects model). A high level of statistical heterogeneity was present (I² = 69%, P < 0.0001).

Subgroups: outcome A3

(i) Adults in low‐income countries

Riley 1977 was the only study included in this subgroup, involving 11,958 participants with 133 events in the vaccine group and 170 in the control group (OR 0.79, 95% CI 0.62 to 0.99).

(ii) Adults with chronic illness in high‐income countries

Six studies involving 3603 participants were included in this subgroup analysis with 263 events in the vaccinated group compared to 231 events in the control group. Whilst there were more deaths in the vaccinated group than in the control group the pooled estimate failed to demonstrate protective efficacy (or harm) in this subgroup (OR 1.13, 95% CI 0.90 to 1.43; random‐effects model, I² = 6%).

(iii) Adults in high‐income countries

Seven studies involving 32,023 participants were included in this subgroup analysis with 622 events in the vaccinated group and 638 events in the control group. There was no evidence of a protective effect against all‐cause mortality with a pooled estimated OR of 0.88 (95% CI 0.67 to 1.17; random‐effects model, I² = 79%).

As with IPD and all‐cause pneumonia, the difference between the heterogeneity for all studies compared to groups i, ii and iii was statistically significant (Chi² = 7.49, 1 df, P < 0.05), indicating that the pooled estimate may not be representative for each population group.

Secondary outcomes

IPD (subgroups)

Outcome A4. Vaccine‐type IPD

We included five studies involving 31,223 participants for this outcome, with 14 events in the vaccinated group and 140 in the control group. The pooled estimate showed vaccination to be effective for this very specific outcome (OR 0.18, 95% CI 0.10 to 0.31). Statistical heterogeneity was absent (I² = 0%, P = 0.70).

Outcome A5. Definitive pneumococcal pneumonia

We included 10 studies involving 35,483 participants for this outcome, with 15 events in the vaccinated group compared to 60 events in the control group. PPV reduced the risk of definitive pneumococcal pneumonia, with a pooled estimated OR of 0.26 (95% CI 0.15 to 0.46; random‐effects model). The protective vaccine efficacy of 74% (95% CI 54% to 85%) was very similar to the size of the effect for outcome A1, which is likely to be due to the large amount of overlap for studies contributing to both outcomes. Statistical heterogeneity was absent (I² = 0%, P = 0.48).

Outcome A6. Definitive PPV (vaccine‐types only)

We included four studies involving 30,561 participants for this outcome, with three events in the vaccinated group and 30 events in the control group. The pooled estimate showed the vaccine to be highly effective for this outcome (OR 0.13, 95% CI 0.05 to 0.38; random‐effects model). Statistical heterogeneity was absent (I² = 0%, P = 0.92).

All‐cause pneumonia (subgroups)

Outcome A7. Presumptive pneumococcal pneumonia

We included nine studies involving 20,335 participants for this outcome, with 100 events in the vaccinated group and 276 in the control group. The pooled estimate showed the vaccine to be effective against this outcome (OR 0.46, 95% CI 0.25 to 0.84; random‐effects model). As with all‐cause pneumonia, a high level of statistical heterogeneity was present among the studies included for presumptive pneumococcal pneumonia (I² = 75%, P < 0.0001).

Outcome A8. Presumptive pneumococcal pneumonia (vaccine‐types only)

We included five studies involving 18,568 participants for this outcome, with 19 events in the vaccinated group and 130 in the control group. The pooled estimate showed vaccination was effective against this outcome with an OR of 0.27 (95% CI 0.08 to 0.87; random‐effects model). Again a high level of statistical heterogeneity was present (I² = 70%, P = 0.01).

All‐cause mortality (subgroups)

Outcome A9. Mortality due to pneumonia

We included nine studies involving 30,723 participants for this outcome, with 135 events in the vaccinated group and 221 events in the control group. However, the pooled estimate failed to demonstrate protective efficacy against pneumonia‐related mortality (OR 0.71, 95% CI 0.44 to 1.16; random‐effects model) with a high level of statistical heterogeneity again present (I² = 72%, P = 0.0004).

Outcome A10. Mortality due to pneumococcal infection

We included three studies involving 2445 participants for this outcome, with five events in the vaccinated group compared to one event in the control group. Given the small number of events for the sample size, there was insufficient power to demonstrate either a reduction or increase in the risk of death from pneumococcal infection among those who had been vaccinated (OR 2.51, 95% CI 0.45 to 14.13; random‐effects model; I² = 0%).

Outcome A11. Pneumococcal nasopharyngeal colonisation

Although three studies reported outcomes for pneumococcal nasopharyngeal colonisation (Austrian 1976a; Riley 1977; Simberkoff 1986) neither could be included in the meta‐analysis of this outcome. Riley 1977 reported on a subgroup of healthy persons (151/2713 vaccine recipients and 181/2660 placebo recipients) but gave no details on how this subgroup had been selected. Simberkoff 1986 reported data on 10,961 throat swabs taken from 2295 participants. Fifty‐four vaccine recipients and 56 placebo recipients were reported as having transient colonisation but no details were given for prevalence of colonisation across the two groups for one time period. Klugman 2011 reported data from the Austrian study (Austrian 1976a) demonstrating a lower rate of vaccine type carriage as a proportion of any pneumococcal carriage (excluding serotype 3) in six and 13‐valent vaccine recipients compared to both the meningococcal and saline control group respectively (11/160, 6.9% compared to 33/144, 22.9% (P < 0.001) and 26/113 23% (P = 0.002) and 6/33, 18.3% compared to 10/28, 35.7% (P = 0.15) and 18/35, 51% (P = 0.0054) respectively). These data could not be included in a meta‐analysis as no individual denominator data are provided.

Sensitivity analysis on RCTs

Quality score

For the sensitivity analysis based on trial risk of bias, only Austrian 1980b, Maruyama 2010, Ortqvist 1998 and Riley 1977 scored as overall low risk of bias. Retaining only these trials deemed to have a ‘low risk of bias’ did not alter the significantly protective effect against IPD but the significance of the effect against all‐cause pneumonia was lost. For all‐cause pneumonia, the level of statistical heterogeneity remained high. All‐cause mortality remained non‐significant when limited to those trials deemed to have a ‘low risk of bias’.

Outcome A1. IPD

OR 0.15 (95% CI 0.05 to 0.43; random‐effects model, I² = 0%).

Outcome A2. All‐cause pneumonia

OR 0.82 (95% CI 0.59 to 1.14; random‐effects model, I² = 76%).

Outcome A3. All‐cause mortality

OR 0.93 (95% CI 0.77 to 1.13; random‐effects model, I² = 22%).

Influenza vaccine as a control

Five included studies used influenza vaccine. Two studies (Koivula 1997; Leech 1987) gave the vaccine to both PPV intervention and control participants and as such are only able to measure the incremental benefit of pneumococcal vaccination. Furomoto 2008, Gaillat 1985 and Kawakami 2010 used influenza vaccine in control participants only. Following the exclusion of these three studies, the estimate of protective effects of PPV against IPD and pneumonia remained significant. The result for all‐cause mortality was only slightly affected. The level of statistical heterogeneity remained high for both all‐cause pneumonia and all‐cause mortality.

Outcome A1. IPD

OR 0.25 (95% CI 0.14 to 0.45; random‐effects model, I² = 0%).

Outcome A2. All‐cause pneumonia

OR 0.73 (95% CI 0.55 to 0.97; random‐effects model, I² = 87%).

Outcome A3. All‐cause mortality

OR 0.88 (95% CI 0.69 to 1.11; random‐effects model, I² = 75%).

Publication (small study) bias

We assessed publication bias using funnel plots and presented this for the three primary outcomes of IPD, all‐cause pneumonia and all‐cause mortality. For each of these outcomes the funnel plots were asymmetrical.

The funnel plot of studies contributing to the outcome of IPD (Figure 3) showed asymmetry around the pooled point estimate. However, it was not the smaller studies contributing larger treatment effects and, therefore, asymmetry may be due to other factors such as heterogeneity between the population groups.

3.

Funnel plot for outcome 1. Invasive pneumococcal disease

The funnel plot of studies contributing to the outcome of all‐cause pneumonia (Figure 4) also showed asymmetry around the pooled point estimate. There was an absence of smaller studies contributing less effective outcomes. This did not appear to be having a large impact on the pooled result as the larger studies appeared to be symmetrical around no effect.

4.

Funnel plot for outcome 2. Pneumonia, all causes

The funnel plot of studies contributing to the outcome of all‐cause mortality (Figure 5) also showed asymmetry around the pooled point estimate. Kaufman 1947 was a clear outlier and two other smaller studies show a larger effect but again these studies did not appear to have a large impact on the pooled results.

5.

Funnel plot for outcome 3. Mortality, all causes

B. Non‐RCTs

Outcome B1. IPD

Seven studies were included in this outcome. PPV reduced the risk of all IPD with a pooled estimate OR of 0.48 (95% CI 0.37 to 0.61; random‐effects model). Statistical heterogeneity was low (I² = 31.4%, P = 0.19).

Subgroup outcome B1. Immunocompetent

A subgroup analysis of immunocompetent participants included six studies. PPV reduced the risk of all IPD in immunocompetent adults with a pooled estimated OR of 0.41 (95% CI 0.32 to 0.52; random‐effects model). Statistical heterogeneity was low (I² = 18%, P = 0.30).

Subgroup outcome B1. Immunocompetent older adults

A subgroup analysis of immunocompetent older adults included five studies. PPV reduced the risk of all IPD in immunocompetent older adults with a pooled estimated OR of 0.32 (95% CI 0.22 to 0.47; random‐effects model). Statistical heterogeneity was absent (I² = 0%, P = 0.68).

Subgroup analysis of study type

Subgroup analysis for IPD (all types) was conducted by study type (case‐control or cohort). For case‐control studies, the pooled estimate of vaccine effectiveness was not significantly different from that determined from the cohort studies (OR 0.47, 95% CI 0.32 to 0.68; and OR 0.57, 95% CI 0.36 to 0.89, respectively). The difference between the heterogeneity for all studies compared to case‐control and cohort studies did not reach statistical significance (Chi² = 2.13, 1 df, P > 0.1).

Outcome B2. Vaccine‐type IPD

Three studies were included in this outcome, with a pooled estimated OR of 0.45 (95% CI 0.38 to 0.54; random‐effects model; I² = 0%). The estimate for this outcome was similar to that for all studies, although with a narrower CI.

Subgroup outcome B2. Immunocompetent

Of the two studies that considered vaccine‐type IPD in immunocompetent participants, pooled results showed an OR of 0.40 (95% CI 0.29 to 0.54; random‐effects model; I² = 0%).

Subgroup outcome B2. Immunocompetent older adults

Only one study reported on vaccine‐type IPD in immunocompetent older adults, with a non‐significant protective effect (OR 0.66, 95% CI 0.14 to 3.03).

Discussion

In this review we aimed to determine the efficacy/effectiveness of the pneumococcal polysaccharide vaccine (PPV), a vaccine that in its current format has been licensed for nearly 30 years in adults. Given that most large randomised controlled trials (RCTs) had been conducted in populations with significant differences in disease susceptibility and serotype distribution, the clinical relevance to adult populations today appears to be uncertain. Nevertheless, we have attempted to define clinically relevant primary outcomes and conducted analysis in distinct population groups that are likely to differ in disease susceptibility and disease exposure.

Summary of main results

Invasive pneumococcal disease (IPD)

This meta‐analysis demonstrates strong evidence of protection against IPD, with a correlate of efficacy from the RCTs of 74% (95% confidence interval (CI) 56% to 85%). This result remained robust in sensitivity analyses based on trial quality and influenza vaccine use.

Although there was no statistical heterogeneity when all RCTs were considered, there was a statistical difference (P < 0.01) between the heterogeneity of the studies conducted among different population groups (otherwise healthy adults in low‐income countries, adults with chronic illness in high‐income countries and otherwise healthy adults in high‐income countries). This indicates that the estimate of efficacy generated from pooling all of the studies is not representative for each of these population groups. In particular, vaccine efficacy amongst the subgroup of adults with chronic disease appears poor in comparison to that of otherwise healthy adults, in developed or low‐income countries. Whilst there was insufficient power to demonstrate protective efficacy among the subgroup of adults with chronic disease, it should be noted that the disease risk amongst the control group was 124 per 100,000 population (2/1619). This is within the general estimate of IPD for older adults (50 cases per 100,000 population) (Fedson 1999) and indicates these studies are underpowered due to the number of participants recruited rather than low disease incidence. Evidence for vaccine effectiveness against IPD in this subgroup population is considered in the non‐RCTs.

The inclusion of non‐RCTs in this meta‐analysis provides a valuable contribution to the understanding of the effectiveness of the PPV, used more recently in large population groups for whom the vaccine is currently recommended. The meta‐analysis of the non‐RCTs demonstrated protective effectiveness against IPD with a pooled estimate corresponding to a slightly lower measure of vaccine effectiveness of 52% (95% CI 37% to 61%) for all serotypes and 55% (95% CI 38% to 54%) for vaccine‐type disease.

Effectiveness among both immunocompetent adults and immunocompetent older adults was also shown, with a non‐significantly higher estimate of effectiveness of 59% for all serotypes (95% CI 48% to 68%). We assessed case‐control studies separately from cohort studies and showed that the estimate of effectiveness from pooled case‐control studies, at 53% (95% CI 32% to 68%), was not dissimilar to that obtained from the two large cohort studies (43%, 95% CI 11% to 64%).

The case‐control studies had similar estimates for vaccine effectiveness (70%, 67% and 70%) with the exception of the Benin 2003 study among Navajo adults. Benin 2003, involving persons aged from 18 years who had a medical condition that placed them at higher risk of pneumococcal disease, or others from age 65 years, found vaccine effectiveness of only 27%. The inferior estimate of effectiveness among Navajo adults may be due to population characteristics. It was reported that 75% of cases and 78% of controls were assigned as risk level 2 (alcoholism, diabetes, chronic cardiac, lung, renal and liver disease). This is higher than the equivalent disease risk ratings of the other case‐control studies. Alcoholism in particular is very prevalent in participants in the Benin 2003 study (43% and 45% of cases and controls), which has been associated with poor vaccine immunogenicity (Butler 1993) and vaccine failures (Hanna 2000; McMahon 1993).

Both the Jackson 2003 and Vila‐Corcoles 2006 cohort studies report very similar multivariate adjusted hazard ratios (HRs) for all‐type IPD of 0.56 (95% CI 0.33 to 0.93) and 0.60 (95% CI 0.22 to 1.65) respectively. Both studies adjusted for receipt of influenza vaccination.

All‐cause pneumonia

All‐cause pneumonia was the most reported outcome in this review with data from 16 RCTs. The pooled estimate of vaccine efficacy was 28% (95% CI 7% to 44%) but there was substantial variability in the effect estimate due to heterogeneity (I² = 85%, P < 0.0001). Given the substantial proportion of variability due to heterogeneity, the available evidence does not demonstrate that PPVs prevent all‐cause pneumonia in adults.

In contrast, we did find evidence of benefit against all‐cause pneumonia among otherwise healthy adults in low‐income countries (subgroup analysis). Here the pooled estimate of vaccine efficacy was supported by low heterogeneity (vaccine efficacy = 46%, 95% CI 33% to 57%, I² = 19%, P value for heterogeneity was 0.29). The point estimate for other population subgroups, adults with chronic illness and otherwise healthy adults from high‐income countries, were inconclusive. For those with chronic illness there was a low level of heterogeneity (I² = 10%) but vaccine efficacy was 7% (‐19% to 27%). A comparison of heterogeneity between these groups and the total pooled estimate was statistically significant (P < 0.001), which also supports the conclusion that the overall estimate of effectiveness is not applicable to all population groups, who are at different risk and susceptibility to disease.

With the upper limit of the CI of the odds ratio (OR) at 0.93, excluding any study from this analysis results in a loss of significance (including sensitivity analysis based on trial quality). It should be acknowledged that the meta‐analysis is inadequately powered to exclude a protective efficacy of less than 44%. This has been a consistent criticism of previous meta‐analyses that remains valid in this updated review.

All‐cause mortality

This meta‐analysis has failed to demonstrate evidence for PPV effectiveness against mortality (all‐cause or pneumococcal‐related). All‐cause mortality was reported in 14 studies with no evidence of a protective benefit from the meta‐analysis (OR 0.90, 95% CI 0.74 to 1.09). The statistical heterogeneity (P < 0.0001) of these pooled studies was again investigated by population group. We found the differences between otherwise healthy adults in low‐income countries, adults with chronic illness in high‐income countries and otherwise healthy adults in high‐income countries to be significant (P < 0.05), suggesting that the pooled estimate of effectiveness is not applicable to all population groups at different levels of risk and susceptibility to disease. It should be acknowledged that the meta‐analysis is inadequately powered to exclude a protective efficacy of less than 26%, or a harmful effect less than 9%.

The use of influenza vaccine in three included studies may have influenced the ability of this meta‐analysis to determine the efficacy of PPV against all‐cause mortality. Furomoto 2008, Gaillat 1985 and Kawakami 2010 administered influenza vaccine to control participants only. The removal of these studies had only a slight effect on the pooled estimate of all trials in sensitivity analysis. Koivula 1997 and Leech 1987 gave influenza vaccine to both PPV intervention and control participants, thereby only had the ability to show the benefit of PPV against all‐cause mortality in addition to that achieved by influenza vaccination.

Subgroup analysis

Otherwise healthy adults in low‐income countries

The combined studies from this relatively homogenous subgroup of young African miners and Papua New Guinean Highlanders consistently demonstrate elevated vaccine efficacy against various outcomes. Of note, vaccine efficacy has been demonstrated for all‐cause pneumonia (four studies) and all‐cause mortality (one study) for this population only. It is also important to highlight that the settings of these trials meant pneumococcal disease was likely to be caused by a limited number of serotypes, particularly in the mines, where pneumococcal outbreaks were a significant cause of mortality. In addition, the vaccines utilised in these trials had fewer serotypes and higher antigen content than the currently licensed 23‐valent PPV. These findings highlight potential benefits of vaccination against pneumonia and death for otherwise healthy adults in low‐income countries. Whether the findings from these early studies remain directly applicable today will depend on a range of factors including whether the circulating serotypes match those in the vaccine, the health status of the population, susceptibility to disease and risk of exposure.

Adults with chronic illness in high‐income countries

The subgroup analysis of RCTs in adults recruited on the basis of the presence of chronic illness in high‐income countries consistently failed to demonstrate evidence of protective benefit of vaccination. For each of the outcomes included in subgroup analysis there were few events in either the intervention or control group and CIs were wide. This indicates that the combined studies remain underpowered. Whilst our review incorporated observational studies which included control for potential confounding factors such as chronic disease, we did not directly assess vaccine effectiveness against IPD for the chronic disease group.

Otherwise healthy adults in high‐income countries

The combined studies for this subgroup included participants recruited on the basis of older age or who were likely to be a disadvantaged population in high‐income countries and demonstrated consistent evidence for protection against IPD. However, for the outcomes of all‐cause pneumonia and all‐cause mortality, there was a high level of statistical heterogeneity. The heterogeneity within all‐cause pneumonia appears to be predominantly from Gaillat 1985 and Kaufman 1947. The remaining four studies do not show evidence for protection against this outcome. The pooled estimate for all‐cause mortality is also statistically heterogeneous, with Kaufman 1947 being the only study to show protective benefit. Again, the remaining six studies do not show any evidence for protection against death within this population group.

Quality of the evidence

RCTs contributing data to this meta‐analysis were conducted over a considerable period of time (1947 to 2010) and within distinct population groups, utilising various valencies of the vaccine (with differing amounts of antigen content). The current 23‐valent PPV has been available since 1983 and was used in 10 of the 24 trials (with known vaccine valency) considered. The quality of the evidence and applicability of these findings to population groups currently recommended to receive the 23‐valent PPV is consistently strong for protection from invasive pneumococcal disease but less clear with respect to less specific outcomes, including all‐cause pneumonia.

Potential biases in the review process

The main strengths of the review are the specificity of outcomes considered. Whilst the review has lost events from studies that considered less specific definitions, this was unlikely to have substantially altered the power of the meta‐analysis. Considering the length of time that these vaccines have been licensed, the review is also strengthened by the inclusion of high‐quality non‐RCTs for the further consideration of culture confirmed IPD; which provided consistent evidence to support findings from the RCTs.

The main weakness of the review remains the lack of power with which to determine vaccine efficacy against pneumonia. This review also excluded two studies that were available only in abstract form in order to maintain the quality of the review. Whilst exclusion of non‐published data may lead to an over‐estimation of the intervention effect (where less significant results remain unpublished), this is unlikely to have had an influence on this review as both studies had positive findings for vaccine efficacy against pneumonia (Teramoto 2007; Ya Tseimakh 2006).

This review did not consider vaccine safety, as it was outside the scope of the review. The assessment of adverse events following PPV administration has been assessed in more recent studies (Cook 2007; Jackson 1999; Musher 2010).

Agreements and disagreements with other studies or reviews

In addition to the previous Cochrane Review (Moberley 2008), at least seven previous meta‐analyses of PPV in adults have been published. However, depending on the selection criteria, results from these meta‐analyses have been variable. Controversy continues to surround the effectiveness and value of the vaccine, particularly with respect to different disease outcomes and within different population groups.

A recent meta‐analysis by Huss 2009 was based on reporting of trial quality and has led to calls for withdrawal of the use of 23‐valent PPV in high‐risk groups (Jefferson 2009). The most recent and best‐quality clinical trials, as determined by Huss 2009, were conducted post‐licensure, largely among populations with chronic illness or severe immunosuppression, or both. In these trials, there were very few cases of IPD: seven cases of definitive pneumococcal pneumonia from two studies and 44 cases of bacteraemia from six studies (most of which were among HIV‐infected adults in Uganda). Overall, the findings from the Huss 2009 study were similar to those presented in this updated Cochrane Review, with respect to effectiveness against presumptive pneumonia, all‐cause pneumonia and death. However, the primary point of difference in this updated Cochrane Review is that the protective effect against IPD remained significant under sensitivity analysis, when the lower‐quality trials were excluded.

Cornu 2001, Moore 2000 and Fine 1994 concluded that the vaccine was effective against bacteraemic pneumococcal pneumonia in 'low‐risk' healthy adults but the RCTs failed to demonstrate vaccine efficacy in those at 'high risk', a heterogeneous group which included older adults (55 years and above), those with chronic disease, or the immunosuppressed. Our results also failed to show evidence for protective efficacy in adults with chronic illness (discussed below). However, we have further refined this assessment and have found evidence of efficacy against IPD amongst otherwise healthy adults in developed country settings. This population consisted predominantly of older adults and institutionalised people.

Hutchison 1999 concluded that there was no evidence that the vaccine was less efficacious for older adults (55 years and above), institutionalised people or people with chronic disease. Whilst we did not directly measure vaccine efficacy against older adults or institutionalised people, the results of our meta‐analysis do highlight apparent differences in efficacy amongst those participants selected on the basis of chronic illness, as will be discussed later.

Watson 2002 found the vaccine was effective against mortality and all‐cause pneumonia in non‐industrialised countries but not in industrialised countries and noted that the small number of cases of pneumococcal bacteraemia made it difficult to draw any firm conclusions for this outcome. Like Watson 2002, our results highlight differences in vaccine efficacy amongst different population groups. Of particular note is the absence of heterogeneity within the population subgroups where IPD has been the outcome of interest but when these subgroups where compared against the total population there was strong evidence of heterogeneity, suggesting the pooled estimate may not be representative of each population group.

Puig‐Barbera 2002 concluded there was no evidence supporting pneumococcal vaccine effectiveness to reduce or avoid S. pneumoniae disease in older adults (55 years and above) but this meta‐analysis lacked power to adequately assess this outcome, with only four RCTs included.

Conaty 2004 conducted a systematic review of non‐RCTs and compared results with those from RCTs, finding elevated point estimates of effectiveness from non‐RCTs and non‐significant results from pooled RCTs. Like Conaty 2004 and as with the previous Cochrane Review (Moberley 2008), this update again includes non‐RCTs and shows supportive evidence of effectiveness against IPD. In contrast to both of these meta‐analyses, however, our updated review also demonstrates protective efficacy against IPD from pooled RCTs.

Authors' conclusions

Implications for practice.

This meta‐analysis supports the use of pneumococcal polysaccharide vaccine (PPV) to prevent invasive pneumococcal disease (IPD) in adults, particularly otherwise healthy adults, in high‐income and low‐income countries. The evidence from our meta‐analysis of randomised controlled trials (RCTs) does not support the routine use of PPV to prevent all‐cause pneumonia or mortality in adults.

Implications for research.

Given the effectiveness of the vaccine in protecting individuals against IPD, commencing new RCTs in populations at risk where vaccine effectiveness and disease burden is known would face ethical difficulties, however, the 23‐valent PPV may have a place as a control treatment in RCTs of conjugate or potential protein vaccine candidates, which this review does not consider.

We have highlighted the potential differences in vaccine effectiveness across population groups, where evidence of protective efficacy from RCTs is less clear with respect to adults with chronic illness. Given adults with chronic illness are the same population who are targeted for vaccination, further trials assessing vaccine efficacy against IPD amongst those with chronic disease appear warranted. However, such trials would need to be large given that this meta‐analysis of five pooled studies remained underpowered against the rare event of IPD (2/1619 participants in the control group had the outcome of interest).

Feedback

Best systematic of 23‐valent pneumococcal vaccine

Summary

Dear Authors,

The inclusion of non‐controlled studies in the current systematic review is clearly a step forward. The case‐control studies enables an analysis of invasive pneumococcal disease that is not possible to do with the prospective studies that have been performed, due to lack of power. However, there is also a well performed cohort study, adjusted for background factors, that showed the same preventive efficacy against IPD (Jackson NEJM MAy 2003). Why was that not included? The search strategy stated that you included papers up to June 2003.

For your information, there are some new data from the study that was published as an early report in Lancet 2001 by Christenson et al. This, however, was published during the fall of 2003. (Hedlund J, Christenson B, Lundbergh P, Örtqvist Å. Effects of a large‐scale intervention with influenza and 23‐valent pneumococcal vaccines in elderly people: a one‐year follow‐up. Vaccine 2003; 21: 3906‐11). Although we are still working with a "complete" background adjustment of the groups to minimise biases, the results of this paper was sex and age adjusted. In addition, a comparison was made between influenza season and non‐influenza. In that comparison it can be seen that there was a significant prevention against both influenza, pneumonia and IPD during the influenza season in patients who had received influenza and/or pneumococcal vaccine. During the non‐influenza season, however, there was no difference concerning influenza, whereas there was still a significant protection against pneumonia. For IPD the RR was the same as during the influenza season (0.47) but there was to few cases to make it significant.

Finally, in your conclusions you make a mistake by stating that typical figures of IPD in developed countries is about 10 per 100.000 per year. That may be so for the whole population but in the elderly where this calculation is of interest, the correct figure is about 50 per 100.000 per year, thereby reducing the NNT to about 4000 vaccinations per infection avoided.

With the best wishes,

Ake

I certify that I have no affiliations with or involvement in any organisation or entity with a direct financial interest in the subject matter of my criticisms.

Reply

See reply to comment # 2

Contributors

Ake Ortqvist

Biased assessment of pneumococcal vaccine effect

Summary

The assessment of pneumococcal vaccine in preventing invasive pneumococcal disease (IPD)of this review is biased in favour of the vaccine and some graphs are misleading.

Take notice that: 
 No quality assessment has ben made of the observational studies included.

Results of heterogeneous studies are displayed in forest plots (see autoco 06 for instance) where results obtained in young adults are displayed with results obtained in the elderly. This is not appropriate and is misleading for the not expert.

Assessment of effectiveness of IPV rest on results of Kaufman 1947, no random assignment, no blind researchers, no placebo group, only three serotypes vaccine (?); Shapiro 1984, results go in favour of the vaccine when data is unmatched(!). Simm 1988, excluded 46% of subjects because of lack of information and did not provide information on pneumococcus serotypes; Shapiro 1991, excluded 121 cases because were originated by one of the 23 serotypes included in the 23 valent vaccine but nevertheless reports a non biased effectiveness in the elderly of 0,6 (IC95% 0,29 yo 1,23); and Butler 1993, no exposure information on 36% of subjects included, a rate that would invalidate any observational study.

This could go on but you can go to Puig‐Barbera et al to get a much more "Cochrane" description and analysis of the data available.

In our current state of knowledge it cannot be assured that the polysaccharide non‐conjugated vaccine is free of deleterious effects in the elderly. Applying the precautionary principle this possibility should be clearly discarded. Meanwhile influenza vaccine does a tremendous good job preventing pneumonia in the elderly.

I certify that I have no affiliations with or involvement in any organisation or entity with a direct financial interest in the subject matter of my criticisms.

Reply

We entirely agree that any positive assessment of the vaccine rests on the very old pseudo‐randomised trial of Kaufman (1947), and on the observational studies. However, we do point out the crucial importance of the Kaufman study in the RCT part of our review. Our analyses are presented both with and without this study for this reason. Indeed our conclusions from this part of the review are decidedly negative: for example we suggest that a large RCT carried out now would fail to show any benefit. We include a table showing how the apparent efficacy of the vaccine increases monotonically as one progressively includes the poorer quality, older studies, and point out that several recent high quality randomised studies consistently showed no effect against all‐cause pneumonia. Fedson and Liss have argued that the failure of other meta‐analyses to demonstrate a benefit against this outcome should be seen as an inconclusive rather than negative result on the grounds that, if 30 to 50% of all pneumonias are pneumococcal, a VE of 50% against pneumococcal pneumonia equates to a VE of 15 to 25% against all‐cause pneumonia.(Fedson, 2004) Our results cannot discount this possibility.

We note your concern about the inclusion criteria for observational studies but believe a more fundamental issue was the decision to incorporate observational studies within the review. Our initial review was conducted in accordance with the protocol. That version of the review reached a largely negative assessment of the efficacy of polysaccharide pneumococcal vaccines in preventing pneumonia and death. It was felt by the Editors that this would deliver an unbalanced message, since it made no mention of other important medical endpoints that the RCTs do not address, in particular invasive pneumococcal disease. The review was therefore expanded specifically in order to include the observational studies and it could be argued that the impact of this decision was to bias the review in favour of the vaccine. Whether such a process is proper for a Cochrane Systematic Review is perhaps doubtful but we have still attempted to assess the evidence fairly. We note that Ave Ortqvist has indicated support for this approach in other comments posted on this review.

Dr Puig‐Barbera suggests we should read the paper by Puig‐Barbera et al to "get a much more "Cochrane" description and analysis of the data available". The clear implication here is that our results are biased by the manner in which we included and analysed the non‐randomised studies. It is true that we have not included a formal, numerical assessment of quality of the observational studies. Probably there is no scale for such studies that would serve as well as the Jadad scale does for RCTs. We have noted the criteria used for assessment of observational studies by Puig‐Barbera et al but do not agree that these criteria are any more valid than the approach used by us. We note with interest that according to the criteria of Puig‐Barbera et al, the study by Forrester et al was rated ahead of a number of other observational studies and was included when others were excluded. In our review, we have described a number of serious flaws in Forrester et al, not least of which was the failure to conduct a matched analysis on a matched case‐control study.

From the paper by Puig‐Barbera et al, it appears that the observational studies have been analysed in an unmatched fashion. We believe this is invalid since, as we have explained in our review, all but one of the observational studies included were matched case‐control studies. For our analysis we combined the estimates of OR based on conditional logistic regression in each study (which accounts for the matching) and calculated a weighted average log‐OR using Stata. Furthermore, it is simplistic to assume that an unmatched analysis of such studies will always approach the null and that failure to do so represents bias in the study.

We do acknowledge that it might be of value to report the high exclusion rates in the observational studies and are grateful for the (implied) suggestion. As for the forest plots, they can be stratified in many ways. We suggest that the date of publication is more important in this regard than the age range of subjects.

We agree with Dr Ortqvist's suggestion that the NNT should be calculated for the older age group rather than for all adults.

We thank Dr Puig‐Barbera and Dr Ortqvist for their comments, and thank the Comments Editor for permitting this response.

Ross Andrews 
 John Holden 
 David Tatham 
 Keith Dear

I certify that I have no affiliations with or involvement in any organisation or entity with a direct financial interest in the subject matter of my criticisms.

Contributors

Joan Puig‐Barbera

Reply to comment by Dr Puig‐Barbera

Summary

Dear Dr Puig‐Barbera,

There are small or big flaws in all studies. The results of the case‐control studies included in the Cochrane analysis are corroborated by the results of the two most recent prospective studies (Honkanen and our own) where there was a clear trend for a 70‐80% protection of the 23‐valent vaccine against bacteraemic pneumococcal pneumonia. A similar finding was published in a cohort study, adjusted for background factors, by Lisa Jackson in NEJM 2003.

Although I agree with you concerning the good effect of influenza vaccine, you're of course aware of that there is no prospective controlled study in the elderly showing that influenza vaccine prevents against severe influenza or pneumonia? The only controlled study showing a protection of the vaccine against clinical (irrespective of severity) and serological influenza.

With the best wishes,

Ake

I certify that I have no affiliations with or involvement in any organisation or entity with a direct financial interest in the subject matter of my criticisms.

Reply

See reply to comment #2

Contributors

Ake Ortqvist

What's new

Date	Event	Description
22 June 2012	New citation required but conclusions have not changed	Review conclusions have not significantly altered.
22 June 2012	New search has been performed	Updated search conducted. We included three new randomised controlled trials (RCTs) (Furomoto 2008; Kawakami 2010; Maruyama 2010), excluded three new RCTs (Steentoft 2007; Teramoto 2007; Ya Tseimakh 2006) and excluded 13 non‐RCTs (Blay 2007; Chang 2012; Christenson 2008; Gilbertson 2011; Lamontagne 2008; Manzur 2011; Ochoa‐Gondar 2008; Rodriguez‐Barradas 2008; Schembri 2009; Skull 2007; Sumitani 2008; Vila‐Corcoles 2009; Vila‐Corcoles 2010).

History

Protocol first published: Issue 2, 1996
 Review first published: Issue 4, 2003

Date	Event	Description
15 July 2010	New search has been performed	Searches conducted.
9 June 2008	Amended	Converted to new review format.
25 July 2004	Feedback has been incorporated	Feedback comment added.
23 June 2003	New search has been performed	Searches conducted.

Notes

Included studies

We included non‐randomised controlled trials (non‐RCTs) provided adjustment had been made for important confounding factors. This essentially excluded studies of vaccine effectiveness utilising the indirect‐cohort method.

Methods

We revised the outcomes assessed. Primary outcomes are invasive pneumococcal disease (IPD) (previously, bacteraemic pneumococcal pneumonia was considered separately), all‐cause pneumonia and all‐cause mortality.

We conducted subgroup analysis according to prespecified characteristics of trial participants that were considered clinically relevant and would lead to recommendations for vaccination according to different populations at risk of pneumococcal disease. These included otherwise healthy adults in low‐income countries, adults with chronic illness in high‐income countries and otherwise healthy adults in high‐income countries.

Findings

When considering all invasive pneumococcal disease, this meta‐analysis found strong evidence of protective efficacy.

Appendices

Appendix 1. MEDLINE (Ovid) search strategy for randomised trials

MEDLINE (Ovid)

1 Streptococcus pneumoniae/ 
 2 streptococcus pneumoniae.tw. 
 3 "s. pneumoniae".tw. 
 4 exp Pneumococcal Infections/ 
 5 (pneumococcal adj2 (infection* or disease*)).tw. 
 6 (pneumococc* adj5 (pneumon* or sepsis or sinusit* or meningit* or otitis media)).tw. 
 7 bacteraemic pneumon*.tw. 
 8 (invasive pneumococcal disease or ipd).tw. 
 9 or/1‐8 
 10 exp Vaccines/ 
 11 exp Vaccination/ 
 12 Immunization/ 
 13 immunoprophylaxis.tw. 
 14 (immuni* or inocul* or vaccin*).tw. 
 15 or/10‐14 
 16 9 and 15 
 17 Pneumococcal Vaccines/ 
 18 pneumococcal polysaccharide vaccin*.tw,nm. 
 19 ppv*.tw,nm. 
 20 pneumovax*.tw,nm. 
 21 or/16‐20

Appendix 2. EMBASE (Elsevier) search strategy for randomised trials

20. #15 AND #19 
 19. #16 OR #17 OR #18 
 18. ((singl* OR doubl*) NEAR/2 (blind* OR mask*)):ab,ti 
 17. random*:ab,ti OR placebo*:ab,ti OR factorial*:ab,ti OR crossover*:ab,ti OR 'cross‐over':ab,ti OR 'cross over':ab,ti OR volunteer*:ab,ti OR assign*:ab,ti OR allocat*:ab,ti 
 16. 'randomised controlled trial'/exp OR 'single blind procedure'/exp OR 'double blind procedure'/exp OR 'crossover procedure'/exp 
 15. #12 OR #13 OR #14 
 14. 'pneumococcal polysaccharide vaccine':ab,ti OR 'pneumococcal polysaccharide vaccines':ab,ti OR ppv*:ab,ti OR pneumovax*:ab,ti 
 13. 'pneumococcus vaccine'/exp 
 12. #8 AND #11 
 11. #9 OR #10 
 10. immuni*:ab,ti OR vaccin*:ab,ti OR inocul*:ab,ti 
 9. 'vaccine'/exp OR 'vaccination'/de OR 'immunization'/de OR 'immunoprophylaxis'/de 
 8. #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7 
 7. 'invasive pneumococcal disease':ab,ti OR ipd:ab,ti 
 6. 'bacteraemic pneumonia':ab,ti 
 5. (pneumococc* NEAR/5 (pneumon* OR sepsis OR sinusit* OR meningit* OR 'otitis media')):ab,ti 
 4. (pneumococcal NEAR/2 (infection* OR disease*)):ab,ti 
 3. 'pneumococcal infection'/exp 
 2. 'streptococcus pneumoniae':ab,ti OR 's. pneumoniae':ab,ti 
 1. 'streptococcus pneumoniae'/de

Appendix 3. MEDLINE (Ovid) search strategy for non‐randomised studies

1 Comparative Study/ 
 2 Follow‐Up Studies/ 
 3 Time Factors/ 
 4 chang*.tw. 
 5 evaluat*.tw. 
 6 reviewed.tw. 
 7 prospective*.tw. 
 8 retrospective*.tw. 
 9 baseline.tw. 
 10 cohort.tw. 
 11 case series.tw. 
 12 case control.tw. 
 13 (compare* or compara*).tw. 
 14 or/1‐13 
 15 Pneumococcal Vaccines/ 
 16 pneumococcal polysaccharide vaccin*.tw. 
 17 (ppv* adj10 (pneumococc* or pneumon*)).tw. 
 18 pneumovax*.tw. 
 19 or/15‐18 
 20 14 and 19

Appendix 4. EMBASE (Elsevier) search strategy for non‐randomised studies

#17. #5 AND #16 
 #16. #6 OR #7 OR #8 OR #9 OR #10 OR #11 OR #12 OR #13 OR #14 OR #15 
 #15. retrospective:ab,ti 
 #14. consecutive:ab,ti 
 #13. baseline:ab,ti 
 #12. reviewed:ab,ti 
 #11. evaluat*:ab,ti 
 #10. chang*:ab,ti 
 #9. 'clinical trial'/de 
 #8. 'major clinical study'/de 
 #7. 'treatment outcome'/de 
 #6. 'controlled study'/de 
 #5. #1 OR #2 OR #3 OR #4 
 #4. pneumovax*:ab,ti AND [embase]/lim 179 23 Jun 2011 
 #3. (ppv* NEAR/10 (pneumococ* OR pneumon*)):ab,ti 
 #2. 'pneumococcal polysaccharide vaccine':ab,ti OR 'pneumococcal polysaccharide vaccines':ab,ti 
 #1. 'pneumococcus vaccine'/de

Appendix 5. Details of previous searches

The initial review (Dear 2003) included electronic searches of the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library 2003, Issue 2) which includes the Cochrane Acute Respiratory Infections (ARI) Group Specialised Register; MEDLINE (January 1966 to June 2003); and EMBASE (1974 to June 2003). Search terms included 'pneumococcal vaccine' or 'pneumococcal immunisation' and 'trials' or 'controlled trials'. Other search terms were not specified.

For this updated review, we searched the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library 2007, Issue 2); MEDLINE (January 1966 to June 2007); and EMBASE (1974 to June 2007). The following MEDLINE search terms were run over CENTRAL and adapted for EMBASE.

MEDLINE (OVID) 
 1 exp Pneumococcal Vaccines/ 
 2 pneumococcal vaccine$.mp. 
 3 (pneumococcal adj (immunis$ or immuniz$)).mp. 
 4 exp Streptococcus pneumoniae/ 
 5 exp Vaccination/ 
 6 (immunis$ or immuniz$ or vaccin$).mp. 
 7 or/5‐6 
 8 4 and 7 
 9 or/1‐3,8 
 10 RANDOMISED CONTROLLED TRIAL.pt. 
 11 CONTROLLED CLINICAL TRIAL.pt. 
 12 RANDOMISED CONTROLLED TRIALS.sh. 
 13 RANDOM ALLOCATION.sh. 
 14 DOUBLE BLIND METHOD.sh. 
 15 SINGLE‐BLIND METHOD.sh. 
 16 or/10‐15 
 17 Animals/ 
 18 Humans/ 
 19 17 not 18 
 20 16 not 19 
 21 CLINICAL TRIAL.pt. 
 22 exp Clinical Trials/ 
 23 (clin$ adj25 trial$).ti,ab. 
 24 ((singl$ or doubl$ or trebl$ or tripl$) adj25 (blind$ or mask$)).ti,ab. 
 25 PLACEBOS.sh. 
 26 placebo$.ti,ab. 
 27 random$.ti,ab. 
 28 or/21‐27 
 29 28 not 19 
 30 20 or 29 
 31 9 and 30

The following terms were used for reports of non‐randomised trials within MEDLINE:

1 exp Pneumococcal Vaccines/ 
 2 pneumococcal vaccine$.mp. 
 3 (pneumococcal adj (immunis$ or immuniz$)).mp. 
 4 exp Streptococcus pneumoniae/ 
 5 exp Vaccination/ 
 6 (immunis$ or immuniz$ or vaccin$).mp. 
 7 or/5‐6 
 8 4 and 7 
 9 or/1‐3,8 
 10 exp Cohort Studies/ 
 11 exp Case‐Control Studies/ 
 12 exp Intervention Studies/ 
 13 exp Prospective Studies/ 
 14 exp Longitudinal Studies/ 
 15 observational stud$.mp 
 16 uncontrolled stud$.mp. 
 17 latin square$.mp. 
 18 factorial.mp. 
 19 Evaluation Studies.pt. 
 20 or/10‐19 
 21 9 and 20

Data and analyses

Comparison 1. RCTs of vaccination versus placebo.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Invasive pneumococcal disease	11		Odds Ratio (M‐H, Random, 95% CI)	Subtotals only
1.1 All studies	11	36489	Odds Ratio (M‐H, Random, 95% CI)	0.26 [0.14, 0.45]
1.2 i) Adults in low‐income countries	1	5373	Odds Ratio (M‐H, Random, 95% CI)	0.14 [0.03, 0.61]
1.3 ii) Adults in high‐income countries with chronic illness	5	3230	Odds Ratio (M‐H, Random, 95% CI)	1.56 [0.35, 6.94]
1.4 iii) Adults in high‐income countries	5	27886	Odds Ratio (M‐H, Random, 95% CI)	0.20 [0.10, 0.39]
2 Pneumonia, all causes	16		Odds Ratio (M‐H, Random, 95% CI)	Subtotals only
2.1 All studies	16	47734	Odds Ratio (M‐H, Random, 95% CI)	0.72 [0.56, 0.93]
2.2 i) Adults in low‐income countries	4	14562	Odds Ratio (M‐H, Random, 95% CI)	0.54 [0.43, 0.67]
2.3 ii) Adults in high‐income countries with chronic illness	6	4010	Odds Ratio (M‐H, Random, 95% CI)	0.93 [0.73, 1.19]
2.4 iii) Adults in high‐income countries	6	29186	Odds Ratio (M‐H, Random, 95% CI)	0.71 [0.45, 1.12]
3 Mortality, all causes	14		Odds Ratio (M‐H, Random, 95% CI)	Subtotals only
3.1 All studies	14	47560	Odds Ratio (M‐H, Random, 95% CI)	0.90 [0.74, 1.09]
3.2 i) Adults in low‐income countries	1	11958	Odds Ratio (M‐H, Random, 95% CI)	0.79 [0.62, 0.99]
3.3 ii) Adults in high‐income countries with chronic illness	6	3603	Odds Ratio (M‐H, Random, 95% CI)	1.13 [0.90, 1.43]
3.4 iii) Older adults in high‐income countries	7	32023	Odds Ratio (M‐H, Random, 95% CI)	0.88 [0.67, 1.17]
4 Invasive pneumococcal disease (vaccine types only)	5	31223	Odds Ratio (M‐H, Random, 95% CI)	0.18 [0.10, 0.31]
5 Definitive pneumococcal pneumonia	10	35483	Odds Ratio (M‐H, Random, 95% CI)	0.26 [0.15, 0.46]
5.1 All studies	10	35483	Odds Ratio (M‐H, Random, 95% CI)	0.26 [0.15, 0.46]
6 Definitive pneumococcal pneumonia (vaccine types only)	4	30561	Odds Ratio (M‐H, Random, 95% CI)	0.13 [0.05, 0.38]
7 Presumptive pneumococcal pneumonia	9	20335	Odds Ratio (M‐H, Random, 95% CI)	0.46 [0.25, 0.84]
8 Presumptive pneumococcal pneumonia (vaccine types only)	5	18568	Odds Ratio (M‐H, Random, 95% CI)	0.27 [0.08, 0.87]
9 Mortality due to pneumonia	9	30723	Odds Ratio (M‐H, Random, 95% CI)	0.71 [0.44, 1.16]
10 Mortality due to pneumococcal infection	3	2445	Odds Ratio (M‐H, Random, 95% CI)	2.51 [0.45, 14.13]

1.1. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 1 Invasive pneumococcal disease.

1.2. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 2 Pneumonia, all causes.

1.3. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 3 Mortality, all causes.

1.4. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 4 Invasive pneumococcal disease (vaccine types only).

1.5. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 5 Definitive pneumococcal pneumonia.

1.6. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 6 Definitive pneumococcal pneumonia (vaccine types only).

1.7. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 7 Presumptive pneumococcal pneumonia.

1.8. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 8 Presumptive pneumococcal pneumonia (vaccine types only).

1.9. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 9 Mortality due to pneumonia.

1.10. Analysis.

Comparison 1 RCTs of vaccination versus placebo, Outcome 10 Mortality due to pneumococcal infection.

Comparison 2. Non‐randomised studies.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Invasive pneumococcal disease (all types)	7		OR (Random, 95% CI)	Subtotals only
1.1 All studies	7		OR (Random, 95% CI)	0.48 [0.37, 0.61]
1.2 Immunocompetent	6		OR (Random, 95% CI)	0.41 [0.32, 0.52]
1.3 Immunocompetent older adults	5		OR (Random, 95% CI)	0.32 [0.22, 0.47]
1.4 Cohort studies	2		OR (Random, 95% CI)	0.57 [0.36, 0.89]
1.5 Case‐control studies	4		OR (Random, 95% CI)	0.47 [0.32, 0.68]
2 Invasive pneumococcal disease (vaccine type)	3		OR (Random, 95% CI)	Subtotals only
2.1 All studies	3		OR (Random, 95% CI)	0.45 [0.38, 0.54]
2.2 Immunocompetent	2		OR (Random, 95% CI)	0.40 [0.29, 0.54]
2.3 Immunocompetent older adults	1		OR (Random, 95% CI)	0.66 [0.14, 3.03]

2.1. Analysis.

Comparison 2 Non‐randomised studies, Outcome 1 Invasive pneumococcal disease (all types).

2.2. Analysis.

Comparison 2 Non‐randomised studies, Outcome 2 Invasive pneumococcal disease (vaccine type).

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Alfageme 2006.

Methods	RCT
Participants	Participants diagnosed with chronic obstructive airways disease by spirometry at the University Hospital of Valme, Spain (n = 600). Participants excluded if received previous pneumococcal vaccination, pregnant, immunosuppressed, in dialysis, HIV+, asplenia
Interventions	23‐valent PPV (n = 300) or no vaccine (n = 300). The study ran from 1999 to 2004 and follow‐up was 6 monthly, up to three years
Outcomes	A1. IPD A2. Pneumonia (all causes) A5. Definitive pneumococcal pneumonia A7. Presumptive pneumococcal pneumonia
Notes	Case ascertainment: participant to contact doctor if temperature greater than 38 C Primary outcome was time to first episode of CAP Patients assigned vaccine/no vaccine from a computer‐generated random number sequence. No reference to blinding No reference to sample size calculation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated random number sequence
Allocation concealment (selection bias)	Unclear risk	Method not described (nor a placebo used, potentially high risk of bias)
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not described. Participants probably aware due to receipt of 2 vaccines
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Physicians participating in the 3‐year follow‐up were unaware of the group to which individuals were assigned
Incomplete outcome data (attrition bias) All outcomes	Low risk	No evidence of incomplete outcome data
Selective reporting (reporting bias)	Low risk	No evidence of selective outcome reporting

Austrian 1976a.

Methods	RCT
Participants	Young adult male miners in South Africa
Interventions	6‐valent PPV first year, 13‐valent second year, or Group A meningococcal vaccine or saline placebo. Total participants across 3 studies n = 12,000
Outcomes	A2. Pneumonia (all causes) A4. IPD (vaccine‐type)
Notes	Follow‐up for 2 years Participants assigned vaccine/control from table of random numbers 3 trials conducted and data combined where relevant. Data for outcome 2 and 4 confirmed by correspondence with author (from the previous version of this review). Not able to include outcome 1 (IPD) as data not able to be extracted from figure 6 with certainty
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Table of random numbers
Allocation concealment (selection bias)	Unclear risk	Not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not described
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not described
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Unclear risk	Not described

Austrian 1980a.

Methods	RCT
Participants	Adult inpatients, Dorothea Dix Mental Hospital, USA
Interventions	6‐valent PPV (n = 607), or saline placebo (n = 693)
Outcomes	A2. Pneumonia (all causes) A3. Mortality (all causes) A9. Mortality due to pneumonia
Notes	Follow‐up continued for 3 years (average 2.2 years) Participants randomly assigned 'in double‐blind fashion', method not specified
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not described
Allocation concealment (selection bias)	Unclear risk	Method not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	"The trial was conducted in a double‐blind fashion"
Blinding of outcome assessment (detection bias) All outcomes	Low risk	As above
Incomplete outcome data (attrition bias) All outcomes	High risk	Participant withdrawals not described. Discrepancies in outcomes reported, e.g. deaths
Selective reporting (reporting bias)	Low risk	No evidence of selective outcome reporting

Austrian 1980b.

Methods	RCT
Participants	Adults, Kaiser Permanente Health Plan, USA
Interventions	12‐valent PPV (n = 6782) or saline placebo (n = 6818)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A5. Definitive pneumococcal pneumonia A6. Definitive pneumococcal pneumonia (vaccine‐type) A9. Mortality due to pneumonia
Notes	Average follow‐up 2.5 years Participants randomly allocated by colour codes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated random sequence
Allocation concealment (selection bias)	Low risk	Participants randomly assigned by colour codes
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	"Contents of vials not known to person administering injection, nor to participants"
Blinding of outcome assessment (detection bias) All outcomes	Low risk	As above
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Participant withdrawals not described
Selective reporting (reporting bias)	Low risk	No evidence of selective outcome reporting

Benin 2003.

Methods	Matched case‐control study
Participants	Navajo adults with IPD aged 18 and over with medical risk factor or above 65 years
Interventions	108 IPD case patients recruited in 1996 or 1997 330 control patients without prior IPD or pneumonia 23‐valent PPV
Outcomes	B1. IPD B2. Vaccine‐type IPD
Notes	Matched 1: up to 7 according to age, sex, chronic medical condition (duration, number of conditions and severity) Cases more likely to have underlying disease and multiple underlying disease conditions (P = 0.0002). Cases may have had prior IPD and controls excluded if prior IPD or pneumonia in previous 10 years Likely bias against vaccine effectiveness AIDS patients not excluded Vaccinated defined as receiving any prior dose (23‐valent PPV)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐randomised study
Allocation concealment (selection bias)	Low risk	Study investigators determining participant inclusion unlikely to be aware of vaccination status
Confounding All outcomes	Low risk	4/5 important confounding factors. Study did not control for smoking due to low prevalence
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to the specificity of outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not clear whether there were withdrawals
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

Davis 1987.

Methods	Double‐blind RCT
Participants	Adults with chronic obstructive pulmonary disease, recruited from chest clinics in New York, USA Participants excluded if malignant neoplasms, sickle cell disease, severe renal failure, hepatic impairment
Interventions	14‐valent PPV (n = 50) administered subcutaneously (2 brands used), or saline placebo (n = 53)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A5. Definitive pneumococcal pneumonia A7. Presumptive pneumococcal pneumonia A9. Mortality due to pneumonia A10. Mortality due to pneumococcal infection
Notes	Average follow‐up 32 months Groups appear similar apart from higher proportion of vaccinees currently smoking (53% compared to 33%). More placebo participants had prior pneumonia and pneumococcal pneumonia (14 and 8 compared to 10 and 5) Random allocation by table of random numbers
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Table of random numbers
Allocation concealment (selection bias)	Unclear risk	Method not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Staff and participants blinded
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessors were not aware of study participant group allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No evidence of selective outcome reporting
Selective reporting (reporting bias)	Unclear risk	No reason to suspect other bias

Dominguez 2005.

Methods	Retrospective, matched case‐control study
Participants	Cases hospitalised with IPD aged >= 65 years, conducted in Spain Controls: 2 hospital, 1 outpatient control per case
Interventions	149 cases, 447 controls 23‐valent PPV
Outcomes	B1. IPD (immunocompetent and immunocompetent elderly)
Notes	Matched 3:1 according to age, date of hospitalisation, medical condition and most important risk factor. The outpatient control was also matched by risk conditional profile of case 33 case patients (18%) and 59 control patients were excluded as vaccination status could not be determined Cases were more likely to have longer mean hospital length of stay and death during hospitalisation Considered vaccinated (23‐valent PPV) from 14 days following administration
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐randomised study
Allocation concealment (selection bias)	Low risk	Study investigators determining participant inclusion unlikely to be aware of vaccination status
Confounding All outcomes	Low risk	3/6 important confounding factors
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to specificity of outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not clear whether there were withdrawals
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

Furomoto 2008.

Methods	Randomised, open‐label controlled study
Participants	Patients with stable chronic lung disease recruited from respiratory clinic
Interventions	23‐valent PPV and influenza vaccine (n = 87), or influenza vaccine alone (n = 80)
Outcomes	A2. Pneumonia (all causes) A3. Mortality (all causes)
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Low risk	Sealed envelopes
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Participants probably and clinical assessors definitely knew allocation to pneumococcal vaccination, or not, during the study
Blinding of outcome assessment (detection bias) All outcomes	High risk	Outcome assessors aware of participant study group allocation
Incomplete outcome data (attrition bias) All outcomes	Low risk	Those lost to follow‐up or early termination of follow‐up reported but reasons unclear beyond participant choice
Selective reporting (reporting bias)	Unclear risk	No evidence of selective outcome reporting

Gaillat 1985.

Methods	RCT
Participants	Older adults living in hospices and retirement homes in France aged between 55 and 85 years Participants excluded if considered very high risk (> 85 years with major visceral defect/or 2 minor, bedridden or immunocompromised)
Interventions	14‐valent PPV (n = 937) administered subcutaneously, or influenza vaccine (n = 749)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A5. Definitive pneumococcal pneumonia
Notes	Followed up for 2 years Case notification to researchers by pre‐paid cards then researcher conducted follow‐up visits Randomised by residential home according to proportion of high‐risk patients in each facility
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not described
Allocation concealment (selection bias)	Unclear risk	Not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not described
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not described
Incomplete outcome data (attrition bias) All outcomes	High risk	141 participants excluded from results "due to insufficiently strict monitoring over the two years in the six establishments involved. Doctors participated moderately and irregularly"
Selective reporting (reporting bias)	High risk	40 participants not accounted for in results

Jackson 2003.

Methods	Retrospective cohort study
Participants	Cohort of older adults aged 65 years and above, enrolled in a Group Health Co‐operative in Washington State
Interventions	47,365 people in cohort ‐ study period March 1998 until February 2001 23‐valent PPV
Outcomes	B1. IPD (immunocompetent and immunocompetent elderly)
Notes	Followed up until death, disenrollment from HMO or end of study (maximum 3 years) Model adjusted for age, sex, nursing home residence, influenza vaccination, smoking, chronic illness, previous hospitalisation for pneumonia in last 12 months, number of outpatient visits. Follow‐up until death, disenrolled from HMO or end of study (3 years) Considered vaccinated from 14 days following administration Confirmation IPD confirmed by sterile site culture by personal communication with author
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐random study
Allocation concealment (selection bias)	Low risk	Study investigators determining participant inclusion unlikely to be aware of vaccination status
Confounding All outcomes	Low risk	All important confounding factors considered
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to specificity of outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

Kaufman 1947.

Methods	RCT
Participants	Patients of New York City Home, USA, aged above 40 years
Interventions	2‐valent PPV given from 1937 to 1938 and 3‐valent PPV given later types 1939 to 1942 (n = 5750), or no intervention (n = 5153)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A4. IPD (vaccine‐type) A5. Definitive pneumococcal pneumonia A6. Definitive pneumococcal pneumonia (vaccine‐type) A7. Presumptive pneumococcal pneumonia A8. Presumptive pneumococcal pneumonia (vaccine‐type) A9. Mortality due to pneumonia
Notes	A continuation of Kaufman 1947 Details regarding patient histories were taken but not reported in paper (age comparison between groups "was about the same, except that there were somewhat more persons in the oldest and somewhat less persons in the youngest group among the controls"). Non‐pneumonia mortality was 91/1000 in the vaccination group and 89/1000 in the control group Average follow‐up for each group unclear (pneumonia incidence presented over 6‐year period)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not described
Allocation concealment (selection bias)	Unclear risk	Not described
Confounding All outcomes	Unclear risk	Not described
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not described. However, likely to be high risk ("5,153 kept as controls") and no placebo used
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not described
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Unclear risk	Not described

Kawakami 2010.

Methods	Open‐label RCT
Participants	Participants were diagnosed with chronic respiratory or cardiac illness/hypertension receiving long‐term care from a physician
Interventions	23‐valent PPV and influenza vaccine (n = 394), or influenza vaccine alone (n = 392)
Outcomes	A2. Pneumonia (all causes) A3. Mortality (all causes) A9. Mortality due to pneumonia
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Low risk	Sealed envelope
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Participants probably (due to receipt of 1 or 2 vaccines) and clinical assessors definitely knew allocation to pneumococcal vaccination, or not
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Pneumonia was diagnosed by the participant's physician (not study personnel)
Incomplete outcome data (attrition bias) All outcomes	Low risk	Those lost to follow‐up reported and numbers low (1% of participants)
Selective reporting (reporting bias)	Unclear risk	No evidence of selective outcome reporting

Klastersky 1986.

Methods	RCT
Participants	Patients with bronchogenic carcinoma prior to receiving radiotherapy/chemotherapy, aged 42 to 47, Brussels, Belgium
Interventions	17‐valent PPV (n = 26) administered subcutaneously, or identically packaged saline placebo (n = 21)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A5. Definitive pneumococcal pneumonia A7. Presumptive pneumococcal pneumonia A9. Mortality due to pneumonia A10. Mortality due to pneumococcal infection
Notes	3 participants lost to follow‐up (no details of which group). Length of follow‐up unclear
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Low risk	"The vaccine was prepared in numbered boxes, prepared according to prior randomisation"
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	However, staff and participants likely blinded due to method of allocation concealment
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not described
Incomplete outcome data (attrition bias) All outcomes	Low risk	No evidence of incomplete outcome data, however, 3 participants lost to follow‐up (no details of which group)
Selective reporting (reporting bias)	Unclear risk	No evidence to suspect selective outcome reporting

Koivula 1997.

Methods	RCT
Participants	Older adults residing in town in eastern Finland, aged 60 years or above
Interventions	14‐valent PPV and influenza vaccine (n = 1364), or influenza vaccine alone (n = 1473)
Outcomes	A3. Mortality (all causes)
Notes	Followed up for 3 years; 1983 to 1985. Randomisation appears to have occurred prior to consent to participate. Following randomisation, participants with high‐risk medical conditions also received annual influenza vaccination Unable to include all‐cause pneumonia as not all pneumonia radiologically typical (appears to include radiological uncertain)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomised by computer
Allocation concealment (selection bias)	High risk	Although participants not aware of assignment, not possible to conceal as either 1 or 2 vaccines given
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	High risk	Described as single‐blinded
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessors did not know group allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No evidence of incomplete outcome data
Selective reporting (reporting bias)	Unclear risk	No evidence to suspect reporting bias due to selective outcome reporting

Leech 1987.

Methods	RCT
Participants	Participants aged 40 to 89 years, with chronic obstructive lung disease (with FEV1 < 1.5 L), recruited from outpatient department at Montreal Chest hospital, Canada between January and June 1981
Interventions	14‐valent PPV (n = 92) or saline placebo (n = 97); both groups were given influenza vaccine at the same time and annually thereafter
Outcomes	A1. IPD A3. Mortality (all causes) A5. Definitive pneumococcal pneumonia
Notes	Potential participants stratified by age, FEV and sex, then randomised Follow‐up for 2 years at 6‐monthly intervals 23 participants lost to follow‐up (which group is unclear)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Unclear risk	Method not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Staff and participants blinded
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessors did not know participant study group allocation
Incomplete outcome data (attrition bias) All outcomes	High risk	Participants lost to follow‐up with no rationale provided
Selective reporting (reporting bias)	Unclear risk	No evidence to suspect selective outcome reporting

Maruyama 2010.

Methods	RCT
Participants	Nursing home residents aged 55 to 106 years
Interventions	23‐valent PPV (n = 502) or saline placebo (n = 506)
Outcomes	A1. IPD A2. All‐cause pneumonia A3. Mortality (all causes) A7. Presumptive pneumococcal pneumonia A9. Mortality due to pneumonia
Notes	The pneumococcal urinary antigen test was used to assist in the diagnosis of presumptive pneumococcal pneumonia (in addition to documented new infiltrate on chest x‐ray). Study results unable to be included in outcome A10, as unable to differentiate those diagnoses that did not fit definition of confirmed pneumococcal pneumonia
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomisation using a random number table
Allocation concealment (selection bias)	Low risk	Vaccine and placebo were presented in identical single dose syringe and needle combinations, labelled with sequential study numbers only
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Staff and participants blinded
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Pneumonia was diagnosed by medical staff of respiratory unit of the affiliated hospital, who were unaware of participant group allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No evidence of incomplete outcome data
Selective reporting (reporting bias)	Unclear risk	No evidence of selective outcome reporting

Ortqvist 1998.

Methods	Double‐blind RCT
Participants	Non‐immunocompromised adults aged 50 to 85 years who had been inpatients for CAP in Sweden. Potential participants excluded if assumed poor compliance, previous receipt of PPV, allergy to PPV or immunocompromised
Interventions	23‐valent PPV (n = 339) administered intramuscularly (subcutaneous for patients on anticoagulant therapy), or saline placebo (352)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A4. IPD (vaccine‐type) A5. Definitive pneumococcal pneumonia A6. Definitive pneumococcal pneumonia (vaccine‐type) A7. Presumptive pneumococcal pneumonia A8. Presumptive pneumococcal pneumonia (vaccine‐type) A9. Mortality due to pneumonia
Notes	Average follow‐up 2.5 years Case ascertainment: participant to contact doctor if temperature above 38 C or cause to suspect pneumonia recurrence. 1 participant withdrawn (n = 2, as randomised twice)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Specific method not described, however, text states: "randomisation was done by the vaccine manufacturer and the code was not disclosed until follow‐up had ended"
Allocation concealment (selection bias)	Low risk	Vaccine and placebo were marked with random numbers
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Staff and participants blinded
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessors did not know participant study group allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No evidence of incomplete outcome data
Selective reporting (reporting bias)	Unclear risk	No evidence of selective outcome reporting

Riley 1977.

Methods	Double‐blind RCT
Participants	Adults from Tari in Papua New Guinea Highlands
Interventions	14‐valent PPV (n = 5946), or saline placebo (6012). Surveillance subset of 2713 vaccinees, 2660 controls for disease outcomes
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A4. IPD (vaccine‐type) A5. Definitive pneumococcal pneumonia A6. Definitive pneumococcal pneumonia (vaccine‐type)
Notes	Follow‐up 3 years for mortality and 16 months morbidity 540 records lost during transport, unclear which group, report likely equal numbers
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The vials of vaccine and placebo were then randomly assigned consecutive numbers
Allocation concealment (selection bias)	Low risk	Distinguishing labels were removed from the vaccine and placebo vials
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Study was double‐blind
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Study was double‐blind
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	High risk	"Records were lost of 540 participants known to have received an injection"

Shapiro 1984.

Methods	Matched case‐control study
Participants	Adults aged 18 years and above with indication for vaccination or above 65 years (n = 180)
Interventions	Cases (n = 90) hospitalised with IPD from 1978 to 1982 (no previous pneumococcal disease) Controls (n = 90) selected from roster of hospitalised patients with no previous episode of pneumonia 14‐valent PPV
Outcomes	B1. IPD (immunocompetent and immunocompetent elderly)
Notes	Matched 1:1 according to age, date of hospitalisation, condition identified and vaccination indication (duration, severity and number of illnesses) Vaccination history reviewed from 1978 onwards Vaccinated defined as receiving dose at least 2 weeks prior to selected hospitalisation (14‐valent PPV)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐random study
Allocation concealment (selection bias)	Low risk	Study investigators determining participant inclusion unlikely to be aware of vaccination status
Confounding All outcomes	Low risk	2/6 important confounding factors considered, however critical factors covered
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to specificity of the outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

Shapiro 1991.

Methods	Matched case‐control study
Participants	Adults aged 18 years and above with indication for vaccination or above 65 years (n = 2108)
Interventions	Cases (n = 1054) hospitalised with IPD from 1984 to 1990 (no previous pneumococcal disease) and pneumococcal isolate serotyped Controls (n = 1054) selected from roster of hospitalised patients with no previous episode of IPD during the study period Both 14‐valent and 23‐valent PPV
Outcomes	B1. IPD (immunocompetent) B2. Vaccine‐type IPD (immunocompetent and immunocompetent elderly)
Notes	Matched 1:1 according to age, date and site of hospitalisation, condition of vaccination indication (duration, severity and number of illnesses) Less cases were white (862/921) and had a private physician (829/903). More cases were residents and chronic care facility (181/117) and died during hospitalisation. Likely bias against vaccine effectiveness Analysis according to indirect cohort method was conducted in a subgroup analysis of cases with a serotype contained in the 14 valent vaccine compared to non‐23 valent or 14 valent serotypes (23 valent non‐14 valent serotype cases excluded) Vaccinated defined as receiving dose at least 2 weeks prior to selected hospitalisation
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐random study
Allocation concealment (selection bias)	Low risk	Study investigators determining participant inclusion unlikely to be aware of vaccination status
Confounding All outcomes	Low risk	2/6 important confounding factors considered, however critical factors covered
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to specificity of outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

Simberkoff 1986.

Methods	Double‐blind RCT
Participants	High‐risk ambulatory patients 55 years and above, referred from clinics, in the USA (with chronic renal, hepatic, cardiac or pulmonary disease, alcoholism or diabetes) Excluded if previous receipt of PPV, multiple myeloma, lymphoma or malignant disease, immunosuppressive treatment, history of splenectomy or functional asplenia
Interventions	14‐valent PPV (n = 1175) administered subcutaneously, or saline placebo (1179)
Outcomes	A1. IPD A2. Pneumonia (all causes) A3. Mortality (all causes) A4. IPD (vaccine‐type) A5. Definitive pneumococcal pneumonia A7. Presumptive pneumococcal pneumonia A8. Presumptive pneumococcal pneumonia (vaccine‐type) A9. Mortality due to pneumonia A10. Mortality due to pneumococcal infection
Notes	Mean follow‐up period 2.9 years More vaccinees had prior pneumococcal disease compared to placebo recipients (48 and 30 respectively, P = 0.035) Additional low‐risk patients (n = 59) included for antibody response only Loss to follow‐up not reported (statement in text satisfactory)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Unclear risk	Method not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Staff and participants blinded
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Outcome assessors did not know participant study group allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No evidence of incomplete outcome data
Selective reporting (reporting bias)	Unclear risk	No evidence of selective outcome reporting

Sims 1988.

Methods	Matched case‐control
Participants	Immunocompetent older adults aged 55 years and above
Interventions	Cases (n = 122) IPD cases from Jan 1980 to July 1986 Controls (n = 244) consecutive hospital admissions Vaccine valency not specified
Outcomes	B1. IPD (immunocompetent and immunocompetent elderly)
Notes	Matched 2:1 according to hospital, admission date, comorbid conditions (number and severity) Sample size based on vaccine efficacy at 50%, power 0.80, error 0.05 (one tailed) and 20% vaccination coverage in controls requires 164 cases and 328 controls Controls excluded if they had radiographically proven pneumonia during the study period Vaccinated definition (time since dose) not specified
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐random study
Allocation concealment (selection bias)	Low risk	Study investigators determining participant inclusion unlikely to be aware of vaccination status
Confounding All outcomes	Low risk	3/6 important confounding factors covered, however, critical factors covered
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to the specificity of outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

Smit 1977a.

Methods	RCT
Participants	Young adult gold miners in South Africa (recruited within 1 day of arrival)
Interventions	6‐valent PPV (n = 983), or meningococcal group A vaccine (n = 1051), or saline (n = 985). Total control n = 2036
Outcomes	A2. Pneumonia (all causes) A7. Presumptive pneumococcal pneumonia A8. Presumptive pneumococcal pneumonia (vaccine‐type)
Notes	Mean length of follow‐up not stated Recruited between February 1973 to June 1974, with follow‐up until May 1975 Maximum duration of follow‐up 2.3 years
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Unclear risk	Method not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not specified
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"Clinical observations were made blindly without knowledge of whether the subject had received pneumococcal vaccine or control material"
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Unclear risk	Not described

Smit 1977b.

Methods	RCT
Participants	Young adult gold miners in South Africa (recruited within 1 day of arrival)
Interventions	12‐valent PPV (n = 540), or meningococcal types A and C vaccine (n = 585); saline (n = 550). Total control = 1135
Outcomes	A2. Pneumonia (all causes) A7. Presumptive pneumococcal pneumonia A8. Presumptive pneumococcal pneumonia (vaccine‐type)
Notes	Mean length of follow‐up not stated Recruited between July 1974 to January 1976, with follow‐up until February 1976 Maximum duration of follow‐up 1.6 years
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method not described
Allocation concealment (selection bias)	Unclear risk	Method not described
Confounding All outcomes	Unclear risk	NA
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	Not described
Blinding of outcome assessment (detection bias) All outcomes	Low risk	"Clinical observations were made blindly without knowledge of whether the subject had received pneumococcal vaccine or control material"
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Unclear risk	Not described

Vila‐Corcoles 2006.

Methods	Prospective cohort study
Participants	Community‐dwelling older adults aged 65 years and above, assigned to primary healthcare centres
Interventions	11,241 members of the cohort. Study period from January 2002 until April 2005 23‐valent PPV
Outcomes	B1. IPD (immunocompetent and immunocompetent elderly) B2. Vaccine‐type IPD (immunocompetent and immunocompetent elderly)
Notes	Cohort observed until study completion or first occurrence of each outcome (maximum 3 years, 4 months) IPD incidence of study participants 0.64/1000 person‐years Model adjusted for age, sex, number of outpatient visits, history of hospitalisation for pneumonia in previous 24 months, influenza vaccination status, underlying medical conditions, current smoking status and immunocompromised status. Vaccination was a time variable factor; considered vaccinated from 14 days following administration Results for IPD by vaccine‐type only serotypes provided by personal communication with first author
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Non‐random study
Allocation concealment (selection bias)	Low risk	Investigators reviewing clinical data unaware of vaccination status
Confounding All outcomes	Low risk	6/6 important confounding factors considered
Blinding of participants and personnel (performance bias) All outcomes	Unclear risk	NA
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Due to specificity of outcome
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not described
Selective reporting (reporting bias)	Low risk	Limited extent to which analysis could have been manipulated to bias the findings

CAP: community‐acquired pneumonia 
 FEV: forced expiratory volume 
 FEV₁: forced expiratory volume in first second 
 HMO: health maintenance organisation 
 IPD: invasive pneumococcal disease 
 NA: not applicable 
 PPV: pneumococcal polysaccharide vaccine 
 RCT: randomised controlled trial

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Ammann 1977	Outcome was changes in antibody titres
Ansaldi 2005	Observational study with non‐IPD outcome
Austrian 1976b	No useable data due to inadequate reporting
Bentley 1981	Cohort study, results not controlled for confounding factors
Blay 2007	Observational study with non‐IPD outcome
Bolan 1986	Data are included in a subsequent report: Butler 1993
Brieman 2000	HIV‐positive participants
Broome 1980	Data are included in a subsequent report: Butler 1993
Butler 1993	Indirect cohort study (no control of confounding factors)
Chang 2012	Observational study with non‐IPD outcome
Chiba 2004	All outcomes defined by ICD codes
Chintu 1983	Non‐randomised clinical trial, case series only
Christensen 2001	Only early interim results are available
Christenson 2004	Cases of IPD identified by ICD codes. No adjustment for confounding factors
Christenson 2008	Observational study with non‐IPD outcome
Douglas 1984	Trial measures efficacy of vaccine in children
Douglas 1986	Trial measures efficacy of vaccine in children
Dworkin 2001	Cohort study in HIV positive participants
Farr 1995	Age of participants from 2 years. No subgroup analysis for adults able to be included
Fletcher 1997	Outcome was changes in antibody titres
Forrester 1987	This study was designed as a case‐control study but VE estimate given according to indirect cohort method, therefore no control of confounding factors in this methodology
Franzen 2000	Observational study with non‐IPD outcome
Gilbertson 2011	Observational study with non‐IPD outcome
Hedlund 2003	Interim results of Christenson 2004. Methodological issues of case identification by ICD codes and no control for confounding factors
Honkanen 1999	Could not able be included with RCTs as this trial is not randomised (initially year of birth allocation but participants also able to choose group). Could not be included in observational review due to lack of control of confounding factors
Karma 1985	Trial measures efficacy of vaccine in children
Kaufman 1941	All results included in second report (Kaufman 1947)
Lamontagne 2008	Observational study with non‐IPD outcome
Lindenburg 2001	Observational study with non‐IPD outcome
MacLeod 1945	No usable data due to inadequate reporting
Manzur 2011	Observational study with non‐IPD outcome
MMWR 2001	Not all cases confirmed as IPD (7 of 9 confirmed by blood culture)
Mykietiuk 2006	Case series, no comparison group
Nichol 1999	Cohort study, primary outcome pneumonia defined by ICD codes
Ochoa‐Gondar 2008	Observational study with non‐IPD outcome. Results for bacteraemic pneumococcal pneumonia were presented but all participants were involved in the general cohort of the EVAN‐65 Study (included study Vila‐Corcoles 2006). Clarified by personal communication with the author
Rodriguez‐Barradas 2008	Observational study with non‐IPD outcome
Rosen 1983	Trial measures efficacy of vaccine in children
Schembri 2009	Observational study with non‐IPD outcome
Skull 2007	Observational study with non‐IPD outcome
Steentoft 2007	Unable to separate the role of steroids within this study; comparison of vaccination + steroids versus no vaccination and most likely a mix of +/‐ steroids during the 3 months prior to vaccination (49 COPD patients of whom 13 had no systemic steroids during the preceding 3 months). Unable to contact author to clarify
Sumitani 2008	Observational study with non‐IPD outcome
Teramoto 2007	Study only available in abstract form. Unable to ascertain full methodology details or conduct quality assessment
Vila‐Corcoles 2009	Participants were involved in the general cohort of the EVAN‐65 Study (included study Vila‐Corcoles 2006). Clarified by personal communication with the author
Vila‐Corcoles 2010	IPD cases were identified on the basis of ICD‐9 discharge codes
Wagner 2003	Observational study with non‐IPD outcome
Ya Tseimakh 2006	Study only available in abstract form. Unable to ascertain full methodology details or conduct quality assessment

COPD: chronic obstructive pulmonary disease 
 ICD: international classification of diseases 
 IPD: invasive pneumococcal disease 
 VE: vaccine effectiveness

Characteristics of ongoing studies [ordered by study ID]

Ryan 2001.

Trial name or title	Ryan 2001
Methods	Double‐blind RCT
Participants	Over 191,000 military recruits
Interventions	23vPPV compared to placebo
Outcomes	All‐cause pneumonia, acute respiratory illness, hospitalisation and outpatient encounters
Starting date	October 2000
Contact information
Notes

RCT: randomised controlled trial

Contributions of authors

Holden (JH) and Tatham (DT) were involved in the preparation of the initial protocol, the searches for studies and the assessment of new randomised controlled trials (RCTs) for inclusion in this updated review. JH conducted the updated 'Risk of bias' assessment for the RCTs.

Andrews (RA) joined in May 2001 to assist with incorporating non‐RCTs in the review. For this update, RA reviewed observational studies for inclusion in the review, conducted the updated 'Risk of bias' assessment for observational studies and contributed to writing this updated review.

Moberley (SM) joined in 2006 to assist with the update of the review, conducted electronic searches, selected observational studies and re‐extracted data for the additional outcomes in the RCTs. For this update, SM reviewed observational and randomised studies for inclusion in the review, conducted the updated 'Risk of bias' assessment for observational and randomised studies, conducted the data analysis and wrote this updated review.

Sources of support

Internal sources

• Menzies School of Health Research, Australia.

• St. Helens Multidisciplinary Audit Advisory Group, UK.

External sources

• No sources of support supplied

Declarations of interest

None known.

New search for studies and content updated (no change to conclusions)