Skip to main content
PMC home page
Bulletin of the World Health Organization logoLink to Bulletin of the World Health Organization
. 2012 May 11;90(8):568–577. doi: 10.2471/BLT.11.098178

Sentinel versus population-based surveillance of pneumococcal conjugate vaccine effectiveness

Suivi comparé d'un échantillon sentinelle et de la population quant à l'efficacité du vaccin conjugué contre le pneumocoque

Efectividad de la vacuna antineumocócica conjugada: comparativa entre un sistema de vigilancia centinela y un sistema de vigilancia poblacional

الترصد الخفري مقابل الترصد السكاني لفاعلية اللقاح المتقارن المضاد لالتهاب المكورات الرئوية

肺炎球菌疫苗有效性的疾病哨点和人群监测对比

Подходы к эпиднадзору за эффективностью пневмококковой конъюгированной вакцины: дозорный против популяционного

Lee M Hampton a,✉, Elizabeth R Zell b, Stephanie Schrag b, Adam L Cohen b
PMCID: PMC3417785  PMID: 22893740

Abstract

Objective

To compare sentinel and population-based surveillance of the effect of seven-valent pneumococcal conjugate vaccine (PCV7), introduced in 2000, on the hospitalization of children aged under 5 years with invasive pneumococcal disease (IPD) in the United States of America.

Methods

Population surveillance data were used to identify children hospitalized between 1998 and 2006 with IPD caused by Streptococcus pneumoniae serotypes. The change from 1998 and 1999 (baseline) to 2006 in the number of hospitalized IPD cases recorded by sentinel surveillance systems involving single hospitals or groups of hospitals was compared with the change in the incidence of hospitalized IPD cases measured by population-based surveillance.

Findings

The change in incidence in the eight surveillance areas varied from −37 to −82% for IPD caused by any serotype and from −96 to −100% for IPD caused by serotypes contained in PCV7. All individual sentinel hospitals with more than three cases annually at baseline reported a decrease in cases by 2006. In addition, over 95% of sentinel systems with an average of more than 30 cases annually at baseline recorded a change by 2006 in the number of cases caused by any serotype that fell within the 95% confidence interval for the change in the incidence of hospitalized cases in the corresponding population surveillance area. The change in cases caused by PCV7 serotypes was accurately measured by 93% and 100% of sentinel systems with ≤ 20 and > 20 cases annually at baseline, respectively.

Conclusion

Sentinel surveillance can accurately measure the effect of PCV7 on the number of children hospitalized with IPD, provided sufficient cases are detected at baseline. Serotyping increases accuracy.

Introduction

Every year Streptococcus pneumoniae, or pneumococcus, causes an estimated 826 000 deaths worldwide in children under 5 years of age.1 In 2000, a seven-valent pneumococcal conjugate vaccine (PCV7) was introduced in the United States of America. The result was a dramatic, sustained reduction in invasive pneumococcal disease (IPD) in young children.2,3 Moreover, a very similar nine-valent pneumococcal conjugate vaccine (PCV9) that contains the seven S. pneumoniae serotypes in PCV7 (i.e. serotypes 4, 6B, 9V, 14, 18C, 19F and 23F) plus serotypes 1 and 5 has been shown to be efficacious in developing countries4,5 and the World Health Organization (WHO) has recommended that all countries introduce pneumococcal conjugate vaccines into their routine childhood immunization programmes.6 As with other new vaccines, measuring the effect of pneumococcal conjugate vaccines helps policy-makers determine whether the benefits of their introduction and sustained use outweigh their costs.7

Trends in the occurrence of a vaccine-preventable disease after a vaccine is introduced can be assessed by either population-based or sentinel surveillance. Population-based surveillance involves identifying all new cases of the disease under surveillance in a defined population. The data obtained can be used to calculate the disease incidence rate since the size of the population under surveillance is known. In contrast, sentinel surveillance involves monitoring a disease at a single facility or a small number of facilities. Generally, incidence rates cannot be derived from sentinel surveillance data since the population covered is rarely known and the data obtained may not be representative of the catchment area. However, sentinel surveillance usually requires fewer resources than population-based surveillance.8 The feasibility of sentinel surveillance has led WHO to recommend that countries adopt the approach for monitoring the effect of newly introduced vaccines, including pneumococcal conjugate vaccines.9 Sentinel surveillance systems have, nonetheless, several limitations and it is not clear whether even well-functioning systems can provide the data required for evaluating the effect of a new vaccine. However, studies have shown that sentinel systems involving hospitals in the United States can provide a reasonably accurate assessment of trends in antibiotic resistance10 and that sentinel surveillance in antenatal clinics in developing countries can be used to assess the prevalence of human immunodeficiency virus infection.11

In this study, we used data from the Active Bacterial Core surveillance system in the United States to determine whether a sentinel surveillance system involving either individual hospitals or groups of hospitals can provide estimates similar to those based on data from a population-based surveillance system when used to quantify the effect of introducing PCV7 on the occurrence of laboratory-confirmed IPD requiring hospitalization among children aged under 5 years.

Methods

The Active Bacterial Core surveillance system in the United States performs active, population-based surveillance for IPD, which is defined as being present when pneumococcus is isolated from normally sterile body fluid or tissue, such as blood, cerebrospinal fluid or pleural fluid, in an individual who is resident in a surveillance area on the date of culture.3,12 We obtained data from this surveillance system on all hospitalized IPD cases identified in children aged under 5 years between 1 January 1998 and 31 December 2006. We adopted 2006 as the endpoint of the study because that year recorded the lowest incidence of IPD to date in the age group.3 Throughout the study period, the Active Bacterial Core surveillance system monitored IPD in all eight counties in the state of Connecticut and in 49 counties overall in the states of California, Georgia, Maryland, Minnesota, New York, Oregon and Tennessee.12 The total population aged under 5 years that was under surveillance averaged 1 140 372 during 1998 and 1999 and was 1 261 188 in 2006 (Table 1). In the study, data were analysed for eight surveillance areas, which corresponded to areas monitored in the eight states. Demographic and clinical information on individual cases of IPD were obtained from medical records. A patient was regarded as having meningitis when S. pneumoniae was isolated from cerebrospinal fluid, irrespective of the clinical diagnosis, or when a clinical diagnosis of meningitis was accompanied by the isolation of S. pneumoniae from blood.

Table 1. Active Bacterial Core surveillance system areas, United States of America, 1998–1999 and 2006.

Surveillance area Average population in 1998–1999
 % change in population aged < 5 years from 1998–1999 to 2006 No. of hospitals reporting
Total no. No. of children < 5 years ≥ 1 IPD cases in 1998 or 1999 ≥ 5 IPD cases in 1998 and 1999 combined
California 746 276 34 370 19 6 0
Connecticut 3 278 050 214 726 −6 17 8
Georgia 3 801 578 288 607 34 18 6
Maryland 2 447 423 163 176 2 18 9
Minnesota 2 535 790 180 369 9 18 8
New York 1 104 857 78 018 −19 5 2
Oregon 1 373 146 93 559 13 4 2
Tennessee 1 318 865 87 547 14 12 5
All 16 605 985 1 140 372 11 98 40
Open in a new tab

IPD, invasive pneumococcal disease.

We calculated the observed, or crude, annual incidence of IPD requiring hospitalization in children aged under 5 years for each of the eight surveillance areas using the total number of cases identified in that age group by the Active Bacterial Core surveillance system in a given year and the area’s population of under 5-year-olds as reported in National Center for Health Statistics bridged-race population estimates, which are based on United States census data.13 We calculated the percentage change in the annual incidence of hospitalized IPD cases in children aged under 5 years using the formula: (RR − 1) × 100%, where RR was defined as the incidence in the most recent time period divided by the incidence at baseline. We determined 95% confidence intervals (CIs) for the percentage change using standard interval calculations in SAS version 9.2 (SAS Institute Inc., Cary, USA).

With sentinel surveillance, the effect of the vaccine was quantified by estimating the percentage change between baseline and 2006 in the number of hospitalized IPD cases treated at an individual hospital or group of hospitals participating in the Active Bacterial Core surveillance system. With population-based surveillance, the effect was quantified by estimating the percentage change between the two time periods in the incidence of hospitalized IPD cases in the surveillance area covered by the Active Bacterial Core surveillance system. Subsequently, the effect of the vaccine as assessed by sentinel surveillance was compared with its effect as assessed by population-based surveillance in the area in which sentinel surveillance was carried out. For example, the change in the IPD case count at an individual hospital in Connecticut was compared with the change in the incidence of hospitalized IPD cases for the entire state of Connecticut, as all eight counties in the state were covered by the Active Bacterial Core surveillance system. Since changes in referral patterns within individual surveillance areas could have had a confounding effect on the assessment of the change in hospitalized IPD cases in individual hospitals, all cases that were transferred from one hospital to another were excluded from calculations of the effect of the vaccine made using data from sentinel surveillance.

The effect of PCV7 on the number of children aged under 5 years who were hospitalized with IPD was estimated for three groups, according to the S. pneumoniae serotype causing the disease: (i) IPD due to any serotype, even when it was unknown or not recorded; (ii) IPD caused only by serotypes included in PCV7 (i.e. serotypes 4, 6B, 9V, 14, 18C, 19F and 23F); and (iii) IPD caused only by serotypes not included in PCV7. The period 1998 to 1999 was chosen as the baseline for comparisons and the years 2002, 2004 and 2006 were the end points.

Applying a method similar to that used by Montana et al.,11 we judged that sentinel surveillance was in agreement with population-based surveillance in assessing the effect of the vaccine on hospitalized IPD cases in children aged under 5 years when the magnitude of the change in the number of hospitalized IPD cases determined by sentinel surveillance fell within the 95% CI of the magnitude of the change in the incidence of hospitalized IPD cases in the corresponding surveillance area.

We also examined whether particular characteristics of the individual hospital or group of hospitals involved in sentinel surveillance were associated with a greater likelihood that the sentinel surveillance system’s assessment of the magnitude of the effect of PCV7 would agree with the population-based determination for the corresponding surveillance area. Every possible combination of hospitals in a given surveillance area in which each hospital had at least five hospitalized IPD cases in children aged under 5 years in 1998 and 1999 combined was examined in the analysis of groups of sentinel hospitals. Specifically, we determined whether the number of prevaccine cases in 1998 and 1999 or the proportion of cases in the surveillance area in 1998 and 1999 that was dealt with by a given sentinel system predicted which sentinel systems would produce an assessment of the effect of PCV7 that was in agreement with that obtained in the corresponding surveillance area. We used the Cochran–Armitage trend test to determine whether either of these two characteristics of the sentinel system or the increase in the amount of time since the introduction of PCV7 from 2 to 4 to 6 years was associated with a greater likelihood that a sentinel system would produce an assessment in agreement with that of its surveillance area.

Active Bacterial Core surveillance case reporting and isolate collection were considered to be public health surveillance activities that were exempt from institutional review by the Centers for Disease Control and Prevention in the United States. In addition, the surveillance protocol was evaluated at each participating surveillance site and either it was decided that the protocol was exempt from review or approval was obtained from the appropriate local institutional review board. Neither the Centers for Disease Control and Prevention nor institutional review boards at individual sites required informed consent.

Results

The number of hospitals in each surveillance area that reported at least one hospitalized IPD case in a child aged under 5 years in either 1998 or 1999 varied from 4 to 18 (Table 1) and the average number of cases in 1998 and 1999 combined for each of these hospitals ranged from 1 to 28 (median: 2). The proportion of all hospitalized IPD cases in a given surveillance area in 1998 and 1999 that was treated at a single hospital ranged from 0.01% to 58%. The average baseline population of children aged under 5 years in the surveillance areas during 1998 and 1999 ranged from 34 370 to 288 607 and the change in population from 1998 and 1999 to 2006 ranged from −19% to 34%.

We identified 1667 hospitalized IPD cases among children aged under 5 years between 1998 and 2006: 685 (41%) were cases of bacteraemia of unknown source, 558 (33%) involved invasive pneumonia, 218 (13%) involved meningitis and 206 (12%) involved infections at other locations. The annual number of hospitalized IPD cases caused by any S. pneumoniae serotype in all eight surveillance areas combined declined from an average of 360 in 1998 and 1999 to 148 in 2006, which corresponds to a decrease in incidence of 63% (95% CI: 55–69), from 31.4 to 11.7 cases per 100 000 children aged under 5 years (Table 2). The decrease in hospitalized IPD cases caused by any serotype in individual surveillance areas varied from 37 to 82%. Moreover, the decline in the incidence of hospitalized IPD cases caused by a serotype included in PCV7 in all surveillance areas combined was 99% (95% CI: 97–100), with the decline in individual surveillance areas varying from 96 to 100%. In contrast, the incidence of hospitalized IPD cases caused by a serotype not included in PCV7 increased by 93% (95% CI: 53–144) in all surveillance areas combined, with individual surveillance areas showing an increase ranging from 17 to 342%.

Table 2. Children aged under 5 years hospitalized with invasive pneumococcal disease (IPD), United States of America, 1998–1999 and 2006a.

S. pneumoniae serotype and surveillance areab Annual average 1998–1999
 2006a
 Change in incidence
No. of cases Incidencec No. of cases Incidencec % 95% confidence intervalc
All serotypes
California 7 20.4 3 7.3 −64 −90 to 25
Connecticut 55 25.4 23 11.3 −55 −72 to −30
Georgia 90 31.0 37 9.6 −69 −78 to −56
Maryland 56 34.3 36 21.5 −37 −57 to −9
Minnesota 81 44.9 27 13.7 −69 −80 to −54
New York 16 20.5 8 12.7 −38 −71 to 34
Oregon 16 16.6 6 5.7 −66 −86 to −18
Tennessee 39 44.0 8 8.0 −82 −91 to −62
All 360 31.4 148 11.7 −63 −69 to −55
Serotypes in PCV7d
California 6 16.0 0 0.0 −100 NA
Connecticut 45 21.0 1 0.5 −98 −100 to −83
Georgia 72 24.8 1 0.3 −99 −100 to −93
Maryland 49 29.7 2 1.2 −96 −99 to −84
Minnesota 62 34.1 0 0.0 −100 NA
New York 13 16.7 0 0.0 −100 NA
Oregon 13 13.9 0 0.0 −100 NA
Tennessee 33 37.1 0 0.0 −100 NA
All 293 25.5 4 0.3 −99 −100 to −97
Serotypes not in PCV7d
California 2 4.4 3 7.3 68 −66 to 733
Connecticut 10 4.4 22 10.8 145 33 to 353
Georgia 18 6.2 36 9.3 50 −6 to 138
Maryland 8 4.6 34 20.3 342 141 to 712
Minnesota 20 10.8 27 13.7 27 −22 to 108
New York 3 3.8 8 12.7 230 15 to 851
Oregon 3 2.7 6 5.7 113 −35 to 597
Tennessee 6 6.9 8 8.0 17 −52 to 187
All 70 5.9 144 11.4 93 53 to 144
Open in a new tab

NA, not applicable; PCV7, seven-valent pneumococcal conjugate vaccine.

a PCV7 was introduced in the United States in 2000.

b Areas covered by the Active Bacterial Core surveillance system.

c Cases per 100 000 population.

d PCV7 includes Streptococcus pneumoniae serotypes 4, 6B, 9V, 14, 18C, 19F and 23F.

All invasive pneumococcal disease

Of the 98 individual hospitals that reported at least one hospitalized IPD case among children aged under 5 years in either 1998 or 1999, 93 (95%) recorded a decrease in the number of cases by 2006 (data not shown). In particular, all 26 hospitals with an annual average of more than three cases in 1998 and 1999 saw a decrease. However, the magnitude of the decrease observed at individual hospitals agreed with the magnitude of the decrease observed in the corresponding surveillance area for only 21% of hospitals (Table 3). Twelve of the 16 (75%) hospitals with more than five IPD cases a year during 1998 and 1999 and twelve of the 19 (63%) hospitals that treated more than 10% of their surveillance area’s cases in that period recorded a decrease that agreed in magnitude with that in the corresponding surveillance area. In contrast, only 11% of the hospitals with five or fewer IPD cases a year at baseline and 11% that treated 10% or less of their surveillance area’s cases recorded a decrease in agreement with that in the corresponding surveillance area (Table 3).

Table 3. Proportion of hospitals recording a change in the number of children aged under 5 years with IPD from 1998–1999 to 2006a that agreedb with the change recorded in their surveillance areas,c United States of America.

IPD cases at each hospital IPD caused by any serotype
 IPD caused by serotypes in PCV7d
 IPD caused by serotypes not in PCV7d
Hospitals in agreement/all hospitalse
(No./no.)		 Hospitals in agreement
(%)		 Hospitals in agreement/all hospitalse
(No./no.)		 Hospitals in agreement
(%)		 Hospitals in agreement/all hospitalse
(No./no.)		 Hospitals in agreement
(%)
Annual average of cases in 1998 and 1999f
0 to 5 9/82 11 72/72 100 6/32 19
6 to 10 5/7 71 6/7 86 3/5 60
11 to 15 5/5 100 5/5 100 2/5 40
16 to 20 0/0 NA 0/0 NA 0/0 NA
21 to 25 2/3 67 3/3 100 3/3 100
26 to 30 0/1 0 1/1 100 1/1 100
All 21/98 21 87/88 99 15/46 31
Proportion (%) of the surveillance area's cases during 1998 and 1999f
0–10 9/79 11 68/69 99 5/30 17
11–20 3/4 75 4/4 100 1/3 33
21–30 7/11 64 11/11 100 6/10 60
31–40 0/2 0 2/2 100 1/1 100
41–50 1/1 100 1/1 100 1/1 100
51–60 1/1 100 1/1 100 1/1 100
All 21/98 21 87/88 99 15/46 33
Open in a new tab

IPD, invasive pneumococcal disease; NA, not applicable; PCV7, seven-valent pneumococcal conjugate vaccine.

a PCV7 was introduced in the United States in 2000.

b The change in the number of hospitalized cases of IPD in children aged under 5 years from the average in 1998 and 1999 to 2006 was judged to be in agreement with the change observed in the corresponding surveillance area when the magnitude of the change in the number recorded at the hospital fell within the 95% confidence interval of the magnitude of the change in the incidence of hospitalized IPD cases for the corresponding surveillance area.

c Areas covered by the Active Bacterial Core surveillance system.

d PCV7 includes Streptococcus pneumoniae serotypes 4, 6B, 9V, 14, 18C, 19F and 23F.

e The denominators vary across the columns because some sentinel hospitals either did not have any IPD cases caused by serotypes in PCV7 or did not have any cases caused by serotypes not in PCV7 in 1998 or 1999.

f P < 0.01 for trend (Cochran-Armitage test) for IPD caused by any serotype and for IPD caused by serotypes not in PCV7.

Fig. 1 illustrates the relationship between the number of children aged under 5 years hospitalized with IPD due to all serotypes in either 1998 or 1999 that was covered by a sentinel surveillance system in Tennessee and the likelihood that the magnitude of the decrease in cases recorded by the sentinel system by 2006 would agree with the magnitude of the decrease in the corresponding surveillance area, which here included five counties in Tennessee. Each sentinel system comprised either a single hospital with at least one hospitalized IPD case in either 1998 or 1999 or a group of hospitals, each of which had at least five cases during 1998 and 1999 combined. Overall, in all surveillance areas, more than 95% of sentinel systems that treated either more than 30 cases per year during 1998 and 1999 or more than 50% of cases in the corresponding surveillance area during that period recorded a decrease in all hospitalized IPD cases by 2006 that agreed with the decrease recorded in the corresponding surveillance area (Table 4).

Fig. 1.

Change in number of children aged under 5 years hospitalized with invasive pneumococcal disease (IPD) from 1998–1999 to 2006a recorded by sentinel surveillance systems,b by number of cases in 1998 and 1999, Tennessee, United States of America

PCV7, seven-valent pneumococcal conjugate vaccine.

a PCV7 was introduced in the United States in 2000.

b Each sentinel system comprised either a single hospital with at least one hospitalized IPD case in either 1998 or 1999 or a group of hospitals, in which each hospital had at least five cases during 1998 and 1999 combined.

c A case was a child aged under 5 years who was hospitalized with IPD caused by any Streptococcus pneumoniae serotype.

d The change in the number of IPD cases recorded by the sentinel surveillance system was judged to be in agreement with the change observed in the corresponding surveillance area when the magnitude of the change in number recorded by the surveillance system fell within the 95% confidence interval of the magnitude of the change in the incidence of hospitalized IPD cases for the corresponding surveillance area.

e The whole surveillance area was the area covered by the Active Bacterial Core surveillance system in the United States in which all the sentinel surveillance systems were located; here the area included five counties in Tennessee.

Note: The vertical line indicates the change in the incidence of hospitalized IPD cases recorded in the whole surveillance area.

Fig. 1

Open in a new tab

Table 4. Proportion of sentinel surveillance systemsa recording a change in the number of children aged under 5 years with invasive pneumococcal disease (IPD) from 1998–1999 to 2006b that agreedc with the change recorded in their surveillance area,d United States of America.

IPD cases seen by each sentinel systema IPD caused by any serotype
 IPD caused by serotypes in PCV7e
 IPD caused by serotypes not in PCV7e
Sentinel systems in agreement/all sentinel systemsf
(No./no.)		 Sentinel systems in agreement
(%)		 Sentinel systems in agreement/all sentinel systemsf
(No./no.)		 Sentinel systems in agreement
(%)		 Sentinel systems in agreement/all sentinel systemsf
(No./no.)		 Sentinel systems in agreement
(%)
Annual average of cases in 1998–1999g
0 to 10 57/181 31 164/171 96 41/114 36
11 to 20 230/330 70 301/330 91 210/323 65
21 to 30 280/328 85 328/328 100 204/328 62
31 to 40 218/223 98 223/223 100 205/223 92
41 to 50 42/43 98 43/43 100 41/43 95
51 to 60 46/46 100 46/46 100 46/46 100
61 to 70 25/25 100 25/25 100 25/25 100
71 or more 3/3 100 3/3 100 3/3 100
All 901/1179 76 1133/1169 97 775/1105 70
Proportion (%) of surveillance area's cases in 1998–1999g
0–10 17/103 17 92/93 99 12/50 24
11–20 65/124 52 109/124 88 56/111 50
21–30 114/178 64 158/178 89 108/172 63
31–40 188/231 81 231/231 100 156/229 68
41–50 216/236 92 236/236 100 165/236 70
51–60 148/153 97 153/153 100 125/153 82
61–70 101/102 99 102/102 100 101/102 99
71–80 42/42 100 42/42 100 42/42 100
81–90 10/10 100 10/10 100 10/10 100
91–100 NA NA NA NA NA NA
All 901/1179 76 1133/1169 97 775/1105 70
Open in a new tab

NA, not applicable; PCV7, seven-valent pneumococcal conjugate vaccine.

a Each sentinel system comprised either a single hospital with at least one hospitalized IPD case in either 1998 or 1999 or a group of hospitals in which each hospital had at least five cases during 1998 and 1999 combined.

b PCV7 was introduced in the United States in 2000.

c The change in the number of hospitalized cases of IPD in children aged under 5 years from the average in 1998 and 1999 to 2006 seen by a sentinel surveillance system was judged to be in agreement with the change observed in the corresponding surveillance area when the magnitude of the change in number recorded by the surveillance system fell within the 95% confidence interval of the magnitude of the change in the incidence of hospitalized IPD cases for the corresponding surveillance area.

d Areas covered by the Active Bacterial Core surveillance system.

e PCV7 includes Streptococcus pneumoniae serotypes 4, 6B, 9V, 14, 18C, 19F and 23F.

f The denominators vary across the columns because some sentinel hospitals either did not have any IPD cases caused by serotypes in PCV7 or did not have any cases caused by serotypes not in PCV7 in 1998 or 1999.

g P < 0.0001 for trend (exact Cochran-Armitage test) for IPD due to all serotypes, serotypes in PCV7 and serotypes not in PCV7.

Consequently, it appears that the likelihood that the magnitude of the decrease in all hospitalized IPD cases recorded by a given sentinel surveillance system agreed with that recorded in the corresponding surveillance area increased as the number of cases at baseline increased, whether determined by the total number of cases treated during 1998 and 1999 or by the percentage of cases in the surveillance area treated during that period.

It was also found that the likelihood of agreement between a sentinel surveillance system and its surveillance area in determining the decrease in IPD cases increased as the length of time from the introduction of PCV7 increased from 2 to 4 to 6 years (P < 0.001 for all comparisons, Cochran–Armitage test for trend; data not shown).

Invasive pneumococcal disease by serotype

When the effect of PCV7 on only hospitalized IPD cases in children aged under 5 years that were caused by S. pneumoniae serotypes included in PCV7 was considered, it was found that 97% of all sentinel surveillance systems produced an estimate of the decrease in cases from 1998 and 1999 to 2006 that agreed with the decrease determined for the corresponding surveillance area: the proportion in agreement rose from 93% of sentinel systems with an annual average of 20 or fewer cases caused by all serotypes at baseline to 100% of sentinel systems with an annual average of more than 20 baseline cases (Table 4). Similarly, the proportion in agreement rose from 91% of sentinel systems that treated 30% or less of their surveillance area’s baseline cases caused by all serotypes to 100% of those that treated more than 30% of baseline cases.

In contrast, when the effect of PCV7 on only hospitalized IPD cases caused by serotypes that were not included in PCV7 was considered, only 70% of sentinel systems produced an estimate of the change in cases from 1998 and 1999 to 2006 that agreed with the change determined for the corresponding surveillance area. Moreover, only sentinel systems that treated an annual average of more than 40 IPD cases caused by all serotypes at baseline or more than 60% of their surveillance area’s baseline cases caused by all serotypes produced estimates that consistently agreed with those for the surveillance area (Table 4).

The likelihood of agreement between a sentinel surveillance system and its corresponding surveillance area in determining the change in hospitalized IPD cases, whether or not caused by serotypes included in PCV7, increased significantly as the length of time from the introduction of PCV7 increased from 2 to 4 to 6 years (P < 0.001 for all comparisons, Cochran–Armitage test for trend; data not shown).

Discussion

Our study of sentinel hospital surveillance of the effect of PCV7 in the United States indicates that sentinel surveillance can play a useful role in monitoring the effect of pneumococcal conjugate vaccines on IPD. We recommend that sentinel surveillance systems for monitoring the effect of these vaccines should: (i) establish the baseline burden of IPD before the introduction of the vaccine; (ii) ensure that the number of cases recorded at baseline is sufficient for monitoring future trends; (iii) determine which S. pneumoniae serotypes are responsible for individual cases; and (iv) measure the effect of the vaccine for at least 2 years after its introduction.

The number of IPD cases that should be recorded by a sentinel surveillance system at baseline depends on the goals of surveillance. In our study, almost all individual hospitals in the Active Bacterial Core surveillance areas correctly showed a decline in the number of children aged under 5 years who were hospitalized with IPD due to any serotype. However, only sentinel systems that included an annual average of more than 30 hospitalized IPD cases caused by any serotype during the 1998 to 1999 baseline period and those that included more than 50% of their surveillance area’s cases caused by any serotype during that period reliably measured the magnitude of the decline between baseline and 2006. That a sentinel system needed to have so many baseline cases to ensure that the magnitude of the effect of PCV7 was reliably assessed suggests that many potential sentinel systems may not be able to provide a dependable measure of a vaccine’s effect. For example, in its efforts to measure the burden of IPD around the world, the PneumoADIP programme14 found that, while many sentinel systems with large populations in their catchment areas identified more than 30 hospitalized, laboratory-confirmed cases a year in children aged under 5 years,7,15,16 others identified substantially fewer.7,17–20 However, the alternative approach of establishing sentinel surveillance at hospitals that treat a large proportion of their areas’ IPD cases has the disadvantage that, without a population-based surveillance system, it is difficult to estimate the percentage of cases in the population seen at an individual hospital. Pooling several individual hospitals to create a sentinel system with a large enough total number of baseline cases may be more practical.

We found that sentinel systems performed best when measuring the effect of PCV7 on IPD caused by S. pneumoniae serotypes included in the vaccine: overall, 97% of sentinel systems produced results in agreement with those of their surveillance areas. This was largely because almost no patients were hospitalized with IPD caused by vaccine serotypes in any of the surveillance areas by 2006. The observation that it is important to know which S. pneumoniae serotypes are responsible for the IPD cases is consistent with studies that demonstrated the elimination of invasive Haemophilus influenzae type b in the Gambia21 and substantial reductions in the infection in South Africa,22 despite, respectively, the continued occurrence of and reported increases in invasive non-type-b H. influenzae. In practice, serotyping pneumococcal isolates is even more important than serotyping H. influenzae isolates for assessing the effect of a vaccine since, historically, H. influenzae type b has accounted for almost all invasive H. influenzae disease.23 In contrast, the serotypes in PCV7, PCV10 and PCV13 account for substantially smaller proportions of IPD.24 Moreover, serotyping pneumococcal isolates makes it possible to monitor any increase in IPD caused by serotypes not included in the vaccine, which is a potentially important problem.3,25 In our study, however, the sentinel systems included in the Active Bacterial Core surveillance system were generally less reliable in assessing the magnitude of the change in disease due to non-PCV7 serotypes than the magnitude of the change due to PCV7 serotypes. Although serotyping pneumococcal isolates is difficult in many settings with limited laboratory facilities, the use of techniques based on the polymerase chain reaction may provide a practical solution.26–28

Population-based surveillance systems have shown a decline in IPD within 1 to 2 years of vaccine introduction2,29 and sentinel systems may do the same. However, in our study, a sentinel system’s reliability in measuring the vaccine’s effect on IPD, whether or not caused by serotypes included in PCV7, increased with the passage of time.

Although careful selection of sentinel sites is important, it cannot compensate for fundamental problems in the operation of the surveillance system. A high-quality laboratory is essential for consistently identifying possible cases of the disease of interest, for carrying out accurate confirmatory tests and for recording and communicating case information. Potential problems with staff, supplies, support or funding30 mean that surveillance systems monitoring the effect of a vaccine, especially laboratory-based systems performing new tests, should be operating reliably before the target vaccine is introduced. In addition, staff must be alert to the possibility that any change in the occurrence of an invasive bacterial disease may result from a change in the size of the population under surveillance or a change in the operating or referral pattern of the local health-care area rather than a change in the amount of disease in the population.

Even when a sentinel system provides high-quality data on a vaccine’s effect, the findings should not be overgeneralized. For example, the effect of a vaccine measured by a single hospital that treats more than 30 IPD cases per year should not automatically be used to deduce the vaccine’s effect across a country with tens of millions of people.

This study has several limitations that may limit the generalizability of its findings. Other countries may differ from the United States in care-seeking behaviour among the population, clinical practice, laboratory capacity and pneumococcal epidemiology, all of which would affect the ability of sentinel surveillance to reflect accurately population-based surveillance. In addition, staff at the Active Bacterial Core surveillance system actively seeks out cases, so our findings may not apply to systems that collect case reports passively. Also, since the Active Bacterial Core surveillance system’s data are based on laboratory testing, they may not be useful for the surveillance of clinical syndromes, such as meningitis. Finally, although we expect our findings will apply to other pneumococcal conjugate vaccines, our study addressed only the effect of PCV7 and not that of higher-valence pneumococcal conjugate vaccines such as PCV10 or PCV13.

In addition to monitoring the effect of a vaccine on IPD, sentinel surveillance can also help identify the serotypes causing disease and can record the clinical and demographic characteristics of individual cases. However, the recent global push towards using pneumococcal conjugate vaccines suggests that measuring the effect of a vaccine is a key reason for carrying out surveillance. Although laboratory-based sentinel systems can be useful for monitoring the direction of the effect of a vaccine on the incidence of invasive bacterial disease, if they are to measure reliably the magnitude of the effect, they will have to either include a large number of cases at baseline before the vaccine’s introduction or have the ability to identify the serotype and not just the species of the invasive bacteria. While it is more expensive to create sentinel systems that involve several hospitals or that have a laboratory capable of identifying specific serotypes, it is worthwhile finding the additional resources.

Acknowledgements

We thank the following individuals for their assistance: Pam Daily Kirley and Arthur Reingold at the California Emerging Infections Program; Zack Fraser, James L Hadler and Susan Petit at the Connecticut Emerging Infections Program; Kathryn E Arnold, Wendy Baughman, Monica Farley, Amy Holst and Stephanie Thomas at the Georgia Emerging Infections Program; Lee H Harrison, Rosemary A Hollick and Kim Holmes at the Maryland Emerging Infections Program; Richard Danila, Billie Anne Juni, Lindsey Lesher, Catherine Lexau, Ruth Lynfield and Lori Triden at the Minnesota Emerging Infections Program; Nancy M Bennett, Geetha Nattanmai, Glenda Smith, Suzanne Solghan and Nancy Spina at the New York Emerging Infections Program; Karen Stefonek and Ann Thomas at the Oregon Emerging Infections Program; Brenda G Barnes and William Schaffner at the Tennessee Emerging Infections Program; and Felicita David, Melissa Lewis, Matt Moore, Tamara Pilishvili, Karrie Ann Toews, Chris Van Beneden, and Carolyn Wright at the Centers for Disease Control and Prevention.

Funding:

This work was supported by the Emerging Infections Program and the Office of Antimicrobial Resistance of the Centers for Disease Control and Prevention.

Competing interests:

None declared.

References

  • 1.O’Brien KL, Wolfson LJ, Watt JP, Henkle E, Deloria-Knoll M, McCall N, et al. Burden of disease caused by Streptococcus pneumoniae in children younger than 5 years: global estimates. Lancet. 2009;374:893–902. doi: 10.1016/S0140-6736(09)61204-6. [DOI] [PubMed] [Google Scholar]
  • 2.Whitney CG, Farley MM, Hadler J, Harrison LH, Bennett NM, Lynfield R, et al. Decline in invasive pneumococcal disease after the introduction of protein-polysaccharide conjugate vaccine. N Engl J Med. 2003;348:1737–46. doi: 10.1056/NEJMoa022823. [DOI] [PubMed] [Google Scholar]
  • 3.Pilishvili T, Lexau C, Farley MM, Hadler J, Harrison LH, Bennett NM, et al. Sustained reductions in invasive pneumococcal disease in the era of conjugate vaccine. J Infect Dis. 2010;201:32–41. doi: 10.1086/648593. [DOI] [PubMed] [Google Scholar]
  • 4.Cutts FT, Zaman SM, Enwere G, Jaffar S, Levine OS, Okoko JB, et al. Efficacy of nine-valent pneumococcal conjugate vaccine against pneumonia and invasive pneumococcal disease in the Gambia: randomised, double-blind, placebo controlled trial. Lancet. 2005;365:1139–46. doi: 10.1016/S0140-6736(05)71876-6. [DOI] [PubMed] [Google Scholar]
  • 5.Klugman KP, Madhi SA, Huebner RE, Kohberger R, Mbelle N, Pierce N, et al. A trial of a 9-valent pneumococcal conjugate vaccine in children with and those without HIV infection. N Engl J Med. 2003;349:1341–8. doi: 10.1056/NEJMoa035060. [DOI] [PubMed] [Google Scholar]
  • 6.World Health Organization Pneumococcal conjugate vaccine for childhood immunization – WHO position paper. Wkly Epidemiol Rec. 2007;82:93–104. [PubMed] [Google Scholar]
  • 7.Knoll MD, Moisi JC, Muhib FB, Wonodi CB, Lee EH, Grant L, et al. Standardizing surveillance of pneumococcal disease. Clin Infect Dis. 2009;48:S37–48. doi: 10.1086/596480. [DOI] [PubMed] [Google Scholar]
  • 8.Lynfield R, Van Beneden CA. Public health surveillance for infectious diseases. In: Lee LM, Teutsch SM, Thacker SB, St Louis ME, editors. Principles and practice of public health surveillance, 3rd edn. Oxford: Oxford University Press; 2010. pp. 236–54. [Google Scholar]
  • 9.Global framework for immunization monitoring and surveillance Geneva: World Health Organization; 2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Schrag SJ, Zell ER, Schuchat A, Whitney CG. Sentinel surveillance: a reliable way to track antibiotic resistance in communities? Emerg Infect Dis. 2002;8:496–502. doi: 10.3201/eid0805.010268. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Montana LS, Mishra V, Hong R. Comparison of HIV prevalence estimates from antenatal care surveillance and population surveillance surveys in sub-Saharan Africa. Sex Transm Infect. 2008;84:i78–84. doi: 10.1136/sti.2008.030106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Schuchat A, Hilger T, Zell ER, Farley MM, Reingold A, Harrison L, et al. Active Bacterial Core surveillance of the emerging infections program network. Emerg Infect Dis. 2001;7:92–9. doi: 10.3201/eid0701.010114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.US census populations with bridged race categories [Internet]. Atlanta: Centers for Disease Control and Prevention; 2011. Available from: http://www.cdc.gov/nchs/nvss/bridged_race.htm [accessed 28 March 2012].
  • 14.Levine OS, Cherian T, Hajjeh R, Knoll MD. Progress and future challenges in coordinated surveillance and detection of pneumococcal and Hib disease in developing countries. Clin Infect Dis. 2009;48:S33–6. doi: 10.1086/596479. [DOI] [PubMed] [Google Scholar]
  • 15.Kisakye A, Makumbi I, Nansera D, Lewis R, Braka F, Wobudeya E, et al. Surveillance for Streptococcus pneumoniae meningitis in children aged <5 years: implications for immunization in Uganda. Clin Infect Dis. 2009;48:S153–61. doi: 10.1086/596495. [DOI] [PubMed] [Google Scholar]
  • 16.Saha SK, Naheed A, El Arifeen S, Islam M, Al-Emran H, Amin R, et al. Surveillance for invasive Streptococcus pneumoniae disease among hospitalized children in Bangladesh: antimicrobial susceptibility and serotype distribution. Clin Infect Dis. 2009;48:S75–81. doi: 10.1086/596544. [DOI] [PubMed] [Google Scholar]
  • 17.Batuwanthudawe R, Karunarathne K, Dassanayake M, de Silva S, Lalitha MK, Thomas K, et al. Surveillance of invasive pneumococcal disease in Colombo, Sri Lanka. Clin Infect Dis. 2009;48:S136–40. doi: 10.1086/596492. [DOI] [PubMed] [Google Scholar]
  • 18.Williams EJ, Thorson S, Maskey M, Mahat S, Hamaluba M, Dongol S, et al. Hospital-based surveillance of invasive pneumococcal disease among young children in urban Nepal. Clin Infect Dis. 2009;48:S114–22. doi: 10.1086/596488. [DOI] [PubMed] [Google Scholar]
  • 19.Shah AS, Knoll MD, Sharma PR, Moisi JC, Kulkarni P, Lalitha MK, et al. Invasive pneumococcal disease in Kanti Children’s Hospital, Nepal, as observed by the South Asian Pneumococcal Alliance network. Clin Infect Dis. 2009;48:S123–8. doi: 10.1086/596490. [DOI] [PubMed] [Google Scholar]
  • 20.Falade AG, Lagunju IA, Bakare RA, Odekanmi AA, Adegbola RA. Invasive pneumococcal disease in children aged < 5 years admitted to 3 urban hospitals in Ibadan, Nigeria. Clin Infect Dis. 2009;48:S190–6. doi: 10.1086/596500. [DOI] [PubMed] [Google Scholar]
  • 21.Adegbola RA, Secka O, Lahai G, Lloyd-Evans N, Njie A, Usen S, et al. Elimination of Haemophilus influenzae type b (Hib) disease from the Gambia after the introduction of routine immunisation with a Hib conjugate vaccine: a prospective study. Lancet. 2005;366:144–50. doi: 10.1016/S0140-6736(05)66788-8. [DOI] [PubMed] [Google Scholar]
  • 22.von Gottberg A, de Gouveia L, Madhi SA, du Plessis M, Quan V, Soma K, et al. Impact of conjugate Haemophilus influenzae type b (Hib) vaccine introduction in South Africa. Bull World Health Organ. 2006;84:811–8. doi: 10.2471/BLT.06.030361. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Gessner BD, Adegbola RA. The impact of vaccines on pneumonia: key lessons from Haemophilus influenzae type b conjugate vaccines. Vaccine. 2008;26(Suppl 2):B3–8. doi: 10.1016/j.vaccine.2008.04.013. [DOI] [PubMed] [Google Scholar]
  • 24.Johnson HL, Deloria-Knoll M, Levine OS, Stoszek SK, Freimanis Hance L, Reithinger R, et al. Systematic evaluation of serotypes causing invasive pneumococcal disease among children under five: the pneumococcal global serotype project. PLoS Med. 2010;7:e1000348. doi: 10.1371/journal.pmed.1000348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Hicks LA, Harrison LH, Flannery B, Hadler JL, Schaffner W, Craig AS, et al. Incidence of pneumococcal disease due to non-pneumococcal conjugate vaccine (PCV7) serotypes in the United States during the era of widespread PCV7 vaccination, 1998-2004. J Infect Dis. 2007;196:1346–54. doi: 10.1086/521626. [DOI] [PubMed] [Google Scholar]
  • 26.Morais L, Carvalho Mda G, Roca A, Flannery B, Mandomando I, Soriano-Gabarro M, et al. Sequential multiplex PCR for identifying pneumococcal capsular serotypes from South-Saharan African clinical isolates. J Med Microbiol. 2007;56:1181–4. doi: 10.1099/jmm.0.47346-0. [DOI] [PubMed] [Google Scholar]
  • 27.Dias CA, Teixeira LM, Carvalho Mda G, Beall B. Sequential multiplex PCR for determining capsular serotypes of pneumococci from Brazilian children. J Med Microbiol. 2007;56:1185–8. doi: 10.1099/jmm.0.47347-0. [DOI] [PubMed] [Google Scholar]
  • 28.Njanpop Lafourcade BM, Sanou O, van der Linden M, Levina N, Karanfil M, Yaro S, et al. Serotyping pneumococcal meningitis cases in the African meningitis belt by use of multiplex PCR with cerebrospinal fluid. J Clin Microbiol. 2010;48:612–4. doi: 10.1128/JCM.01402-09. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Vestrheim DF, Lovoll O, Aaberge IS, Caugant DA, Hoiby EA, Bakke H, et al. Effectiveness of a 2+1 dose schedule pneumococcal conjugate vaccination programme on invasive pneumococcal disease among children in Norway. Vaccine. 2008;26:3277–81. doi: 10.1016/j.vaccine.2008.03.087. [DOI] [PubMed] [Google Scholar]
  • 30.Amos B, Kisakye A, Makewa D, Mudhune S, Mwamtemi H, Nansera D, et al. Behind the data: establishing the Network for Surveillance of Pneumococcal Disease in the East African Region. Clin Infect Dis. 2009;48(Suppl 2):S162–71. doi: 10.1086/596496. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Bulletin of the World Health Organization are provided here courtesy of World Health Organization

RESOURCES