Evaluation strategies for measuring pneumococcal conjugate vaccine impact in low-resource settings

Abstract

Background

Pneumococcal conjugate vaccines (PCVs) are effective in reducing pneumococcal disease. We measured 13-valent PCV (PCV13) effect on different pneumococcal outcomes using diverse studies in Lao People’s Democratic Republic (Laos).

Methods

Studies included: pre-PCV13 population-based record review of hospitalized childhood pneumonia cases; acute respiratory infection (ARI) study post-PCV13 to demonstrate effectiveness (VE) against hypoxic pneumonia; invasive pneumococcal disease (IPD) surveillance in all ages (2004-2018); carriage studies in children hospitalized with ARI (2013-2019); community carriage surveys pre- and post-PCV13.

Findings

Annual pneumonia incidence rate in children 2-59 months pre-PCV13 was 1,530 (95% confidence interval [CI] 1,477-1,584) per 100,000. Adjusted VE against hypoxic pneumonia was 37% (95% CI 6-57%). For IPD, 85% (11/13) of cases were due to vaccine-types prior to PCV13, and 43% (3/7) post-PCV13 in children aged <5 years; for ≥5 years, this was 61% (27/44) and 42% (17/40), respectively. For ARI cases, adjusted VE for vaccine-type carriage was 39% (95% CI 4-60) in <5 year olds; slightly higher than community surveys (23% [95% CI 4-39%] in 12-23 month olds).

Conclusion

Despite limited baseline data, we found evidence of PCV13 impact on disease and carriage. Our approach could be used in similar settings to augment existing WHO PCV evaluation guidelines.

1. Introduction

Pneumococcal disease is a common cause of childhood mortality, with 318,000 deaths estimated globally in young children in 2015 [1]. Most pneumococcal deaths are due to pneumonia, with the highest rates in Africa and South-East Asia [1]. Pneumococcal conjugate vaccines (PCVs) are effective against pneumococcal disease and pneumonia in a variety of settings but there are few published studies from low- or middle-income (LMIC) settings in Asia [2].

PCV impact studies should be conducted in different countries to provide geographical representation and show effectiveness in specific populations, as serotype distribution [3] and pneumonia etiology may vary. The World Health Organization (WHO) has published guidelines for countries introducing PCV to assess vaccine impact using a standardized framework and different strategies [4]. At least two years of pre- and five years of post-PCV introduction surveillance data is recommended to determine the population effects of PCV on IPD, in order to account for secular disease trends [4]. Moreover, the measurement of PCV’s impact on childhood pneumonia hospitalizations requires a large sample size of many thousands of admissions each year with a defined catchment to have sufficient power to show evidence of a reduction in disease rates. LMICs may not have robust and complete datasets accessible to undertake this evaluation and establishing surveillance may be costly and resource intensive. It is therefore important to explore other assessment approaches that are feasible in resource-poor settings and with limited baseline data. Advances in implementation science methodology, especially for LMIC, are required.

Governments often want to measure the impact of PCV on pneumonia due to the impact of this condition on health sector human resources, government budgets and child mortality. Measuring impact of PCV on vaccine-serotypes demonstrates evidence of a biological effect specific to the vaccine. We have been undertaking PCV13 impact evaluation studies in Lao People’s Democratic Republic (Laos) since 2013 with the Laos Ministry of Health (MoH) and partners. Here, we describe methodological approaches to evaluate PCV13 impact on disease and serotypes in Laos as a case study for other countries. The Laos MoH introduced the 13-valent PCV (PCV13) in October 2013 [5]. The MoH requested a PCV13 impact evaluation as Laos was undergoing transition from Gavi financing, and they required local evidence of impact particularly given the lack of available data from similar settings in the region. An impact evaluation was not able to be undertaken as per the WHO guidelines as there were no baseline data and vaccine implementation was imminent. In this context, we designed and implemented a series of studies to evaluate the impact of PCV13 in Laos on childhood pneumonia and circulating pneumococcal serotypes.

2. Methods

2.1. Burden of hospitalized pneumonia

We undertook a population-based study to estimate the incidence of hospitalizations due to severe pneumonia among children aged 2-59 months. We conducted a retrospective review of children admitted to all hospitals in a defined catchment of the capital city, Vientiane: five central and nine district hospitals in the pre-PCV13 (2011-2013) period. Vientiane was the only site in Laos with sufficient case numbers to undertake a population-based study. We determined the percentage of all admissions due to pneumonia, clinical features and the percentage with hypoxic pneumonia (see supplementary material for full methods).

The 2013 WHO case definition was used to define pneumonia [6, 7]. Data to determine all-cause and pneumonia hospitalization numbers for children aged 2-59 months were extracted from pediatric admission books for central hospitals and from hospital administrative data for district hospitals. At central hospitals available individual medical records were reviewed to ascertain pneumonia severity and outcomes. Admission books were not available for all facilities for varying durations. To account for missing data, we assumed that the number and pattern of pneumonia and all-cause hospitalizations were stable within hospitals throughout each three-year period (see supplementary material for assumptions).

Data from admission books and administrative data were used to estimate the annual incidence of hospitalized pneumonia in the pre-PCV13 period. For incidence calculations, patients with a known residential address within the defined Vientiane Capital catchment area, and a percentage of those with missing data (based on preexisting data), were included in the numerator from central hospitals. All admissions to district hospitals within Vientiane Capital were included in the incidence calculations for hospitalized pneumonia. The denominator was based on population statistics and annual population growth data from the MoH.

2.2. Vaccine effectiveness against hypoxic pneumonia

We undertook a prospective hospital-based acute respiratory infection (ARI) study in children 2-59 months admitted to any pediatric or intensive care ward from Monday to Friday in a single hospital (Mahosot Hospital) in Vientiane to determine the effectiveness of PCV against hypoxic pneumonia. This hospital admits approximately 400 childhood pneumonia cases annually. This study, using data up until July 2018, has been published [8] and was nested within an ongoing ARI surveillance program commenced in December 2013 [9]. Participants had information recorded on clinical presentation, potential risk factors for carriage and documented PCV13 vaccination status. Children were classified as vaccinated if they had received ≥2 doses of PCV13 (for those aged 0-11 months), or ≥1 PCV13 dose (for those aged ≥12 months) [10]. Children were considered undervaccinated if they did not meet these dose criteria [11].

A test-negative design was used with hypoxic pneumonia (test-positive) defined as pneumonia (2013 WHO definition [6]) with an oxygen saturation of <90% in room air on admission, or requiring oxygen supplementation during hospitalization [8], and all other non-hypoxic enrolled pneumonia cases were the hospital controls (test-negative). To determine the PCV13 VE against hypoxic pneumonia compared to non-hypoxic pneumonia admissions, we used odds ratios from a logistic regression model [VE = (1-OR)*100]. Propensity score matching using inverse probability of treatment weighting was used to adjust for confounding variables and multiple imputation was used to handle missing data. An additional analysis to estimate PCV13 VE against a control condition of total pneumococcal carriage was also conducted. A sample size of 256 cases and 511 controls was determined to have 90% power to detect a VE of 40% assuming that 60% of controls were vaccinated and a 2:1 control-to-case ratio [8].

2.3. PCV13 impact on pneumococcal serotypes

2.3.1. IPD surveillance

IPD surveillance was commenced at Mahosot Hospital, Vientiane in all age groups in 2004 by the Lao Oxford Mahosot Hospital Wellcome Trust Research Unit (LOMWRU) [12, 13]. Blood cultures, lumbar punctures and/or other clinically relevant samples are taken with consent from patients (or their families) admitted with suspected sepsis and meningitis in all age groups who enrolled in the surveillance [12]. A case of IPD was defined as isolation of pneumococcus from a normally sterile site [14]. A case of pneumococcal meningitis was defined as laboratory-confirmation by culture, or identification (i.e. by Gram stain, antigen detection methods) of pneumococcus in the cerebrospinal fluid (CSF) or blood in a child with a clinical syndrome consistent with meningitis [14]. IPD isolates were sent to the Murdoch Children’s Research Institute in Australia for serotyping by latex agglutination/Quellung reaction [15, 16]. IPD case numbers were summarized by year and serotype (PCV13 VT/NVT) for children <5 years and older individuals. Case numbers were compared between the pre- and post-PCV period

2.3.2. Hospital-based pneumococcal carriage surveillance

A hospital-based observational study was conducted using clinical and pneumococcal carriage data [17] from children 2-59 months of age admitted with ARI (fever and one or more respiratory symptoms) as part of ongoing hospital-based surveillance [9]. Data for children admitted to Mahosot Hospital with ARI, methods as previously described for hypoxic pneumonia, was used for children enrolled from December 2013 to June 2019 [17]. Study staff collected data from medical records and immunization registers for enrolled children.

Nasopharyngeal swabs were tested by lytA qPCR to screen for pneumococcal carriage and determine pneumococcal load. Samples that were positive (or equivocal) by lytA qPCR were cultured on selective agar (horse blood agar containing 5 µg/ml gentamicin). Samples with alpha-hemolytic growth had DNA extracted, and molecular serotyping was done by microarray [18].

To demonstrate trends in VT carriage among vaccinated and under-vaccinated children over time, moving carriage prevalence rates within 7-month rolling intervals were calculated using a direct adjustment method [17]. Associations between both vaccine effectiveness and village-level PCV13 coverage and VT carriage were estimated using generalized estimating equations, adjusted for potential confounders. VE was calculated as one minus the odds ratio of PCV13 carriage in vaccinated vs. under-vaccinated children, as previously defined [17].

2.3.3. Community pneumococcal carriage surveys in children

Two cross-sectional community nasopharyngeal carriage surveys were conducted and have been published [19]. These were undertaken at the start of PCV13 introduction (November 2013 - February 2014) and two years following (November 2015 - February 2016) introduction to assess VT and NVT serotype changes among healthy children aged 12-23 months and infants too young to be vaccinated (aged 5-8 weeks). A convenience sample of healthy infants and children were recruited from urban and rural immunization clinics. Both surveys were performed over the same months in the relevant years to account for seasonality [19].

3. Results

3.1. Burden of hospitalized pneumonia

In the pre-PCV period, 2011-2013, hospital admission books were available from three of the central hospitals for the entire period (Table 1). Administrative data were available for eight out of nine district hospitals. Four district hospitals (Pak Ngum, Santhong, Naxaithong and Hadxaifong District Hospitals) were included in the calculations of the percentage of all-cause hospitalizations due to pneumonia, and in annual incidence calculations of hospitalized pneumonia in Vientiane. Four district hospitals (Sikkotabong, Sisatanak, Xaysettha and Chanthabouly District Hospitals) had no pneumonia hospitalizations recorded for children <5 years for 2011-2013. Based on assumptions using identified data (see supplement), we estimated a total of 18,687 all-cause hospitalizations with 3,801 (20%) due to pneumonia over three years (Table 1). The district hospitals contributed very few numbers to the total all-cause hospitalizations (n=1,154, 6%) and total pneumonia hospitalizations (n=165, 4%).

Table 1.

All-cause hospitalizations (n = 18,687) and pneumonia hospitalizations (n = 3,801) of children aged 2-59 months in Vientiane, by hospital and year from available and estimated data.

Hospital		2011	2012	2013
Central Hospitals
Settathirath Hospital	Number of pneumonia hospitalizations	133	134	157
	Number of all-cause hospitalizations	720	780	837
	% of all-cause hospitalizations due to pneumonia	18.5%	17.2%	18.8%
Mahosot Hospital	Number of pneumonia hospitalizations	325	333	256
	Number of all-cause hospitalizations	1321	1435	1607
	% of all-cause hospitalizations due to pneumonia	24.6%	23.2%	15.9%
National Child Hospital	Number of pneumonia hospitalizations	N/A	603a	603a
	Number of all-cause hospitalizations	N/A	2762a	2762a
	% of all-cause hospitalizations due to pneumonia	N/A	21.8%a	21.8%a
Mother and Child Hospital	Number of pneumonia hospitalizations	603a	N/A	N/A
	Number of all-cause hospitalizations	2762a	N/A	N/A
	% of all-cause hospitalizations due to pneumonia	21.8%a	N/A	N/A
Hospital 103	Number of pneumonia hospitalizations	163a	200a	126
	Number of all-cause hospitalizations	849a	943a	755
	% of all-cause hospitalizations due to pneumonia	19.2%a	21.2%a	16.7%
District Hospitals
	Pneumonia hospitalizations	47	55	63
	All-cause hospitalizations	274	443	437
	% pneumonia hospitalizations	17.2%	12.4%	14.4%
Total	Pneumonia hospitalizations	1271	1325	1205
	All-cause hospitalizations	5926	6363	6398
	% pneumonia hospitalizations	21.4%	20.8%	18.8%

^aEstimated data: accounted for missing data from hospital admission books using assumptions based on a number of consistent intra-hospital and inter-hospital patterns in monthly and annual admissions from the available records. It was assumed that the number of pneumonia and all-cause hospitalizations followed the same pattern within hospitals throughout the three-year period. N/A: not available

Out of a total of 1,999 identified pneumonia patients, 931 (46.6%) individual medical records were retrieved for review. Of 931 pneumonia cases with available medical records, 513 (55%) were severe, 146 (16%) were hypoxic, 136 (15%) required ICU admission and 45 (5%) were unwell at discharge. Only 3 children were recorded as having died during admission (Table 2). The estimated annual incidence of hospitalized pneumonia in Vientiane was 1,530 per 100,000 children aged 2-59 months (95% confidence interval [CI] 1477-1584) for the 2011-2013 pre-PCV13 period.

Table 2.

All-cause hospitalizations (n = 9,674) and severity classification and clinical characteristics of pneumonia hospitalizations with medical records (n = 931) by age group, for children aged 2-59 months in central hospitals, Vientiane, 2011-2013

	2-59 months	2-5 months	6-11 months	12-23 months	24-59 months
All-cause hospitalizationsa	n = 9674	n = 1167	n = 2243	n = 2844	n = 3420
All pneumonia hospitalizationsa, n [% of all-cause hospitalizations]	1999 [20.7%]	386 [48.5%]	457 [25.0%]	623 [27.7%]	533 [18.2%]
Pneumonia hospitalizations with medical recordsb, n [% 2-59m pneumonia hospitalizations]	n = 931	n = 141 [15.1%]	n = 202 [21.7%]	n = 335 [36.0%]	n = 253 [27.2%]
Severity classification,n [% by age group]
Pneumonia (non-severe)c	418 [44.9%]	54 [38.3%]	82 [40.6%]	158 [47.2%]	124 [49.0%]
Severe pneumoniad	513 [55.1%]	87 [61.7%]	120 [59.4%]	177 [52.8%]	129 [51.0%]
Clinical features and management, n [% by age group]
Cyanosis or hypoxiae	146 [15.7%]	45 [31.9%]	36 [17.8%]	45 [13.4%]	20 [7.9%]
ICU admission	136 [14.6%]	45 [31.9%]	39 [19.3%]	32 [9.6%]	20 [7.9%]
Supplemental oxygen required	159 [17.1%]	54 [38.3%]	39 [19.3%]	39 [11.6%]	27 [10.7%]
Outcomes, n [% by age group]
Alive and well	883 [94.8%]	125 [88.7%]	191 [94.6%]	324 [96.7%]	243 [96.0%]
Unwell at discharge/discharged home to dief	45 [4.8%]	15 [10.6%]	10 [5.0%]	10 [3.0%]	10 [4.0%]
Dead	3 [0.3%]	1 [0.7%]	1 [0.5%]	1 [0.3%]	0 [0.0%]

^aFrom available central hospital admission books

^bFrom available individual medical records from central hospitals

^cPneumonia defined as: cough or difficulty breathing, and one of either tachypnea (>50 breaths per minute (bpm) if 2-11 months old; or >40 bpm if 12-59 months old); or chest indrawing

^dSevere pneumonia defined as: criteria for pneumonia, plus either central cyanosis, or oxygen saturation < 90% in room air, severe respiratory distress, inability to drink, lethargy, unconsciousness, or convulsions

^eSpO₂ <90% on admission

^fIncludes patients discharged for the following documented reasons: discharged against medical advice, child remained unwell or in severe condition, evidence of severe pneumonia at time of discharge or in preceding 24 hours (cyanosis or hypoxia, oxygen use at time of discharge, severe respiratory distress, inability to drink, lethargy or reduced consciousness or convulsions)

3.2. Vaccine effectiveness against hypoxic pneumonia

Of 1,375 participants with ARI, 826 children had pneumonia, with about one-third being hypoxic (n=285, 34.5%). Of the pneumonia cases, 682 (82.5%) had PCV13 vaccination status available, and of these 55.3% (n=377) were appropriately vaccinated. The adjusted VE against hypoxic pneumonia was 37% (95% CI 6, 57%) using propensity score analysis. Using multiple imputation to account for missing variables, the adjusted VE against hypoxic pneumonia was similar [35% (95% CI 7, 55%)]. The VE against the control condition, total pneumococcal carriage, was -6% (95% CI: -54, 28%) [8].

3.3. PCV13 impact on pneumococcal serotypes

3.3.1. IPD surveillance

From 2004-2019, there were 104 IPD cases; of these 28 had meningitis, 41 had pneumonia, 12 had sepsis and 23 had another primary diagnosis. Most patients (81%, 84/104) were aged ≥5 years. Figures 1 and 2 show the numbers of IPD cases identified by year and serotype in both children and older individuals. A higher percentage of VT cases were identified in the pre-PCV compared with the post-PCV period, for both those aged <5 years (from 85%, 11/13 to 43%, 3/7) and ≥5 years (from 61%, 27/44 to 42%, 17/40). In young children, the individual serotypes identified in the post-PCV13 period (Figure S1) were 10A (n=1), 14 (n=2), 15A (n=1), 15C (n=1), 35B (n=1) and 6B (n=1). In individuals aged ≥5 years, the most common individual serotypes (Figure S2) in the post-PCV13 period (from 2013) were 15A (n=4), 33B (n=3), 19A (n=4), 1 (n=3), 3 (n=3) and 4 (n=3).

Figure 1.

Number of invasive pneumococcal disease cases in participants <5 years, by year and PCV13 serotype, MahosotHospital, Vientiane, 2004-2019 (n=20)

Figure 2.

Number of invasive pneumococcal disease cases in participants >5 years, by year and PCV13 serotype, MahosotHospital, Vientiane, 2004-2019 (n=84)

3.3.2. Hospital-based pneumococcal carriage surveillance

Hospital-based surveillance recruited 1,423 eligible cases with ARI from December 2013 to June 2019; of these cases 455 (32%) had severe pneumonia, pneumococcal carriage prevalence was 35.8% (509/1,421) with VT carriage detected in 42.5% (189/445) of patients with serotyping results [17]. Approximately 60% (724/1209) of participants, with a known PCV13 status, were appropriately vaccinated. Monthly VT carriage prevalence (7-month rolling intervals) declined in both vaccinated and undervaccinated cases between 2013 and 2019. VT carriage prevalence reduced from 20.0% (95% CI 15.2-24.9%) to 12.8% (95% CI 8.5-17.1%) with increasing vaccine coverage. Adjusted PCV13 effectiveness was 38.1% (95% CI 4.1-60.0) against VT carriage [17].

3.3. Community carriage surveys

As reported previously [19], a total of 1,000 infants (5-8 weeks) and 1,010 children (aged 12-23 months) participated in the pre- and post-PCV13 community carriage surveys. In the pre-PCV13 period, 7% (95% CI 5-9) of infants (n=498) and 33% (95% CI 29-37) of children (n=503) carried a PCV13 serotype. In the post-PCV13 period, 5% (95% CI 3-8) of infants (n=502) and 20% (95% CI 16-24) of children (n=507) carried a PCV13 serotype. Although there was no significant reduction between the two surveys in VT carriage rates in the infant age group, there was a 23% (95% CI 4-39%) reduction in the carriage of VT serotypes in the toddler age group. There was no significant change in NVT carriage in either age group [19].

4. Discussion

Using a variety of methods and limited preceding baseline data, we were able to demonstrate that PCV13 had direct and indirect effects on pneumonia, and pneumococcal vaccine-type carriage and disease in Laos. Combined, the results from the different studies provide sufficient evidence of the impact of PCV13 introduction in the context of limited baseline data.

In LMICs, determining the effect of PCV introduction on non-specific outcomes such as pneumonia using impact or VE studies is challenging. Advances in implementation science methodology, especially for LMICs, are needed. Countries often rely on routinely collected administrative data on pneumonia hospitalizations to document the pneumonia burden before and after PCV introduction. There are few published studies demonstrating the use of administrative data in LMICs to assess the impact of PCV. A study from Rwanda [20] collected meningitis and pneumonia pediatric admission information retrospectively from hospital registers and a meningitis surveillance database, demonstrating feasibility of this approach with appropriate data. Similarly, in Brazil the use of administrative data was shown to be a feasible approach to document a decline in pneumonia hospitalizations post-PCV10 introduction [21]. In preparation for a PCV impact evaluation in Fiji, an audit was undertaken to assess the completeness and consistency of routinely collected hospital admission data [22]. The audit found that completeness of admission data and coding was high, and suitable for impact evaluation, although consistency of complete reporting varied, with the highest rates in tertiary hospitals using an electronic database. Missing data is common in routine administrative data. A study from Fiji using administrative data imputed missing data by multiple imputation [23].

Care needs to be taken with using routinely collected data, however, given that admission criteria and clinician practice may change over time. We have previously found [24] that the change in WHO pneumonia case definition resulted in an apparent reduction in the percentage of all-cause pneumonia hospitalizations attributable to severe pneumonia, as well as a lower annual incidence of severe pneumonia hospitalizations, when comparing the 2013 definition with the 2005 definition. This has implications for the interpretation of pneumonia intervention impact evaluations [24]. Missing data may also affect study results and effectiveness estimates. Despite having missing data in Laos, we were able to estimate, for the first time in this setting, the incidence of hospitalized pneumonia using local administrative data. We also demonstrated how common pneumonia is as a cause for all childhood hospitalizations and the burden in the youngest age groups. This information provides important supportive evidence to recommend the ongoing use of PCV in Laos. However, we were not able to do a hospital administrative review for pneumonia cases post-PCV with a PCV impact evaluation as we had insufficient power to undertake this; a simulation analysis based on pre-PCV13 data suggested that with the case load in the population selected, we had only 63% power to detect a 20% decline in pneumonia admissions in the three years post-PCV13, using an interrupted time series analysis [25]. It is important to undertake power calculations prior to embarking on pneumonia impact evaluations as there is a danger of drawing erroneous conclusions if the study is undertaken but underpowered [26].

Case-control studies are often used to evaluate vaccine effectiveness following vaccine introduction. There are however a number of challenges and limitations when conducting these types of studies [27, 28]. Using cases and controls admitted to a single hospital and a novel approach using a test negative design, we were able to determine the effect of PCV on pneumonia in a low-resource setting by determining PCV13 VE against hypoxic pneumonia compared with non-hypoxic pneumonia admissions in a single hospital [8]. This was the first study in Asia showing PCV VE against hypoxic pneumonia. It employed simple methodology and is feasible to be undertaken in other LMICs using a single hospital. To adjust for confounding we used propensity score analysis to balance covariates between exposure groups, which strengthened internal validity of results. These results are important as child mortality in Laos is one of the highest in the region, pneumonia is common, and hypoxic pneumonia is the usual cause of death. Thus, this vaccine is likely to contribute to improved child survival in Laos. In addition, PCV introduction has been found to coincide with a decline in all-cause pneumonia mortality which is an added benefit [29].

IPD surveillance has been ongoing in Laos since 2004, however the reported IPD numbers are few. These numbers are likely to be a considerable underestimate of the true disease burden in the population, as there are few available bacteriology laboratories, and challenges to accessing health care [30] with antibiotics frequently given prior to hospital admission [31]. The surveillance is also sentinel at one hospital in the capital city. This makes it difficult to draw any firm conclusions from the data, although PCV13 serotypes have declined since PCV13 introduction. There are a number of obstacles to establishing robust surveillance; these include both the cost and difficulty in starting up and maintaining standardized clinical and laboratory surveillance practices [32].

The impact of PCV on VT serotypes in IPD can be augmented using carriage studies. A study from Cambodia [33] using data from six outpatient clinic carriage surveys and a hospital-based invasive pneumococcal disease (IPD) program demonstrated a vaccine effectiveness (VE) of 39.2% (95% CI 26.7–46.1%) for vaccine type (VT) carriage and a 42.3% (95% CI 67.3 to −1.5%) decline in VT IPD. Combining colonization and IPD results, translated into a VT IPD decline of 36.3% (95% CI 23.8–46.9) [33]. In Laos, using data from two cross-sectional carriage surveys, we demonstrated impact on vaccine-type nasopharyngeal carriage in children aged 12-23 months, but no indirect effects in infants too young to be vaccinated. However, given the post-PCV13 survey was undertaken two years following vaccine introduction, this may be too early to show indirect effects [19]. Infants too young to be vaccinated are at high risk of pneumococcal disease, especially meningitis. Carriage surveys allow us to measure the impact of PCV13 in this young age group and reflect herd protection [34]. It is assumed that a reduction in pneumococcal carriage would translate into a reduced incidence of pneumococcal disease. Vaccine schedules, immunization coverage and catch-up campaigns play a role in producing indirect effects in a population [35]. Catch-up campaigns accelerate indirect PCV effects through the reduction of VT carriage in toddlers who are the major reservoir for VT transmission to unvaccinated and undervaccinated groups [36]. Laos had a limited catch-up campaign to 12 months of age which relied on children being brought to the health center (for 9 month immunization visits and other non-immunization visits). Uptake was therefore limited in the older age group, allowing for continued circulation of VTs in the community. For VT pneumococcal carriage to be interrupted in the community with a suboptimal catch-up campaign requires approximately 3-4 years with vaccination of multiple birth cohorts.

In our hospital surveillance program, vaccinated ARI cases had lower VT pneumococcal carriage rates than undervaccinated cases. In contrast total pneumococcal carriage rates were unchanged, which is to be expected as VT decline while NVT increase following PCV13 introduction [37]. While we did not observe indirect effects in infants from the community carriage surveys, our hospital-based carriage surveillance indicates increasing indirect effects as vaccine coverage increases – with reductions in VT carriage from 20% to 12.8% among ARI cases as PCV13 coverage increases from 0-60% four years after PCV introduction.

PCV causes an increase in disease from NVT serotypes (serotype replacement). The extent to which replacement occurs in LMICs, especially in Asia, is not well described. It is also unclear whether replacement disease could threaten pneumococcal disease control. Given the challenges with conducting robust IPD surveillance in LMICs, it is important to show whether pneumococcal carriage can predict replacement disease in these settings.

The strength of evidence for PCV impact provided by different study designs, study endpoints and sample types are variable. Although national comprehensive IPD surveillance is considered the “gold standard”, IPD is a rare event and case identification is usually through culture-based detection methods which results in the under-detection of cases. Detection may be improved through the use of molecular methods but often this is not available. Pneumonia is the most common cause of pneumococcal disease mortality, however undertaking pneumonia surveillance to determine the population effect of PCV is challenging and resource intensive. We present an alternative implementation science method using the test negative design that despite not demonstrating the population effect of PCV, shows that PCV is effective against the most severe form of pneumonia, hypoxic pneumonia, using a single hospital approach. Additionally, to demonstrate a biological impact of PCV on circulating pneumococcal serotypes, monitoring pneumococcal nasopharyngeal carriage in children with pneumonia or healthy people from the community may be a useful and relatively inexpensive way to monitor vaccine impact in children and demonstrate the indirect effects in adults and other age groups.

Laos introduced PCV13 in a 3+0 schedule in 2013 based on the WHO recommendations for introduction strategies in LMICs which were in place at this time. This schedule has previously been shown in other countries to have indirect effects on vaccine-type IPD when introduced as part of a national campaign. The described methods could be used to augment existing WHO PCV evaluation guidelines for countries with limited resources and baseline data, and imminent vaccine introduction [4]. This is important for LMICs, and countries in the Asian region with limited access to resources to establish surveillance and undertake impact evaluations. However, further work is required to understand how to measure and monitor replacement in LMICs in the post-PCV era with imperfect surveillance systems.

Our evaluation provides important data to support the ongoing use of PCV in Laos as it was used for evidence-based decision making by government decision makers to continue to fund the continuation of PCV13 as Lao PDR transitions from Gavi support.. It demonstrates how vaccine impact can be measured using methods that do not require pre-existing surveillance. Most program components are relatively simple and could be applied to document vaccine impact in other LMIC settings without complex laboratory-based studies. We do not recommend that these methods replace WHO guidelines but augment them for countries that are unable to undertake traditional evaluations. On their own, some components, such as the retrospective record review, did not have sufficient power to demonstrate a population effect on all-cause hospitalized pneumonia but this could be alleviated in other settings with higher case numbers. However, in conjunction with other supporting data, the impact of PCV13 in this setting could be determined and has provided sufficient evidence to support the ongoing use of PCV in Laos.

5. Conclusion

Despite limited baseline data in Laos, we found adequate evidence of PCV13 impact on disease and vaccine-type serotypes using a combination of studies. Our evaluation approach could be used in similar settings to augment existing WHO PCV evaluation guidelines and document vaccine effects.