Abstract
Background
The Global Polio Eradication Initiative (GPEI) Strategic Plan for 2019–2023 includes commitments to monitor the quality of immunization campaigns using lot quality assurance sampling surveys (LQAS) and to support poliovirus surveillance in Pakistan and Afghanistan.
Methods
We analyzed LQAS and poliovirus surveillance data between 2016 and 2020, which included both acute flaccid paralysis (AFP) case-based detection and the continued expansion of environmental surveillance (ES). Using updated estimates for unit costs, we explore the costs of different options for future poliovirus monitoring and surveillance for Pakistan and Afghanistan.
Results
The relative value of the information provided by campaign quality monitoring and surveillance remains uncertain and depends on the design, implementation, and performance of the systems. Prospective immunization campaign quality monitoring (through LQAS) and poliovirus surveillance will require tens of millions of dollars each year for the foreseeable future for Pakistan and Afghanistan.
Conclusions
LQAS campaign monitoring as currently implemented in Pakistan and Afghanistan provides limited and potentially misleading information about immunization quality. AFP surveillance in Pakistan and Afghanistan provides the most reliable evidence of transmission, whereas ES provides valuable supplementary information about the extent of transmission in the catchment areas represented at the time of sample collection.
Keywords: dynamic modeling, eradication, polio, surveillance
The Global Polio Eradication Initiative (GPEI) 2019–2023 Strategic Plan recognizes the necessity of achieving very high immunization coverage to stop and prevent the transmission of wild polioviruses (WPVs) and circulating vaccine-derived polioviruses (cVDPVs) [1]. Efforts to intensify immunization activities remain challenging, but they have successfully led to a record low number of cases in Pakistan [2]. However, beginning in 2018 a deterioration in overall supplementary immunization activity (SIA) coverage and quality led to a resurgence in WPV1 cases in Pakistan and emergence of cVDPV2 in 2019 [1, 3]. Since 2017, Pakistan and Afghanistan have been the last remaining reservoirs of serotype 1 WPV (WPV1) transmission and the main obstacle to global polio eradication [3–6]. Several prior studies modeled poliovirus transmission in Pakistan and Afghanistan assuming 1 epidemiological block with preferentially mixing subpopulations of undervaccinated individuals in each country [7–11]. These studies supported the need for intensified efforts aimed at delivering oral poliovirus vaccine (OPV) using SIAs and specifically targeting undervaccinated children [7–10]. A statistical analysis designed to compare the effectiveness of bivalent OPV (bOPV) and serotype 1 monovalent OPV (mOPV1) based on observations in young children in Pakistan and Afghanistan reported poor and declining immunization coverage and suggested that reaching the persistently missed children and achieving SIA coverage of at least 80% would make WPV1 eradication feasible [12].
Issues with poor program performance (discussed in the supplement) led to substantial investment in the use of lot quality assurance surveys (LQAS), a statistical method developed in 1929 to optimize the inspection of manufactured goods [13]. The original manufacturing context assumed random sampling from a static production process and sought to develop a barrier to ensure quality defects below a specific tolerance level with a minimum cost for inspection [13]. Thus, by design, LQAS provides a low-cost sampling strategy for each population sampled at the time of sampling, and it involves collecting a small random sample from the specific population (or lot) and testing each item in the sample for failure to determine whether the lot is acceptable. LQAS provides information about the specific lot(s) sampled but is not designed to and does not provide a precise estimate of population parameters, for example, production defect rates. The GPEI uses LQAS to monitor the quality of routine immunization (RI) [14] and SIAs [15] in some countries. Analyses of the value of the information provided by LQAS suggest some potential help with respect to identifying program weaknesses (eg, poor quality of delivery of RI or SIA doses in the sampled population) [15]. However, the increasing use of LQAS [16, 17] prompted some debates about the validity of its use and information value due to the design of the monitoring studies and their substantial departures from the typical uses and assumptions required for valid LQAS inferences [18, 19].
In addition to immunization, poliovirus surveillance remains one of the primary pillars of the GPEI [1]. However, as cases of polio become increasingly rare, countries may question the value of their continued investments in poliovirus surveillance activities. Prior global policy modeling assumed that high-quality acute flaccid paralysis (AFP) surveillance would continue throughout the completion of polio eradication and for some time beyond, which would ensure rapid detection of any reintroduced live polioviruses (if these occurred) [20]. However, anticipating its success, the GPEI began transitioning resources, including funds used to support surveillance activities, out of some polio-free countries starting in the mid-2010s [21]. At the same time, the GPEI also increasingly recognized the potential value of conducting environmental surveillance (ES), which involves collecting and testing sewage samples [22], particularly for high-risk areas. This led to substantial investments in ES in some countries, particularly in Pakistan and Afghanistan (see the Supplementary Data for a brief review of prior statistical analyses) [23–30]. During 2016–2020, Pakistan and Afghanistan evaluated the quality of their ES sites, which led to the closure of some sites and opening of new sites [31]. The main limitation of ES remains that it cannot be conducted in many locations due to the absence of appropriate sampling sites, often in the same areas and populations with suboptimal AFP surveillance.
Program monitoring and surveillance activities require financial investments to support field and laboratory infrastructure and activities. Several recent studies provide updated estimates for cost and valuation model inputs [32], costs of the Global Polio Laboratory Network (GPLN) activities [33], and the economics of the GPEI as of early 2020 [34]. In 2020, the emergence of severe acute respiratory syndrome coronavirus 2, which led to the coronavirus disease 2019 (COVID-19) pandemic, affected global poliovirus transmission dynamics [35], and this motivated updated modeling of poliovirus transmission dynamics for Pakistan and Afghanistan that consider the impacts of COVID-19 [36]. This study uses recent LQAS, AFP, and ES data for Pakistan and Afghanistan to explore trends and unit cost inputs to estimate expected costs for LQAS, AFP, and ES activities for 2019–2023.
METHODS
A companion paper with an updated Pakistan and Afghanistan model [36] explores the immunization coverage estimates required to allow outcomes that approximate the same retrospective transmission dynamics as observed during 2016 and 2020 in Pakistan and Afghanistan. In support of the updated modeling, we performed statistical analyses of the available LQAS data for Pakistan and Afghanistan to explore areas of real or perceived inconsistencies between the modeling [36] and expectations based on epidemiological data. To perform this analysis, we listed all completed 2016–2020 SIAs using unique code names and matched these with reported LQAS campaign monitoring results (as of November 12, 2020). We identified 173 SIAs (ie, 93 in Pakistan and 80 in Afghanistan) with unique code names in this time frame, 152 of which appeared completed (ie, 81 in Pakistan and 71 in Afghanistan). We identified 86 total LQAS campaign monitoring activities for the same time period (ie, 45 in Pakistan and 43 in Afghanistan). Each LQAS lot targeted 60 children, except for some lots in April and May of 2017 in Pakistan that targeted 50 children. Table 1 summarizes the information and shows that both SIA target populations and the number of monitoring lots per SIA vary considerably. We performed regression analyses to explore trends in the data.
Table 1.
Lot Quality Assurance Sampling Summary Table for Supplementary Immunization Activities in Afghanistan (n = 43) and Pakistan (n = 45) in 2016–2020a
| Fraction of Age Group Targeted By SIA | No. of Lots per SIA | No. of Children Checked | No. of Children Found Unvaccinated | Percent Unvaccinated Children | |
|---|---|---|---|---|---|
| Afghanistan, n = 43 | |||||
| Average | 0.65 | 97 | 5791 | 415 | 7.3 |
| Median | 0.60 | 95 | 5700 | 408 | 6.7 |
| Minimum | 0.02 | 2 | 120 | 7 | 2.3 |
| Maximum | 1.17 | 163 | 9780 | 880 | 20.0 |
| Pakistan, n = 45 | |||||
| Average | 0.60 | 498 | 29 843 | 1103 | 3.7 |
| Median | 0.58 | 596 | 35 760 | 1296 | 3.6 |
| Minimum | 0.02 | 8 | 480 | 24 | 1.3 |
| Maximum | 1.15 | 1317 | 79 020 | 2538 | 9.6 |
Abbreviation: SIA, supplementary immunization activity.
aData as of November 12, 2020.
We also analyzed the available AFP surveillance and ES data for Pakistan and Afghanistan (Table 2) to estimate the level of surveillance between 2016 and 2020 and explore possible surveillance options for 2021–2023. Specifically, we listed all reported AFP cases and calculated nonpolio AFP (NPAFP) rates per 100 000 children under 15 years of age using the median variant of the 2019 revision of the UN World Population Prospects [37]. For ES, we used a list of collected samples by site and assessed the sampling frequency and the number of positive samples per site.
Table 2.
Acute Flaccid Paralysis Surveillance and Environmental Surveillance Summary Table for Afghanistan and Pakistan in 2016–2020a
| Reported AFP Cases per Year | Reported Nonpolio AFP Cases per Year | Nonpolio AFP Rate per 100 000 Children <15 Years Old per Year | No. of Active ES Sampling Sites per Year | Sampling Frequency per ES Site per Year | No. of Positive WPV1 or VDPV2 Samples per ES Site per Year | |
|---|---|---|---|---|---|---|
| Afghanistan | ||||||
| Average | 3279 | 3225 | 20 | 20 | 15 | 3 |
| Median | 3251 | 3080 | 19 | 20 | 13 | 2 |
| Minimum | 2905 | 2891 | 18 | 15 | 2 | 0 |
| Maximum | 3768 | 3739 | 23 | 23 | 26 | 16 |
| Pakistan | ||||||
| Average | 11 073 | 10 997 | 15 | 63 | 11 | 4 |
| Median | 10 330 | 10 322 | 14 | 59 | 12 | 2 |
| Minimum | 7848 | 7826 | 11 | 53 | 1 | 0 |
| Maximum | 15 216 | 15 046 | 20 | 72 | 36 | 15 |
Abbreviations: AFP, acute flaccid paralysis; ES, environmental surveillance; EV, enterovirus; VDPV2, serotype 2 vaccine derived poliovirus; WPV1, serotype 1 wild poliovirus.
aData as of November 12, 2020.
To estimate the costs for continued surveillance and campaign quality monitoring in Pakistan and Afghanistan for 2019–2023, we applied recent unit cost inputs for poliovirus surveillance expressed in 2019 US dollars (US$2019) [32], and we estimated the cost per LQAS lot (Table 3). Table 3 shows AFP surveillance unit cost per child under 15 without adjustment (ie, assuming perfect surveillance that detects every case, defined as a detection threshold of 1) [32] and with adjustment for less-than-perfect AFP surveillance assuming an average detection threshold of 3.6 for Pakistan and 4.3 for Afghanistan [6]. We estimate the monitoring and surveillance costs for 2019–2020 using the unit costs and reported activities. To project the costs for 2021–2023, we explore several possible monitoring and surveillance options. Specifically, for the monitoring we assume that LQAS will continue for all national immunization days (NIDs) and subnational immunization days (SNIDs), and we assume AFP surveillance at the same annual level. Although the actual ES sampling frequency will likely depend on experience with WPV1 and cVDPV2 transmission, we retain the number of sampling sites per country as of Q4 2020 (ie, 23 in Afghanistan and 68 in Pakistan). We consider 3 scenarios for ES sampling frequency at the 23 sites in Afghanistan: (i) remaining at the same level (“Baseline”), (ii) reverting to the default ES sampling frequency of 12 samples per site per year (“Default”), and (iii) increasing all ES sampling frequencies to 24 samples per site per year (“Reactive”) (Table 4).
Table 3.
Surveillance and Campaign Quality Monitoring Unit Cost Estimates (US$2019)
| Input | Afghanistan | Pakistan |
|---|---|---|
| LQAS campaign monitoring cost per lot | $1600.00 | $1600.00 |
| AFP surveillance cost per child under 15 | ||
| Unadjusted [32] | $0.16 | $0.22 |
| Adjusted for detection threshold | $0.04 | $0.06 |
| ES cost per sample | $1000.00 | $2000.00 |
Abbreviations: AFP, acute flaccid paralysis; ES, environmental surveillance; LQAS, lot quality assurance survey.
Table 4.
Projected Sampling Frequency per Environmental Surveillance Site per Year for Afghanistan and Pakistan for 2020–2023 for Different Scenarios
| No. of Active ES Sampling Sites of Given Sampling Frequency | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 2020 | Baseline | Default | Reactive | |||||||
| Sampling Frequency per ES Site per Year | 2021 | 2022 | 2023 | 2021 | 2022 | 2023 | 2021 | 2022 | 2023 | |
| Afghanistan | ||||||||||
| 12 | 6 | 6 | 6 | 6 | 23 | 23 | 23 | 0 | 0 | 0 |
| 18 | 11 | 11 | 11 | 11 | 0 | 0 | 0 | 0 | 0 | 0 |
| 24 | 6 | 6 | 6 | 6 | 0 | 0 | 0 | 23 | 23 | 23 |
| Pakistan | ||||||||||
| 12 | 68 | 68 | 68 | 68 | 68 | 68 | 68 | 68 | 68 | 68 |
Abbreviations: AFP, acute flaccid paralysis; ES, environmental surveillance.
RESULTS
LQAS occurred for 56% of SIAs for Pakistan and 61% of SIAs for Afghanistan. Assuming a uniform distribution of the results for lots over the population covered by the SIAs implies an average (range) of 3.7% (1.3%–9.6%) and 7.3% (2.3%–20.0%) unvaccinated children found per SIA in Pakistan and Afghanistan, respectively. For all of the SIAs performed with LQAS, we consider the fraction of the total children in the age group targeted by each SIA as a basis for comparison with the corresponding LQAS data. For the fraction targeted by SIA, NIDs imply a value of 1, whereas SNIDs imply a value <1. One SIA in each country (in 2018 in Pakistan and in 2020 in Afghanistan) reported a fraction targeted by SIA that exceeded 1, which reflected the overlapping bOPV and mOPV1 use. Figure 1 compares the fraction of the total children in the age group targeted by each SIA (x-axis) vs the absolute fraction of children in that age group targeted by LQAS campaign monitoring for Afghanistan (A) and Pakistan (B) or vs the relative fraction of that age group targeted by LQAS campaign monitoring (ie, the fraction of children targeted by the SIA who were also targeted by LQAS) for Afghanistan (C) and Pakistan (D). Figure 1 shows that LQAS campaign monitoring always captures a very small fraction of the age group targeted by a given SIA. Regression analyses show statistically significant relationships between the y-axis variable and the fraction targeted by SIA (x-axis). However, the results suggest nonuniform distributions, which presumably occur due to more intense monitoring in high-risk areas and/or in areas safe enough to access. Figure 1 also shows considerable variability in the fraction of children checked by LQAS during an NID, with averages of 0.15 % (range of 0.11%–0.17% in Afghanistan and 0.05%–0.29% in Pakistan). Based on this finding, we consider the average and the ranges as the basis for our assumptions about prospective monitoring coverage for NIDs and scale proportionally for the SNIDs. Exploratory data analyses and regression of the LQAS data including year as a variable did not show year as a significant predictor in any models (not shown).
Figure 1.
Analysis of lot quality assurance sampling (LQAS) coverage in Afghanistan (A, B) and Pakistan (B, D) in 2016–2020*, comparing fraction targeted by supplementary immunization activities (SIAs) versus absolute fraction targeted by LQAS (A, B) and versus relative fraction targeted by LQAS (C, D). *Data as of November 12, 2020 only for SIAs with LQAS monitoring performed (43 of 71 in Afghanistan and 45 of 81 in Pakistan). Abbreviations: LQAS, lot quality assurance sampling; SIA, supplementary immunization activities.
Table 2 summarizes the information about AFP surveillance and ES performed in Pakistan and Afghanistan in 2016–2020 (as of November 12, 2020). Over this period, Pakistan and Afghanistan reported an annual average of 11 073 and 3279 AFP cases, respectively, primarily classified as NPAFP (ie, 10 997 [99%] for Pakistan and 3225 [98%] for Afghanistan). Because of the high average national NPAFP rates (range) for both countries (ie, 15 [11–20] for Pakistan and 20 [18–23] for Afghanistan) (see Supplementary Table 1 for details), we also consider NPAFP rates by province to investigate regional variabilities. The data indicate high regional variability of NPAFP rates ranging from 2.6 to 40.4 in provinces of Pakistan and 8.4 to 48.8 in provinces of Afghanistan (see Supplementary Table 1 for details [a] by year and [b] by year by province). At the same time, Pakistan and Afghanistan used the ES information from 63 (range, 53–72) and 20 (range, 15–23) sites, respectively, with the number of sites changing from year to year. Specifically, the number of sites only increased over time in Afghanistan, while Pakistan explored a total of 89 locations and both closed poor-performing sites as well as opened new sites. The data suggest average sampling frequencies per site per year (range) of 11 (1–36) for Pakistan and 15 (2–26) for Afghanistan, although the variations depend on the timing of site openings and closures (see Supplementary Table 2 for details by year). The GPEI reported applying a benchmark for a good-quality ES site as detection of any enteroviruses (EVs), including polioviruses and nonpolio enteroviruses (NPEVs), in 50% of the ES samples collected during a 6–12-month period [31]. For 2016–2020, all ES sites that operated for at least 6 months met that benchmark (Supplementary Table 3). The sampling data imply variable NPEV rates per site over time (range, 0–1 for ES sites in both countries), with an average of 0.61 for Pakistan and 0.57 for Afghanistan (Supplementary Table 3). Overall, 43%–98% of ES sites in Pakistan and 13%–96% of ES sites in Afghanistan reported at least 1 sample positive for WPV1 or VDPV2 in any given year (Supplementary Table 3).
Table 5 gives the estimated costs of (A) LQAS campaign monitoring, (B) AFP surveillance, and (C) ES for 2019–2023 in Pakistan and Afghanistan. For LQAS, we estimate that Pakistan and Afghanistan will need $32 million (range, $19–50 million) for 2019–2023, or ~$6.4 million per year. For AFP, we estimate costs ranging from $27 to $99 million for 2019–2023, or ~$5–$20 million per year. Our analysis of the ES sampling frequency (Supplementary Table 2) showed 12 samples per site per year for Pakistan, and consequently we use only the “Default” scenario. In contrast, the ES sampling frequency in Afghanistan appears more variable over time and site location (Supplmentary Table 2) due to an apparent policy to increase sampling frequency at a site to 2 times per month following the detection of a positive ES signal, and consequently we applied all 3 scenarios. For ES, we estimate an average cost of $10 million for 2019–2023, or ~$2 million per year (Table 5). Overall, this cost analysis for Pakistan and Afghanistan suggests an expected annual budget between approximately $10 and $32 million.
Table 5.
Estimated Costs for Continued Surveillance and Supplementary Immunization Activity Monitoring in Afghanistan and Pakistan for Modeled Scenarios for 2019–2023
| A, Campaign Quality Monitoring Costs | ||||||
|---|---|---|---|---|---|---|
| 2019–2020 | No. of SIAs With LQAS Monitoring | No. of Lots | Costs | |||
| Afghanistan | 13 | 1207 | 1 931 200 | |||
| Pakistan | 7 | 4974 | 7 958 400 | |||
| 2021–2023 | Fraction Targeted by LQAS, % | No. of Lots | Costs | |||
| Afghanistan | 0.15 | 2374 | 3 798 400 | |||
| Average | 0.11 | 1741 | 2 785 600 | |||
| Minimum | 0.17 | 2690 | 4 305 000 | |||
| Maximum | ||||||
| Pakistan | 0.15 | 11 701 | 18 721 600 | |||
| Average | 0.05 | 3900 | 6 240 000 | |||
| Minimum | 0.29 | 22 622 | 36 195 200 | |||
| Maximum | ||||||
| 2019–2023 | No. of Lots | Costs | ||||
| Total | 20 256 | 32 409 600 | ||||
| Average | 11 822 | 18 915 200 | ||||
| Minimum | 31 493 | 50 388 800 | ||||
| Maximum | ||||||
| B, AFP Surveillance Costs | ||||||
| Year | Afghanistan | Pakistan | Total Cost | |||
| Adjusted | Unadjusted | Adjusted | Unadjusted | Adjusted | Unadjusted | |
| 2019 | 646 288 | 2 585 152 | 4 554 938 | 16 701 438 | 5 201 225 | 19 286 589 |
| 2020 | 651 234 | 2 604 936 | 4 614 825 | 16 921 025 | 5 266 059 | 19 525 960 |
| 2021 | 656 459 | 2 625 836 | 4 679 238 | 17 157 205 | 5 335 697 | 19 783 041 |
| 2022 | 660 942 | 2 643 766 | 4 735 644 | 17 364 027 | 5 396 585 | 20 007 794 |
| 2023 | 664 871 | 2 659 484 | 4 785 670 | 17 547 457 | 5 450 541 | 20 206 942 |
| Total | 3 279 793 | 13 119 173 | 23 370 314 | 85 691 152 | 26 650 107 | 98 810 325 |
| C, ES Costs | ||||||
| Year | Afghanistan | Pakistan | Total Cost | |||
| No. of Samples | Costs | No. of Samples | Costs | |||
| 2019 | 264 | 264 000 | 860 | 1 720 000 | 1 984 000 | |
| 2020 | 414 | 414 000 | 816 | 1 632 000 | 2 046 000 | |
| 2021 | ||||||
| Baseline | 414 | 414 000 | 816 | 1 632 000 | 2 046 000 | |
| Default | 276 | 276 000 | 1 908 000 | |||
| Reactive | 552 | 552 000 | 2 184 000 | |||
| 2022 | ||||||
| Baseline | 414 | 414 000 | 816 | 1 632 000 | 2 046 000 | |
| Default | 276 | 276 000 | 1 908 000 | |||
| Reactive | 552 | 552 000 | 2 184 000 | |||
| 2023 | ||||||
| Baseline | 414 | 414 000 | 816 | 1 632 000 | 2 046 000 | |
| Default | 276 | 276 000 | 1 908 000 | |||
| Reactive | 552 | 552 000 | 2 184 000 | |||
| Total | ||||||
| Baseline | 1920 | 1 920 000 | 4124 | 8 248 000 | 10 168 000 | |
| Default | 1506 | 1 506 000 | 9 754 000 | |||
| Reactive | 2334 | 2 334 000 | 10 582 000 | |||
Abbreviations: AFP, acute flaccid paralysis; ES, environmental surveillance; LQAS, lot quality assurance survey; SIA, supplementary immunization activity.
DISCUSSION
LQAS campaign monitoring represents a tradeoff between affordable costs and achieving the desired precision in immunization coverage measurement, as well as the requirement for random sampling to obtain valid results. Current implementation of LQAS campaign monitoring in Pakistan and Afghanistan provides limited and potentially misleading information about immunization quality. Although the relatively large number of lots sampled over time provide substantial amounts of data, these data do not reflect random samples from static production processes and may not use an appropriate sampling frame; additionally, LQAS surveys do not occur with every SIA or for every targeted population. The limited number of children checked for vaccination relative to the size of vaccinated population and nonuniform geographic distribution of sampling lots lead to potentially biased data in addition to the very low precision estimation inherent in LQAS, despite providing some information about the specific lots sampled. Combined with unclear bias in the actual choice of the lots and potential nonrandom selection of lot sampling locations and of the children sampled, the LQAS data cannot and should not be generalized as representative of overall provincial (or even district) immunization coverage at any point in time. In addition, the dynamic nature of performance, particularly as resource allocations change, implies nonstatic underlying processes, which again contrast substantially with the original manufacturing foundation of LQAS [13]. In addition, the type and training of surveyors and the quality of supervision they receive also impact the value of the information, with convenience reassessments of selected clusters by World Health Organization staff in both countries yielding different (and often lower) results compared with those reported by the regular surveyors. Thus, as currently implemented in Pakistan and Afghanistan, LQAS campaign monitoring does not provide as much information about SIA coverage as some people may perceive, as sampling may be substantially biased against selection of the highest-risk, recalcitrant, and mobile populations. Improved information about immunization coverage may require highly functional independent monitoring and external reviews, implying an even larger investment. Simply enlarging the sample sizes in LQAS does not solve the problem of biased sampling, and LQAS surveys with larger samples marginally improve precision, but not sufficiently to justify the added effort [38]. Knowing that a problem exists does not solve it, as the chronic detection of AFP cases in known poliovirus transmission reservoirs continues to demonstrate despite LQAS results that suggest that this should not occur.
Our review of the available campaign quality monitoring informed our modeling [36] with respect to overall confidence about the immunization coverage data. Specifically, ongoing transmission of WPV1 and cVDPV2s provides evidence of gaps in immunization coverage that might appear inconsistent with the LQAS data overall from high-risk areas. Fitting the model to the available immunization data without consideration of the issues with data quality would lead to die out of transmission in the model instead of the observed ongoing transmission [36]. Thus, while the quality of modeling inputs always represents a concern, models can help to elucidate biases in mental models that stem from assumptions about data quality and associated inferences (in this case, biases that LQAS data suggest better immunization coverage than appears consistent with ongoing detection of cases by AFP and transmission by ES).
AFP case detection provides the most reliable evidence of transmission over time and supports the identification of reservoirs of undervaccinated individuals. However, AFP surveillance also offers a low value of information in terms of predicting prospective transmission risk outside of known reservoirs given the dynamics of transmission and the ability of polioviruses to rapidly spread. AFP only serves to confirm chronic performance failures in known reservoirs, although it remains the best tool for detecting cases that occur outside the known reservoirs. Highly variable NPAFP rates, with values exceeding 40 in some provinces of Pakistan and Afghanistan, suggest that AFP surveillance likely identifies many children lacking AFP symptoms due to incentives for surveillance officers to meet quotas, which makes it hard to assess overall quality. The identification of “orphan” viruses provides evidence of historical lack of quality surveillance over time for both countries [39]. While ES provides potentially valuable information about the existence and extent of transmission in the catchment areas represented at the time of sample collection, even in the absence of AFP case detection, the relative physical proximity of ES collection sites and the catchment areas of undervaccinated populations (ie, those responsible for maintaining poliovirus transmission) remains unclear. The observed changes in number of sites and sampling frequency suggest likely improvements in the ES system and its quality over time. However, AFP and ES observations are not intended for pinpointing proactive immunization efforts that could prevent WPV1 and/or cVDPV2 cases before their detection or for correcting immunization program deficiencies in known reservoirs.
Eradication represents an unforgiving target of permanent prevention, which requires raising population immunity above the levels required to stop and prevent transmission in all areas simultaneously. This means overcoming the weak links in vaccination delivery in transmission reservoirs and then maintaining the gains until all transmission dies out everywhere. Our modeling with pre-COVID-19 pandemic contact mixing implied threshold population immunity levels of over 91% (with seasonal variation of 89%–92%) for WPV1 and 90% (with seasonal variation of 88%–91%) for cVDPV2 for Pakistan and Afghanistan to stop transmission [7–11], with lower thresholds associated with recent population responses (ie, lower contact mixing) to the pandemic [35]. Ending transmission requires achieving this level of population immunity everywhere at the same time and sustaining it for long enough for the existing chains of transmission to die out [40, 41]. Modeling studies that focus on targeting small geographies that emerge as issues over time (based on analyses of retrospective surveillance data [27, 28]) may also inadvertently provide somewhat mixed messaging relevant to prospective risks. For example, by identifying small areas with recent poor performance and suggesting a temporary intensification of efforts in these small areas, the national programs may fail to ensure both the achievement and maintenance of sufficiently high coverage in all areas at risk until completion.
The estimates of NPAFP rates (Table 2) depend on the assumptions regarding the size of the populations under 15 years of age [37] in the defined areas and may differ from the national estimates. Average provincial NPAFP rates mask substantial variability over locations and time, and viral genetic analyses suggest that areas of both countries exist where transmission is missed for considerable periods of time by both AFP surveillance and ES [5, 39]. The very high national NPAFP rates shown in Table 2 may provide false confidence of good surveillance, but the AFP system needs to meet the minimum quality, and ideally perform well, in all geographies. As AFP already provides a high sensitivity at the rate of 2 NPAFP cases per 100 000 children under 15 years old per year, sampling rates on the order of 10 times higher provide very little additional value of information, although they add substantially to surveillance costs. However, when areas report meeting surveillance standards, that does not mean sufficiently high quality or sensitive surveillance, and therefore ongoing monitoring and supervision of activities should occur, along with external review in low-quality areas. The estimated costs of continued surveillance and campaign quality monitoring in Pakistan and Afghanistan depend on our assumptions about unit prices for these activities, sampling quality and efficiency, and any prospective decisions made regarding the number of SIAs and their quality, as gauged by LQAS monitoring, potential expansion of the ES system (number of ES sites), and the future of AFP surveillance.
Ongoing campaign quality monitoring and surveillance in Pakistan and Afghanistan will continue to cost tens of millions of dollars per year for the foreseeable future. If Pakistan and Afghanistan face choices related to the allocation of limited funds for their polio programs, then AFP and ES appear to represent a comparatively better value of information than LQAS as currently implemented in Pakistan and Afghanistan. However, the value of these activities depends on the quality of implementation, and our analysis suggests some opportunities to better focus AFP surveillance resources on creating the right incentives to achieve higher-quality surveillance at lower NPAFP rates (eg, with better training and supervision). This may imply cost savings for data collection, but cost increases to invest in more supervision, validation, and external review. The relative value of ES compared with AFP depends on the quality of the AFP system and ES sites [11] and will likely change over time as AFP and ES quality differentially evolve. Specifically, as countries move away from AFP, the expected value of ES in detecting cases will increase, although the value of ES will depend on the system design [11].
CONCLUSIONS
Given the importance of high-quality data to inform program activities in Pakistan and Afghanistan, this analysis anticipates the need for continued substantial financial investments in immunization campaign quality monitoring and poliovirus surveillance at least through 2023. Improving the value of the information will require a better focus of AFP surveillance resources on creating the right incentives to achieve higher-quality surveillance at lower NPAFP rates (eg, with better training and supervision), optimal selection of ES site locations, and greater independent review of campaign monitoring efforts.
Supplementary Data
Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Acknowledgments
We thank Arie Voorman, Steven Wassilak, and 2 independent reviewers for helpful discussions and comments.
Financial support. The first and last authors acknowledge support for this publication under Cooperative Agreement Number 5NU2RGH001913-05-00 funded by the Centers for Disease Control and Prevention. The views expressed are solely those of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.
Potential conflicts of interest. The authors do not have a commercial or other association that might pose a conflict of interest (eg, pharmaceutical stock ownership, consultancy, advisory board membership, relevant patents, or research funding). All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
Patient consent. Not applicable; this please study does not include factors necessitating patient consent.
Prior presentation. This material has not been presented at any professional meetings.
References
- 1. World Health Organization Global Polio Eradication Initiative. Polio eradication and endgame strategic plan (2019–2023). Available at: http://polioeradication.org/wp-content/uploads/2019/05/polio-endgame-strategy-2019–2023.pdf. Accessed 4 June 2019.
- 2. Khan AW, Taylor-Robinson AW. Major shift in national health policy towards poliomyelitis in Pakistan reaps rewards. Infect Dis Health 2017; 22:153–5. [DOI] [PubMed] [Google Scholar]
- 3. Hsu CH, Rehman MS, Bullard K, et al. Progress toward poliomyelitis eradication—Pakistan, January 2019-September 2020. MMWR Morb Mortal Wkly Rep 2020; 69:1748–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. World Health Organization. Polio this week as of 9 December 2020. Available at: http://polioeradication.org/polio-today/polio-now/this-week/. Accessed 10 December 2020.
- 5. Martinez M, Akbar IE, Wadood MZ, et al. Progress toward poliomyelitis eradication—Afghanistan, January 2019-July 2020. MMWR Morb Mortal Wkly Rep 2020; 69:1464–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kalkowska DA, Pallansch MA, Wassilak SGF, et al. Global transmission of live polioviruses: updated dynamic modeling of the polio endgame. Risk Anal 2021; 41:248–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Duintjer Tebbens RJ, Pallansch MA, Cochi SL, et al. Modeling poliovirus transmission in Pakistan and Afghanistan to inform vaccination strategies in undervaccinated subpopulations. Risk Anal 2018; 38:1701–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Duintjer Tebbens RJ, Thompson KM. Evaluation of proactive and reactive strategies for polio eradication activities in Pakistan and Afghanistan. Risk Anal 2019; 39:389–401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Kalkowska DA, Duintjer Tebbens RJ, Pallansch MA, Thompson KM. Modeling undetected live poliovirus circulation after apparent interruption of transmission: Pakistan and Afghanistan. Risk Anal 2019; 39:402–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Kalkowska DA, Thompson KM. Insights from modeling preventive supplemental immunization activities as a strategy to eliminate wild poliovirus transmission in Pakistan and Afghanistan. Risk Anal 2021; 41: 266–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Kalkowska DA, Duintjer Tebbens RJ, Thompson KM. Environmental surveillance system characteristics and impacts on confidence about no undetected serotype 1 wild poliovirus circulation. Risk Anal 2019; 39: 414–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. O’Reilly KM, Durry E, ul Islam O, et al. The effect of mass immunisation campaigns and new oral poliovirus vaccines on the incidence of poliomyelitis in Pakistan and Afghanistan, 2001-11: a retrospective analysis. Lancet 2012; 380:491–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Dodge HF, Romig HG. A method of sampling inspection. Bell Syst Tech J 1929; 8:613–31. [Google Scholar]
- 14. Mushtaq MU, Majrooh MA, Ullah MZ, et al. Are we doing enough? Evaluation of the Polio Eradication Initiative in a district of Pakistan’s Punjab province: a LQAS study. BMC Public Health 2010; 10:60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Brown AE, Okayasu H, Nzioki MM, et al. Lot quality assurance sampling to monitor supplemental immunization activity quality: an essential tool for improving performance in polio endemic countries. J Infect Dis 2014; 210(Suppl 1):S333–40. [DOI] [PubMed] [Google Scholar]
- 16. Robertson SE, Anker M, Roisin AJ, et al. The lot quality technique: a global review of applications in the assessment of health services and disease surveillance. World Health Stat Q 1997; 50:199–209. [PubMed] [Google Scholar]
- 17. Robertson SE, Valadez JJ. Global review of health care surveys using lot quality assurance sampling (LQAS), 1984–2004. Soc Sci Med 2006; 63:1648–60. [DOI] [PubMed] [Google Scholar]
- 18. Rhoda DA, Fernandez SA, Fitch DJ, Lemeshow S. LQAS: user beware. Int J Epidemiol 2010; 39:60–8. [DOI] [PubMed] [Google Scholar]
- 19. Pagano M, Valadez JJ. Commentary: understanding practical lot quality assurance sampling. Int J Epidemiol 2010; 39:69–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Duintjer Tebbens RJ, Pallansch MA, Wassalik SGF, et al. An economic analysis of poliovirus risk management policy options for 2013–2052. BMC Infect Dis 2015; 15:389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. World Health Organization. Report on polio transition planning - advance draft as of 18 April, 2017. Available at: http://www.who.int/mediacentre/events/2017/advance-report-polio-transmission.pdf?ua=1. Accessed 12 March 2018.
- 22. Duintjer Tebbens RJ, Zimmermann M, Pallansch MA, Thompson KM. Insights from a systematic search for information on designs, costs, and effectiveness of poliovirus environmental surveillance systems. Food Environ Virol 2017; 9:361–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. O’Reilly KM, Verity R, Durry E, et al. Population sensitivity of acute flaccid paralysis and environmental surveillance for serotype 1 poliovirus in Pakistan: an observational study. BMC Infect Dis 2018; 18:176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Kroiss SJ, Ahmadzai M, Ahmed J, et al. Assessing the sensitivity of the polio environmental surveillance system. PLoS One 2018; 13:e0208336. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Cowger TL, Burns CC, Sharif S, et al. The role of supplementary environmental surveillance to complement acute flaccid paralysis surveillance for wild poliovirus in Pakistan - 2011–2013. PLoS One 2017; 12:e0180608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Scott RP, Cullen AC, Chabot-Couture G. Disease surveillance investments and administration: limits to information value in Pakistan polio eradication. Risk Anal 2021; 41:273–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Molodecky NA, Blake IM, O’Reilly KM, et al. Risk factors and short-term projections for serotype-1 poliomyelitis incidence in Pakistan: a spatiotemporal analysis. PLoS Med 2017; 14:e1002323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mercer LD, Safdar RM, Ahmed J, et al. Spatial model for risk prediction and sub-national prioritization to aid poliovirus eradication in Pakistan. BMC Med 2017; 15:180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Imran H, Raja D, Grassly NC, et al. Routine immunization in Pakistan: comparison of multiple data sources and identification of factors associated with vaccination. Int Health 2018; 10:84–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Alam MM, Sharif S, Shaukat S, et al. Genomic surveillance elucidates persistent wild poliovirus transmission during 2013-2015 in major reservoir areas of Pakistan. Clin Infect Dis 2016; 62:190–8. [DOI] [PubMed] [Google Scholar]
- 31. World Health Organization Global Polio Eradication Initiative. Global polio surveillance status report 2019 (WHO/POLIO/19.08). Available at: http://polioeradication.org/wp-content/uploads/2020/02/Polio-surveillance-status-report-2019.pdf. Accessed 23 November 2020.
- 32. Thompson KM, Kalkowska DA. Potential future use, costs, and value of poliovirus vaccines. Risk Anal 2021; 41:349–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Duintjer Tebbens RJ, Diop OM, Pallansch MA, et al. Characterising the costs of the Global Polio Laboratory Network: a survey-based analysis. BMJ Open 2019; 9:e023290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Thompson KM, Kalkowska DA. An updated economic analysis of the Global Polio Eradication Initiative. Risk Anal 2021; 41:393–406. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Kalkowska DA, Voorman A, Pallansch MA, et al. The impact of disruptions caused by the COVID-19 pandemic on global polio eradication. Vaccine 2021;S0264-410X(21)00473-4. doi: 10.1016/j.vaccine.2021.04.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Kalkowska DA, Pallansch MA, Cochi SL, Thompson KM. Updated characterization of poliovirus transmission in Pakistan and Afghanistan and the impacts of different outbreak response options. J Infect Dis 2021. doi: 10.1093/infdis/jiab160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Population Division of the Department of Economic and Social Affairs of the United Nations Secretariat. World population prospects. The 2019 revision. Available at: https://population.un.org/wpp/. Accessed 17 July 2019.
- 38. Okayasu H, Brown AE, Nzioki MM, et al. Cluster lot quality assurance sampling: effect of increasing the number of clusters on classification precision and operational feasibility. J Infect Dis 2014; 210(Suppl 1):S341–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Snider CJ, Diop OM, Burns CC, et al. Surveillance systems to track progress toward polio eradication—worldwide, 2014–2015. MMWR Morb Mortal Wkly Rep 2016; 65:346–51. [DOI] [PubMed] [Google Scholar]
- 40. Thompson KM, Pallansch MA, Tebbens RJ, et al. Modeling population immunity to support efforts to end the transmission of live polioviruses. Risk Anal 2013; 33:647–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Thompson KM, Tebbens RJD. Eradication versus control for poliomyelitis: an economic analysis. Lancet 2007; 369:1363–71. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.