Costing approaches for vaccine-preventable disease surveillance: lessons from Ethiopia and Nepal

Abstract

Introduction:

There is limited information about the vaccine-preventable disease (VPD) surveillance cost.

To address this gap, retrospective micro-costing studies of pre-COVID-19 pandemic VPD surveillance were conducted in Nepal and Ethiopia. Based on these evaluations—the sole cost evaluations on comprehensive VPD surveillance —this article provides methodological considerations and recommendations for other countries planning to conduct VPD surveillance costing studies to inform planning and budgeting.

Methods:

The methods used for each study were systematically compared by key themes: costing perspective, cost categories, costing approach, allocation of shared costs, sampling criteria, extrapolation strategies, data collection, and analytic adjustments. For each theme, investigators identified methodologic challenges and potential strategies to address them, compared study methodologies to surveillance costing guidelines, and recommended practices for future such studies.

Results:

The studies used similar perspectives and VPD inclusion criteria. Costs in Nepal were collected and analyzed by a subset of surveillance core and support functions, whereas the Ethiopia study categorized costs using surveillance support functions from the Global Strategy on Comprehensive VPD Surveillance.

A mix of random and purposive sampling of surveillance sites was used in both studies. Surveillance sites were selected considering the strata of interest at each administrative level. Results from both studies were extrapolated country-wide using sampling weights and assumptions about the representativeness of purposively sampled units.

Discussion:

The review highlighted potential methodologic tradeoffs in utility and precision of results based on the lessons learned from two country VPD surveillance cost studies. The advantages of collecting and using cost estimates by VPD surveillance core versus support function for program budgeting for varied audiences should be explored in future studies. Sampling strategies should be developed with consideration for the precision needed for the intended use of costing results. The resulting recommendations can improve and standardize the conduct and interpretation of future such studies.

1. Introduction

Vaccine-preventable disease (VPD) surveillance is important for identifying outbreaks for immediate action, identifying un- and under-immunized populations, estimating disease burden to inform vaccine introductions, generating evidence on vaccine impact, and monitoring progress toward global and regional disease elimination and eradication goals [1]. Comprehensive VPD surveillance further requires global, regional, and country systems to meet the World Health Organization (WHO)-recommended standards for surveillance of all priority VPDs (as defined by each country). These standards stipulate integration of surveillance functions across VPDs and other diseases; addition of VPDs and geographic areas not currently included in national or sentinel VPD surveillance systems; and emphasis on laboratory confirmation of disease, case-based data collection and reporting, epidemiological investigation, data management and analysis, and use of VPD surveillance data for routine program monitoring, optimization, decision-making, and response [1]. However, despite its vital role in public health infrastructure, there is limited evidence about the cost of conducting VPD surveillance [2].

A previous systematic review of VPD surveillance costing studies in low- and middle-income countries published from 2000 to 2017 found only 11 such surveillance costing studies, covering 19 VPDs and 11 countries (plus one study of 85 countries in aggregate for acute flaccid paralysis [AFP] surveillance only). These few studies were insufficient to estimate the resource needs for establishment and maintenance of comprehensive VPD surveillance systems at country, regional, and global levels [3]. The review also identified variability in methodology, cost categorization, and inclusion criteria that made comparison and synthesis of costs difficult across studies [3]. To contribute to filling the identified evidence gap on the cost of comprehensive VPD surveillance, WHO and the U.S. Centers for Disease Control and Prevention supported VPD surveillance costing studies in Ethiopia and Nepal as case study countries in the WHO African and South-East Asia Regions, respectively [4, 5]. Nepal and Ethiopia conducted VPD surveillance costing studies in 2018 and 2022, respectively.

Based on the experiences from the Nepal and Ethiopia VPD surveillance cost evaluations, this article discusses the methodologies of the studies and provides considerations and recommendations for other countries planning to conduct VPD surveillance costing studies. While WHO previously published generic guidelines and recommendations for costing general disease surveillance systems in 2005, this article aims to provide updated and more granular recommendations specifically for costing VPD surveillance with greater precision, which often is more resource-intensive than more general event-based or syndromic surveillance as it includes individual-level case-based surveillance with laboratory confirmation [1, 6, 7]. Although these recommendations are intended to support or inform costing of comprehensive VPD surveillance systems, these recommendations could also apply to costing surveillance of individual VPDs. Improved estimates of country-level surveillance costs, including component costs, are important to enable national programs to budget sufficient funds for VPD surveillance activities and development partners to target support as needed to implement comprehensive VPD surveillance.

2. Methods

In order to determine methodological recommendations for future country-level VPD surveillance costing studies, the authors first systematically compared the Ethiopia and Nepal studies by key themes: costing perspective, timeframe (time period during which costs of the program were incurred), analytic horizon (time period over which costs and effects of the program are evaluated), study design, cost categories, costing approach, allocation of shared resources, data collection activities, sampling criteria, extrapolation strategies, and analytic adjustments.

For each theme and its respective comparison between the two country studies, the authors characterized the methodology in each country evaluation and then identified related strengths and challenges or limitations based on gold standard research design principles for economic evaluation [8]. Following these characterizations and comparisons, the authors proposed recommendations for future VPD surveillance costing studies and compared these with the recommendations from the 2005 WHO general surveillance costing guidelines (titled Evaluating the Costs and Benefits of National Surveillance and Response Systems) [6].

3. Results

The two VPD surveillance cost studies in Ethiopia and Nepal shared multiple similar study design elements but differed in a few elements in comparison to each other and to WHO surveillance guidelines as a reference standard (Table 1).

Table 1.

Comparison of study design elements between country costing studies of vaccine-preventable disease surveillance in Ethiopia and Nepal.

Methods Elements	Description	Ethiopia	Nepal	Reference Standard – 2005 WHO surveillance costing guidelines [6]
Costing perspective	Perspective of the public health sector, including government and non-governmental and international organization implementing partners	X	X	Dependent on intended study use and audience
Timeframe	Cost timeframe of one fiscal year in-country	X	X
Analytic horizon	Same as timeframe: one fiscal year in-country	X	X
Study design	Retrospective cross-sectional full costing study	X	X
Cost categories
Resource inputs	Labor	X	X	X
	Equipment	X	X	X
	Building/Utilities	X	X	X
	Supplies	X	X	X
	Vehicles	X	X	X
	Fuel	X	X	X
	Travel, Lodging, Per Diem	X	X	X
	Contracted Services	X	X	X
	Event (In Ethiopia, event costs were included and allocated across multiple resource inputs, e.g., supplies, travel, lodging, per diem)	X	X
	Other	X		X
Support function / activity categories	See Table 2
	Inclusion of surveillance reporting as part of routine health services to higher service delivery levels	X	X	X
Vaccine-preventable diseases (VPDs)	Inclusion of all diseases in the national VPD surveillance systems or for which routine vaccines are included in the National Immunization Program (NIP) or recommended by the National Immunization Technical Advisory Group for inclusion in the NIP	Xb	Xc	Dependent on intended study use and audience
Costing subcomponents
Resource allocation	Retrospective allocation of time / use of resource to VPD surveillance support function / activity category for shared activities or items	X	X	Xa
	Further allocation of resource time / use to activity category to each VPD	X		Not specified
Non-cost data / effectiveness of surveillance	Data collection of costs of programmatic quality of surveillance at national sites		X	Dependent on intended study use and audience
Costing approach	Bottom-up, ingredients-based approach measuring empirical costs, i.e., resources actually purchased and/or consumed during timeframe	X	X	X
Inclusion and exclusion criteria for specimen shipping	Exclusion of operations and testing costs of specimens at international laboratories	X	X	Not specified
	Inclusion of specimen shipping costs from country to international laboratory for testing		X	Not specified
Data collection procedures
Piloting data collection tool	Pilot of data collection tools in non-sampled sites	X	X	X
Validating data	Daily data validation checks	X	X	Not specified
Sampling and extrapolation strategies
Sampling strategy: federal level	Inclusion of all relevant cost centers in-country (Ministry of Health HQ, national laboratories, WHO Country Office, other implementing partner offices)	X	X	X
Sampling strategy: subnational levels	Combination of purposively selected and randomly selected sites	X	X	Dependent on intended study use and audience
	Costs captured from all administrative levels and strata of interest to account for variability of costs by geographical area type and/or administrative level	X	X	X
	Pre-selection of backup sites due to inaccessibility of primary sampled sites (e.g., attributed to insecurity, limited availability of staff)	X
	Sampled surveillance facilities needed to satisfy certain criteria: (a) administrative and financial records for last fiscal year would be available in the health facility (b) health facility chief or surveillance focal point had been in the position since at least the last financial year		X
Extrapolation strategy for countrywide estimates	For randomly sampled cost centers: • Multiplied each cost by the product of the administrative-level weights for that cost center level and higher levels within a region • Sampling weight determined by inverse probability of selection	X X		See alternative reference standard d
	For purposively sampled cost centers: • Averaged costs across sampled cost centers in the same administrative level and geographic area • Sampling weight determined by number of cost centers of the same type within administrative level and geographic area • Multiplied average costs for each variable by product of administrative level weights within a region	X X X		See alternative reference standard d
	For both purposive and randomly sampled methods: • Costs were summed to regional level and then extrapolated to similar regions	X		See alternative reference standard d
	For provinces with two WHO-Immunization Program Division field offices, cost for the selected field office was used as proxy for the unselected office in the same province		X
	For government district health offices, costs of the selected district health offices were averaged by ecological zone and applied to all remaining district health offices in the same ecological zone		X
	Average costs were calculated for informing units and reporting units by hospital and health facility and were multiplied by the total number of units in their respective categories to obtain the nationwide cost at the hospital and health facility level		X
Data analysis
Analytic adjustments	Annuitization of economic costs	X		X
	Straight-line depreciation of financial costs	X	X	X
	Application of average monthly exchange rate for each activity by month of resource use during the study timeframe	X		Dependent on exchange rate data available
	Application of average exchange rate during the study timeframe		X
	Inflation adjustments	X	X	X

^aApplicable at time of study. The Global Strategy on Comprehensive VPD Surveillance with definitions of support functions was released in 2020, after the completion of data collection in Nepal. Activities in surveillance standards at the time of the Nepal included subsets of core and support functions.

^bVPDs assessed in Ethiopia evaluation: cholera, congenital rubella syndrome, human rabies, invasive-bacterial VPDs (Haemophilus influenzae type b (Hib), streptococcus pneumoniae), measles and rubella (MR), meningococcal meningitis, neonatal tetanus, polio/AFP, rotavirus, rubella, yellow fever, and other (diphtheria, pertussis, typhoid, smallpox).

^cVPDs assessed in Nepal evaluation: congenital rubella syndrome, diphtheria, invasive-bacterial diseases (Hib, streptococcus pneumoniae, meningococcal meningitis), Japanese encephalitis, MR, neonatal tetanus, pertussis, polio/AFP, and rotavirus.

^dExtrapolation strategy not recorded in WHO surveillance costing guidelines. Ethiopia study followed extrapolation strategies used by Janusz et al. and suggested in guidelines for costing immunization programs: [10, 11].

3.1. Perspective, Timeframe, and Analytic Horizon

Similarities between the Ethiopia and Nepal studies included a retrospective cross-sectional costing study design from the costing perspective of the public health sector over one fiscal year as both the timeframe and analytic horizon. The results of the costing studies in Ethiopia and Nepal were intended for use by the government of each country to inform domestic resource allocation and more sustainable resource mobilization in the context of a potentially changing donor landscape with progress towards polio eradication. Therefore, the public health sector was used as the perspective in both studies, defined not only as the government that funds and maintains the VPD surveillance system, but also including non-governmental and international organization implementing partners that operationalize and support VPD surveillance through technical assistance and supplemental funds. To align with the timeframes of government budgeting and disbursements, costs of VPD surveillance were collected for one fiscal year, defined according to the respective country government calendar (Ethiopia: July 1, 2018 –June 30, 2019; Nepal: July 15, 2016 –July 14, 2017). Both studies collected cross-sectional, retrospective data through a bottom-up, ingredients-based approach. The resource costs included those that were either purchased or used by sampled cost centers during the selected fiscal year. In order to limit potential biases from COVID-19 surveillance costs, the timeframe for the Ethiopia study covered the fiscal year 2018–19, which ended prior to the start of the COVID-19 pandemic. Data were collected in Nepal and Ethiopia during the periods of March through August 2018 and March through May 2022, respectively.

3.2. Cost Categories: Resource Inputs, Costing Activities, and Vaccine-Preventable Diseases

Costs were categorized by resource input, i.e., the type of goods or services purchased or used by sampled cost centers (in-country offices, laboratories, hospitals, and health facilities at which VPD surveillance costs were incurred by the public health sector and implementing partners) [6]. These resource input categories included labor, equipment, building/utilities, supplies, vehicles, fuel, travel/lodging/per diem, contracted services, and other (not otherwise categorized) for the Ethiopia study. The Nepal study included the same resource inputs grouped slightly differently (e.g., travel/lodging/per diem costs were not considered as a single resource input while event-related costs were considered as single resource input).

Both the Ethiopia and Nepal studies identified the key surveillance activities; however, the Ethiopia study grouped activities by surveillance support functions as listed in the 2020 Global Strategy on Comprehensive VPD Surveillance, which was published after the Nepal study (Table 2). Costs in Ethiopia were classified by the surveillance support functions (coordination, data management and use, field logistics and communication, governance, laboratory, program management, supervision, workforce capacity) while those in Nepal were classified by activities, based on subsets of surveillance core functions (i.e., active case search, case detection, case investigation and specimen collection, laboratory testing, data analysis) and support functions (i.e., training, monitoring and supervision, coordination) [1]. Although the 2005 WHO surveillance costing guidelines listed core and support functions separately, the 2020 Global Strategy on Comprehensive VPD Surveillance embedded the core functions within the support function definitions. The core functions measure the processes and outputs from the surveillance system, while the support functions facilitate the implementation of the core functions for surveillance [12]. Surveillance activities defined by budget categories of a country government may align more appropriately with core function definitions, whereas surveillance activities defined for integrated VPD surveillance systems, may align more aptly with support function definitions. Thus, costs collected in Ethiopia capture the cross-cutting resources needed for implementation of the core functions of case detection, case investigation, testing, analysis, etc. Costs of both core and support functions were included in each country study but grouped slightly differently by activity category (Table 2).

Table 2.

Comparison of activities across WHO surveillance costing guidelines, Ethiopia vaccine-preventable disease (VPD) surveillance costing support functions, and Nepal VPD surveillance costing activities.

2005 WHO surveillance costing guidelines core and support functions [6]a	Ethiopia support functionsb									Nepal activities
	Workforce capacity	Field logistics and communication	Laboratory	Data management and use	Governance	Supervision	Program management	Coordination	Active case search	Case detection, registration, and reporting	Case investigation and specimen collection	Laboratory test	Data analyses	Training	Monitoring and supervision	Coordination
Case detectionc	X	X							X	X
Case registrationc	X	X							X	X
Case confirmationc	X	X	X									X
Reportingc	X	X		X					X	X
Data analysis and interpretationc				X				X					X
Feedbackc	X	X							X		X
Standards, norms, and guidelinesd					X											X
Trainingd	X													X
Supervisiond						X									X
Resourcesd							X									X

^aSurveillance support and core functions in WHO surveillance costing guidelines based on surveillance functions enumerated in McNabb et al., 2002 [12].

^bEthiopia support functions based on surveillance functions enumerated in Global Strategy on Comprehensive VPD Surveillance, 2020 [1].

^cCategorized as core function in 2005 WHO surveillance costing guidelines.

^dCategorized as support function in 2005 WHO surveillance costing guidelines.

The activity definitions from the two studies directly aligned with each other in areas concerning field logistics and communication / case detection and investigation, laboratory, data management, and monitoring and supervision. All costs concerning program management, coordination, and governance were solely considered as those related to “coordination” for the Nepal study. “Workforce capacity” for VPD surveillance activities in Ethiopia not only included training, but also staff time for core functions.

Both studies included the VPDs in the national surveillance system or for which routine vaccines were included in the National Immunization Program (NIP) or recommended by the National Immunization Technical Advisory Group (NITAG) for inclusion in the NIP. Variances in VPDs included in the costing scopes of the Ethiopia and Nepal studies primarily stemmed from differences in disease incidence in the geographic region of each country (e.g., of yellow fever and cholera in Ethiopia, and of Japanese encephalitis in Nepal).

Although both countries conduct surveillance for influenza as part of global influenza surveillance networks (for pandemic preparedness and vaccine strain selection), neither country offers influenza vaccination as part of its routine immunization schedule; therefore, costs for influenza surveillance were not included in either study. While cases of certain other VPDs (e.g., Ebola) would be notifiable events in the general disease surveillance system in the event of an outbreak, routine surveillance for these other VPDs is not conducted by countries. Thus, their costs were not included in either country study scope.

3.3. Data Collection Procedures

To ensure ease of use of electronic, Excel-based data collection tools, both studies piloted the data collection tools in non-sampled sites in their national capital cities prior to the start of primary data collection. The data collection tools were then adapted and refined based on feedback from enumerators and pilot respondents. During the primary data collection periods of both studies, completed data collection tools were submitted to a password-protected cloud-based drive. These submitted data collection tools were monitored, validated, and reviewed by core co-investigators.

3.4. Inclusion and Exclusion Criteria for the Costs Related to Use of an International Reference Laboratory

Countries may periodically require shipment of specimens from national laboratories to international laboratories for further testing and analysis. Costs to ship these specimens from a national laboratory to an international laboratory were included in the scope of the Nepal study while excluded from that of the Ethiopia study. Both studies excluded operations and testing costs of disease specimens in international laboratories because they were borne by international laboratories and donor organizations outside the scope of the national public health sector perspective used for these studies.

3.5. Sampling and Extrapolation Strategies

To capture federal-level costs of VPD surveillance in Ethiopia and Nepal, both studies collected data from all federal-level cost centers in-country, such as the Ministry of Health headquarters, national laboratories, WHO Country Office, and implementing partner national offices. For subnational costs, both studies sampled a combination of purposively selected and randomly selected sites, considering sites from all administrative levels and strata of interest (e.g., urban-rural, geographical zone) to account for variability of costs by geographical area type and/or administrative level. Purposively selected sites were laboratories, sentinel sites for specific VPDs, and referral health facilities (which may have different cost structures), which were deliberately selected for sampling due to their unique functions in VPD surveillance systems. The remaining sites needed to complete the study sample size were randomly selected for inclusion in the sample. The Nepal study further required sampled surveillance facilities to satisfy criteria concerning availability of administrative / financial records for the last fiscal year and presence of the current surveillance focal point since at least the last fiscal year. While the Ethiopia study did not utilize sampling criteria concerning availability of records or personnel, study co-investigators selected backup sites in advance of data collection in the event of inaccessibility to primary sampled sites (attributed to insecurity or limited availability of staff). Neither study employed statistically representative sampling at country-level due to budget and feasibility constraints.

To extrapolate costs from randomly sampled cost centers in Ethiopia to non-sampled sites in-country, costs were calculated by administrative level (i.e., region, zone, woreda/district, health facility/hospital) using the product of administrative level weights; the sampling weight was determined by the inverse probability of selection. For purposively sampled cost centers in Ethiopia, the costs were averaged across sampled cost centers in the same administrative level and geographic area. The sampling weight was determined by the inverse of the number of cost centers of the same type (e.g., health facility, general hospital, specialized hospital, district health offices) within an administrative level and geographic area. Using these standardized methods for extrapolation, costs from randomly and purposively sampled sites were summed to regional level and then extrapolated to similar regions. The regional extrapolation groups were based on the proportions of population in the lowest wealth quintile, except for the regions consisting of the large cities Dire Dawa, Harari, and Addis Ababa. These three regions were grouped together due to their urbanized composition and largest proportions of population in the highest wealth quintile [9].

In contrast, the Nepal study extrapolated health facility / hospital costs from sampled sites to non-sampled sites across the entire country by health facility / hospital type. Costs from sampled sites in Nepal were used to generate nationwide estimates by averaging costs for each type of health facility and hospital and then multiplying by the total number of units in-country in their respective categories to obtain the nationwide costs at the hospital and health facility levels. Higher-level sampled sites in Nepal were extrapolated by geographic area, akin to methodologies used in the Ethiopia study. For government district health offices in Nepal, the costs of the selected district health offices were averaged by ecological zone (i.e., geographic terrain) and applied to all remaining district health offices in the same ecological zone. For provinces with two WHO Immunization Program Division field offices, the cost for the selected field office was used as a proxy for the unselected office in the same province in Nepal.

3.6. Data Analysis

Regarding cost analysis approaches, both studies employed straight-line depreciation of financial costs, assuming asset loss value at a constant rate over its useful life. Straight-line depreciation was similarly applied to economic costs in Nepal, while economic costs of capital assets in Ethiopia were annuitized using a discount rate of 5% [6]. Financial and economic costs collected from Ethiopia were further adjusted for inflation to recent values based on local consumer price indices. Data collected in local currency were converted to U.S. dollars (USD) using an average exchange rate during the entirety of the timeframe of resource use (fiscal year 2016–17) for Nepal, while monthly exchange rates based on the month of resource use were used for Ethiopia. Neither study collected programmatic / epidemiological data, such as the number of cases or case investigations, from subnational sampled sites. Therefore, neither study was able to compare the quality (e.g., non-polio AFP rate, measles-rubella discard rate) or quantity (e.g., number of reported cases for VPDs, number of case investigations) of VPD surveillance activities with the costs of surveillance at subnational levels [7]. However, the Nepal study collected programmatic/epidemiologic data in aggregate at national sites and at country level.

The methods used in the Ethiopia and Nepal VPD surveillance costing studies met eight out of 11 primary recommendations from the WHO surveillance costing guidelines. The two VPD surveillance cost studies in Ethiopia and Nepal differed from the WHO recommendations in that they did not consider surveillance and response systems together for analysis, and accordingly, excluded activities for which the main objective was outbreak response (e.g., supplementary immunization activities / vaccination campaigns, emergency activities, or cholera treatment camps). Additionally, the two studies did not select a baseline surveillance (and response) system for comparison of costs due to the consideration of full cost evaluations, rather than incremental cost evaluations.

4. Discussion

National health authorities, researchers, or partner organizations planning to conduct similar VPD surveillance costing studies should consider methodological recommendations based on lessons learned from the Ethiopia and Nepal studies. (Table 3). These recommendations provide considerations for countries aiming to conduct robust costing studies, with suggestions concerning collection of programmatic data, costing perspective and approach, inclusion and exclusion criteria for outbreak response costs, VPD categories, resource allocation and mapping strategies, data collection procedures, sampling and extrapolation strategies, analytic adjustments, and sensitivity and scenario analyses.

Table 3.

Methodological recommendations for country VPD surveillance costing studies based on lessons learned from VPD surveillance costing Ethiopia and Nepal studies

Methods Element	Recommendation	Explanatory Notes
1. Surveillance output data	1. Simultaneously collect programmatic data on surveillance outputs at each sampled site, including at subnational levels, to contextualize costs.	1. Programmatic data examples: number of reported cases for VPDs included in the study, number of case investigations conducted, testing accuracy, system sensitivity, timeliness, non-polio acute flaccid paralysis rate, measles-rubella discard rate, or other surveillance performance indicators and epidemiological information about the VPDs in the surveillance system. Data will allow for characterizing the relationship, if any, between costs and quality and volume of surveillance activities for different settings at subnational levels to assess cost drivers and determine costs to improve surveillance to meet performance targets. The VPD surveillance system should be studied holistically from both the costing and program performance lenses.
2. Costing perspective	2. Cost VPD surveillance system from the perspective of the public health sector, including any external donors or technical assistance agencies involved in the funding, implementation, and performance of the VPD surveillance system.
3. Outbreak response costs	3a. Determine the intended use of results to determine the necessity to collect costs for outbreak response, e.g., supplementary immunization activities / campaigns.	3a. Regarding inclusion of outbreak response costs, countries can cost immunization campaigns, for instance, to obtain inputs for estimation of their desired response estimates. Outbreak response costs may differ annually depending on the number of cases, disease, seasonality of disease, and geographic composition of outbreaks and related responsive campaigns.
	3b. If outbreak response costs are to be included in the study scope, outbreak response costs should be reported separately from routine VPD surveillance system costs.a Outbreak investigation costs, such as case investigation or case reporting costs, should be included in surveillance system costs.	3b. Outbreak investigation includes preliminary timely measures leading to confirmation of the outbreak and application of appropriate measures.
4. Costing approach	4. Utilize an ingredients-based approach during data collection to ensure that all costs can be disaggregated and allocated to each variable of interest.	4. The consistent use of costing activity / support function definitions and development of methods to identify categories in national budgets corresponding with those in Global Strategy on Comprehensive VPD Surveillance will also allow for comparability of VPD surveillance costs across countries and assist in determining potential areas of additional resource needs based on costs of surveillance in similar countries, and across those with different surveillance quality.
5. Cost item mapping	5a. For one variable of interest, such as the surveillance activity for which resources were used, it is recommended to structure the resource mapping at first by identifying the key surveillance functions that are relevant in the country. Depending on study use, potential activities / functions could be defined based upon support functions and related standardized definitions in the Global Strategy on Comprehensive VPD Surveillance. For each activity, map the resources requirement by resource input types.	5a. Depending on intended use of study, consider whether appropriate to use core functions as compared to support functions to define activity categories. For instance, if the primary purpose of the study is to inform budgeting, then consider using activities defined by the budget categories of the government, which may be in terms of core functions. If aiming to cost integration of VPD surveillance systems, consider using activities defined by support functions.
	5b. Include reporting of health events to higher service delivery levels as part of routine health services and surveillance.
6. Cost categories: vaccine-preventable diseases	6a. Define VPD inclusion criteria to meet the objective of the study. VPD inclusion criteria could be: i) All diseases included in the national VPD surveillance systems or for which routine vaccines are included or recommended by the National Immunization Technical Advisory Group (NITAG) for inclusion in the National Immunization Program ii) All diseases included in the national VPD surveillance systems or for which routine vaccines are included or recommended by the NITAG for inclusion in the National Immunization Program, as well as all diseases with pandemic potential included in global respiratory virus surveillance (e.g., influenza (flu), coronaviruses) iii) All diseases included in the national VPD surveillance systems or for which routine vaccines are recommended by WHO for inclusion in all National Immunization Programs, even if vaccines are not available in-country iv) All diseases included in the national VPD surveillance systems or for which vaccines are available globally
	6b. Based on integrated surveillance approaches at lower administrative levels, consider costing certain VPDs together as a group, such as measles and rubella, and all invasive-bacterial VPDs.	6b. For example, since surveillance for rubella is integrated with surveillance for measles, it may be implausible to separate measles costs from rubella costs. Therefore, consider a cost category for joint measles-rubella surveillance activities in the structure of data collection tools, as well as during analysis.
7. Resource allocation	7a. Resource allocation of time or use by support function and shares of support function time or use allocated toward each VPD requires establishing operational definitions for time allocated to a specific activity or VPD, as well as determining the final audience and use case for VPD-specific estimates to determine the importance of accurate allocation.
	7b. If VPD-specific estimates are critical to the study objectives, then explore time-motion, time sampling, or other techniques to obtain more accurate estimates of the share of resource use by VPD.	7b. Depending on country use case, analysis of costs by VPD may be useful as programmatic outputs are delineated by VPD and certain inputs may be VPD-specific. Analysis of costs by VPD may also help to inform expansion of the surveillance system to add new VPDs or sentinel sites. If analysis of costs by VPD is not considered essential, consider pooling costs together to reduce respondent burden.
	7c. If feasible, conduct intensive prospective data collection in a small number of sites with clear operational definitions and inclusion and exclusion criteria per VPD to obtain more accurate estimates of the share of resource use by VPD.	7c. Could require estimation of time allocation only for diseases of greatest policy or budget relevance.
8. Sampling strategy	8a. To ensure representative results, if budget allows, randomly sample surveillance sites in each administrative level (e.g., region, zone) and multiple strata of interest (e.g., rural-urban, ecological zones, health facility type) to account for drivers of VPD surveillance costs in-country.	8a. For instance, the capital region and rural districts further from regional laboratories and national laboratories should be included in the sample to account for variation in specimen transportation costs.
	8b. In case some sites may be inaccessible during data collection, preselect primary back-up sites in accordance with in-country contextual factors and budget for potential increased travel or data collection costs for back-up sites.
	8c. All relevant national level cost centers in-country should be included the sample to obtain a census of costs at national level.
	8d. Preferably, depending on country context, add a selection criterion that subnational sampled sites should be those that maintained financial records to avoid data collection obstacles.
9.Extrapolation strategy	9a. If sampling is non-representative at country level due to budget limitations or inaccessibility concerns, decide the basis for extrapolation of sampled to non-sampled units in advance of data collection and analysis, following in-country statistical or economic expert advice.
	9b. Extrapolation of the cost of sentinel sites should be conducted separately from the general extrapolation strategy. Costs from sentinel sites are often specific to a disease and applicable only to a limited number of similar sentinel sites.
10. Data collection procedures	10a. Prior to data collection, develop supplementary materials for interviewers and respondents with complete definitions and examples of variables of interests, including: surveillance support or core functions, resource inputs, VPDs, and funding sources. 10b. Develop structured questionnaires to standardize content inquired by interviewers and provide context and explanations of economic terms to aid interviewers and respondents with limited experience in economic evaluation in understanding desired study information.
	10c. Using the data collection tool, questionnaire, and supplementary material, teams should pilot the questionnaire in non-sampled sites to ensure data will be captured with the tool with desired level of accuracy.	10c. Based on feedback from interviewers and respondents, adjust the tool as necessary before employing the tool for cost data collection at sampled sites.
	10d. Depending on the objective of the study, if knowledge of funding sources is essential, then data collection should begin at national level to identify original funding sources before distribution of funds to subnational levels.	10d. If collecting data in a country with a decentralized health governance and financing system, all resources may not flow into the national level, and it may be more optimal to begin data collection at the first administrative level, i.e., state or province. Subnational data collection should then be tailored to validate costs collected at national level and to complement national cost collection with a focus on resource use funded from other sources at subnational levels and any allocation of time / resource use that is not evident from national level records.
	10e. Data collection and data quality assurance / control should occur simultaneously.	10e. If possible, tool submission and data review by primary study team members should occur daily to ensure that interviewers can retrieve any missing data and / or clarify categories or costs with respondents while still located in the vicinity of sampled site and prior to further advancement of movement plans.
11. International shipping costs for specimen	11a. If international shipping costs for various specimens are borne by the country or implementing partner, specimen shipping costs from the country to international laboratory for testing of suspected cases of major diseases (polio, measles, rubella, etc.) should be included in the VPD surveillance system cost estimate.	11a. The exclusion of international shipping costs borne by the country or implementing partner should be clearly reported and interpreted as an underestimate of the total costs of VPD surveillance for the country.
	11b. Costs related to the operations and testing in international laboratories should be excluded if costs are not borne by the country government or in-country implementing partner.
12. Analytic adjustments	12a. Annuitize capital assets, such as buildings, equipment, vehicles, for economic cost estimates, and apply straight-line depreciation for financial cost estimates.
	12b. Use average monthly exchange rate during month of purchase for more accurate estimates, especially in country settings in which there may be substantial variation in exchange rates over the study time frame.
	12c. Use monthly local inflation adjustments (e.g., consumer price indices at country level or subnational levels, if available), rather than international consumer price indices, to inflate costs to present values.
13. Sensitivity analysis	13. Conduct sensitivity analyses for any variables that may have been susceptible to measurement error or when sampling is not representative for organizational / geographic units for which costs are estimated.
14. Scenario analysis for prospective planning	14. For prospective planning, consider scenario analyses to assess variation in input types, prices and quantities of inputs, and overall economic growth and/or inflation in-country.	14. Even with the application of inflation adjustments, costs from a retrospective study may have limitations when used for prospective planning as program implementation or external conditions may change.

^aScope of outbreak response costs should reflect the intended audience of outbreak response cost estimates and align with normative guidance on outbreak response costing.

As programmatic / epidemiological data were not collected at subnational levels in Ethiopia and Nepal, the corresponding studies were unable to determine the quality of VPD surveillance for the assessed costs in each setting. The principal recommendation for future VPD surveillance costing studies derived from this analysis accordingly concerns simultaneously collecting programmatic data at each sampled site, including at subnational levels, to contextualize costs. For instance, researchers and program evaluators conducting costing studies can collect indicators of surveillance sensitivity and performance (e.g., number of reported cases for VPDs included in the study, number of case investigations conducted, testing accuracy, system sensitivity / ability to detect a health event, timeliness, non-polio AFP rate, measles-rubella discard rate) or other epidemiological information about the VPDs in the surveillance system. Programmatic data will allow countries to analyze the relationship, if any, between costs and quality and volume of surveillance activities for different settings at subnational levels to assess cost drivers, and help inform estimating costs of improvements to the surveillance system to meet the standards for a high-quality surveillance system. As cross-sectional analyses only capture the cost of a given quality and volume of surveillance at one point in time, determination of the cost of quality improvement may require cost analysis at multiple points of time or dedicated analyses of interventions designed to improve surveillance quality.

The COVID-19 pandemic further highlighted the necessity for robust VPD surveillance system performance and related resource needs to improve and sustain national surveillance systems. Depending on the objective of the VPD surveillance costing study, countries can evaluate if it is necessary to assess the VPD surveillance system pre- / post- the COVID-19 pandemic, especially with increased surveillance costs during the pandemic. Furthermore, to understand system costs prior to and following the COVID-19 pandemic, it is recommended for countries to consider a retrospective empirical costing study with primary data collection on surveillance resource utilization and costs before, during, and after the height of the COVID-19 pandemic. If countries implemented specific interventions and resources to improve VPD surveillance quality, integration, and / or efficiency during the COVID-19 pandemic or other periods, they can also consider cost data collection on the national VPD surveillance systems not only during a cross-section of the implementation period, but also pre-, during, and/or post- intervention.

As the 2005 WHO surveillance costing guidelines are broad and not focused on VPD-specific surveillance issues, as well as intended to be adaptable and applicable to multiple potential types of cost studies, the recommendations from the Ethiopia and Nepal studies are more applied and detailed than those from the WHO guidelines. The WHO guidelines include other considerations for which there are no parallel recommendations based on the Ethiopia and Nepal studies, such as 1) selecting a baseline surveillance and response system for comparison with the primary system to be costed by countries and 2) prioritizing separate studies of costs and benefits. The selection of a baseline/comparator surveillance and response is only needed when conducting an incremental costing study, such as costing system improvements, as compared to the full costing studies in Ethiopia and Nepal. Separate cost and benefit studies (rather than combining into cost-effectiveness, cost-utility, or cost-benefit studies) were recommended by WHO due to limited data and computational feasibility at the time of publication in 2005. However, combined studies of the costs and benefits of VPD surveillance investments is an important area for future research in the current global era of renewed attention to pandemic preparedness [13].

One recommendation from the WHO surveillance costing guidelines differs from those based on the Ethiopia and Nepal VPD surveillance costing studies: the inclusion of outbreak response costs. The Ethiopia and Nepal studies did not cost outbreak response activities (e.g., supplementary immunization activities / campaigns), but costed surveillance activities, such as outbreak (or case) investigation, in order to ensure more accurate representations of routine VPD surveillance activities. Outbreak responses may vary across years depending on incidence of cases and VPDs. Based on these two country studies, it is recommended to employ resources to cost response activities concurrently with surveillance activities only if response costs are defined and delineated from routine surveillance costs and such estimates are desired by countries. The WHO surveillance costing guidelines generally recommend costing outbreak response concomitantly with surveillance, under all circumstances, as surveillance often instigates response activities, and the cost of response partially depends on the functionality of the surveillance system. For instance, if a functional surveillance system provides timely warnings of potential epidemics, information on VPD morbidity and mortality, and the locations requiring increased access to vaccination, then a timely response and increased vaccination activities can lower disease transmission and ultimately lower response costs [14].

5. Conclusion

The recommendations from the Ethiopia and Nepal evaluations can improve and standardize future country studies of VPD surveillance costs and help researchers and program managers navigate the tradeoffs in utility and precision of results in conducting and interpreting VPD surveillance costing studies. Countries could compare surveillance cost estimates with peer countries, as well as project costs for the addition of other VPDs to the surveillance system and estimate resources needed for progress towards high quality surveillance and comprehensive control, elimination, and eradication of VPDs.

Abbreviations

AFP

acute flaccid paralysis

COVID-19

coronavirus disease 2019

NITAG

National Immunization Technical Advisory Group

NIP

National Immunization Program

VPD

vaccine-preventable disease

WHO

World Health Organization

Data Availability Statement

Data that support the findings of this study are available from the corresponding author upon reasonable request.