Strengthening health security: an intuitive and user-friendly tool to estimate country-level costs

Abstract

Member States of the WHO working to build capacity under the International Health Regulations (IHR) are advised to develop prioritised, costed plans to implement improvements based on the results of voluntary external assessments. Defining the costs associated with capacity building under the IHR, however, has challenged nations, funders and supporting organisations. Most current efforts to develop costed national action plans involve long-term engagements that may take weeks or months to complete. While these efforts have value in and of themselves, there is an urgent need for a rapid-use tool to provide cost estimates regardless of the level of expertise of the personnel assigned to the task. In this paper, we describe a tool that can—in a matter of hours—provide country-level cost estimates for capacity building under the IHR. This paper also describes how the tool can be used in countries, as well as the challenges inherent in any costing process.

Summary box.

• To implement the International Health Regulations (IHR), countries must both be able to assess their current progress of meeting the IHR and determine how much it will cost to build the capacity needed to meet the requirements.

• While the ability to assess current progress is supported, in part, by the Joint External Evaluation process, there was no comprehensive, rapid, easy-to-use tool that supports the evaluation of progress and costs estimation across all indicators.

• To meet this gap, we present here an intuitive, web-based tool, available in both English and French, which supports costing of IHR implementation.

• The tool can be completed within an hour and requires inputs already available to countries.

• Validation is currently under way as the tool is being used for costing exercises by international organisations and national actors.

• New tools being developed by the World Health Organization and other actors complement this tool and will eventually provide of suite of options to countries.

Introduction

When all Member States of the World Health Assembly adopted the International Health Regulations (IHR) in 2005, the global community endorsed the need for every country to build the infrastructure required to prevent, detect and respond to public health emergencies. By 2012, however, only one in five countries had declared that they had met this obligation.¹ By 2014, that number had only increased to one in three.² Countries struggled to build their capacities, in part because resource-constrained governments faced multiple competing priorities, and in part because financial resources were not readily available to provide desperately needed assistance in building public health infrastructure.

National stakeholders and international partners struggled to define the specific actions that were required under IHR to strengthen and maintain health security systems, and therefore had little understanding of the magnitude of costs associated with such an undertaking. As a result, many countries were unable to build a case with domestic policymakers or external partners for the necessary appropriations, grants or loans.

In 2016, the WHO adopted the Joint External Evaluation (JEE) tool as a voluntary process to measure country-specific progress and help set goals in developing the capacities needed to implement the IHR. The JEE identifies 19 technical areas of health security, each characterised by one to four activities, for a total of 48 activities, and measures progress across each of these activities. The response to the JEE in the last couple of years has been impressive: 67 countries have already completed the evaluations and 13 more are in the pipeline (as of January 2018).³ Recognising the need for costing as a necessary step towards developing a country-level strategy to raise domestic and international financing in support of actions to strengthen health security, the United Nations (UN) High Level Panel called for costed action plans to be completed within 3 months of a JEE.⁴ A similar sentiment was voiced by the International Working Group on Financing Preparedness, which recommends that ‘Within nine months of completion of JEE and PVS [Performance of Veterinary Services], national governments should develop and publish a prioritized and costed plan to implement recommendations emerging from the JEE and the PVS assessments, including regional elements where relevant.’⁵ Progress has been very slow, however, and only a handful of countries have so far prepared costed action plans, and compliance with IHR remains elusive for all.⁶

Defining the costs associated with capacity building under the IHR has challenged nations, funders and supporting organisations. There are now a variety of tools and processes that can be consulted to aid nations in building costed national action plan, and countries or regions may wish to use different tools at different stages in the costing and budgeting process. Most current efforts to develop costed national action plans involve long-term engagements that take weeks to months to complete. While these efforts have value in and of themselves, there is an urgent need for a rapid-use tool to provide cost estimates regardless of the level of expertise of the personnel assigned to the task. In this paper, we describe a tool that can—in a matter of hours—provide country-level cost estimates for capacity building under the IHR.

The rest of the paper is organised as follows: the second section introduces the IHR Costing Tool, and provides details on its salient features; the third section introduces how best the tool can be used; the fourth section describes the challenge inherent in any costing process, especially those related to the health sector, and how this particular tool helps address the challenge.

The International Health Regulations Costing Tool

The IHR Costing Tool is based on JEE scores across 48 activities; information on population, currency, administrative organisation details and public health infrastructure; and estimates of unit costs for 19 items as listed in the online supplementary table S1. In-built functions compute the quantity of inputs needed to produce the desired outputs, and five in-built multipliers scale base costs by country-specific parameters to arrive at country-level estimates (table 1). The multipliers can be edited by end users, who can change the in-built parameters as needed.

Table 1.

List of multipliers

Multiplier name	Example usage in costing calculations
Country intermediate area count (eg, provinces, districts)	One information technology officer per intermediate area
Country local area count (eg, counties, cities)	One trainer for annual surveillance training per local area
Country total population	Two veterinary officers per million population
Country national healthcare facilities count	Two mobile phones for surveillance units per healthcare facility
Number of meeting attendees (small, medium, large)	One per diem per policy meeting attendee

Multipliers are used to adjust scale base costs according to country-specific parameters information. These multipliers are used iteratively throughout the costing calculations and can also be edited by the end user.

The costing framework is based on templates developed by our research team following a review of IHR implementation in 14 case study countries spanning multiple regions. The framework is further refined by country feedback on user needs, collected while supporting WHO in developing and pilot-testing an earlier conceptual framework for IHR costing in six countries.^{7 8} This foundational work was performed using the evolving versions of the IHR Core Capacity Monitoring Tool (IHRMT), the monitoring and evaluation tool with 256 attributes used for national-level self-assessments of the IHR prior to the introduction of the JEEs.

The costing framework and associated calculations are based on these findings and aligned with the actions costed with the technical areas identified in the JEE. As noted earlier, the JEE identifies 19 technical areas of health security, each characterised by one to four activities, for a total of 48 activities, which are assessed on a five-point scale ranging from no capacity (1) to sustained capacity (5).⁹ As the JEE and the IHRMT contain slightly different indicators, we reviewed published guidance from WHO, the World Organization for Animal Health (OIE), and other international authorities for specific requirements for capacity building and implementation that would address the new indicators.¹⁰ Finally, the team reviewed published JEE evaluations during Spring 2017 to identify specific activities/capabilities that the assessment process had associated with the scores assigned in each indicator. A data ontology was then defined to cross-walk the actions needed to achieve increased capacity with the costs associated with those actions.

Actions included for costing are aligned to the scores assigned by the JEE. The data ontology links costs to developing the capabilities identified in researching technical and policy documentation, pilot testing, and country feedback indicated would support a given JEE capacity score for a given score step. The ontology covers from development of a ‘Limited Capacity (2)’ from ‘No Capacity (1)’ and continues through step increases from Limited to Developed Capacity (2–3) and from Developed to Demonstrated Capacity (3–4). Costs for developing the capabilities to transition from Demonstrated to Sustainable Capacity (4–5) were omitted to focus efforts on the core goal of supporting countries in earlier stages of capacity development.

Costs were validated by reviewing published guidance from WHO, OIE and other international authorities.^11–14 Existing costing tools were also reviewed and aligned to cost estimates where possible, including the OneHealth Costing Tool and the JEE Action Plan Tool developed by the US Centers for Disease Control and Prevention (CDC) and WHO.^{15 16} Costs in the tool are assigned as start-up costs, capital costs and recurring costs, and sorted by both core capacity and core function, based on this review.

Multiplier-based cost calculations and costing defaults

Costs for each action are assigned by a multiplier. These calculations represent how base costs (eg, salary for a single staff member, cost of a piece of equipment, cost to host a meeting, and so on) scale with population, administrative organisation (because IHR requires capacities to detect, assess and report unusual events from the local, intermediate and national levels) and others so that calculations support costing across different countries (see table 1 for a complete list). Country-specific input data for base costs and multipliers as assigned to each to generate country-specific cost estimates. Costs are tagged as start-up (one-time expenses incurred in the first year), capital costs (non-recurring costs incurred in the first year for construction and durable goods) or recurring costs (annual operating expenses, including salaries).

Functional tags applied to each cost include coordination/leadership; planning including assessment, design, planning, policy, legislation; strengthening human resources capacity; strengthening infrastructure; operations/implementation; analysis including data quality and dissemination; and use and review mechanisms and are aligned to functional categories used in application of the JEE tool. A standardised list of default costs assigns cost defaults for frequently costed items. These defaults can be modified by the users and are applied to costing calculations throughout the costing tool (see online supplementary table S1 for references used to develop default costs).

Online presence

The IHR Costing Tool can be accessed publicly through an online tool (https://ghscosting.org, figure 1) that guides users through the costing process from entering JEE assessment scores to estimating costs associated with enhancing capacities and visualising costing results, and is available in both English and French. The user interface is built in HTML5 and JavaScript and is supported by a static database of costing calculations, country populations (drawn from UN World Population Prospects 2017, available at https://esa.un.org/unpd/wpp/Download/Standard/CSV/) and default cost values and descriptions.

Figure 1.

Home page of online International Health Regulations (IHR) Costing Tool: https://ghscosting.org.

The tool accepts country-specific user inputs (population, currency used for costing, cost assumptions for frequently costed items and costs assigned to individual actions) for use during a single session. It does not store data after the web browser is closed, but provides options to store data locally to support multisession costing efforts and offline work using costing templates (see results for additional description). Results are communicated using custom-built visualisations in JavaScript and d3.js-based interactive graphics with filters by JEE core capacity, function (eg, planning, operations/implementation, coordination/leadership, and so on) and to separate start-up, capital or recurring costs (in addition to total costs). In addition, results can be viewed as 1, 3 and 5-year totals to support budgeting, including rolling start-up and the first year of recurring costs in the 1-year total and accounting for additional recurring costs across 3 and 5-year time horizons. These results can be exported from the tool in a spreadsheet format to support budget deliberations and additional analysis.

The costing tool includes several options to support users in uploading or choosing country scores from completed, publicly reported JEEs. In addition, to support cases where JEE scores are not yet published or where internal assessment is in progress, the IHR Costing Tool includes a workflow to input assessments manually. Each of the 19 JEE core capacities is listed on tabs to the left of the assessment page with each indicator nested under the appropriate Core Capacity. If entering assessments manually, the user is asked to enter assessment values for each of the JEE indicators one by one. Each indicator is described by the language used by the JEE and colour-coded to align with the conventions used in published JEEs.

Managing user data without central storage

Setting priorities for IHR implementation, selecting actions and estimating costs is a highly collaborative process and requires trust that ownership over sensitive data is maintained in order to secure engagement and buy-in from national (state level), multinational and non-governmental stakeholders. Therefore, as noted earlier, the IHR Costing Tool does not store data. Instead, users can download a costing template to work offline and enter information into the online tool later or save a local copy of a costing in progress with the online tool (.ihr file downloaded to the user’s computer). Downloaded data include both JEE score data and costing estimates for actions associated with enhancing capacities around individual JEE indicators.

Standardised method for choosing target capacity level

The tool provides a standardised method for users to choose a target JEE score as the ‘goal’. Users choose either to estimate costs based on progression from current JEE scores to a score of 4—Demonstrated Capacity on all indicators (the default selection)—or on the basis of a single-step increase from current JEE scores to the next capability level (ie, from 1 to 2, 2 to 3, or 3 to 4 on an indicator-by-indicator basis). By focusing on reaching a score of 4—Demonstrated Capacity level, the IHR Costing Tool reflects the goal that all countries demonstrate implementation of the IHR, but supports stepwise progress.

To evaluate the cost of IHR implementation, the user is first asked to assign a set of country-specific inputs required for calculations throughout the tool. Based on the country chosen for costing, defaults are assigned for the population (UN World Population Prospects 2017) and currency (default is the local currency); all defaults can be adjusted by the user to capture changes, choose a different option or correct errors. Country details are also required, including entering the breakdown of the number of units at each level of country administrative organisation and an estimate of the number of healthcare facilities in the country (as shown in figure 2). For the purposes of IHR implementation, healthcare facilities include those participating in IHR-related activities, including biosurveillance programmes, point-of-care diagnostics for priority diseases, prevention of healthcare-associated infections, and biosafety and biosecurity programmes, and are primarily expected to be hospitals and government-run health centres. Additional cost assumptions can also be completed by the end user to capture costs that are used iteratively throughout the costing calculations, including salaries for a range of personnel, per diem costs for workshops, and printing costs, among others. A complete list of the elements that can be costed is listed in online supplementary table S1. The default costs for each are based on the results of a series of workshops completed with countries implementing the IHR and are drawn from standardised public data sets such as the WHO Choice database, published trade organisation reports, published manufacturer/provider pricing, government and non-governmental agency reports, and others.

Figure 2.

Population, currency, administrative organisation details, public health infrastructure and a series of optional cost assumptions are entered prior to the main costing page.

Costs include those for establishing the legal/policy foundation and infrastructure to support basic capacities, as well as the operating costs for personnel, training, processes and consumables based on country-specific data supplied as inputs. In addition, users review line item costs for each indicator and, based on the ‘view details’ description, edit default costs and save the country-specific costs as available from country-specific data sets (figure 3). The costs for each indicator include those required for start-up (eg, cost of establishing legislation), capital (eg, building construction, durable goods) and recurring costs (eg, salaries).

Figure 3.

Example of a costing page from the International Health Regulations (IHR) Costing Tool.

Once completed, the total cost estimates can be visualised in a results page (figure 4). The first analysis allows the end user to evaluate costs based on progress towards an improved JEE score. The total cost can be assessed on a 1, 3 or 5-year basis to capture the cumulative costs of start-up and recurring elements. Alternatively, the costs can be evaluated by whether they can be defined as start-up costs (such as writing legislation), capital costs (such as building laboratories) or recurring costs (such as salaries for long-term personnel). The contributions of estimated costs in each category are totalled and visualised for each core capacity. The costs are calculated in the currency chosen in the costing tab; the currency calculations are automatically updated when currency is updated based on the current exchange rates (uses UN Operational Rates of Exchange, https://treasury.un.org/operationalrates/OperationalRates.php).

Figure 4.

The cost summary shown at the top of the results page.

In addition, costs can be analysed by core capacity, indicator and subindicator in an interactive graphic that allows users to evaluate and assess the relative and total cost contributions of each indicator and subindicator to each core capacity (figure 5A) or by function (figure 5B). A series of filters facilitates analysis and interactive exploration of the results viewed at different levels. A pop-up box can be seen by hovering over any node in the graphic; a link allows users to update costs for each element by returning to the corresponding cost element in the tool.

Figure 5.

Cost visualisation by core capacity and core function.

How the tool could be used

On 26 February 2016, Tanzania completed the JEE assessment, and over a year later became the first country to develop a costed National Action Plan for Health Security. Tanzania was joined by Pakistan, which completed the JEE mission on 6 May 2016 and costing 1 year later, in May 2017. Tanzania and Pakistan are pioneers in progressing to the post-JEE process and should be commended for leading the way. It is noteworthy, though, that it took over a year for both countries to complete the costing exercise. In the case of Tanzania, the costing process brought together partners from all over the global health security landscape, including WHO, US CDC, Finland, US Department of Defense, UN Food and Agriculture Organization, the OIE, Public Health England, Japan International Cooperation Agency, Germany Development Agency, US Department of Agriculture, the World Bank and national governments. It also ‘galvanized multiple stakeholders to work together on health security in the country.’¹⁷ Further, Pakistan’s costed plan reaches across all administrative levels and includes costed items on such specific items as printer toners and steel cabinets.

To have a team of country-based experts engaged in this ambitious exercise and produce a costed report has no rival; the know-how, interest and engagement cannot be replaced by a tool. However, to the extent that this level of complexity and time produces cost estimates with large CIs, the effort is not feasible for most countries. This assessment is not merely prescriptive: many countries have completed the JEE, but are challenged by the costing exercise and have not progressed to the next phase. For these countries, the IHR Costing Tool, which can produce cost estimates in a matter of hours, can be especially helpful.

The accuracy of the cost estimates produced by the IHR Costing Tool is heavily dependent on the unit cost data provided by the end user as well as on the multiplier parameters accepted by the end user. We therefore suggest a three-step use of the tool for sophisticated users, which allows the user to fine-tune the tool using historical cost data, and more accurately provide input costs and multiplier values to estimate final costs: (1) start with a backward iteration on recent historical costs, such as the latest year for which audited values are available, across a sample of five to eight of the 48 JEE activities, and use the tool to generate cost estimates; (2) compare these estimates with actual costs already incurred on those activities, adjust unit costs and multipliers, and re-estimate costs; repeat, until estimated costs are very close to actual costs; and (3) using the adjusted unit costs and multipliers, estimate costs for all 48 activities. We encourage interested readers to go to demo.ghscosting.org to see how the tool costs IHR capacity building in Kenya, and download the full spreadsheets on the results page to examine each individual costed line.

The challenge of costing

There are many challenges associated with costing production and delivery of health services, a task that is further complicated if the services are incremental in nature and are produced and delivered in the public sector, such as those listed under the IHR functions. This section discusses some of the major complexities associated with costing that must be kept in mind while determining the confidence level and use of the cost estimates.

First, governments in all countries, rich or poor, are already involved in the production and delivery of many of the services listed under the 48 activities identified in a JEE. The gap is in the level and quality of services produced, which are typically poor in countries scoring low on the evaluation. A methodology that incrementally estimates costs of activities identified to improve level and quality therefore implicitly assumes that the current production function is optimal; cost estimates produced based on this assumption perpetuate the existing inefficiencies in the system and would be low relative to actual cost. Conversely, if it is assumed that activities to improve level and quality are being introduced for the first time, any activities on the ground already are ignored and cost estimates would be high relative to actual cost.

Second, the cost to the government of producing a higher or better level of service may well be close to zero if staff can be reallocated from a low priority area and assigned to the activities listed in the JEE. Assessing and accounting for this substitution requires a comprehensive knowledge and understanding of the entire system of the health and associated sectors; on the other hand, not accounting for the possibility of this substitution would result in high cost estimates.

Third, the relationship between inputs being costed to produce a desired output is inherently non-linear. Cost of capacity increases can be calculated with higher confidence in scenarios in which defined increases in an input lead to a defined increase in output, but this relationship is linear only in a limited number of areas and does not account for economies of scale. Incorporating these issues in a costing framework is a challenging exercise.

Fourth, there are many joint costs in the production and delivery of health services, which confound the alignment of unique activities to specific line items. For example, an improved surveillance capacity may require a new computer system, which could also be used to address needs for staffing, receiving veterinary reports and handling communications. These shared costs are challenging to capture in a costing exercise. This challenge is directly related to a difference in marginal or average costs.

These challenges notwithstanding, cost estimates are necessary to begin discussions with the Ministry of Finance and other partners for the allocation of resources needed to implement activities targeted to increase health security competencies in the country. Given the inherent challenges involved in estimating costs of publicly produced and delivered health services, countries should not spend too much time on this exercise. Therefore, the IHR Costing Tool presented here is particularly powerful: in just a matter of hours, it can yield very good estimates of what it would cost the country to scale up the necessary activity levels, achieve higher scores on the JEE scale and fully implement obligations under the IHR.