Use of the Staged Development Tool for Assessing, Planning, and Measuring Progress in the Development of National Public Health Institutes

Abstract

The Staged Development Tool (SDT) was created to help National Public Health Institutes (NPHIs) assess their current capacity and develop roadmaps for achieving a higher level of functioning. The paper discusses the current use of the SDT by NPHIs to establish baseline capacity and inform strategic planning, and its proposed use in a three-step sequence for measuring the impact of capacity-building interventions over time. The paper also includes descriptions of how NPHIs have been using the SDT to assess their baseline capacity in management issues and core public health functions.

The first use of the SDT by an NPHI provides essential baseline information on their capacities and levels of functioning, and plans for addressing gaps. By repeating the SDT after time for the plans to be implemented, the SDT can be used to evaluate changes in capacity and the effectiveness of the interventions made. Because the SDT is built to be complementary to existing assessments and public health strengthening tools and guidelines, implementing the SDT provides concrete, complementary information that can help countries achieve global health security goals and address current and future threats to public health.

Background

What are NPHIs and why are they important

There is general acknowledgement that strong health systems require robust public health infrastructures [1, 2]. Achieving the objectives of the Sustainable Development Goals and ensuring global health security depend on systems that promote, maintain and restore health. Health systems are stronger and more effective with integrated core functions of public health, including but not limited to surveillance of population health and well-being; monitoring and response to health hazards and emergencies; advancing public health research to develop the evidence to inform policies and programs; and assuring a sufficient and competent public health workforce [1, 3, 4]. Epidemiologic data and public health research data provide an evidence-base for decisions and policies affecting all aspects of the larger health system. Furthermore, a strong public health workforce increases the capacity of a country to ensure the existence of the conditions in which people can be healthy [4, 5].

In many countries, public health activities are spread across several organizational units, sometimes across several ministries. Even when they are concentrated within the Ministry of Health, public health priorities are often overshadowed by the perceived greater urgency of providing clinical care. Fragmentation of public health efforts among multiple organizational units hinders effective coordination, leadership and management of public health activities (such as public health laboratory services, surveillance, and emergency response and preparedness) for preventing recurring outbreaks. These problems can be mitigated by developing a national public health institute (NPHI) that serves as a single focal point that leads or coordinates public health activities [6–8]. NPHIs are organizations that are typically part of Ministries of Health (MOHs), or closely aligned with them, that promote health by coordinating public health functions and programs to prevent, detect, and respond to public health threats, for both infectious and non-infectious conditions. In some countries, they also serve as the public health focal point for One Health and regularly liaise with Ministries of Agriculture, Environment, and other relevant ministries and partners.

In recent years, many NPHIs have been created or augmented, often in response to health security concerns or events. For example, the Canadian NPHI, the Public Health Agency of Canada, was created after what was perceived as a poor response to the severe acute respiratory syndrome (SARS) outbreaks [7]. Liberia’s NPHI was created very rapidly following lessons learned from the country’s less than optimal response to Ebola.

In contrast, Nigeria’s response to the Ebola Virus Disease outbreak, with guidance from the Nigerian Centre for Disease Control (NCDC, Nigeria’s NPHI) is widely considered a success story [9, 10].

IANPHI and CDC

The International Association of National Public Health Institutes (IANPHI), founded in 2006, is an international membership organization of NPHIs, a global network with more than 100 members from nearly 100 countries. Its major roles include fostering relationships and support among NPHIs and assisting countries in NPHI development. The recent creation of the Africa Centers for Disease Control (Africa CDC) has provided additional impetus for NPHI creation, as the Africa CDC considers NPHIs to be an essential component of its plans for improving public health in Africa.

The US CDC, an active IANPHI member since the network’s inception, supports countries in organizing and consolidating public health expertise and systems within a national public health institute[11].

CDC and IANPHI partner extensively to help build and strengthen NPHIs, particularly in low- and middle-income countries. At the request and invitation of the country, CDC’s NPHI program works to identify and engage with key stakeholders, map and document public health functions, operations, and organizational capacity and conduct strategic planning to elaborate the country’s public health priorities. The specific activities in a country’s NPHI development are customized to each country’s context, national priorities, public health needs, and existing infrastructure, which enhances country ownership and sustainability.

CDC and IANPHI Tools for NPHI Development

As part of their work in helping develop NPHIs, CDC and IANPHI – individually and in partnership – have developed a number of tools to help NPHIs achieve their goals and contribute to international goals for global health security and sustainable development. These include a Taxonomy of Essential Public Health Functions, Operations, and Services (FOS, CDC unpublished data), the IANPHI Peer-Review Framework (P2P, [12]), and the Staged Development Tool (SDT, [11]). These tools are used on their own or in conjunction with other assessment and planning tools. The focus of this article is on the SDT, a unique tool developed specifically for use in NPHIs.

SDT

Reason for developing the SDT

With input from a consultative group of NPHI leaders from around the world, CDC’s NPHI program and IANPHI developed the SDT as a process and tool-kit to support planning needs articulated by NPHIs in early stages of their development: to help NPHIs assess their present capacities; and to identify gaps and barriers that prevent them from moving to a higher level of functioning.

The product of SDT is a road-map to improve capacity and have impact in areas the NPHI considers a priority; therefore, implementing the SDT will help NPHIs function better and have more impact.

The SDT process

Largely based on a capability/maturity model framework adapted around descriptions of essential public health capacities, the SDT includes assessment, prioritization, and work-planning processes. The SDT describes a four-level evolving path of increasingly organized and systematically more mature processes across a set of public health domains, providing detailed descriptors for the different stages/domains that are used to determine current and desired capacity including resources that an NPHI might need, to effectively perform at each level of maturity.

The SDT uses a set of 28 Discussion Guides that provide descriptions of stages of development for a range of topics (Table 1). The Discussion Guides include 11 “internal-facing” and 17 “external-facing” topics. “Internal-facing” refers to operations that are essential for an NPHI to function well as an organization (e.g. Leadership, Human Resources, and Financial Management) while “external-facing” refers to the NPHI’s ability to carry out its vision and deliver services to their stakeholders (e.g. Surveillance for Acute Public Health Problems, Laboratory Reference and Diagnostic Services).

Table 1.

SDT Discussion Guides

Internal-Facing Discussion Guides	External-Facing Discussion Guides
1. Planning	12. Population Health Status (Assessment and Reporting)
2. Leadership and Management	13. Management of Public Health Information
3. Health and Safety	14. Health Communication
4. Laboratory Safety	15. Laboratory Reference and Diagnostic Services, and Support for Quality Improvement
5. Human Resources (HR) Management	16. Surveillance
6. Staff Development	17. Surveillance for Acute Public Health Problems, Including Infectious Diseases
7. Management of Organizational Information	18. Sentinel Surveillance
8. Internal Communication	19. Reporting of Acute Public Health Events
9. External Communication about the NPHI and its Activities	20. Investigation of Acute Public Health Events
10. Information Technology (IT)	21. Emergency Preparedness and Response
11. Financial Management	22. Implementation of International Health Regulations (IHR)
	23. Data-to-Action
	24. Strategic Data Collection and Analysis
	25. Development of Public Health Recommendations
	26. Uptake of Public Health Recommendations
	27. Public Health Workforce Development
	28. Public Health Research

Prior to the SDT implementation, the NPHI reviews the list of Discussion Guides and selects areas for assessment, based on their priorities and interest. The SDT is then facilitated by a person trained in the use of the tool, as well as in facilitation skills. Participants are selected based on the discussion guides being used. Staff from the relevant program need to be present to talk in detail about the NPHI’s capacity in that area.

The facilitator leads participants through a three-step process:

• Assessment, during which the group must:

  • ○
    Have a robust discussion, and try to determine their present stage of operational maturity (current stage) defined by alignment of the perceived capacity of the NPHI to the descriptors of one of the stages (basic; developing; advanced; leading edge) across six domains (strategic direction; systems; resources; quality; stakeholder engagement; impact).
  • ○
    Identify their aspiration for a higher stage of operational maturity (desired stage), to be reached within a specified timeframe (usually one or two years, depending on the NPHI).
  • ○
    Clearly define gaps that need to be addressed for the NPHI to move from the current to the desired stage, and the present barriers to addressing the identified gaps
• Prioritization, during which the group must:

  • ○
    Identify the gaps the NPHI would like to address first, in the time frame specified
  • ○
    Prioritize which gaps are most important to address
• Work-planning, during which the group must:

  • ○
    Define a clear work plan to address the priority gaps, including the actions to be taken, personnel responsible for each activity, deliverables, and timeframe within which the actions will be taken.

Because the NPHI takes the lead in selecting Discussion Guides, a wide range of participants are included, and the process is a facilitated internal assessment and not externally driven, the result is a country-owned plan. Because Discussion Guides include ideas that less developed NPHIs often haven’t previously considered they may result in a more creative plan than processes like the SWOT, which may be limited by the knowledge and experiences of people in the room. The wide range of aspects found in any given Discussion Guide – e.g., such as resource issues and engagement on the topic – provide opportunities for individuals with a range of knowledge and experience to provide input.

An important feature of the SDT is that the Discussion Guides are detailed and the process is country-owned, so extensive experience in the various topics is not required for effective facilitation.

Experiences using the SDT

Early pilots

In 2016, the SDT process was piloted in Togo (in French) and in Guinea Bissau (in Portuguese). The objective of these pilots was to field-test the content of the Discussion Guides as well as to test the process of conducting the SDT. In both pilots, the countries were offered a pre-selected set of discussion guides (both internal- and external-facing), and asked to select participants from across a broad range of technical and administrative expertise. These pilots provided feedback that was critical to the successful implementation of the SDT Process, which went into effect in in early 2017.

Colombia

The Instituto Nacional de Salud (INS), Colombia’s NPHI, is a scientific and technical public institution under the Ministry of Health & Social Welfare (MSPS), whose mandate is the protection of health in Colombia through knowledge management and transfer, monitoring of the health status of population and the provision of goods and services relevant to public health. There are five scientific areas that comprise the health portfolio of the INS: Disease Surveillance, Public Health Laboratories, Public Health Research, the National Health Observatory, and Biological Products & Production.

Support from the CDC’s NPHI Program and IANPHI helped Colombia enhance its public health system to better identify and respond to known and emerging health threats. This support included the establishment of a Public Health Emergency Operations Center at the INS and strengthening laboratory, surveillance, and risk communications capacity. These investments contributed to Colombia’s robust response to the Zika virus outbreak started.[13]

In 2017, the Director prioritized SDT implementation and INS implemented the SDT during two different sessions that focused on 9 Discussion Guides:

1. Planning,

2. Management of organizational information,

3. External communication about the NPHI and its activities,

4. Laboratory services,

5. Surveillance for acute public health problems,

6. Emergency preparedness and response,

7. Strategic data collection and analysis,

8. Development of public health recommendations, and

9. Public health research.

Participants in the various discussions were selected based on their roles in the NPHI. The NPHI leadership identified the key units of the NPHI to participate in each of the discussions, and invited 3–4 people to participate, in addition to a core set of participants who were present throughout the process. Discussions generally involved people with a wide range of experiences, including those with technical expertise and others with knowledge of the broader public health system.

During these two one-week SDT sessions, INS identified a series of gaps in each of the areas they chose to address, and developed a list of upwards of almost 100 activities that would mitigate those gaps. The list of activities was then ranked and prioritized based on importance, feasibility and urgency, and sorted into short- and long-term milestones that comprised a short-term and a longer-term work plan. This work plan informed the organization’s strategic planning and served as the starting point for a collaborative project funded by IANPHI, using funds provided by the US CDC.

At the conclusion of the SDT, the INS Leadership concluded that there have been rare, if any, opportunities in the past, for staff from across different parts of the organization, to have meaningful discussion around the identification of gaps, activities to mitigate them, and work planning to implement these activities. This observation was consistent with implementations of the SDT that followed.

Nigeria

The Nigeria Centre for Disease Control (NCDC) was established in 2011 as an autonomous entity to enhance Nigeria’s preparedness and response to epidemics through prevention, detection and control of communicable and non-communicable diseases. NCDC’s core mandate is to detect, investigate, prevent and control diseases of national and international public health importance.

In 2017, NCDC’s newly appointed Chief Executive Officer identified public health surveillance, emergency response, and health communications as priorities for strengthening using the SDT. Using the SDT, NCDC identified their current level of functioning, desired level of functioning and gaps across the three areas assessed.

Participants in the SDT implementation process were NCDC staff from all three priority areas identified, staff from the African Field Epidemiology Network (AFENET) and a few fellows from the Field Epidemiology Training Program (FETP). Thirty people participated in the process. Although the participants were a good mix of junior, frontline staff and senior staff, the environment afforded everyone the opportunity to freely participate in the assessment process. It was empowering for the frontline staff as their perspective and field experiences provided context that helped participants reach consensus on NCDC’s current level of functioning.

The FETP program staff participation provided a helpful “field” voice,

The discussions uncovered gaps in NCDC’s current level of functioning in the three priority areas identified, some of which came as no surprise to the participants. However, the process also unraveled subtle gaps that were previously unknown. For example, whereas NCDC knew that their communications team was understaffed, with limited tools, they realized during the discussions that their communications efforts may not have been reaching their targeted population. Participants also came to a common understanding that NCDC had not been collaborating with other organizations that have more robust communications structure in place to leverage resources for stronger impact.

Through prioritization, participants identified immediate action steps (“quick wins”), and longer-term activities that formed the basis for work plan development. The process helped to operationalize an already existing strategic plan and provided better understanding of the order in which activities would be implemented based on urgency, importance and available resources. NCDC is currently in the implementation phase, where they are addressing the gaps identified during the baseline SDT assessment.

CDC and IANPHI are using a similar approach to NPHI strengthening in many developing countries throughout the world – supporting them to better collect and use public health data, implement and monitor evidence-based public health programs, and, ultimately, save lives and money.

Use of the SDT to measure progress

There are several ways the SDT can be used to track progress over time. The Discussion Guides results from baseline assessment can form a basis upon follow-up assessment to determine if the NPHI has moved from their baseline score (“current score” at initial assessment) to their desired score in the defined period. This would allow the NPHI not only to assess progress, but also to redefine their aspirations towards a higher stage of operational maturity to be achieved in the next period, as well as develop a work plan to achieve it.

In addition, the work plan developed during baseline assessment can be used as a basis for a monitoring and evaluation to assess progress — that is, to track the completion or progress towards completion of the milestones and deliverables originally identified. The NPHI can also choose to develop short-term (3-month) work plans based on baseline gap prioritization. Short-term work plans tend to be more concrete and measurable than those developed for a longer time frame. The NPHI may choose to develop sequential short-term work plans to address the priority gaps, and to monitor progress of implementation, revising existing and developing new work plans as previous work plans become fully implemented.

Finally, the SDT can be used for continuous monitoring and evaluation of progress made towards strengthening gaps that were identified using other tools, such as the WHO Joint External Evaluation (WHO JEE,[14])or Africa CDC’s Scorecard. By using the results of broader assessment tools (WHO JEE), the SDT can focus on the identified gaps to have greater impact.

In Colombia, the impact of the work planning and the provision of resources will be assessed by repeating the SDT using the same 9 Discussion Guides, in 2019, and comparing the outcomes with the 2017 ones. In Nigeria, NCDC plans to conduct a follow-up SDT assessment 12 months post implementation, to measure progress made in strengthening the three key areas that they set out to improve.

Use of the SDT in conjunction with other assessment tools

The SDT can be used in conjunction with other tools; in fact, many of the SDT Discussion Guides were developed to be explicitly complementary to other assessment tools, such as the WHO JEE. When the SDT is used with these other tools, the combined results provide NPHIs with a clear picture of current operational (internal-facing) and public health (external-facing) function capacities, and feasible work plan for addressing gaps to strengthen their capacity – information critical to effectively fulfilling IHR and global health security responsibilities.

Recent examples of such complementary engagements include Mongolia’s National Center for Public Health and Togo’s Institut National d’Hygiène.

• In Mongolia, the SDT was conducted in conjunction with use of IANPHI’s Peer Review tool. [12] The peer review tool provides an external evaluation of the NPHI that is meant to inform leadership and policy-makers of high-level strengths and issues. The SDT, on the other hand, is a self-evaluation of the NPHI, directly addresses areas of concern; it provides very specific recommendations for improvement in the evaluated areas, as well as work plans for achieving a more advanced developmental stage. Hence, in this process, the SDT focused on addressing gap areas identified by the peer review tool.

• In Togo, the SDT was undertaken shortly after the country had completed the WHO JEE. The JEE is a voluntary multi-sectoral external evaluation process that gives countries a starting point for improving their health security by:

  • (a)
    Identifying their strengths and most urgent needs within their health systems
  • (b)
    Assessing the countries capacity to prevent, detect, and rapidly respond to public health threats.

The Togolese NPHI used the areas of concern identified by the WHO JEE to select their SDT Discussion Guides and broadened the reach of their capacity development efforts.

The Africa CDC is currently in the early stages of developing a tool that can be used alongside the SDT, a high-level scorecard to assess NPHIs in key areas. This scorecard would provide Ministers of Health and NPHI directors with a visual snapshot of their NPHI’s functioning. Africa CDC is exploring how the scorecard could be used to highlight areas for SDT focus, so that concrete work plans are developed to support improvement in areas that receive yellow or red on the scorecard.

Strengths of the SDT

Successful implementation in a number of countries

The SDT Discussion Guides have been used successfully in six countries and piloted in another two, spanning continents and cultures. Feedback has been consistently positive, and several countries where the SDT was used have conducted or planned to conduct work with additional Discussion Guides.

Country ownership

Unlike many external assessments and despite being externally facilitated, the SDT was designed for country ownership. Countries select the areas (Discussion Guides) that they want to focus on, assess themselves, with the aid of a facilitator) and come to a common understanding of their current and desired states. Usually, the SDT is a self-assessment carried out in a closed setting without external stakeholders beyond the facilitator; hence, participants in the assessment process serve as their own judges. This process minimizes defensiveness and promotes open and frank discussions. In some cases, highly trusted stakeholders that are working closely with the NPHI, e.g., WHO, are important participants in discussions of certain topics. The direct link between assessment, identification of gaps, and work planning phases increases the likelihood that work plans will be implemented.

Concrete content

The SDT Discussion Guides were intentionally developed to be very concrete. This design is meant to stimulate participants to envision capacities and ways of working that had not previously occurred to them. It was also meant to reduce the dependency of the SDT on subject matter experts. Rather, a skilled, but generalist facilitator can use the material in the Discussion Guides to generate discussion and help participants envision a preferred future.

Inclusion of work planning

The direct link between assessment and work planning helps ensure that the assessment is not an end-stage document; a successful SDT experience concludes with a detailed work plan that flows directly from the assessment and may be used in the immediate strategic planning of the NPHI, or as a point of departure for future funding opportunities.

Scalability

While the original 28 Discussion Guides were developed around critical domains in public health, in support of the International Health Regulations (2005) and in response to the need for complementarity with current assessment tools, the architecture of the SDT is simple, flexible, and open. Existing Discussion Guides can be adapted or additional Discussion Guides developed to accommodate more public health topics of interest, such as One Health, Antimicrobial Resistance Monitoring, border health issues and others.

Limitations of the SDT

Ability to address systems issues

As is the case with many assessment tools, SDT participants tend to focus on easily measurable or identifiable issues, (e.g. the existence or lack of standard operating procedures or plans, inadequate training, insufficient funding). Participants do not often address systems issues such as the lack of district-level staff to conduct surveillance or inadequate use of technology, which are frequently the root causes of the problems they identify.

A skilled facilitator can use the SDT to identify the underlying issues that must be resolved in order to achieve the preferred state. It is up to the leadership of the NPHI undergoing the SDT to become a champion for change within the health sector governance, and to advocate for the impact that achieving the desired, higher state would have for the country. The SDT outcomes can help articulate this.

Expectation for future use

As more countries choose to develop NPHIs or strengthen existing NPHIs, we expect increasing demand for the SDT for planning and assessing progress over time. We also expect the SDT to increasingly provide support for detailed planning following higher-level assessments. Therefore, CDC and IANPHI are in discussions with other organizations, such as Africa CDC, and working towards developing a cadre of individuals experienced in using the SDT, who can assist in meeting the demand. Towards meeting that demand, CDC and IANPHI are supporting training of NPHI leaders and staff in the use of the SDT and training them in facilitation skills in early 2018. A number of the staff trained in the SDT and facilitation, will then begin participating alongside CDC and IANPHI facilitators to work with countries at all economic levels to implement the SDT, either with other tools (generally focused at a higher level) or on its own. This very important aspect of health system’s capacity building is fundamental for global health security, as is the evaluation of how these supportive tools impact the stakeholders’ ability to prevent, detect and respond to public health threats.