Stakeholders’ assessment of US Centers for Disease Control and Prevention’s contributions to the development of National Public Health Institutes in seven countries

Abstract

National Public Health Institutes (NPHIs) can strengthen countries’ public health capacities to prevent, detect, and respond to public health emergencies. This qualitative evaluation assessed the role of the US Centers for Disease Control and Prevention (CDC) in NPHI development and strengthening of public health functions. We interviewed NPHI staff (N=43), non-NPHI government staff (N=29), and non-governmental organization staff (N=24) in seven countries where CDC has supported NPHI development: Cambodia, Colombia, Liberia, Mozambique, Nigeria, Rwanda, and Zambia. Participants identified four areas of support that were the most important: workforce capacity building, technical assistance for key public health functions, identifying institutional gaps and priorities, and funding to support countries’ priorities. Participants underscored the need for capacity building directed toward country-driven priorities during planning and implementation. Continued support for NPHI development from CDC and other partners is vital to building stronger public health systems, improving population health, and strengthening global health security.

Introduction

National Public Health Institutes (NPHIs) serve as a country’s lead technical public health agency and strengthen countries’ public health capacities to prevent, detect, and respond to public health emergencies [1-3]. Most NPHIs have legal mandates that create distinct leadership and national-level authority, which allows independence from political interference and response in the best interest of public health [2, 4]. They generate data and evidence to inform policies and programs and guide development of a skilled public health workforce. They strengthen countries’ public health systems through disease surveillance, early detection, and monitoring; investigating and controlling outbreaks; conducting research; and providing training in health education, health promotion, and laboratory sciences [1]. NPHIs strive to employ skilled public health staff, including infectious disease specialists, epidemiologists, laboratory microbiologists, statisticians, and health educators, who work together to combat the most salient public health challenges in their countries [5]. NPHIs also play a crucial role in building capacity for global health security through the implementation of the International Health Regulations (IHR 2005), which provide an overarching framework for countries to build core capacities and manage public health emergencies [6].

Support for NPHIs can drive long-term impact of public health interventions, because NPHIs contribute to the development and sustainability of public health infrastructure, such as workforce capacity [7]. Low- and middle-income countries (LMICs) often establish NPHIs in collaboration with their Ministries of Health (MoH) and through technical and financial support from partners, such as the US Centers for Disease Control and Prevention (CDC) and other NPHIs, non-governmental organizations like the International Association of National Public Health Institutes (IANPHI), and multilateral partners like the World Bank. As part of CDC’s Global Health Strategy [8] and the goal to build country and global capabilities to improve public health preparedness, CDC supports NPHI development to facilitate central coordination of essential public health functions, such as surveillance systems, laboratory systems, workforce development, and emergency preparedness and response. Since 2011, CDC has made significant contributions to assist development and strengthening of NPHIs in more than 30 countries and the Africa Centres for Disease Control and Prevention (Africa CDC), primarily through its NPHI Program. CDC’s support has included direct, multi-year financial support and sustained technical assistance for NPHI development and public health function strengthening [9]. However, no systematic evaluation has been conducted to assess the impact of CDC’s support. We conducted this evaluation to assess CDC’s role in supporting the development and strengthening of NPHIs in seven countries.

Methods

Study design

Country selection

Among 30 countries where CDC has partnered on NPHI development and strengthening, we purposively selected countries where CDC provided financial support at a comparatively high level, and accounted for geographic variation, institutional development stage, and the means by which CDC provided support (directly or through partners). We selected seven countries: Cambodia, Colombia, Liberia, Mozambique, Nigeria, Rwanda, and Zambia.

Participant recruitment

We selected potential participants through a collaborative process among CDC’s NPHI program team, and NPHI and CDC staff in the seven countries. We sought to conduct interviews in each country with (a) NPHI leadership (for example, a NPHI director, laboratory leader, emergency operations center leader), (b) other government staff (such as MoH permanent secretaries, public health directors, district health office directors), and (c) external partners collaborating with the NPHIs (for example, universities, non-governmental organizations, UN agencies, international organizations). We selected participants based on their roles as public health leaders who could provide expert opinion about the evaluation’s objectives. After considering human and financial resources available to implement the evaluation, we sought to interview 13–15 participants in each country [10, 11]. We contacted participants by e-mail, shared the evaluation objectives, and requested participation.

Data collection

From August 2019 through January 2020, MAW and KF conducted in-person interviews with all participants. MAW led the evaluation, was unaffiliated with CDC’s NPHI program, and is a PHD researcher with extensive experience in conducting mixed-method and qualitative evaluations in low-income settings. KF was an evaluation fellow with CDC’s NPHI program and had experience working in sub-Saharan Africa, but had not previously worked with the participating countries on NPHI development. MAW provided refresher training to the evaluation team on qualitative methods, including ethical considerations, data collection, data analysis, and writing. Neither of the interviewers knew any of the participants prior to traveling to the seven countries to conduct the interviews. Interviews were semi-structured, lasted 41 to 96 min (average = 53 min) and explored participants’ perceptions about CDC’s role in supporting NPHI development, strengthening public health functions, and their recommendations for how CDC could better support NPHIs. The team member not conducting the interview took detailed written notes. We conducted all interviews in private and in English, except for 14 interviews in Colombia and one interview in Cambodia that we conducted through experienced interpreters in Spanish and Khmer, respectively.

Data analysis

The evaluation team transcribed all recordings verbatim and coded them using MAXQDA Version 20.0.2. We employed content analysis in which coding categories were informed by the interview questions and derived directly from transcribed data [12]. Four team members coded the interviews. We used the first iteration of the codebook to code six transcripts independently and then we checked, refined, and modified, if needed. The team reviewed a random selection of each set of transcripts to ensure consistent application of theme categorizations and used an iterative process to resolve any discrepancies in the coding application. This process established inter-coder reliability, including consistency and consensus coding application within the MAXQDA platform. Upon coding completion, the team reviewed the coded excerpts for key themes and identified themes through well-established techniques, including repetition (if a theme was expressed multiple times) [13].

To assess the validity of our conclusions, we used triangulation (collecting data from multiple sources, using more than one interviewer), and “member checking” (sharing and soliciting feedback on the results and conclusions from a small group of representatives at each of the seven NPHIs) [14, 15]. To prevent potential researcher bias during data analysis [16], the team debriefed regularly to discuss the results, emerging themes, and potential conclusions.

Ethical considerations

CDC’s project determination team reviewed the evaluation and determined that it was non-research and did not require review from an institutional review board. We notified participants of data confidentiality, data safeguarding procedures, their rights regarding participation, and that all information they provided would remain confidential and anonymized. All participants provided written consent before the interview; we recorded interviews if the participant agreed.

Results

Among 108 individuals contacted, all agreed to participate. Twelve persons (11%) who had initially agreed to participate had to cancel due to last-minute conflicts and we successfully completed 96 (89%) interviews. Of the 96 persons interviewed, 43 (45%) were NPHI staff, 29 (30%) were non-NPHI government staff, and 24 (25%) worked in non-governmental or international organizations (Table 1). Sixty-six (69%) participants were male. A full list of participants’ positions or institutions they represented appears in Table 2.

Table 1.

Characteristics of participants in the National Public Health Institute (NPHI) evaluation, August 2019–January 2020

	Participant type
Country	NPHI Staff		Non-NPHI government staff		Non-government partner staff		Total (N)
	n	Male	n	Male	n	Male	N	Male
Cambodia	4	4	5	5	5	5	14	14
Colombia	9	4	6	3	1	0	16	7
Liberia	7	6	4	3	0	0	11	9
Mozambique	7	4	5	5	7	5	19	14
Nigeria	5	2	1	1	6	2	12	5
Rwanda	6	6	2	2	2	1	10	9
Zambia	5	3	6	4	3	1	14	8
Total	43	29	29	23	24	14	96	66

Table 2.

Characteristics of participants by position or institution in the National Public Health Institute (NPHI) evaluation, August 2019–January 2020

Participant type	Position or institution	Countries represented	Number of participants	Total
NPHI staff	NPHI Directors, Deputy Directors, and Division Directors	Colombia, Cambodia, Liberia, Nigeria, Mozambique, Rwanda, Zambia	43	43
Non-NPHI government staff	Ministry of Health (MoH)	Cambodia, Colombia, Liberia, Mozambique, Rwanda, Zambia	14	29
	District (Provincial) Level Health Department	Colombia, Mozambique, Zambia	6
	Ministry of Foreign Affairs	Colombia	1
	President Office	Colombia and Zambia	2
	Ministry of Agriculture	Colombia, Mozambique, Nigeria, Rwanda, Zambia	5
	Ministry of Statistics	Cambodia	1
Non-government partner staff	Pasteur Institute	Cambodia	1	24
	German Development Agency (GIZ)	Cambodia, Nigeria	3
	World Health Organization (WHO) and Pan American Health Organization (PAHO)	Cambodia, Colombia, Mozambique, Nigeria, Rwanda, Zambia	9
	University	Cambodia, Zambia	2
	Clinton Health Access Initiative (CHAI)	Mozambique	2
	United Nations International Children’s Emergency Fund (UNICEF)	Mozambique	2
	Canadian High Commission	Mozambique	1
	Public Health England	Nigeria	1
	Public Health International	Nigeria	1
	World Bank	Zambia	1
	West African Health Organization (WAHO)	Nigeria	1
Total				96

CDC’s role in strengthening NPHIs

Forty-two (98%) NPHI staff, 13 (45%) non-NPHI government staff, and 10 (42%) non-government partner staff discussed CDC’s contributions towards strengthening NPHIs in their countries. Four themes emerged describing CDC’s most prominent contributions (Supplementary Materials Table A):

1. enhancing countries’ public health workforce by building capacity of NPHI and government staff;

2. strengthening public health systems through technical assistance for key public health functions;

3. using tools and technologies to help NPHIs identify priorities and gaps in their functions; and

4. providing funding (through CDC’s NPHI program) that could be directed towards country-defined priorities.

Building staff capacity

NPHI staff noted that CDC’s support to the Field Epidemiology Training Program (FETP) has been one of the most impactful collaborations with NPHIs and critical to building a trained public health workforce. As a result of CDC’s assistance, FETP has trained staff in emergency preparedness and response, including outbreak investigation. FETP graduates have become the building blocks of NPHIs’ workforces and operate at the national and local levels to strengthen countries’ health systems. Participants from all seven countries noted that NPHIs’ roles included defining strategic direction for disease control, providing leadership during public health emergencies, and coordinating public health activities. To meet these responsibilities, participants underscored NPHIs’ roles in training countries’ public health workforce to address emerging and re-emerging public health threats. As such, participants considered NPHI collaboration with sub-national governments and various partners, including CDC, to be critical to building competencies among staff at all government levels.

Technical assistance for key functions

Participants from all seven countries identified CDC-supported training (laboratory techniques, surveillance methodologies, and emergency preparedness and response) as among the most important contributions to their NPHI development. In three countries (Rwanda, Colombia, Zambia), participants mentioned that CDC-supported training on specific laboratory techniques, including influenza molecular diagnosis, gave NPHI staff the ability to run tests and calibrations independently. Participants also mentioned that CDC helped improve NPHIs’ capabilities with data collection, analysis, and reporting, and the use of new and better technologies to identify unusual events. As a result, participants reported that NPHIs have improved surveillance reporting, epidemic response, and real-time reporting at the local, sub-national, and national levels.

Identifying priorities and gaps

According to participants across all countries, CDC helped NPHIs identify gaps and priorities using various tools, including the Staged Development Tool (SDT), a maturity model-based tool that assesses organizational capacity and public health functions [17]. These assessments helped NPHIs improve internal operations and resource management and to identify gaps in training, research, and workforce development. Participants also shared that CDC-led assessment exercises helped institutes identify weaknesses and opportunities for improvement and provided guidance for effective strategic and operational planning, budget creation and management, and risk communication.

NPHI resources

NPHI staff across all seven countries noted the challenges of constrained financial resources to implement their strategies. Most of the government funding for NPHIs is based on what country governments can allocate rather than what the NPHIs need. Funds from partners are often earmarked for specific programs, public health functions, or public health areas. Participants noted that donor funding that allows NPHIs to address country priorities, rather than pre-defined funder priorities is essential.

Recommendations for CDC on how to better support or continue supporting NPHIs

Thirty-nine (91%) NPHI staff, 20 (69%) non-NPHI government staff, and 14 (58%) non-government staff offered at least one suggestion for how CDC could improve its support to NPHIs. Three major themes emerged (Supplementary Materials Table B): (1) continuing to build NPHIs’ staff capacity and supporting platforms for experience sharing; (2) providing more flexible funding for NPHIs; and (3) prioritizing countries’ needs during planning and implementation.

Continue to build NPHIs’capacity

Participants underscored the importance of CDC continuing to build the capacity of NPHI staff to perform their jobs, rather than delegating key activities to donor countries or organizations. Specifically, participants suggested that CDC continue supporting capacity training for NPHI staff in epidemiology, surveillance, laboratory, and research. Participants also noted the importance of training the next generation of public health leaders as critical to continuation of programs and institutions. Additional recommendations included providing trainings in specific skills to NPHI staff and health workers at the district level. They emphasized the critical importance of sharing information and transferring knowledge, experiences, and challenges with sub-national regions, and said CDC’s support as a partner would be important during these exchanges. Participants also highlighted need for information sharing and for building a platform to allow NPHIs to share experiences and lessons learned from different NPHI models around the world. Some examples included collaborating on scientific manuscripts, sharing pertinent data to inform policy and public health actions in countries, and establishing a mechanism to exchange continuous feedback among all partners and share lessons learned.

Consider support of NPHIs’ priorities

Participants mentioned that most NPHIs have institutional priorities, including enhancing administrative operation, that require training additional staff so NPHIs can make effective use of donor and partner financial support. Most NPHI staff noted that at least some donor funding NPHIs receive is focused narrowly on a specific area of capacity development or disease. Participants emphasized that without the proper administrative support, most program implementation, including emergency preparedness and response efforts, could fail.

Prioritize countries’ needs during planning and implementation

Participants across all countries and groups described the importance for CDC and other partners, including IANPHI and donor countries, to prioritize countries’ needs when planning and programming funds and providing technical assistance. NPHI staff underscored the importance of NPHI staff engagement in the program, project and activity planning process, rather than CDC informing NPHIs about activities after decisions have already been made. NPHI staff also emphasized the importance of having national staff at the forefront of outbreak responses and other public health emergencies, with CDC serving in a supporting role. They emphasized that a successful partnership would assure the countries lead the response to their nations’ problems, with support of partners.

Discussion

We found that CDC played critical roles in support of NPHIs’ key public health functions in the seven participating countries. These contributions included strengthening workforce development, surveillance, laboratory, and emergency preparedness and response. CDC also assisted NPHIs to identify institutional gaps and priorities and directed its assistance to those priorities. In addition to financial support, CDC’s technical experience as the US NPHI for the past 75 years has positioned it to provide technical assistance that has helped countries build public health capacity and improve health systems at the global level. We found evidence of the importance of CDC’s support reflected among participants who perceived CDC’s engagement with the seven NPHIs to be essential and successful.

Although not an explicit objective of our evaluation, input from participants from all seven countries affirmed that the role of NPHIs included training their countries’ public health workforce, coordinating public health activities, such as research, surveillance, and outbreak investigations, and providing leadership during times of public health emergencies. Through training and service programs like FETP, NPHIs develop a workforce able to collect and analyze data and employ evidence-based decision making to address national public health priorities, including not only infectious diseases, but importantly non-communicable diseases and injuries. These findings align with those of previous studies about the role of NPHIs in countries, including Liberia, Cambodia, Zambia, Mozambique, and Nigeria [18-22].These findings also support that NPHIs serve as a single focal point for linking public health activities essential to coordination and implementation of effective response strategies during emergencies [23].

Participants provided suggestions on how CDC could enhance its support to NPHIs. These included: building capacities, providing funding that can be applied to institutional or program development priorities, and emphasizing partner countries’ needs during all processes. First, CDC could continue to build capacity of NPHIs in LMICs using CDC expertise in virtual or in-person trainings in leadership and management, epidemiology, surveillance, laboratory, and research. CDC could also support administrative structures and provide grants management training. Second, CDC could engage partner NPHIs in a participatory planning process to identify key priorities and gaps that would allow them to target specific funding or technical assistance to support sustainability improvements in priority areas. NPHIs emphasized the importance of resources to support NPHI administration, operations, and country priorities to complement donor funding that focused on a specific disease or technical area. Third, CDC could improve its own staff’s awareness of the role of partner support in country-led emergency preparedness and response, and of the importance for CDC to facilitate country-focused decision-making for a successful partnership.

Our findings must be interpreted in light of several limitations. First, we focused solely on the impact of CDC support of NPHIs and did not assess other partners’ support of NPHI development in these seven countries. We believe that our findings still offer important information for other organizations, agencies, and partners that want to support NPHI strengthening in LMICs. Second, our data were collected only from seven countries that were purposively selected and might not be representative of all NPHIs that CDC supports. It is unclear to what extent our findings reflect opinions shared by participants beyond countries included in this study. Third, the participants interviewed for this evaluation might have been more likely to give positive feedback, given that their institutions are longstanding CDC partners that have received financial and technical support. We tried to minimize this potential for bias by using interviewers who were not part of CDC’s NPHI program and did not have a history of past collaboration on NPHI development. Although we tried to understand the impact of CDC’s contributions from various perspectives by interviewing individuals from different organizations, we could not fully account for other factors (economic trends, political changes) that may have influenced participants’ perspectives. Finally, although CDC’s NPHI Program works closely with other CDC programs working in these countries, we focused this evaluation specifically on the work of CDC NPHI Program and not on other CDC programs.

Conclusions

CDC plays a critical role in the development and strengthening of NPHIs in countries where it has made significant contributions. NPHIs and workforce development go hand-in-hand, and this linkage is crucial during times of emergency. In addition, NPHIs need fully developed internal functions, such as communication, finance systems, and human resource development to maximize strong science and overall public health impact. Finally, continued support in NPHI development from CDC and other partners are vital to building stronger public health systems, improving population health, and strengthening global health security [24].

Key Messages.

• National Public Health Institutes (NPHIs) identify CDC-supported training as critical to improved surveillance reporting, epidemic response, and real-time reporting at the local, sub-national, and national levels.

• CDC assistance for identifying gaps and priorities helps NPHIs strengthen the organization internally and improve operations and resource management.

• NPHIs and workforce development go hand-in-hand, and this linkage is crucial during times of public health emergencies.

Biographies

Mahlet A. Woldetsadik Ph.D., M.P.H., is Senior Service Fellow and Health Economist on the Applied Research Team, Office of the Director, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Kaitlin Fitzpatrick M.P.H., is Cooperative Agreement Specialist on the Program Budget and Extramural Branch, Division of Global HIV and TB, Center for Global Health at the US Centers for Disease Control and Prevention. At the time of authorship, she was an Oak Ridge Institute for Science and Education (ORISE) Fellow in the Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Lisetta Del Castillo M.P.H., is Manager for the National Office Performance Evaluation at Planned Parenthood Federation of America. At the time of authorship, she was RTI contractor in the Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Bridget Miller M.P.H., is Public Health Institute Fellow in the Division of Global HIV and TB, Center for Global Health at the US Centers for Disease Control and Prevention. At the time of authorship, she was ORISE Fellow in the Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Dennis Jarvis M.P.H., is Public Health Advisor with the National Public Health Institute Program in the Workforce and Institute Development Branch, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Caroline Carnevale M.S.P.H., is Public Health Analyst for the National Public Health Institute Program in the Workforce and Institute Development Branch, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Fatima Ravat M.P.H., is Program Manager with the National Public Health Institute Program in the Workforce and Institute Development Branch, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Cynthia H. Cassell Ph.D., is Lead for Applied Research in the Office of the Director, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Alice Williams M.P.H., is Senior Consultant at Booz Allen Hamilton. At the time of authorship, she was an ORISE Fellow in the Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Stephanie K. Young M.P.H., is Consultant at Guidehouse serving the US Centers for Disease Control and Prevention. At the time of authorship, she was an ORISE Fellow in the Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Jacob Clemente M.P.H., is Public Health Analyst on the Consultation and Training Team, Office of the Director, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases at the US Centers for Disease Control and Prevention. At the time of authorship, he was a student intern in the Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Henry C. Baggett M.D., M.P.H., is Chief of the Workforce and Institute Development Branch, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.

Shelly Bratton M.P.H., is Lead for the NPHI Program in the Workforce and Institute Development Branch, Division of Global Health Protection, Center for Global Health at the US Centers for Disease Control and Prevention.