Evaluating the impact of research investments in the creation of knowledge in any area of work is challenging; even more so is evaluating the impact of research programs in emerging fields because of a lack of preexisting data and parameters. In the case of public health emergency preparedness (PHEP), the major difficulty is that this field of research is vast in terms of the number of disciplines included, and it comprises a variety of methods. Such methods are often tailored to address the diversity of local practice needs within specific public health systems, frequently leading to limited applicability and transferability to other geographic areas or response settings.
In 2008, the Centers for Disease Control and Prevention (CDC) created the Preparedness Emergency Response Research Centers (PERRCs) program to address public health preparedness knowledge gaps using a public health systems perspective. This program represents the first major federal investment in public health systems research in preparedness and to our knowledge is the largest worldwide. It’s been a decade since the start of this initiative, and we believe evaluation efforts are necessary to determine how to plan, organize, and advance future research in PHEP. In this editorial, we embrace the challenge to evaluate and discuss the contribution of the PERRCs program to knowledge production in this field.
HISTORY OF PERRCs
In 2008, the CDC commissioned the Institute of Medicine to develop a report identifying knowledge gaps in PHEP with the purpose of creating a research agenda meeting the requirements of the Pandemic and All-Hazards Preparedness Act of 2006. Knowledge gaps were identified in four priority areas: (1) enhancing the usefulness of training: research that creates best practices for the design and implementation of trainings (e.g., simulations, drills, and exercises) and facilitates the translation of their results into improvements in public health preparedness; (2) improving communications in preparedness and response: research that identifies and develops communications in relation to preparedness and response that effectively exchange vital and accurate information in a timely manner with diverse audiences; (3) creating and maintaining sustainable preparedness and response systems: research that identifies the factors that affect a community’s ability to successfully respond to a crisis with public health consequences, and the systems and infrastructure needed to foster constructive responses in a sustainable manner; and (4) generating criteria and metrics to measure effectiveness and efficiency: research that generates criteria for evaluating public health emergency preparedness, response, and recovery efforts and metrics for measuring their efficiency and effectiveness.1 Based on the recommendations provided by the Institute of Medicine, the CDC established a network of nine centers, the PERRCs, within accredited schools of public health in the United States. The network was awarded approximately $56.6 million over six years to conduct research in domestic-focused public health preparedness.
HOW MUCH KNOWLEDGE WAS PRODUCED?
A previous analysis conducted by Leinhos et al.2 reported that the network of PERRCs conducted 34 research projects, resulting in more than 130 peer-reviewed publications and 80 tools; trained more than 30 new investigators; and engaged more than 500 research partners. By reexamining a literature review previously published by Savoia et al,3 we further assessed the contribution of the PERRCs to the US scientific knowledge production. We retrieved the articles included in the review and determined the PERRCs’ contribution based on authorship, grant acknowledgments (direct contribution), and citations to PERRCs’ studies in the references (indirect contribution).
As described in Figure 1, our observations suggest that the PERRCs played a substantial role in the majority of the research conducted in this field and that without this program at least half of the knowledge produced during this period would be missing. For example, the PERRCs contributed to more than half of the studies produced in the area of creating and maintaining sustainable preparedness and response systems, the research area with the largest knowledge produced. By appraising the content of the PERRCs studies we generated lists of topic areas and applied methods to understand what areas the PERRCs focused on the most. Our observations indicate that among all preparedness capabilities, PERRCs studies prevailed in the area of community preparedness. The majority of PERRCs used quantitative methods and embraced an all-hazards approach. Across all four priority areas determined by the Institute of Medicine, the PERRCs contributed to: generating a better understanding of how to evaluate emergency preparedness trainings, and how to identify a population’s knowledge, attitudes, and practices in relation to emergency communication efforts; investigating organizational and structural characteristics that influence a public health system’s preparedness; and exploring the development and use of measures of organizational assessment to determine a system’s readiness.
DEVELOPING A WAY OF KNOWING
The paucity of research in specific areas and the diversity of research goals limit the ability of aggregating findings across PERRCs’ studies, as well as comparing the knowledge produced by the PERRCs to the research conducted in sites other than PERRCs. In the previous review, the quality of research in PHEP was assessed focusing on study methods, relevance, coherence, and adequacy of findings across studies within specific areas of research.3
We accentuate that the contribution of the PERRCs, as well as other organizations, should be valued not only for the body of knowledge produced but also for the way of knowing they have developed. They developed a systems-oriented way of studying preparedness that uses data from actual events to methodically examine public health system organization, performance capabilities, effectiveness, and social determinants of system outcomes.4 Another important characteristic of the PERRCs’ work has been the focus on knowledge transfer. The PERRCs dedicated considerable resources toward developing tools derived from their research efforts and in testing each tool’s applicability in various practice settings to make research findings more accessible and available to public health practitioners.5
PUBLIC HEALTH IMPLICATIONS
Developing a future agenda in PHEP observations on knowledge production, as reported in this editorial, should complement consultation processes commonly used by stakeholders to identify knowledge gaps in practice.6,7 Combining both approaches—knowledge synthesis and stakeholders’ input—provides a means to determine if knowledge gaps are because of a lack of translation of knowledge into practice or lack of research, or to the fact that some practice questions cannot be answered through research. A better understanding of the nature of knowledge gaps can help inform priorities for future research and for facilitating the transfer of relevant research findings into practical applications.
ACKNOWLEDGMENTS
This study was supported under a cooperative agreement with the Centers for Disease Control and Prevention’s (CDC’s) Collaboration With Academia to Strengthen Public Health Workforce Capacity (grant 3 U36 OE000002-04 S05), funded by the CDC and the Office of Public Health and Preparedness and Response through the Association of Schools and Programs of Public Health (ASPPH).
Note. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the CDC, the Department of Health and Human Services, or the ASPPH.
REFERENCES
- 1.Institute of Medicine. Research Priorities in Emergency Preparedness and Response for Public Health Systems. A Letter Report. Washington, DC: The National Academies Press; 2008. [Google Scholar]
- 2.Leinhos M, Qari SH, Williams-Johnson M. Preparedness and Emergency Response Research Centers: using a public health systems approach to improve all-hazards preparedness and response. Public Health Rep. 2014;129(suppl 4):8–18. doi: 10.1177/00333549141296S403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Savoia E, Lin L, Bernard D, Klein N, James LP, Guicciardi S. Public health system research in public health emergency preparedness in the United States (2009–2015): actionable knowledge base. Am J Public Health. 2017;107(S2):e1–e6. doi: 10.2105/AJPH.2017.304051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Qari SH, Abramson DM, Kushma JA, Halverson PK. Preparedness and Emergency Response Research Centers: early returns on investment in evidence-based public health systems research. Public Health Rep. 2014;129(suppl 4):1–4. doi: 10.1177/00333549141296S401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. PERRCs toolkit inventory. Available at: https://www.hsph.harvard.edu/preparedness/toolkits/perrcs-toolkit-inventory. Accessed January 16, 2018.
- 6.Khan Y, Fazli G, Henry B et al. The evidence base of primary research in public health emergency preparedness: a scoping review and stakeholder consultation. BMC Public Health. 2015;15(1):432. doi: 10.1186/s12889-015-1750-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Siegfried AL, Carbone EG, Meit MB, Kennedy MJ, Yusuf H, Kahn EB. Identifying and prioritizing information needs and research priorities of public health emergency preparedness and response practitioners. Disaster Med Public Health Prep. 2017;11(5):552–561. doi: 10.1017/dmp.2016.198. [DOI] [PMC free article] [PubMed] [Google Scholar]