Abstract
The objective of this article is to describe factors which contribute to successful translation of science into evidence-based practices and their implementation in public health practice agencies, based on a review of the literature and evidence from a series of case studies. The case studies involved structured interviews with key informants in four health departments and with four corresponding partners from academic institutions. Interviews were recorded and transcribed, coded by two independent, trained coders, using a standard codebook. A thematic analysis of codes was conducted. Coding was entered into Atlas TI software for further analysis.
Results from the literature review indicated that only approximately half of programs implemented in state and local health departments were evidence-based. Lack of time, inadequate funding, and absence of cultural and managerial support – including incentives - are among the most commonly cited barriers to implementing evidence-based practices.
Findings from the case studies suggest that these health departments, successful in implementing evidence-based practices, have strong relationships and good communication channels established with their academic partner(s). There is strong leadership engagement from within the health department and in the academic institution. Implementation of evidence-based programs was most often related to high priority community needs and the availability of resources to address these needs. The practice agencies operate with a culture of quality improvement throughout the agency.
Information technology, training, how the interventions are bundled, including their complexity and ability to be customized and resource requirements are all fruitful avenues for further research.
Introduction
This article describes factors which contribute to successful implementation of public health science. Public health practitioners increasingly have a vast array of information at their fingertips as they seek to improve their practice. One way to achieve improvements is to build the practice of evidence-based public health, which is defined as a process of integrating science-based interventions with community preferences to improve the health of populations.1 It is difficult to estimate precisely how widely evidence-based approaches are being applied. In a survey of U.S. public health practitioners, an estimated 58% of programs in their agencies were deemed evidence-based (i.e. based on the most current evidence from peer-reviewed research).2 This finding mirrors the use of evidence-based approaches in clinical care. A random study of adults living in selected metropolitan areas within the United States found that 55% of overall medical care was based on what is recommended in the medical literature.3
To accelerate the movement of evidence to practice, public health systems and services research (PHSSR) and dissemination and implementation (D&I) research seek to identify the underlying knowledge and approaches for applying evidence-based approaches in a wide range of settings.4 Numerous studies have shown that dissemination of evidence-based interventions using passive methods (e.g. publication of consensus statements in professional journals, mass mailings) are ineffective, resulting in only small changes in practice.5 Targeted, active approaches to D&I are needed that take into account many factors, including the characteristics and needs of users, types of evidence needed, administrative skills, organizational climate, and culture.
Relevancy of evidence appears to influence the uptake and use of evidence-based approaches. For example, information that is highly research-oriented and commonly used among public health experts may not be useful to policy audiences. A survey of 292 state level policy makers suggests that officials are overwhelmed with the volume of information they receive and prefer data that are concise and relevant to current debates.6
The Consolidated Framework for Implementation Research
A literature review identified several models of dissemination and implementation.7,8,9 The Consolidated Framework for Implementation Research (CFIR) by Damschroder et al., was most applicable to the case studies reported in this article. This framework integrates 19 models of innovation dissemination and implementation, and organizes the constructs into five domains:7
intervention characteristics - such as perceived source and evidence strength and quality
outer setting - such as community needs and resources and external policies or incentives
inner setting - such as perceived need for change and available resources
characteristics of individuals – such as knowledge and beliefs about the intervention
implementation process - such as quality of planning and engaging staff
Green et al. point out that public health research has often examined narrow and specific research problems and focused on establishing internal validity, to the neglect of external validity, or the degree to which practitioners in typical settings might be able to use the knowledge.8 While this may be appropriate for certain kinds of technical interventions, it is less well suited for the complex multifactorial problems and varied community contexts in which public health agencies operate. Green et al. advocate that researchers partner with practice organizations to identify, create, and evaluate practices that the organizations can implement.8 Collaborative approaches to intervention development and evaluation offer the promise of addressing many barriers to implementation in the CFIR.7
Organizations seeking to disseminate research-based interventions might also use the CFIR to plan their communication and change efforts.7 By doing so, they might avoid many of the “top ten dissemination mistakes” described by Dearing, such as assuming that evidence rather than costs, compatibility, and simplicity matter most to decision-makers, or that information will influence decisions, when social influence from professional associations or within organizations may be more important.9
We report findings of our case studies, aimed to evaluate specific constructs that influence implementation, through application of the CFIR.7 The purpose of this study was to better understand how public health practitioners identify evidence and use these findings in their practice settings, and to identify what works across various practice settings, with a focus on four local public health agencies.
Methods
Four health departments with special knowledge or insight into the issue of interest were identified through discussions with key informants from practice-based research networks, the Centers for Disease Control and Prevention (CDC) Prevention Research Centers, academic health departments, and the CDC. Health directors from these sites identified academic partners to include in the study based on their working relationship with these partners. Two sets of interview questions were developed based on the literature review. One set of interviews was conducted with faculty to determine how they conduct research about effective public health programs and practices, how they disseminate this research into practice, and how they help practice organizations implement and sustain evidence-based programs. The second set of interviews was conducted with leadership in the four health departments. The interviews focused on factors that facilitate implementation of evidence-based practices or programs, and whether and how they obtained training or technical assistance when implementing or sustaining these evidence-based practices. Health department leaders were also asked to share any barriers they encountered to implementing and sustaining evidence-based programs. Community health assessment documents, strategic plans, and annual reports were analyzed for presence of evidence-based practices. Interviews averaged approximately one hour, and were conducted by two former local health directors with doctoral degrees in public health. Interviews were recorded and transcribed. Two team members were trained on coding to ensure reliability among raters. Coders were assigned transcripts to code independently using a codebook based on CFIR. A thematic analysis of codes was conducted to determine which codes appeared most frequently. The research proposal was approved by the IRB at the University of North Carolina at Chapel Hill. Informed consent was obtained from all interviewees.
Description of Sites for Interviews
County A is home to a large state university housing a College of Medicine and a state-authority local public health agency with a budget of 48 million dollars and 640 positions serving a southeastern city with over 1 million residents.10,11 Median household income was approximately $50,000 in 2010, with 14.2% of individuals living in poverty and 8.7% unemployed. Nearly one-third of residents are Black/African American. County B is home to a large state university housing a College of Public Health and a local public health agency employing 45 individuals in a Midwestern city serving 240,000 residents.10,11 The population is largely Caucasian (80%) and median household income in 2010 was approximately $40,000, with 17% living below the poverty level and 11% unemployed. The agency has a budget of 5 million dollars, with local funding contributing 67% to the overall budget. City C is home to a local public health agency employing 258 individuals serving 600,000 metropolitan in the Midwest.10,11 A large state university housing a School of Medicine and Public Health is located in another city. Caucasian and Black/African American residents make up 45% and 40% of the population respectively. Nearly 18% identify themselves as Hispanic/Latino. Median household income is $36,000 with 26% of individuals living in poverty and an estimated 12% unemployed. The agency has an annual budget of 26 million dollars, with 50% of funding from local taxes. County D is home to a local public health agency employing 134 individuals serving 300,000 residents in a medium-size metropolitan area in the northeast.10,11 A large private university housing a School of Public Health is located in an urban city approximately two hours from the health department. The population is largely Caucasian (80%) with a median household income of $70,000, eight percent of individuals living below poverty and an unemployment rate under 7%. The annual budget is approximately 36 million dollars.
Findings
Eight faculty members from academic institutions and 14 public health executives and senior leaders were interviewed across the four sites. Several themes emerged that were consistent with the CFIR7 and the most common are reported here.
Outer Setting
Both academic partners and health department employees were widely networked with external organizations such as NACCHO, statewide associations, professional organizations, peer counties, and practice-based research networks. Opportunities existed for boundary spanning roles for staff and faculty, including adjunct appointments of health department staff at universities and staff roles for faculty within health departments (primarily negotiated through contracts between the academic organization and the health department). These opportunities and networks contributed to the cosmopolitanism of the organizations and facilitated supportive leadership, training, access to information and better communication between research and practice. The academic partners stated that their organizations tended to make investments in faculty engaging in practice-related activities and they were able to attend meetings with health department leadership and offer assistance in grant writing, data analysis and reporting, and adaptation of evidence-based practices to meet the contextual needs of the agency.
Moreover, health departments and academic partners consistently reported that their emphasis on dissemination, adoption, and implementation of evidence-based research was directly related to meeting community needs and identifying additional resources to meet those needs. They regularly looked to key peer or comparable organizations to determine how similar community needs were being addressed by those groups. The Health Commissioner in County A noted, “An example would be expedited partner therapy for STDs, which has been adopted by several states. It’s when the patient is given medications to give to the partner instead of having the partner have to come in for a clinic visit and STD treatment. We’re pursuing legislation that would authorize that to be done in [this state] but it’s as a pilot with research to be done here and in a rural county overseen by our research unit.”
External policies and incentives were also relevant to whether health departments adopted and implemented evidence-based practice. Agencies cited federal and state program mandates that required implementation of evidence-based practice in order to receive or continue to receive funding. A senior administrator in County D noted, “As much as we are independent, we do get a lot of directives from the state in terms of our mandates. Some of those mandates come with specific programmatic guidance, exactly how they want you to do it […] a lot of these are evidence-based because the state’s been pushing for that.”
Health department leaders also mentioned that voluntary public health accreditation and other formal benchmark reporting activities, such as the county health rankings, led their organizations to identify and implement approaches that would result in desired outcomes.
The sociopolitical context within which the academic institutions and health departments functioned appeared to directly influence their organizations’ ability to disseminate, adopt and implement evidence-based approaches. Respondents consistently cited the importance of support from administrators, elected officials and other community leaders as critical to successfully disseminating and implementing evidence-based practice. A senior health administrator from County B noted, “The ultimate decision always comes down to the health commissioner. I think if it was a project that I was working on, I would try and put together the preliminary data, obviously be able to show […] why we need to change a practice or why we need to move in a different direction based on some of the information that I found, and then I would provide that information to the health commissioner.” A faculty member from County B noted that “…our faculty are very supportive of practice-based research and nearly all of them have engaged in it themselves and so there’s been a strong commitment from the day the college was founded that we weren’t going to be an ivory tower, academic institution; that we really did want to work with the community and with our practice partners and engage in scholarly research that could advance the practice of public health. So we already have that commitment and the Dean is committed.”
Inner Setting
Two practice sites identified funding to establish internal units specifically to provide data support for programs, identify grant opportunities, research evidence-based practices and provide evaluation services. The remaining two practice sites noted that personnel trained in epidemiology and biostatistics were available for research and data-related activities. Some health departments noted that this research and data capacity was extended to community organizations. Academic partners shared that they had formal structures within their institutions to facilitate communication and collaborative work with local health departments. A senior faculty member from County B commented, “We have first set up a center for public health practice within the college and one of the goals of that center is to identify practice-based research opportunities and practice-based partners that we can engage in research with. So we have faculty who are interested in applied research and we try to link them up with the practice world.”
Internal networks and communication mechanisms were also important factors in promoting communication and integration across organizational lines. Communication within the health department and with academic partners was strengthened if there was information sharing, open feedback and clear communication of mission and goals.
Health departments described various ways to gain knowledge about evidence-based practices and how to incorporate these approaches into their work tasks. Some health departments specifically distributed research articles through department or professional list serves and some utilized common databases. A senior administrator from County D noted this change over time, “…I get a bunch of list serves and whenever I get something that I see, I just forward it to the staff that I know is working on that particular issue as just a reminder. And what I’ve noticed over the years is, at first, it was like, ‘Oh, wow. This is great. Thank you.’ And now, I’m getting, ‘Thanks, I already get this.’ And that, to me, is the kind of reply I want to get from them, because it tells me that they’ve now gotten the message…”
A researcher from County D in describing how to make more research resources readily available commented, “…I just got an iPad and I’ve been exploring what’s available in apps, because that makes much more sense to me. For public health workers we’re trying to ascertain how many of them do have devices that they could use for this kind of information…”
Implementation Climate (as part of Inner Setting)
Health departments were more likely to implement evidence-based practices in response to a particular driver for change, such as poor health outcomes, low rankings, or budget reductions. If the problem at hand was a high priority, there tended to be more of a shared perception of the need for the implementation within the organization and among stakeholders.
Available Resources to Implement (as part of Inner Setting)
Both LHD leaders and academic partners noted that evidence-based practices are more likely to be adopted and implemented if funds are available. A senior manager in City C stated, “… when there’s new money, new grant funds that’s when we really start thinking what can we best do with this money to make sure that the program that we create is evidenced-based.” Other resources that were necessary for successful identification, adoption and implementation of evidence-based practice included funds for staff training, computer technology, and time. The health commissioner in County A noted, “We try to train all of our leaders-the management team-to train their staff to look for the evidence as they […] operate and renew their programs and to work with their funding sources to try to assure that is happening.”
Implementation Process
Health departments and academic partners noted the importance of engaging individuals to implement evidence-based practices. Important team members included experts and opinion leaders within the organization, formally appointed team leaders, issue champions, and external change agents, such as community leaders. Successful implementation was typically integrated with a quality improvement approach within a culture that embraced using public health science and quality improvement to assure better outcomes. Leadership engagement, particularly identifying core funding to maintain some capacity to implement evidence-based practice, was critical to sustaining this work.
In summary, results from this study indicate that the most common elements of the constructs associated with disseminating, adopting and implementing evidence-based practices in local health departments were:
Linking the practice to a visible, high-priority community need (outer setting),
Maintaining strong relationships with professional organizations and peers (outer setting),
Requiring evidence-based practice through federal and state program mandates, private funding organizations and public health agency accreditation (outer setting),
Leveraging strong political support for application of science (outer setting),
Investing in dedicated resources and staff for research and data capacity (inner setting),
Establishing strong communication channels between researchers and practitioners and between agency staff and agency leaders (inner setting),
Identifying resources to implement evidence-based practices (inner setting construct).
Future Directions
The gap between research and practice underscores the need to understand the barriers to dissemination and uptake of evidence. Studies have reported personal and institutional barriers to utilizing evidence-based practices. Lack of time, inadequate funding, and absence of cultural and managerial support are among the most commonly cited barriers.12,13,14,15 Other studies have found a strong correlation between the perception of institutional priority for evidence-based practices and actual use of research to inform program adoption and implementation.16,14 An understanding of multiple environmental barriers to moving research to practice helps identify opportunities to improve the likelihood that evidence will be implemented. Moreover, among US public health practitioners, those who lacked skills to develop evidence-based programs were likely to have a lower level of education, suggesting that some barriers are modifiable through training.13 Workshops might be appropriate for target groups who are not aware of evidence-based practices or are not inclined to alter their practices, whereas more passive approaches (e.g., websites) may be more effective with educated audiences.
Our case studies and literature review suggest several areas for future PHSSR:
Making use of theory and frameworks. As illustrated in this article, implementation research frameworks can be used to frame case study research on local health departments. Future research is needed to better understand which theories are best suited for various settings (smaller vs. larger health departments) and levels (policy, organizational change) and how theories may need to be adapted for implementation research on health disparities. Conducting more extensive case studies or developing large-scale surveys can be informed by the identification of the seven factors associated with D & I as noted above.
Developing measures. Many of the qualitative factors identified in our case studies can also be measured quantitatively, yet many of the needed measures do not currently exist. PHSSR develop reliable and valid metrics for tracking health department implementation of evidence-based practices.
Developing “tracer conditions” which can be measured and tracked over time, and may serve as direct or indirect measures of implementation of evidence-based public health practices. For example, an evidence review of PHSSR may identify specific organizational, structural, financial, workforce and governance-related changes that a) meet the criteria for evidence (consistent with literature review standards), b) are translatable at the health department level, and c) may be measured and tracked over time.
Conducting systems research. Further research on systems factors would help identify where practitioners and researchers can make organizational changes to create cultures that facilitate D & I of evidence-based practices.
Acknowledgments
Funding: Funding provided in part by a sub-award through the University of Kentucky National Coordinating Center for Public Health Systems and Services Research, funded by the Robert Wood Johnson Foundation.
The authors would like to thank the local health departments and academic researchers who were interviewed as part of this study. We would also like to thank Matthew Schnupp, MSPH, BSN, RN for his assistance with the literature review and data collection activities.
Biographies
Dr. Dorothy Cilenti is Deputy Director of the North Carolina Institute for Public Health and Clinical Assistant Professor in the Department of Maternal and Child Health at the UNC Gillings School of Global Public Health at Chapel Hill. Previously she was the Local Health Director of Alamance and Chatham counties in North Carolina and served as the Deputy Director of the NC Division of Public Health. She received her post graduate degrees from UNC-Chapel Hill.
Dr. Ross C. Brownson is a Professor at Washington University in St. Louis and co-director of the Prevention Research Center in St. Louis. He is involved in numerous community-level studies designed to understand and reduce modifiable risk factors such as physical inactivity, obesity, and tobacco use. In particular, he is interested in the impacts of environmental and policy interventions on health behaviors and he conducts research on dissemination of evidence-based interventions.
Dr. Karl E. Umble has been with the NC Institute for Public Health for more than 10 years. His major work has involved designing and evaluating continuing professional development programs for public health professionals, including many programs related to leadership and management. Umble has also performed program evaluations as a consultant with the CDC and other organizations, and teaches leadership and program evaluation at the UNC Gillings School of Global Public Health.
Dr. Paul Erwin is Professor and Head of the Department of Public Health, University of Tennessee. Having worked as Regional Director and Health Officer for the Tennessee Department of Health for 16 years, Dr. Erwin has expertise in public health practice, community-based health assessment and strategic planning, and has a particular interest in Public Health Systems and Services Research.
Dr. Rosemary Summers currently is a private consultant. She recently retired from 13 years as a local health director in NC and previously served as an Assistant Dean at the UNC Gillings School of Global Public Health. As a founding member of the NC PBRN Dr. Summers has a passion for stronger practice and research partnerships. She received her undergraduate degree from the University of Iowa and her post graduate degrees from UNC-Chapel Hill.
Footnotes
Disclosures: No disclosures.
Contributor Information
Dorothy Cilenti, Deputy Director, NC Institute for Public Health and Clinical Assistant Professor, Department of Maternal and Child Health, UNC Gillings School of Global Public Health, Chapel Hill, NC.
Ross C. Brownson, Professor and Co-Director, Prevention Research Center in St. Louis Brown School, Division of Public Health Sciences and Siteman Cancer Center, Washington University School of Medicine, Washington University in St. Louis.
Karl Umble, Program Evaluator, NC Institute for Public Health and Adjunct Assistant Professor, Department of Health Behavior, UNC Gillings School of Global Public Health, Chapel Hill, NC.
Paul Campbell Erwin, Professor and Department Head, Department of Public Health and Director of Center for Health Policy and Services Research, University of Tennessee, Knoxville, TN.
Rosemary Summers, Independent Contractor for NC Institute for Public Health, UNC Gillings School of Global Public Health, Chapel Hill, NC and Retired Health Director, Orange County Health Department, Hillsborough, NC.
References
- 1.Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: an evolving concept. Am J Prev Med. 2004;27:417–21. doi: 10.1016/j.amepre.2004.07.019. [DOI] [PubMed] [Google Scholar]
- 2.Dreisinger M, Leet TL, Baker EA, Gillespie KN, Haas B, Brownson RC. Improving the public health workforce: evaluation of a training course to enhance evidence-based decision making. J Public Health Manag Pract. 2008 Mar-Apr;14(2):138–143. doi: 10.1097/01.PHH.0000311891.73078.50. [DOI] [PubMed] [Google Scholar]
- 3.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003 Jun 26;348(26):2635–2645. doi: 10.1056/NEJMsa022615. [DOI] [PubMed] [Google Scholar]
- 4.Colditz GA. The promise and challenges of dissemination and implementation research. In: Brownson R, Colditz G, Proctor E, editors. Dissemination and Implementation Research in Health: Translating Science to Practice. New York: Oxford University Press; 2012. [Google Scholar]
- 5.Bero L, Grillr R, Grimshaw J, Harvey E, Oxman AD, Thompson M. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998;317:465–468. doi: 10.1136/bmj.317.7156.465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sorian R, Baugh T. Power of information: closing the gap between research and policy. When it comes to conveying complex information to busy policy-makers, a picture is truly worth a thousand words. Health Aff (Millwood) 2002 Mar-Apr;21(2):264–273. doi: 10.1377/hlthaff.21.2.264. [DOI] [PubMed] [Google Scholar]
- 7.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science. 2009 Aug;4(50) doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Green LW, Ottoson JM, Garcia C, Hiatt RA. Diffusion theory, knowledge utilization, and integration in public health. Annual Review of Public Health. 2009;30:151–174. doi: 10.1146/annurev.publhealth.031308.100049. [DOI] [PubMed] [Google Scholar]
- 9.Dearing JW. Applying diffusion of innovation theory to intervention development. Research on Social Work Practice. 2009;19(5):503–518. doi: 10.1177/1049731509335569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.NACCHO National Profile of Local Health Departments. [Accessed 2/19/2012]; ( http://www.naccho.org/topics/infrastructure/profile/resources/2010report/upload/2010_Profile_main_report-web.pdf)
- 11.National Center for Health Statistics. [Accessed 2/19/2012]; ( http://www.cdc.gov/nchs/data_access/urban_rural.htm)
- 12.Baker EA, Brownson RC, Dreisinger M, McIntosh LD, Karamehic-Muratovic A. Examining the role of training in evidence-based public health: a qualitative study. Health Promot Pract. 2009 Jul;10(3):342–348. doi: 10.1177/1524839909336649. [DOI] [PubMed] [Google Scholar]
- 13.Jacobs JA, Dodson EA, Baker EA, Deshpande AD, Brownson RC. Barriers to evidence-based decision making in public health: a national survey of chronic disease practitioners. Public Health Rep. 2010 Sep-Oct;125(5):736–742. doi: 10.1177/003335491012500516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dobbins M, Cockerill R, Barnsley J, Ciliska D. Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. Int J Technol Assess Health Care. 2001 Fall;17(4):467–478. [PubMed] [Google Scholar]
- 15.Maylahn C, Bohn C, Hammer M, Waltz E. Strengthening epidemiologic competencies among local health professionals in New York: teaching evidence-based public health. Public Health Rep. 2008;123(Suppl 1):35–43. doi: 10.1177/00333549081230S110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Brownson RC, Ballew P, Dieffenderfer B, et al. Evidence-based interventions to promote physical activity: what contributes to dissemination by state health departments. Am J Prev Med. 2007 Jul;33(1 Suppl):S66–73. doi: 10.1016/j.amepre.2007.03.011. [DOI] [PubMed] [Google Scholar]
- 17.Scullion PA. Effective dissemination strategies. Nurse Res. 2002;10(1):65–77. doi: 10.7748/nr2002.10.10.1.65.c5880. [DOI] [PubMed] [Google Scholar]