Abstract
Context
The aim of evidence-based decision-making in public health involves the integration of science-based interventions with community preferences to improve population health. Although considerable literature is available on the development and adoption of evidence-based guidelines and barriers to their implementation, the evidence base specific to public health administration is less developed. This article reviews the literature from public health and related disciplines to identify administrative evidence-based practices (A-EBPs; i.e., agency-level structures and activities that are positively associated with performance measures).
Evidence acquisition
A “review of reviews” was carried out to assess the evidence for the effectiveness of A-EBPs covering the time frame January 2000 through March 2012. The following steps were used: (1) select databases; (2) determine search parameters and conduct the search; (3) screen titles and abstracts; (4) obtain selected documents; (5) perform initial synthesis; (6) abstract data; and (7) synthesize evidence.
Evidence synthesis
In both the reviews and original empiric studies, the most common outcome reported was performance of the local health department or local public health system. On the basis of a synthesis of data from 20 reviews, a total of 11 high-priority A-EBPs were identified (i.e., practices that local public health systems potentially can modify within a few years). The A-EBPs covered five major domains of workforce development, leadership, organizational climate and culture, relationships and partnerships, and financial processes.
Conclusions
As the body of practice-based research continues to grow and the ability to measure administrative evidence-based practices improves, this initial list can be further developed and improved.
Context
Evidence-based decision-making (EBDM) in public health has been defined as the integration of science-based interventions with community preferences to improve population health.1 The scientific evidence for effective population-level interventions has grown rapidly over the past few decades, as summarized in systematic reviews such as the Cochrane Collaboration2 and the Community Guide.3 In addition, there is a growing body of literature on dissemination and implementation (D&I) research,4 which seeks to understand the processes and factors associated with widespread use of an evidence-based practice or the successful integration of evidence-based interventions within a particular setting (e.g., schools, clinics).5 The need for greater emphasis on EBDM is highlighted in the Public Health Accreditation Board Standards that seek to “contribute to and apply the evidence base of public health.”6 This standard highlights the importance of using the best available evidence and also the role of health departments in adding to the body of evidence for promising approaches.
Numerous studies have examined the barriers to EBDM in state and local health departments (LHDs). The most commonly identified barriers include lack of time/competing demands, inadequate funding/high cost, the absence of organizational support, and the chasm between researchers and practitioners.7-12 In a national survey9 of public health practitioners, absence of incentives within the organization was the largest barrier to EBDM. In another study13 of EBDM in Kansas and Mississippi, participants identified communication with policymakers, use of economic evaluation, and translation of research to practice as top competency gaps limiting the movement of evidence to practice in state and LHDs.
Other research8,14 has shown a strong correlation between the perception of institutional priority for EBDM and actual use of research to inform program adoption and implementation. Another related body of inquiry has focused on the barriers to uptake of effective intervention strategies such as those identified in the Community Guide. Based on this growing body of D&I research,4,15-18 several lessons are now apparent: (1) dissemination of an evidence-based practice generally does not occur spontaneously; (2) passive approaches to dissemination largely are ineffective; and (3) single-source prevention messages generally are less effective than comprehensive approaches.
Although considerable literature exists on the development of evidence-based guidelines, barriers to their adoption, and methods for enhancing the uptake of evidence-based practices, the evidence base specific to public health administration is less developed. Public health services and systems research (PHSSR) is particularly situated to inform the development of guidance for effective public health administration, providing the evidence base for what is and should be implemented at the state and local levels. In PHSSR, investigators explore the association between the investment of resources in public health, agency and systems performance, and the impact such inputs may have on the health of communities served—how such interactions take place is tied directly to administrative practices.19 The National Public Health Performance Standards Program, Public Health Accreditation Board, and local quality-improvement and accreditation processes are drawing increasing attention to administrative practices.20-23 However, the majority of PHSSR studies to date are cross-sectional and descriptive, which often do not reach the level of evidence required for EBDM. Only recently has the body of PHSSR research begun to produce findings that can be translated to practice and policy.19,24,25
One important challenge relates to how to build capacity, allowing practitioners to identify such research and then incorporate it into their practices. A notable need exists to identify and act on administrative evidence-based practices (A-EBPs), which are agency (health department)–level structures and activities that are positively associated with performance measures (e.g., achieving core public health functions, carrying out evidence-based interventions). This article reviews the literature from public health and related disciplines to identify a set of A-EBPs that might be acted on to improve practice.
Evidence Acquisition
Combining methods from rapid review26,27 and snowball sampling,28,29 a secondary search of the literature, focusing on representative existing evidence reviews from peer-reviewed journals, was conducted to identify A-EBPs. This “review of reviews” was carried out to assess the current level of evidence for the effectiveness of A-EBPs. The review followed seven steps.
Step 1: Select Database(s) Most Likely to Yield the Desired Document Types
To begin the process, the following databases were searched: PubMed, Web of Science (Social Science Citation and all fields), Academic Search Premier, EconLit, Business Source Complete, PsycINFO, Social Work Abstracts, and ERIC. Because it is likely the largest source of articles on this topic and keyword searching can be imprecise, a manual search was conducted of the Journal of Public Health Management and Practice for January 2009 through March 2012 to capture original research studies too recent for inclusion in most reviews. Additionally, an author search was conducted in PubMed for January 2000–January 2012 publications by selected PHSSR authors (Erwin, Halverson, Handler, Mays, Scutchfield, Turnock). One indicator that a sufficient number of databases had been searched was that new searches did not identify additional articles.
Step 2: Determine Search Parameters and Conduct the Search
The evidence resources reviewed and abstracted were limited to those published between January 2000 and March 2012 plus articles accepted for publication in English-language peer-reviewed journals. Search terms included (“performance” or “health”) AND “local”; “public health performance”; “public health administration”; “public health practice”; “evidence-based”; “public health professional”; “capacity building”;“work force development”;“staff development”;“employee training”; “public health workforce”; “employees—training” and “public health”; “organization”; “partnership”; “interorganization”; “collaboration”; or “relationship.” The study team focused on identifying relevant reviews of studies that had quantitatively tested relationships of A-EBP with performance or health. The team used the Washington University library system to conduct the search.
Step 3: Screen the Titles and Abstracts to Determine Potential Relevance
One reviewer examined the databases and included all reviews plus key original quantitative empiric studies that met the inclusion criteria, and those for which the applicability of the inclusion criteria cannot be determined. The entire team was polled to find useful sources of reviews. In addition, the initial list of articles was cross-referenced with the database on PHSSR housed at the University of Kentucky.30
Step 4: Obtain Selected Documents
The team worked with the Washington University library system to obtain documents. Most documents were available online.
Step 5: Perform an Initial Synthesis to Determine Inclusion
The goal in this stage was to determine if each selected document met the inclusion criteria: was the source of an A-EBP, had relevance to local public health practice, and included an outcome linked to EBDM (e.g., increased performance of a LHD, higher trainee knowledge).
Step 6: Abstract Selected Documents and Summarize
When a set of reviews was identified, the type of review (systematic, narrative) was summarized, along with review methods, number of included studies, publication years, study populations and settings, A-EBP independent variables, dependent variables, and findings related to A-EBP (overall and by EBP domain). For original research, articles were summarized according to study year, study design, study population and setting, independent variables, dependent variables, and results. The team also noted potential survey items and additional articles mentioned by the authors. Detailed evidence tables were created to summarize the reviews (using a spreadsheet with 20 column headings) and original articles (using a spreadsheet with nine column headings). (Detailed tables are available from the first author on request.)
Step 7: Evidence Synthesis
In the final step, evidence was synthesized. Two sets of A-EBPs were created. High priority A-EBPs were those that were (1) associated with a dependent variable of interest in numerous original research articles; (2) associated with a dependent variable of interest in at least one review article; (3) focused on micro-level administrative or management changes; and (4) deemed modifiable by the research team. For high-priority A-EBPs, the study team estimated the time frame for modification of a given practice. Moderate-priority A-EBPs were those that had been associated with a dependent variable of interest in at least one original research article but have either not yet been part of a narrative or systematic review or were thought to take longer to modify.
Evidence Synthesis
Study Characteristics
After screening for relevance, articles were categorized as reviews (n=30); original empirical articles (n=65); and conceptual articles (n=49; i.e., articles that did not meet the inclusion criterion in Step 7 but were nonetheless helpful in framing the review). Most reviews were from the PHSSR, EBDM, or other public health literature, but several reviews of administrative practices came from evidence-based medicine,31-34 public administration,35,36 or the broader organizational literature.37,38 Although the present review focused on local-level organizations, a few relevant studies of state health departments also were included.
Most reviews were of studies conducted in the U.S. Several reviews focused on U.S. studies but also included relevant studies from Canada, the United Kingdom, Australia, or Europe.12,34,35,39 Three reviews32,40,41 included United Kingdom studies exclusively, and two42,43 reviewed Australian research. Of the 65 original studies, 63 originated in the U.S., whereas two were conducted in Canada and one in Australia. Most original articles were published in 2009–2012.
In both the reviews and original empiric studies, the most common outcome reported was performance of the LHD or local public health system as measured in the National Public Health Performance Standards Program or from earlier survey instruments.44 Adoption of evidence-based medicine or healthcare best practices was the next most common outcome, including best practices in mental health and drug addiction treatment services. In several recent reviews and original studies, researchers tested relationships of A-EBPs with health outcomes. Some studies focused on local collaborative service-delivery or policy advocacy efforts. A few studies tested LHD workforce capacity outcomes. Performance of recommended topic-specific practices was reported in several studies. These included meeting program or service-delivery objectives in immunization, maternal and child health, chronic disease prevention, and mental health. Organizational literature outcomes commonly involved the implementation of innovations.
Macro-Level Administrative Evidence-Based Practices
Although the focus of the present review is high-priority, locally modifiable A-EBPs, macro (system)–level elements are presented as background information (Table 1). These largely are derived from the PHSSR literature and relate to the infrastructure for local public health practice. The elements in Table 1 were associated with performance or health outcomes across multiple reviews and original studies.
Table 1.
Macro-level administrative evidence-based practices
Domain and evidence- based practices |
Description |
---|---|
Health department oversight and infrastructure | |
Jurisdiction | Population size of jurisdictions served Type of jurisdictions served (counties, cities) |
Governance and authority | Local health board presence |
Local health board with policy-making role, not just advisory role, at least in large population jurisdictions Centralization of authority at state level or shared state and local control (mixed findings) Statutory authority and responsibilities |
|
Financial | |
Allocation and expenditure of resources |
Total LHD expenditures per capita LHD expenditures per staff FTE Diversity of funding sources Per capita taxes or allocation percentage of local taxes to public health |
Workforce size and composition | |
Staff size and composition | Staffing FTEs per capita Pre-service educational background, licensing, and certification Mix of disciplines |
FTE, full-time equivalent; LHD, local health department
Among the A-EBPs listed in Table 1, the strongest evidence for predicting performance has been shown for allocation and expenditure of resources. The number of LHD staff full-time equivalents, LHD jurisdiction population size, and presence of a local health board also were tested frequently and were positively associated with performance or health. Centralization of authority within the state health department or shared state and local authority was associated with performance in some but not all studies. To affect the elements in Table 1, system changes may be needed in LHD governance; federal, state, and local funding streams; or how schools train professionals that make up the public health workforce.
High-Priority Administrative Evidence-Based Practices
The present review prioritizes A-EBPs that local public health systems potentially can modify within a few years at relatively low cost within any type of LHD governance, jurisdiction, and funding infrastructure. A total of 11 high-priority A-EBPs were identified on the basis of 20 reviews12,24,31,34,35,37,42-55 (Table 2). The A-EBPs covered five major domains of workforce development, leadership, organizational climate and culture, relationships and partnerships, and financial processes.
Table 2.
High-priority, locally modifiable, administrative evidence-based practices
Domain and evidence- based practice |
Description | Time frame for modificationa |
Supporting evidence reviews |
---|---|---|---|
Workforce development | |||
Training | In-service training in quality improvement or evidence-based decision- making Skills-based training (e.g., organization and systems change) Multidisciplinary in-service training Training aligned with essential services and usual job responsibilities |
Short | Adams (2006)37; Alsop (2009)31; Dilley (2012)45; Mays (2009)24; Orton (2011)12; Potter (2003)49; Potter (2010)50; Prentice (2007)51; Roche (2009)34 |
Access to technical assistance |
Access and use of knowledge brokersb Use of process-improvement activities (e.g., accreditation, performance assessment) Face-to-face meetings to share lessons, compare experiences, and provide updates |
Short | Adams (2006)37; Dobbins (2009)46 |
Leadership | |||
Skills and background of leaders |
Leadership skill development Leadership experience Quality of leadership Leadership influence Manager competency to manage change |
Short to medium | Bagley (2008)42; Hyde (2012)48 |
Values and expectations of leaders |
Leadership support of quality improvement, national performance standards, evidence-based decision-making, innovation, accreditation Intend to hire well-educated, experienced staff including specialists (e.g., lab scientists, epidemiologists, environmental health professionals, financial systems experts) |
Short to medium | Orton (2011)12 |
Participatory decision- making |
Management team Leaders and middle managers seek and incorporate employee input Nonhierarchical decision-making |
Medium | Erwin (2008)44 |
Organizational climate and culture | |||
Access and free flow of information |
Communication flow Tailored messaging for evidence-based decision-making 360-degree employee performance reviews geared to evidence-based practices (with extensive feedback) Ready access to high-quality information |
Short | Dilley (2012)45; Dobbins (2009)46; Waters (2003)55 |
Support of innovation and new methods |
Leadership/management and employee training in evidence-based decision-making that includes new methods Employees perceiving that management supports innovation Conscious creation of environments conducive to innovation Organizational capacity to be in both business-as-usual state and state of exploration |
Short | Adams (2006)37; Klein (2005)38; Orton (2011)12 |
Learning orientation | Shared employee perceptions Project management teams that encourage communication and collaboration Presence of multidisciplinary, diverse management teams |
Short to medium | Boyne (2003)35 |
Relationships and partnerships | |||
Interorganizational relationships |
Build and/or enhance partnerships with schools, hospitals, community organizations, social services, private businesses, universities, law enforcement Cooperative agreements with state and/or local health departments; quality improvement |
Medium | Bagley (2008)42; Provan (2007)52; Liberato (2011)43; Varda (2012)54; Roussos (2000)53; Dilley (2012)45 |
Vision and mission of partnerships |
Clear vision and aligned mission of partnerships Capacity building over time |
Medium | Roussos (2000)53 |
Financial | |||
Allocation and expenditure of resources |
Outcomes-based contracting Resources allocated for quality improvement, evidence-based decision- making, innovation, information access, training and implementation Diverse funding sources |
Medium | Dilley (2012)45; Harris (2012)47 |
Time frame definitions: short=<1 year; medium=1–3 years; long=>3 years.
A knowledge broker is defined as a master’s-trained individual available for technical assistance.
Because most workers in public health practice lack formal training in key disciplines,56,57 most workforce-related A-EBPs emphasized on-the-job training across a range of topics to improve EBDM. These include analytic decision-making and specific public health topics (e.g., preparedness, cancer control). Increasingly, these training programs are focusing on competency-based education.10,58-61
Within the leadership domain, A-EBPs included the skills and backgrounds of public health leaders, their values and expectations, and their use of participatory decision-making. Three A-EBPs were identified within the domain of organizational climate and culture: the free flow of information, support for innovation, and an orientation toward learning within the health department. A considerable number of studies focused on relationships and partnerships, resulting in two A-EBPs: the presence of inter organizational relationships and having a clear collaborative vision and mission among partnering organizations. Finally, within the financial domain, the high-priority A-EBP focused on funding allocation and fiscal policies and priorities (e.g., outcomes-based contracting, a foundation of diverse funding sources).
Moderate-Priority Administrative Evidence-Based Practices
A set of moderate-priority A-EBPs also was identified (Table 3).9,13,32,44,62-75 For these practices, the evidence base came from only a small number of studies. There were several domains for these moderate-priority A-EBPs: workforce size and composition, health department oversight and infrastructure, organization relationships, and financial characteristics.
Table 3.
Moderate-prioritya administrative evidence-based practices
Domain and evidence-based practice |
Description | Supporting evidence reviews and articles |
---|---|---|
Workforce development | ||
Staff composition | Educational level of master’s degree or higher | Jacobs (2010)9 |
Staff competencies | Ability to communicate research to policymakers Skill in economic evaluation |
Jacobs (2012)13 |
Staff incentives | Use of incentives and rewards | Jacobs (2010)9; Kennedy (2003)68 |
Health department oversight and infrastructure | ||
LHD accreditation | Identification of gaps Participation in accreditation process |
Davis (2011)64 |
Information systems | Presence of tools for evidence-based decision-making Use of tools for more-rapid access to evidence |
Drabczyk (2012)66 |
Health department characteristics | High job satisfaction and morale Certification of LHD staff Use of common language related to evidence- based decision-making Use of incentives and rewards |
Boyne (2003)35; Dodson (2010)65; Erwin (2008)44; Jacobs (2010)9; Kennedy (2003)68; Ogolla (2007)72; Stetler (2003)74; Swain (2004)75 |
Organization | ||
Organization climate | Common language and terminology | Dopson (2002)32; Merrill (2010)71 |
Relationships and partnerships | ||
Interorganizational relationships | Number and diversity of types of collaborating organizations Percentage of local public health services and activities provided by non-LHD organizations Distribution of authority and effort among collaborating organizations |
Beatty (2010)62; Chen (2010)63; Lovelace (2001)69; Mays (2010)70; Scutchfield (2004)73 |
Financial | ||
Allocation and expenditure of resources |
Program financial risk (program expenditures/ program revenues) |
Honoré (2007)67 |
Financial accountability | Financial transparency practices | Honoré (2007)67 |
These are moderate priority because they are based on original research but have not been part of a narrative or systematic review, or they would take longer to modify than the high-priority A-EBPs.
A-EBP, administrative evidence-based practice; LHD, local health department
Discussion
The need for a greater emphasis on use of EBDM to improve public health practice is well recognized by practitioners and researchers.76-80 There is now a rich knowledge base regarding what to implement (i.e., an array of effective interventions), yet an understanding of how to implement (e.g., the needed management practices in a health department) is lagging. A similar lag in addressing evidence-based management practices in healthcare delivery has been noted.81 The high-priority A-EBPs identified in this review get at the “how” issue and warrant consideration for more systematic use by health departments, funders, and applied researchers.
Across the five A-EBP domains (workforce development, leadership, organizational climate and culture, relationships and partnerships, financial processes), there are numerous opportunities to build on ongoing movements in public health. Performance and quality-improvement initiatives belong in the spectrum of organization-level strategies that have the potential to influence implementation of A-EBPs.20,21,82-84 A related set of activities relates to wide-spread efforts to promote systems change through health department accreditation (i.e., a process of credentialing to mark attainment of a set of standards, a process to measure health department performance against those standards, and recognition for those health departments who meet the standards).6,22,85,86
Administration and management capacity is one of 12 accreditation domains established by the Public Health Administration Board.22 The A-EBPs identified in the current review can be linked with these quality-improvement and accreditation processes. As these A-EBPs are addressed, it will be important to recognize the potential interaction of macro-level elements in Table 1 with the A-EBPs (e.g., lack of resources is likely to hinder the ability to conduct workforce development).
Much of the future success in attaining these A-EBPs will involve capacity building in state and local health departments, often through workforce training. An inadequate commitment to workforce training has been noted for decades.87 Much of the focus of earlier public health training has been on finding and appraising evidence,55,78,88 with less emphasis on A-EBPs. More recently, there have been calls to take a more evidence-based approach to workforce training.89,90
Crawford and colleagues89 have defined a framework for public health workforce research across six areas: definitions and standards, data, methods, evaluation, policy, and dissemination/translation. This final area crosses over with D&I science that has been expanded on by Scharff et al.,59 where 24 competencies were identified for moving research to public health practice.59 Parallel concepts for capacity building and training have been proposed in Australia91 and Canada.92
As reflected in the current A-EBPs, numerous studies7,13,65,74 also show the linkage between health department leadership and EBDM (e.g., leaders who foster a climate supportive of EBDM). There are now well-established leadership training programs to develop the culture for EBDM.93,94 It is also likely that even in the presence of committed leadership, a “critical mass” and a social network in support of EBDM are needed.95,96
An early step in documenting and applying these A-EBPs requires improvements in measurement. A public health adage is “what gets measured, gets done.”97 Progress in defining and changing A-EBPs will require the development of practical measures that are reliable and valid. For use in practice-based research and evaluation of A-EBPs, survey tools need to be user-friendly (i.e., brief, understandable to a broad audience, easy to administer, and easy to analyze). Data can be collected anew from practitioners in health departments, capturing knowledge, attitudes, and perceptions related to A-EBP.
Relevant data may come also from ongoing data collection that provides useful benchmarks on several A-EBPs (e.g., the National Profile of Local Health Departments conducted by the National Association of County & City Health Officials98). Efforts to harmonize surveys conducted by NACCHO (of LHDs); ASTHO (of state health departments); and NALBOH (of local boards of health) also should result in researchable databases linking A-EBPs across multiple domains.99 Over time, it may be useful to improve measurement of A-EPBs via ongoing efforts such as the National Public Health Performance Standards Program.100 Several analytic tools for EBDM can benefit a health department’s attempts to measure progress related to use of these A-EBPs.101,102 Within implementation science, the development of measures for organizational-level characteristics also should be useful in developing metrics for A-EBPs.95,103-105
As these A-EBPs are further elucidated and applied, it may be useful to apply several important concepts from D&I research. Perhaps most importantly, the application of A-EBPs can be informed by Diffusion of Innovations Theory106 and the RE-AIM framework.107 As an example, on the basis of diffusion theory, one would posit that A-EBPs with relative advantage (more beneficial than alternatives) and flexibility (practice is still effective after some level of modification) are more likely to be implemented.
Another core concept of diffusion theory addresses the need for change agents in an organization to champion an administrative innovation.108 A conceptual framework such as RE-AIM can encourage individuals seeking to implement A-EBPs to pay explicit attention to Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance.107,109 There are opportunities to further validate these five A-EBP domains in natural experiments that explore associations between A-EBPs, agency performance, and community health outcomes through the Practice-Based Research Networks funded by the Robert Wood Johnson Foundation.110
Several limitations of the current review should be noted. First, the study team focused on only published literature (i.e., excluded the gray or “fugitive” literature), and because much of the experience in state or local public health practice is not published in peer-reviewed journals, it is likely that the team missed some A-EBPs. Second, the present study did not conduct an assessment of the quality of the studies reviewed, as one would in a systematic review.111 Such an assessment of quality would take into account study design and study execution. The majority of the studies in this review were cross-sectional, which is a design that ranks low in quality in a systematic review.112
Third, the study team did not conduct an exhaustive search of complementary disciplines to public health. For example, in the domain of organizational climate and culture, one might find many useful studies in business, management, or organizational psychology. Fourth, only one reviewer searched and screened the literature. Each study was abstracted by a single reviewer instead of the abstracting team using a consensus process. And finally, although time frames were assigned to the high-priority A-EBPs, these time estimates are affected greatly by local contextual factors (e.g., funding, political climate). Despite the limitations, this review offers local public health systems and researchers a starting point to assess and change administrative and management practices in ways that may improve performance.
It also is worth noting that in this review, the focus was on micro-level A-EBPs (i.e., shorter-term administrative issues that are modifiable within a health department) rather than macro-level A-EBPs (i.e., longer-term policy and budgetary issues that largely are external to an agency). Yet some of these macro-level A-EBPs, including per capita spending in LHDs, presence of a governing Board of Health, and the organizational relationship between local and state health departments, appear to be highly predictive of performance outcomes.113,114 However, such A-EBPs may be modifiable or translatable only in the long term (if at all).44 An important area of research may involve how micro- and macro-level A-EBPs interact to predict performance. Ongoing studies that involve multiple practice-based research center sites (in a manner analogous to multisite clinical trials) are exploring the variability in administrative-related practices, service delivery, and performance, using a common set of metrics, which should provide direct evidence of the relationship between micro- and macro-level E-ABPs.115
The current “review of reviews” builds on ongoing attempts to foster a more evidence-based approach to public health practice,76,78,116 as well as on recent systematic reviews in selected areas of PHSSR.45,47,48,117,118 As the body of practice-based research continues to grow and the ability to measure A-EBPs is strengthened, this initial list can be built on and improved. In part, this can be accomplished by conducting similar reviews of reviews as the literature grows. To fully adopt these A-EBPs, new and different approaches are needed, including a focus on these administrative practices among public health leaders across all levels (national to local) and a recognition of the complex systems present in health departments.119,120 Although implementing these A-EBPs in an era of tight resources will be challenging, there is room for considerable optimism that health departments along with community, professional, and academic partners will be able to adopt and adapt these administrative and management practices, ultimately benefiting the health of the public.
Acknowledgments
This study was initiated by the Robert Wood Johnson Foundation’s initiative to support local evidence-based public health efforts. The authors appreciate the article abstracting and input from Lauren Carothers, MPH/MSW, during her graduate studies at Washington University in St. Louis, and the logistic support provided by Mary Adams and Linda Dix at the Prevention Research Center in St. Louis. The authors are also grateful for the support from the National Coordinating Center for Public Health Services and Systems Research at the University of Kentucky.
This study was supported in part by Robert Wood Johnson Foundation’s grant no. 69964 and by Cooperative Agreement Number U48/DP001903 from the Prevention Research Centers Program at the CDC.
A synopsized version of this paper and others in the area of PHSSR research can be found at the University of Kentucky’s Frontiers in Public Health Services and Systems Research website at uknowledge.uky.edu/frontiersinphssr/.
Footnotes
No financial disclosures were reported by the authors of this paper.
References
- 1.Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: an evolving concept. Am J Prev Med. 2004;27(5):417–21. doi: 10.1016/j.amepre.2004.07.019. [DOI] [PubMed] [Google Scholar]
- 2.The Cochrane Collaboration 2012 Jan 21; www.cochrane.org/
- 3.Zaza S, Briss PA, Harris KW, editors. The guide to community preventive services: what works to promote health? Oxford University Press; New York: 2005. [Google Scholar]
- 4.Brownson R, Colditz G, Proctor E, editors. Dissemination and implementation research in health: translating science to practice. Oxford University Press; New York: 2012. [Google Scholar]
- 5.Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008;14(2):117–23. doi: 10.1097/01.PHH.0000311888.06252.bb. [DOI] [PubMed] [Google Scholar]
- 6.Public Health Accreditation Board . Public health accreditation board standards: an overview. Public Health Accreditation Board; Alexandria VA: 2011. [Google Scholar]
- 7.Baker EA, Brownson RC, Dreisinger M, McIntosh LD, Karamehic-Muratovic A. Examining the role of training in evidence-based public health: a qualitative study. Health Promot Pract. 2009;10(3):342–8. doi: 10.1177/1524839909336649. [DOI] [PubMed] [Google Scholar]
- 8.Dobbins M, Cockerill R, Barnsley J, Ciliska D. Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. Int J Technol Assess Health Care. 2001;17(4):467–78. [PubMed] [Google Scholar]
- 9.Jacobs JA, Dodson EA, Baker EA, Deshpande AD, Brownson RC. Barriers to evidence-based decision making in public health: a national survey of chronic disease practitioners. Public Health Rep. 2010;125(5):736–42. doi: 10.1177/003335491012500516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Maylahn C, Bohn C, Hammer M, Waltz E. Strengthening epidemiologic competencies among local health professionals in New York: teaching evidence-based public health. Public Health Rep. 2008;123(S1):35–43. doi: 10.1177/00333549081230S110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health. 2007;28:413–33. doi: 10.1146/annurev.publhealth.28.021406.144145. [DOI] [PubMed] [Google Scholar]
- 12.Orton L, Lloyd-Williams F, Taylor-Robinson D, O’Flaherty M, Capewell S. The use of research evidence in public health decision making processes: systematic review. PLoS One. 2011;6(7):e21704. doi: 10.1371/journal.pone.0021704. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jacobs JA, Clayton PF, Dove C, et al. A survey tool for measuring evidence-based decision making capacity in public health agencies. BMC Health Serv Res. 2012;12:57. doi: 10.1186/1472-6963-12-57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Brownson RC, Ballew P, Dieffenderfer B, et al. Evidence-based interventions to promote physical activity: what contributes to dissemination by state health departments. Am J Prev Med. 2007;33(1S):S66–S73. doi: 10.1016/j.amepre.2007.03.011. [DOI] [PubMed] [Google Scholar]
- 15.Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA, The Cochrane Effective Practice and Organization of Care Review Group Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998;317(7156):465–8. doi: 10.1136/bmj.317.7156.465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kerner J, Rimer B, Emmons K. Introduction to the special section on dissemination: dissemination research and research dissemination: how can we close the gap? Health Psychol. 2005;24(5):443–6. doi: 10.1037/0278-6133.24.5.443. [DOI] [PubMed] [Google Scholar]
- 17.Rabin BA, Glasgow RE, Kerner JF, Klump MP, Brownson RC. Dissemination and implementation research on community-based cancer prevention: a systematic review. Am J Prev Med. 2010;38(4):443–56. doi: 10.1016/j.amepre.2009.12.035. [DOI] [PubMed] [Google Scholar]
- 18.The Lewin Group I . Factors influencing effective dissemination of prevention research findings by the Department of Health and Human Services. Final report. The Lewin Group, Inc.; Washington DC: Oct 1, 2001. [Google Scholar]
- 19.Van Wave TW, Scutchfield FD, Honore PA. Recent advances in public health systems research in the U.S. Annu Rev Public Health. 2010;31:283–95. doi: 10.1146/annurev.publhealth.012809.103550. [DOI] [PubMed] [Google Scholar]
- 20.Lenaway DC, Buchanan S, Thomas C, Astles R. Quality improvement and performance: CDC’s strategies to strengthen public health. J Public Health Manag Pract. 2010;10(1):11–3. doi: 10.1097/PHH.0b013e3181c115ee. [DOI] [PubMed] [Google Scholar]
- 21.Randolph GD, Stanley C, Rowe B, et al. Lessons learned from building a culture and infrastructure for continuous quality improvement at Cabarrus Health Alliance. J Public Health Manag Pract. 2012;18(1):55–62. doi: 10.1097/PHH.0b013e31822d2e23. [DOI] [PubMed] [Google Scholar]
- 22.Riley WJ, Bender K, Lownik E. Public health department accreditation implementation: transforming public health department performance. Am J Public Health. 102(2):237–42. doi: 10.2105/AJPH.2011.300375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Scutchfield FD, Patrick K. Public health systems research: the new kid on the block. Am J Prev Med. 2007;32(2):173–4. doi: 10.1016/j.amepre.2006.10.005. [DOI] [PubMed] [Google Scholar]
- 24.Mays GP, Smith SA, Ingram RC, Racster LJ, Lamberth CD, Lovely ES. Public health delivery systems: evidence, uncertainty, and emerging research needs. Am J Prev Med. 2009;36(3):256–65. doi: 10.1016/j.amepre.2008.11.008. [DOI] [PubMed] [Google Scholar]
- 25.Scutchfield FD, Marks JS, Perez DJ, Mays GP. Public health services and systems research. Am J Prev Med. 2007;33(2):169–71. doi: 10.1016/j.amepre.2007.03.013. [DOI] [PubMed] [Google Scholar]
- 26.Gannan R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci. 2010;5:56. doi: 10.1186/1748-5908-5-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Watt A, Cameron A, Sturm L, et al. Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care. 2008;24(2):133–9. doi: 10.1017/S0266462308080185. [DOI] [PubMed] [Google Scholar]
- 28.Doreian KL, Woodard K. Fixed list versus snowball selection of social networks. Soc Sci Res. 1992;21:216–33. [Google Scholar]
- 29.Farquharson K. A different kind of snowball: identifying key policymakers. Int J Soc Res Methodol. 2005;8(4):345–53. [Google Scholar]
- 30.Center for Public Health Systems & Services Research Resources. 2012 www.publichealthsystems.org/cphssr/MembershipResources/1411.
- 31.Allsop SJ, Stevens CF. Evidence-based practice or imperfect seduction? Developing capacity to respond effectively to drug-related problems. Drug Alcohol Rev. 2009;28(5):541–9. doi: 10.1111/j.1465-3362.2009.00110.x. [DOI] [PubMed] [Google Scholar]
- 32.Dopson S, FitzGerald L, Ferlie E, Gabbay J, Locock L. No magic targets! Changing clinical practice to become more evidence based. Health Care Manage Rev. 2002;27(3):35–47. doi: 10.1097/00004010-200207000-00005. [DOI] [PubMed] [Google Scholar]
- 33.Ferlie E, Dopson S, Fitzgerald L, Locock L. Renewing policy to support evidence-based health care. Public Adm. 2009;87(4):837–52. [Google Scholar]
- 34.Roche AM, Pidd K, Freeman T. Achieving professional practice change: from training to workforce development. Drug Alcohol Rev. 2009;28(5):550–7. doi: 10.1111/j.1465-3362.2009.00111.x. [DOI] [PubMed] [Google Scholar]
- 35.Boyne GA. Sources of public service improvement: a critical review and research agenda. J Public Adm Res Theory. 2003;13(3):367–94. [Google Scholar]
- 36.Emmons KM, Weiner B, Fernandez ME, Tu SP. Systems antecedents for dissemination and implementation: a review and analysis of measures. Health Educ Behav. 2012;39(1):87–105. doi: 10.1177/1090198111409748. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Adams R, Bessant J, Phelps R. Innovation management measurement: a review. Int J Manag Rev. 2006;8(1):21–47. [Google Scholar]
- 38.Klein KJ, Knight AP. Innovation implementation—overcoming the challenge. Curr Direct Psychol Sci. 2005;14(5):243–6. [Google Scholar]
- 39.Hayes SL, Mann MK, Morgan FM, Kitcher H, Kelly MJ, Weightman AL. Collaboration between local health and local government agencies for health improvement. Cochrane Database Syst Rev. 2011;6:CD007825. doi: 10.1002/14651858.CD007825.pub5. [DOI] [PubMed] [Google Scholar]
- 40.Evans D, Pilkington P, McEachran M. Rhetoric or reality? A systematic review of the impact of participatory approaches by UK public health units on health and social outcomes. J Public Health (Oxf) 2010;32(3):418–26. doi: 10.1093/pubmed/fdq014. [DOI] [PubMed] [Google Scholar]
- 41.Smith KE, Bambra C, Joyce KE, Perkins N, Hunter DJ, Blenkinsopp EA. Partners in health? A systematic review of the impact of organizational partnerships on public health outcomes in England between 1997 and 2008. J Public Health (Oxf) 2009;31(2):210–21. doi: 10.1093/pubmed/fdp002. [DOI] [PubMed] [Google Scholar]
- 42.Bagley P, Lin V. Public health systems research: the state of the field. Aust Health Rev. 2008;32(4):721–32. doi: 10.1071/ah080721. [DOI] [PubMed] [Google Scholar]
- 43.Liberato SC, Brimblecombe J, Ritchie J, Ferguson M, Coveney J. Measuring capacity building in communities: a review of the literature. BMC Public Health. 2011;11:850. doi: 10.1186/1471-2458-11-850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Erwin PC. The performance of local health departments: a review of the literature. J Public Health Manag Pract. 2008;14(2):E9–E18. doi: 10.1097/01.PHH.0000311903.34067.89. [DOI] [PubMed] [Google Scholar]
- 45.Dilley JA, Bekemeier B, Harris JR. Quality improvement interventions in public health systems: a systematic review. Am J Prev Med. 2012;42(5S1):S58–S71. doi: 10.1016/j.amepre.2012.01.022. [DOI] [PubMed] [Google Scholar]
- 46.Dobbins M, Robeson P, Ciliska D, et al. A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implement Sci. 2009;4:23. doi: 10.1186/1748-5908-4-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Harris JK, Beatty KE, Barbero C, et al. Methods in public health services and systems research: a systematic review. Am J Prev Med. 2012;42(5 S1):S42–S57. doi: 10.1016/j.amepre.2012.01.028. [DOI] [PubMed] [Google Scholar]
- 48.Hyde JK, Shortell SM. The structure and organization of local and state public health agencies in the U.S.: a systematic review. Am J Prev Med. 2012;42(5 S1):S29–S41. doi: 10.1016/j.amepre.2012.01.021. [DOI] [PubMed] [Google Scholar]
- 49.Potter MA, Barron G, Cioffi JP. A model for public health workforce development using the National Public Health Performance Standards Program. J Public Health Manag Prac. 2003;9(3):199–207. doi: 10.1097/00124784-200305000-00004. [DOI] [PubMed] [Google Scholar]
- 50.Potter MA, Miner KR, Barnett DJ, et al. The evidence base for effectiveness of preparedness training: a retrospective analysis. Public Health Rep. 2010;125(S5):15–23. doi: 10.1177/00333549101250S504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Prentice B, Flores G. Local health departments and the challenge of chronic disease: lessons from California. Prev Chronic Dis. 2007;4(1):A15. [PMC free article] [PubMed] [Google Scholar]
- 52.Provan KG, Fish A, Sydow J. Inter organizational networks at the network level: a review of the empirical literature on whole networks. J Manag. 2007;33(3):479–516. [Google Scholar]
- 53.Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annu Rev Public Health. 2000;21:369–402. doi: 10.1146/annurev.publhealth.21.1.369. [DOI] [PubMed] [Google Scholar]
- 54.Varda D, Shoup JA, Miller S. A systematic review of collaboration and network research in the public affairs literature: implications for public health practice and research. Am J Public Health. 2012;102(3):564–71. doi: 10.2105/AJPH.2011.300286. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Waters E, Doyle J. Evidence-based public health: Cochrane update. J Public Health Med. 2003;25(1):72–5. doi: 10.1093/pubmed/fdg015. [DOI] [PubMed] [Google Scholar]
- 56.Kennedy VC. Public health workforce employment in U.S. public and private sectors. J Public Health Manag Pract. 2009;15(3):E1–E8. doi: 10.1097/01.PHH.0000349744.11738.25. [DOI] [PubMed] [Google Scholar]
- 57.Turnock BJ. Public health: what it is and how it works. 4th ed Jones & Bartlett Publishers; Sudbury MA: 2009. [Google Scholar]
- 58.Lengerich EJ, Siedlecki JC, Brownson R, et al. Mentorship and competencies for applied chronic disease epidemiology. J Public Health Manag Pract. 2003;9(4):275–83. doi: 10.1097/00124784-200307000-00005. [DOI] [PubMed] [Google Scholar]
- 59.Scharff DP, Rabin BA, Cook RA, Wray RJ, Brownson RC. Bridging research and practice through competency-based public health education. J Public Health Manag Pract. 2008;14(2):131–7. doi: 10.1097/01.PHH.0000311890.91003.6e. [DOI] [PubMed] [Google Scholar]
- 60.Straus SE, Brouwers M, Johnson D, et al. Core competencies in the science and practice of knowledge translation: description of a Canadian strategic training initiative. Implement Sci. 2011;6:127. doi: 10.1186/1748-5908-6-127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Koo D, Miner K. Outcome-based workforce development and education in public health. Annu Rev Public Health. 2010;31:253–69. doi: 10.1146/annurev.publhealth.012809.103705. 1 p following 269. [DOI] [PubMed] [Google Scholar]
- 62.Beatty K, Harris JK, Barnes PA. The role of inter organizational partnerships in health services provision among rural, suburban, and urban local health departments. J Rural Health. 2010;26(3):248–58. doi: 10.1111/j.1748-0361.2010.00285.x. [DOI] [PubMed] [Google Scholar]
- 63.Chen B, Graddy EA. The effectiveness of nonprofit lead-organization networks for social service delivery. Nonprofit Manag Leadership. 2010;20(4):405–22. [Google Scholar]
- 64.Davis MV, Cannon MM, Stone DO, Wood BW, Reed J, Baker EL. Informing the national public health accreditation movement: lessons from North Carolina’s accredited local health departments. Am J Public Health. 2011;101(9):1543–8. doi: 10.2105/AJPH.2011.300199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Dodson EA, Baker EA, Brownson RC. Use of evidence-based interventions in state health departments: a qualitative assessment of barriers and solutions. J Public Health Manag Pract. 2010;16(6):E9–E15. doi: 10.1097/PHH.0b013e3181d1f1e2. [DOI] [PubMed] [Google Scholar]
- 66.Drabczyk A, Epstein P, Marshall M. A quality improvement initiative to enhance public health workforce capabilities. J Public Health Manag Pract. 2012;18(1):95–9. doi: 10.1097/PHH.0b013e31823bca32. [DOI] [PubMed] [Google Scholar]
- 67.Honore PA, Clarke RL, Mead DM, Menditto SM. Creating financial transparency in public health: examining best practices of system partners. J Public Health Manag Pract. 2007;13(2):121–9. doi: 10.1097/00124784-200703000-00007. [DOI] [PubMed] [Google Scholar]
- 68.Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract. 2003;9(3):183–7. doi: 10.1097/00124784-200305000-00002. [DOI] [PubMed] [Google Scholar]
- 69.Lovelace K. Multidisciplinary top management teamwork: effects on local health department performance. J Public Health Manag Pract. 2001;7(1):21–9. doi: 10.1097/00124784-200107010-00006. [DOI] [PubMed] [Google Scholar]
- 70.Mays GP, Scutchfield FD, Bhandari MW, Smith SA. Understanding the organization of public health delivery systems: an empirical typology. Milbank Q. 2010;88(1):81–111. doi: 10.1111/j.1468-0009.2010.00590.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Merrill J, Keeling JW, Carley KM. A comparative study of 11 local health department organizational networks. J Public Health Manag Pract. 2010;16(6):564–76. doi: 10.1097/PHH.0b013e3181e31cee. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Ogolla C, Cioffi JP. Concerns in workforce development: linking certification and credentialing to outcomes. Public Health Nurs. 2007;24(5):429–38. doi: 10.1111/j.1525-1446.2007.00653.x. [DOI] [PubMed] [Google Scholar]
- 73.Scutchfield FD, Knight EA, Kelly AV, Bhandari MW, Vasilescu IP. Local public health agency capacity and its relationship to public health system performance. J Public Health Manag Pract. 2004;10(3):204–15. doi: 10.1097/00124784-200405000-00004. [DOI] [PubMed] [Google Scholar]
- 74.Stetler CB. Roleof the organization in translating research in to evidence-based practice. Outcomes Manag. 2003;7(3):97–103. quiz 104-5. [PubMed] [Google Scholar]
- 75.Swain GR, Schubot DB, Thomas V, et al. Three Hundred Sixty Degree Feedback: program implementation in a local health department. J Public Health Manag Pract. 2004;10(3):266–71. doi: 10.1097/00124784-200405000-00011. [DOI] [PubMed] [Google Scholar]
- 76.Brownson RC, Baker EA, Leet TL, Gillespie KN, True WR. Evidence-based public health. 2nd ed. Oxford University Press; New York: 2011. [Google Scholar]
- 77.Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009;30:175–201. doi: 10.1146/annurev.publhealth.031308.100134. [DOI] [PubMed] [Google Scholar]
- 78.Fielding JE, Briss PA. Promoting evidence-based public health policy: can we have better evidence and more action? Health Aff (Millwood) 2006;25(4):969–78. doi: 10.1377/hlthaff.25.4.969. [DOI] [PubMed] [Google Scholar]
- 79.Glasziou P, Longbottom H. Evidence-based public health practice. Aust NZ J Public Health. 1999;23(4):436–40. doi: 10.1111/j.1467-842x.1999.tb01291.x. [DOI] [PubMed] [Google Scholar]
- 80.Jenicek M. Epidemiology, evidence-based medicine, and evidence-based public health. J Epidemiol Commun Health. 1997;7:187–97. doi: 10.2188/jea.7.187. [DOI] [PubMed] [Google Scholar]
- 81.Walshe K, Rundall TG. Evidence-based management: from theory to practice in health care. Milbank Q. 2001;79(3):429–57. doi: 10.1111/1468-0009.00214. IV-V. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Derose SF, Petitti DB. Measuring quality of care and performance from a population health care perspective. Annu Rev Public Health. 2003;24:363–84. doi: 10.1146/annurev.publhealth.24.100901.140847. [DOI] [PubMed] [Google Scholar]
- 83.Derose SF, Schuster MA, Fielding JE, Asch SM. Public health quality measurement: concepts and challenges. Annu Rev Public Health. 2002;23:1–21. doi: 10.1146/annurev.publhealth.23.092601.095644. [DOI] [PubMed] [Google Scholar]
- 84.Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the U.S.: a framework for change. Milbank Q. 2001;79(2):281–315. doi: 10.1111/1468-0009.00206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Bender K, Benjamin G, Carden J, et al. Final recommendations for a voluntary national accreditation program for state and local health departments: steering committee report. J Public Health Manag Pract. 2007;13(4):342–8. doi: 10.1097/01.PHH.0000278026.49196.40. [DOI] [PubMed] [Google Scholar]
- 86.Bender K, Benjamin G, Fallon M, Jarris PE, Libbey PM. Exploring accreditation: striving for a consensus model. J Public Health Manag Pract. 2007;13(4):334–6. doi: 10.1097/01.PHH.0000278024.03455.d5. [DOI] [PubMed] [Google Scholar]
- 87.Wiesner PJ. Four diseases of disarray in public health. Ann Epidemiol. 1993;3(2):196–8. doi: 10.1016/1047-2797(93)90137-s. [DOI] [PubMed] [Google Scholar]
- 88.Howes F, Doyle J, Jackson N, Waters E. Evidence-based public health: the importance of finding “difficult to locate” public health and health promotion intervention studies for systematic reviews. J Public Health (Oxf) 2004;26(1):101–4. doi: 10.1093/pubmed/fdh119. [DOI] [PubMed] [Google Scholar]
- 89.Crawford CA, Summerfelt WT, Roy K, Chen ZA, Meltzer DO, Thacker SB. Perspectives on public health workforce research. J Public Health Manag Pract. 2009;15(6S):S5–S15. doi: 10.1097/PHH.0b013e3181bdff7d. [DOI] [PubMed] [Google Scholar]
- 90.Thacker SB. Guide for applied public health workforce research: an evidence-based approach to workforce development. J Public Health Manag Pract. 2009;15(6S):S109–S112. doi: 10.1097/PHH.0b013e3181b1eb85. [DOI] [PubMed] [Google Scholar]
- 91.Maxwell ML, Adily A, Ward JE. Promoting evidence-based practice in population health at the local level: a case study in workforce capacity development. Aust Health Rev. 2007;31(3):422–9. doi: 10.1071/ah070422. [DOI] [PubMed] [Google Scholar]
- 92.Kiefer L, Frank J, Di Ruggiero E, et al. Fostering evidence-based decision making in Canada: examining the need for a Canadian population and public health evidence centre and research network. Can J Public Health. 2005;96(3) I1-I40 following 200. [PubMed] [Google Scholar]
- 93.Rowitz L. Management and leadership. J Public Health Manag Pract. 2010;16(2):174–6. doi: 10.1097/PHH.0b013e3181d0d132. [DOI] [PubMed] [Google Scholar]
- 94.Wright K, Rowitz L, Merkle A. A conceptual model for leadership development. J Public Health Manag Pract. 2001;7(4):60–6. doi: 10.1097/00124784-200107040-00010. [DOI] [PubMed] [Google Scholar]
- 95.Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23. doi: 10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 96.Klein K, Sorra J. The challenge of innovation implementation. The Academy of Management Review. 1996;21(4):1055–80. [Google Scholar]
- 97.Thacker SB. Public health surveillance and the prevention of injuries in sports: what gets measured gets done. J Athl Train. 2007;42(2):171–2. [PMC free article] [PubMed] [Google Scholar]
- 98.National Association of County and City Health Officials (NAC-CHO) 2010 National Profile of Local Health Departments. NACCHO; Washington DC: 2011. [Google Scholar]
- 99.Center for Public Health Systems & Services Research Data harmonization. preview.352media.com/UK/PHSSR/dev/data-harmonization.aspx.
- 100.CDC National Public Health Performance Standards Program 2012. www.cdc.gov/nphpsp/
- 101.Center for Public Health Systems & Services Research Synoptic analysis of ASTHO, NACCHO, NALBOH Surveys. www.publichealthsystems.org/cphssr/MembershipResources/1411/SynopticAnalysis.
- 102.Jacobs J, Jones E, Gabella B, Spring B, Brownson R. Tools for implementing an evidence-based approach in public health practice. Prev Chronic Dis. 2012;9:E116. doi: 10.5888/pcd9.110324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103.Glisson C, Landsverk J, Schoenwald S, et al. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health. 2008;35(1-2):98–113. doi: 10.1007/s10488-007-0148-5. [DOI] [PubMed] [Google Scholar]
- 104.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 105.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4 doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 106.Rogers EM. Diffusion of innovations. 5th ed. Free Press; New York: 2003. [Google Scholar]
- 107.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 108.Dearing JW. Evolution of diffusion and dissemination theory. J Public Health Manag Pract. 2008;14(2):99–108. doi: 10.1097/01.PHH.0000311886.98627.b7. [DOI] [PubMed] [Google Scholar]
- 109.Jilcott S, Ammerman A, Sommers J, Glasgow RE. Applying the RE-AIM framework to assess the public health impact of policy change. Ann Behav Med. 2007;34(2):105–14. doi: 10.1007/BF02872666. [DOI] [PubMed] [Google Scholar]
- 110.Center for Public Health Systems & Services Research PHSSR funding: building evidence for decision making, 2012 call for proposals. 2012 www.publichealthsystems.org/cphssr/MembershipResources/1411/PHSSR_Funding.
- 111.Mulrow C, Cook D, editors. Synthesis of best evidence for health care decisions. American College of Physicians; Philadelphia PA: 1998. Systematic reviews. [Google Scholar]
- 112.Briss PA, Zaza S, Pappaioanou M, et al. Developing an evidence-based Guide to Community Preventive Services—methods. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(1S):S35–S43. doi: 10.1016/s0749-3797(99)00119-1. [DOI] [PubMed] [Google Scholar]
- 113.Erwin P, Greene SB, Mays GP, Ricketts TC, Davis MV. The association of changes in local health department resources with changes in state-level health outcomes. Am J Public Health. 2011;101(4):609–15. doi: 10.2105/AJPH.2009.177451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 114.Mays GP, Smith SA. Evidence links increases in public health spending to declines in preventable deaths. Health Aff (Millwood) 2011;30(8):1585–93. doi: 10.1377/hlthaff.2011.0196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115.Center for Public Health Services and Systems Research Public Health Practice-Based Research Networks. 2012 www.publichealthsystems.org/pbrn.
- 116.Briss PA, Brownson RC, Fielding JE, Zaza S. Developing and using the guide to community preventive services: lessons learned about evidence-based public health. Annu Rev Public Health. 2004;25:281–302. doi: 10.1146/annurev.publhealth.25.050503.153933. [DOI] [PubMed] [Google Scholar]
- 117.Beck AJ, Boulton ML. Building an effective workforce: a systematic review of public health workforce literature. Am J Prev Med. 2012;42(5S1):S6–S16. doi: 10.1016/j.amepre.2012.01.020. [DOI] [PubMed] [Google Scholar]
- 118.Hilliard TM, Boulton ML. Public health workforce research in review: a 25-year retrospective. Am J Prev Med. 2012;42(5S1):S17–S28. doi: 10.1016/j.amepre.2012.01.031. [DOI] [PubMed] [Google Scholar]
- 119.Best A. Systems thinking and health promotion. Am J Health Promot. 2011;25(4):eix–ex. doi: 10.4278/ajhp.25.4.eix. [DOI] [PubMed] [Google Scholar]
- 120.Leischow SJ, Milstein B. Systems thinking and modeling for public health practice. Am J Public Health. 2006;96(3):403–5. doi: 10.2105/AJPH.2005.082842. [DOI] [PMC free article] [PubMed] [Google Scholar]