Importance of scientific resources among local public health practitioners

Abstract

Objectives

This study examined the perceived importance of scientific resources for decision-making, among local health department (LHD) practitioners in the U.S.

Methods

This cross-sectional study used data from LHD practitioners (n=849). Respondents ranked important decision-making resources, methods for learning about public health research, and academic journal use. Descriptive statistics were calculated and logistic regression was used to measure associations of individual and LHD characteristics with importance of scientific resources.

Results

Systematic reviews of scientific literature (24.7%) was most frequently ranked as important among scientific resources, followed by scientific reports (15.9%), general literature review articles (6.5%), and one or a few scientific studies (4.8%). Graduate-level education (aORs ranging from 1.7 to 3.5), larger LHD size (aORs ranging from 2.0 to 3.5), and leadership support (aOR = 1.6; 95% confidence interval [CI] = 1.1, 2.3) were associated with a higher ranking of importance of scientific resources.

Conclusions

Graduate training, larger LHD size, and leadership that supports a culture of evidence-based decision-making may increase the likelihood of practitioners viewing scientific resources as important. Targeting communication channels that practitioners view as important can also guide research dissemination strategies.

INTRODUCTION

The pathway through which local health department (LHD) practitioners develop evidence-based programs and policies occurs through a complex decision-making process. Often referred to as evidence-based decision-making (EBDM), this is a process which involves making decisions based on the best available scientific evidence; applying program planning and quality improvement frameworks; engaging the community in assessment and decision-making; and conducting sound evaluation.¹ In order to inform efforts aimed at boosting performance of public health agencies, there should be a clear understanding of decision-making patterns in LHDs, particularly those based on scientific information.

Documenting and implementing improvements in EBDM is increasingly important as a foundation of public health services and systems research (PHSSR)^2-4 and as part of accreditation programs such as The Public Health Accreditation Board Standards (PHAB).⁵ PHAB has directly included EBDM into its set of accreditation standards by recommending and incentivizing LHDs to “contribute to and apply the evidence base of public health”.⁵ Understanding how to enhance EBDM among public health directors, managers of programs or divisions, program coordinators, and other public health staff has become relevant for LHDs seeking or wishing to maintain accreditation status. Tools like the Community Guide-PHAB Crosswalk⁶ aim to increase the use of evidence-based approaches and support accreditation. The tool's purpose is to show how implementing the evidence-based interventions in the Guide to Community Preventive Services (Community Guide) can fulfill the various requirements of PHAB.⁷

The utilization of scientific resources (e.g., systematic reviews, scientific reports, scientific articles, etc.) and capacity building approaches may enhance EBDM in LHDs.^8-10 There is a growing abundance of accessible evidence-based resources available including the Community Guide.¹¹ However, few studies have documented the perceived importance and reported use of scientific resources that are part of the LHD decision-making process, and significant barriers exist. Enhancing access to resources, training, and leadership that fosters a supportive culture have been identified as key components to overcoming barriers that prevent use of evidence-based resources.^1,8,12 Additionally, to improve the translation of science to practice, researchers need new knowledge on the most appropriate methods for communicating their research findings.¹³

The purpose of this study is to 1) describe patterns of the perceived importance and reported use of scientific resources among LHD practitioners, and 2) examine the relationships between individual and organizational characteristics with perceived importance of scientific resources.

METHODS

Data collection

Data on the perceived importance and reported use of resources among LHD practitioners were collected from responses to a nationwide survey of LHDs. Human participant approval was obtained from the sponsoring University's institutional review board. The sampling frame, questionnaire development and testing, and data collection steps have been described previously.^16,17 Briefly, a stratified random sample of 1,067 U.S. LHDs was drawn from the database of 2,565 LHDs maintained by the National Association of County & City Health Officials (NACCHO), with stratification by size of jurisdictional population.

Data collection occurred over a 4-month time period (from October 2012 through February 2013) using an online survey (Qualtrics software¹⁴) that was delivered nationally to valid email accounts of 967 LHD leaders. There were 517 LHD leaders who responded to the survey (53.5% response rate). The conclusion of the survey prompted LHD leaders to identify program managers within their own LHD. The online survey was then sent directly to email accounts of 522 program managers who had been identified by their LHD leaders. There were 332 program managers who responded to the survey (63.6% response rate), representing 196 individual LHDs. Individual characteristics were derived from the survey responses, with additional data on LHDs drawn from linked NACCHO Profile survey data.¹⁵ The separate datasets for LHD leaders and LHD program managers were merged to form a pooled sample. Overall, the online survey was delivered to 1,489 unique email accounts and 849 LHD practitioners responded to the survey (57% response rate).

The survey instrument was based in part on a public health systems logic model and related frameworks^18-21 and previous evidence-based public health (EBPH) research with state and local health departments, where validated and standardized questions existed.^22-27 The questionnaire consisted of six sections (biographical data, administrative practices, diffusion attributes, barriers to EBPH, importance and use of resources, competencies in EBPH), with a total of 66 questions. Seven questions were related to the importance and reported use of resources. Three of these questions used a ranking method to measure the perceived importance of resources and reported use of resources (i.e., academic journals most often read). Respondents were asked 1) “When you make decisions about such things as program planning, policy development, or funding, how important to you are the following?”, 2)“What methods allow you to learn about the current findings in public health research?”, and 3) “Which journals do you most often read to stay up-to-date on current findings in public health?”. For each of these three questions, participants were given a list of pre-determined response options to choose from and rank the top three where 1 is most important or most often read (for the journal question). Using yes/no and multiple choice questions, the remaining four survey items measured reported use of the Community Guide (two questions), reported barriers to journal use, and reported facilitators of journal access.

Data analysis

For the purpose of this analysis, responses listed in any of the top three were given equal weight. Any response option given a first, second, or third ranking was coded or grouped together into a single dichotomous variable. These dichotomous variables were constructed in this manner in order to compare LHD practitioners who perceived a resource as most important or most reportedly used with those who did not perceive the resource to be important or reportedly used.

Descriptive statistics about the characteristics of the LHD practitioners and the departments where they worked (n=849) were calculated. Descriptive statistics were also calculated for all variables about the perceived importance and reported use of resources. A new dependent variable was created to conduct multivariate analysis: perceived importance of scientific resources. Perceived importance of scientific resources was coded as a single dichotomous variable and defined as perceiving any of the following as a top resource: systematic reviews of the body of scientific literature, scientific reports, general literature reviews, or one or a few scientific articles.

Using logistic regression, adjusted odds ratios (aORs) and 95% confidence intervals (CIs) were calculated. Independent variables listed in table 3 that were significant at the p<0.2 level in unadjusted analyses were retained in the final model to calculate adjusted odds ratios. The aORs represent the odds of perceiving a resource to be important.

Table 3.

Predictors of importance of scientific resources among LHD practitioners in U.S.

	Importance of scientific resources1
Characteristic	No. ranked as important2	%3	OR (95% CI)	aOR4 (95% CI)
Individual
Job position
Top executive*	183	49.9	1.0	1.0
Manager or other staff**	118	37.0	0.6 (0.4, 0.8)	0.9 (0.6, 1.2)
Administrator, deputy or assistant director	53	33.5	0.5 (0.3, 0.7)	0.9 (0.5, 1.4)
Age
20-39	50	38.8	1.0	1.0
40-49	68	36.4	0.9 (0.6, 1.4)	1.0 (0.6, 1.6)
50-59	153	43.8	1.2 (0.8, 1.9)	0.8 (0.5, 1.3)
60 or older	83	46.4	1.4 (0.9, 2.2)	1.1 (0.7, 1.7)
Highest degree
Bachelors or less	57	25.1	1.0	1.0
Nursing	54	32.0	1.4 (0.9, 2.2)	1.3 (0.8, 2.2)
Other masters degree	101	49.3	2.9 (1.9, 4.3)	2.0 (1.3, 3.1)
MPH/MSPH***	65	47.4	2.7 (1.7, 4.2)	1.7 (1.1, 2.8)
Doctoral degree	76	69.7	6.9 (4.1, 11.4)	3.5 (1.9, 6.3)
Gender
Female	211	39.7	1.0	1.0
Male	143	45.7	1.3 (0.96, 1.7)	1.0 (0.7, 1.4)
Health Department
Population of jurisdiction
<25,000	39	21.7	1.0	1.0
25,000 to 49,999	70	37.6	2.2 (1.4, 3.5)	2.0 (1.2, 3.4)
50,000 to 99,999	65	43.3	2.8 (1.7, 4.5)	2.2 (1.3, 3.7)
100,000 to 499,999	107	51.0	3.8 (2.4, 5.9)	3.0 (1.8, 5.0)
500,000 or larger	72	59.5	5.3 (3.2, 8.8)	3.5 (1.9, 6.4)
Governance structure
Locally governed	284	42.3	1.0	--
State governed	29	39.7	0.9 (0.6, 1.5)	--
Shared governance	40	39.6	0.9 (0.6, 1.4)	--
Census region
Northeast	55	44.7	1.0	1.0
Midwest	118	37.0	0.7 (0.5, 1.1)	0.8 (0.5, 1.3)
South	107	40.5	0.8 (0.5. 1.3)	0.6 (0.4, 1.04)
West	73	51.4	1.3 (0.8, 2.1)	1.1 (0.7, 2.0)
Leadership structures/practices5
Ability to lead efforts in EBDM	212	46.4	1.4 (1.1, 1.8)	1.0 (0.7, 1.5)
Encourages EBDM use	232	49.7	1.9 (1.5, 2.6)	1.6 (1.1, 2.3)
Fosters participation of staff in decision-making	261	43.3	1.1 (0.8, 1.5)	--
Important to hire people with a public health degree	145	52.2	1.8 (1.3, 2.4)	1.4 (0.9. 2.0)
Important to hire people with public health experience	186	46.9	1.4 (1.05, 1.8)	0.9 (0.6, 1.2)

^*Includes top executives, health directors, health officers, commissioners, or equivalent in “Office of the Director”

^**Includes managers of a division or program, program coordinators, technical expert positions, or other staff

^***MPH (Master of Public Health); MSPH (Master of Science in Public Health)

Notes:

¹Perceived importance of scientific resources defined as: Systematic reviews of the body of scientific literature, scientific reports, general literature reviews, or one or a few scientific articles

²Perceived importance was dichotomized based on whether or not a respondent ranked the resource in any of their top 3 (first, second, or third most important)

³Row percentages are shown

⁴Adjusted odds ratio (95% confidence interval): Variables that were significant at the p<0.2 level in unadjusted analyses were retained in the final model to calculate adjusted odds ratios. The odds ratios represent the odds of perceiving a resource to be important

⁵7-point Likert-scale response option; frequency shown is those who “strongly agree” and “agree.”

RESULTS

In the sample of 849 LHD practitioners, most were top executives (43.5%), followed by managers or other staff (37.8%) and administrators, deputies or assistant directors (18.7%) (Table 1). The most common characteristics of respondents were being female (62.9%), age 50-59 (41.4%), and having a Bachelor's degree or less (26.8%). The average length of public health experience was 18.5 years (SD = 13.0), with an average of 8.2 years (SD = 7.1) in the current position. The highest percent of the sample worked in a LHD located in the Midwest (37.6%). Most LHDs (79.4%) were locally governed, 11.9% had shared governance, and 8.6% had a state governance structure.

Table 1.

Characteristics of the sample of LHD practitioners in the U.S. (n=849)

Characteristic	N	Percent (%)
Individual
Job Position
Top executive*	367	43.5
Manager or other staff**	319	37.8
Administrator, deputy or assistant director	158	18.7
Age (yrs)
20-39	129	15.3
40-49	187	22.2
50-59	349	41.4
60 and older	179	21.2
Highest Degree
Bachelors degree or less	227	26.8
Nursing	169	20.0
Other masters degree	205	24.2
MPH/MSPH***	137	16.2
Doctoral degree	109	12.9
Gender
Female	531	62.9
Male	313	37.1
	N	Mean (Std. Dev)
Length worked in current position	843	8.2 (7.1)
Length worked in public health	844	18.5 (13.0)
	N	Percent (%)
Health Department
Population of Jurisdiction
<25,000	180	21.3
25,000 to 49,999	186	22.0
50,000 to 99,999	150	17.7
100,000 to 499,999	210	24.8
500,000 to larger	121	14.3
Governance Structure
Locally governed	672	79.4
State governed	73	8.6
Shared governance	101	11.9
Census Region
Northeast	123	14.5
Midwest	319	37.6
South	264	31.1
West	142	16.7

^*Includes top executives, health directors, health officers, commissioners, or equivalent in “Office of the Director”

^**Includes managers of a division or program, program coordinators, technical expert positions, or other staff

^***MPH (Master of Public Health); MSPH (Master of Science in Public Health)

Funding guidance from a legislative authority or federal funding source was most often reported as one of the most important resources when making decisions about program planning, policy development, or funding (54.5%) (Table 2). Half of the sample (49.8%) ranked guidance from the state health agency as one of the most important resources, followed by perspectives or priorities of agency leadership (38.9%), success stories and lessons learned from peers (38.4%), and health planning tools (37.2%). The scientific resources most frequently ranked as important were systematic reviews of scientific literature (24.7%), scientific reports (15.9%), general literature review articles (6.5%), and one or a few scientific studies (4.8%).

Table 2.

Perceived importance and reported use of resources among LHD practitioners (n=849)

Perceived importance of resources when making decisions about programs, policy, or funding	No.	%*
Funding guidance (legislative authority or federal funding source)	463	54.5
Guidance from the state health agency	423	49.8
Perspectives or priorities of agency leadership**	330	38.9
Success stories and lessons learned from peers	326	38.4
Health planning tools (e.g., MAPP or Healthy People 2020)	316	37.2
Systematic reviews of the body of scientific literature (Community Guide)	210	24.7
Scientific reports (e.g., IOM reports, Surgeon General reports)	135	15.9
General literature review articles	55	6.5
One or a few scientific studies	41	4.8
Other	37	4.4
Reports to funders	21	2.5
Perceived importance of resources when seeking to learn about current findings in public health research
Seminars or workshops (phone, webinars, or in-person)	447	52.7
Professional associations	410	48.3
Email alerts	288	33.9
Academic journals	279	32.9
Academic conferences	187	22.0
Newsletters	178	21.0
Policy briefs	143	16.8
Other conferences	138	16.3
Press releases	106	12.5
Face-to-face meetings with stakeholders	91	10.7
Targeted mailings	45	5.3
Other	34	4.0
Social Media (Facebook, Twitter)	20	2.4
Media interviews	6	0.7
CD-ROMs	5	0.6
Journals most often read to stay up-to-date on current public health findings
Morbidity and Mortality Weekly Report	194	22.9
American Journal of Public Health	179	21.1
Public Health Reports	87	10.2
Journal of Public Health Management and Practice	82	9.7
Emerging Infectious Diseases	57	6.7
New England Journal of Medicine	44	5.2
Other	43	5.1
Journal of the American Medical Association	39	4.6
American Journal of Preventive Medicine	26	3.1
Health Affairs	13	1.5
Preventing Chronic Disease	13	1.5
Annual Review of Public Health	5	0.6
Preventive Medicine	5	0.6
BMC Public Health	2	0.2
Frontiers in Public Health Services and Systems Research	1	0.1
Implementation Science	0	0.0

^*As a result of equal weighting, it is possible that the percentages within each of the three domains can total up to 300%, since each respondent was able to rank a maximum of three items for each of the three domains. For example, if every single respondent ranked the same three items within one of the three domains (i.e., complete agreement among respondents), then these three items would be 100% each; thus, totaling 300%.

^**The percentage of managers and other staff who ranked “perspectives or priorities of agency leadership” in their top 3 (44%) was slightly higher than top executives (37%) and administrators, deputy or assistant directors (35%). This does not affect the relative ranking for managers and other staff, the group for which this variable is likely to be the most meaningful.

Seminars or workshops had the highest percentage of respondents reporting it as one of the most important ways to learn about current findings in public health research (52.7%) (Table 2). Almost half (48.3%) rated professional associations as one of the most important methods, followed by email alerts (33.9%), academic journals (32.9%), and academic conferences (22%). Twenty-one percent of the sample (21.0%) perceived newsletters as an important resource, followed by policy briefs (16.8%), other conferences (16.3%), and press releases (12.5%). Social media was perceived as important among 2.4% of the sample. Among the 813 respondents who provided data about use of the Community Guide, more than one-third (36.9%) were not familiar with this resource and 21.8% did not personally use this resource in practice.

Respondents were asked to rank their top three most often read journals among a pre-determined list of 15 public health/health journals. Overall, respondents selected less than 1 journal, on average, from the list of 15 journals. Morbidity and Mortality Weekly Report (MMWR) (22.9%), American Journal of Public Health (21.1%), Public Health Reports (10.2%), and the Journal of Public Health Management and Practice (9.7%) were among the highest read journals in the overall sample (Table 2).

Respondents who ranked academic journals as a top resource to learn about public health research were asked to indicate how their agency gained access to journals. Among these respondents (n=279), 68.1% reported that an agency subscription was used, followed by access through a state health agency (22.9%) and access through academic partners (12.5%). Respondents who did not rank academic journals as a top resource to learn about public health research (n=570) were asked to indicate reasons for not utilizing journals. Among these respondents, almost half (47.4%) reported that subscriptions were too expensive, followed by lack of access to journals (32.4%). Thirty percent of respondents (30.2%) also indicated “other” barriers to journals. The “other” responses included: not enough time/competing priorities, information was not practical, and journals were used but are not a top three resource.

Several characteristics of respondents and LHDs were associated with perceived importance of scientific resources in logistic regression models. After adjustment, several variables remained significantly associated (p<.05) with perceived importance of scientific resources (Table 3). For highest degree, other master's degree (adjusted odds ratio (aOR) = 2.0; 95% CI = 1.3, 3.1), MPH/MSPH degree (aOR = 1.7; 95% CI = 1.1, 2.8), and doctoral degree (aOR = 3.5; 95% CI = 1.9, 6.3) were all associated with perceived importance of scientific resources. Population jurisdiction of 25,000 or larger (aORs ranging from 2.0 to 3.5) was also associated with perceived importance of scientific resources. Lastly, leadership that encouraged EBDM was associated with perceived importance of scientific resources (aOR = 1.6; 95% CI = 1.1, 2.3).

DISCUSSION

The findings reveal that a relatively low percent of practitioners reported that systematic reviews of scientific literature were a resource viewed as important in decision-making. Similarly, the study also found that a relatively low percent of practitioners used the Community Guide and were familiar with this resource. These findings are generally consistent with the 2013 National Profile of Local Health Departments which shows that 38% of LHDs do not use the Community Guide.²⁹ Many LHD practitioners may not be using scientific resources to address public health problems locally. However, using the best available scientific evidence is a cornerstone of EBDM and supports the likelihood of programmatic and policy effectiveness which has implications for public health outcomes.

Characteristics of LHD practitioners and the departments where they work may influence the use of scientific resources for EBDM, which is a requirement for LHDs to progress toward or maintain PHAB accreditation.²⁸ Improving the use of scientific resources also aligns with LHDs seeking to boost quality improvement efforts or those wishing to broadly improve performance. Most importantly, applying the best evidence in practice is likely to improve performance toward achieving benchmarks for community health improvement.

Funding guidance and state health departments are also important contributors to decision-making in LHDs, suggesting that the LHD workforce, although largely locally governed, is likely to be influenced by foundations and state/federal partners. Though perhaps not surprising, these results suggest that decision-making practices may be difficult to modify without support from authoritative sources. In order to improve performance in EBPH, funding streams and state health departments should provide incentives that support these practices in the LHD system.

Overall, this study suggests that there is general disagreement among LHD practitioners based on the range of resources viewed as important in both decision-making and when seeking to learn about public health research. Some of this disagreement is likely due to the diversity of the LHD workforce and structure of the LHD system. Scientific resources, in particular, are more likely to be viewed as important in decision-making among specific segments of the LHD workforce. Identifying and addressing workforce characteristics, like educational attainment, may help inform future studies examining potential strategies for improving organizational performance and EBDM in the LHD system. Moreover, practitioners in LHDs serving larger populations were more likely to view scientific resources as important, suggesting that new and creative strategies are needed to reach smaller, rural LHDs.

The implications of these findings for LHDs serving smaller populations should be understood in the context of rural public health. LHDs in these rural areas are often confronted with the challenges of limited health care access.³⁰ Consequently, more prioritization may be placed on delivery of health care services as opposed to population-level services that are the focus of resources such as the Community Guide. These rural LHDs, often led by nurses, may also lack training and education on how to develop and administer population-level services.³⁰

Certain leadership structures and practices have been previously identified as being positively associated with performance measures.³¹ These administrative evidence-based practices (A-EBPs) consist of five major domains, including leadership. The current findings suggest that leadership encouragement may shape decision-making practices, which implies that, although providing training and access to scientific resources is an important first step in the EBDM process among practitioners, having strong leadership may be necessary in order to nudge LHD practitioners to use evidence-based resources. LaPelle found that strong leadership supporting use of evidence-based resources can foster a culture change, which could be especially important for health departments in which EBDM is not a common organizational practice.⁸

Most LHD practitioners rely upon seminars, workshops, and professional associations to learn about public health research, suggesting linkages to scientific evidence do not often occur through traditional academic sources (e.g., journals, professional conferences). Additional efforts are needed to better understand how targeted strategies (e.g., knowledge brokering³²) might contribute to more effective translation of research into local public health practice. The research community should evaluate other media for communicating scientific evidence that are more widely preferred by practitioners. Active dissemination is one approach researchers may adopt that aims to spread evidence-based information and research though determined channels using planned strategies.³³ Researchers can facilitate the use of scientific resources by practicing active dissemination, which may improve EBDM among LHD practitioners. However, studies have suggested that many researchers spend little time on dissemination and lack the infrastructure necessary to support this activity.³⁴ Funding incentives (e.g., from the Centers for Disease Control and Prevention or the National Institutes of Health) may also act as catalysts for supporting dissemination efforts among researchers. To guide the process of dissemination, researchers (and the broader research community, including funders and professional associations) should tailor efforts to specific audiences³⁵ through defined communication messages and channels.^36-38 These results offer insight into the preferred channels of communication that LHD practitioners view as important to find research, informing active dissemination strategies for future evaluation.

Limitations

A few limitations are worth noting. The scope and definition of EBDM used in the analyses for this study does not fully capture the concept in its entirety. EBDM involves several key components including the integration of the best available scientific evidence in public health practice. However, it also includes the practice of community engagement, application of program planning/quality improvement, and evaluation. The data used for this study are based on self-reported questionnaire responses and are cross-sectional, limiting inferences. LHD practitioners may over- or under-estimate perceived importance and reported use of resources in self-report survey responses. It is worth noting that if a respondent did not rank a resource in any of their top three; it does not necessarily mean that the resource is not important or not used. However, the practical aim of the study was to capture the most important and relevant resources that a LHD practitioner considers in the decision-making process. Therefore, we viewed the top three as a reasonable cut-off. Finally, non-response bias is another potential limitation, given the relatively low response rate (57%) to the survey. Moreover, small LHDs and LHDs located in the northeast census region were less likely than other groups to respond; thus, additional caution may be warranted when applying study findings to these groups.

In analyzing characteristics of LHDs, there are multiple potential sources of clustering that may result in unknown quantities of dependency (e.g., multiple LHD responses, overlapping structures of programs within LHDs, state, region). These data are limited such that we were unable to address the impact of these complex structures on our results. In a separate analysis, we restricted the sample to one response per LHD, based on the initial sample of leaders (n=517), to examine results not affected by LHD-level multiple response clustering. Compared to the results with the full dataset, the point estimates and confidence intervals were highly similar.

Conclusions

This study suggests that graduate training and leadership practices may enhance the perceived importance of scientific resources among LHD practitioners. Funding guidance is also an important driver in decision-making and needs to be carefully constructed in order to reflect the principles of EBDM. Additionally, this study has implications for translating research into local public health practice. Only a few academic journals are widely read among LHD practitioners and most individuals rely heavily on seminars, workshops, and their professional associations (e.g., NACCHO) when seeking out research findings. Future research and practice should aim to design and evaluate strategies to improve access and uptake of the use of scientific resources among LHD practitioners.