Learning About and Using Research Evidence Among Public Health Practitioners

Abstract

Introduction

Funders and accreditation standards increasingly call on state and local public health agencies to use the best available science. Using research evidence is a key process in practicing evidence-based decision making (EBDM). This study explored preferences for and uses of research evidence, and examine correlates regarding frequency of use.

Methods

In 2014, eligible staff from 12 state health departments and their partnering agencies were invited to complete an online self-report questionnaire and achieved 82% response (1,237/1,509). The cross-sectional data analyzed in 2015 were baseline to a study on enhancing EBDM capacity and supports.

Results

Webinars/workshops was the most frequently selected method to learn public health findings among those in state and local health departments, whereas academic journals was the top selection by those in universities and healthcare facilities (p<0.001). Several modifiable EBDM practices were associated with more frequent use of research evidence, including direct supervisor expectations for EBDM use and performance evaluation based partially on EBDM use (AOR=2.5, 95% CI=1.9, 3.2 and AOR=2.5, 95% CI=2.1, 2.9, respectively). Increased numbers of EBDM practices were associated with higher odds of frequent research evidence use. Participant characteristics associated with higher research evidence use and adjusted for were job role, education attainment, and gender.

Conclusions

To translate research into public health practice, researchers can tailor evidence on intervention implementation and effectiveness, and disease burden, to accessible and preferred formats for public health workers and partners. Management practices to support evidence-based disease prevention can be instituted and fostered in public health and partnering agencies.

INTRODUCTION

Evidence-based decision making (EBDM) in public health integrates best available scientific findings into making decisions at all stages of public health programing (planning, implementing, evaluating, and disseminating) and can lead to more effective use of scarce funds and improved community health outcomes.^1,2 To maintain robust EBDM processes within organizations, scientific findings must be regularly consumed and applied by staff to day-to-day public health tasks. Retrieval and synthesis of evidence is a core public health competency.³

Although diversity in source and format preferences for receiving evidence has been documented,^4–6 few studies explore factors influencing frequency of research evidence use. These studies suggest that frequency of research evidence use and preferences are contextual to the individual user, including time, access, resources, organizational setting and culture, perceptions of relevance, and position or role within an agency.^7–9 Little is documented regarding research evidence use or preferences for non-governmental agency partners.²

This study examined preferences for receiving evidence, and explored correlates of research evidence use in public health programming among a sample of 12 U.S. state health department disease prevention and health promotion staff and partner health organizations to understand possible touch points for tailoring dissemination practices.

METHODS

As baseline to an ongoing study¹⁰ to support state health departments in enhancing capacity for EBDM, a staggered enrollment survey was conducted with staff in 12 randomly selected state health departments and partners between January and November 2014. State agency staff supplied the authors with health department and partner organization contacts working in disease prevention and health promotion (e.g., local coalition staff from universities, community-based organizations). Data presented here are a cross-sectional snapshot. The Washington University in St. Louis IRB granted human subject exempt approval.

Study staff sent e-mail invitations to participate in the online survey. Overall, 1,237 of 1,509 eligible health department staff and their partners (82.0%) participated. Of the 1,509, a total of 45.4% were from state health departments.

Measures

The 63-item survey was developed from the authors’ previous survey tools assessing EBDM in governmental health departments.¹¹ Most survey items have been tested for reliability.^10,12 Agreement with modifiable organizational supports for EBDM (e.g., work unit access to research)¹³ was assessed with a 7-point Likert scale. Participants indicated their top three preferred methods to learn about public health findings from a list. A 4-point Likert scale assessed frequency of research evidence use for each of six job functions such as planning and implementing programs.

Statistical Analysis

In 2015, descriptive and bivariate statistics were conducted to explore participant characteristics. For frequency of research evidence use, each Likert point was assigned a score and averaged across the six functions for each participant. Individual averages were arranged into three frequency tertiles based on distribution.

The study sought to examine the relationship between modifiable organizational practices¹³ and frequency of research evidence use. Each EBDM practice was entered separately using generalized estimating equations robust estimators to allow for state clustering and adjusted for participant characteristics associated with higher research evidence use (gender, job position, education level).

RESULTS

Nearly half (48.3%) of participants were state health department staff, with others representing a variety of organizations (Table 1). Most (80.3%) were women and 65.2% held a master’s degree or higher in any field. The majority of participants worked in chronic disease prevention (59.5%). Other non-communicable areas such as injury prevention were represented (12.9%), as well as cross-cutting positions (e.g., health equity; 25.4%). Job positions are highlighted in Table 2. Participants reported working in public health for 14.8 (SD=9.5) years.

Table 1.

Preferred Methods to Learn Research Evidence Among State Health Department Staff and Partners^a (n=1,237)

Selected method	State health departmenta (n=596; 48.3%)	Local health department (n=147; 11.9%)	Community based organization (n=225; 18.2%)	University/ academic institution (n=107; 8.7%)	Healthcare facility/ health plan (n=86; 7.0%)	Other state/local/ tribal government agencies (n=51; 4.1%)	p- value
Seminars or workshops (phone, webinars, or in-person)	59.2 (55.3–63.2)	50.4 (42.0–58.7)	45.0 (38.4–51.6)	39.6 (30.2–49.1)	50.6 (39.7–61.4)	39.6 (25.2–53.9)	0.000
Academic journals	50.3 (46.3–54.4)	34.8 (26.8–42.7)	40.1 (33.6–46.6)	68.9 (59.9–77.8)	64.7 (54.3–75.1)	41.7 (27.2–56.1)	0.000
E-mail alerts	40.4 (36.4–44.4)	46.1 (37.8–54.4)	37.4 (31.0–43.8)	29.2 (20.4–38.0)	37.6 (27.1–48.2)	31.3 (17.6–44.9)	0.100
Policy briefs	30.1 (26.4–33.9)	24.1 (17.0–31.3)	30.2 (24.1–36.3)	26.4 (17.9–34.9)	11.8 (4.8–18.8)	22.9 (10.6–35.3)	0.011
Professional associations	24.5 (21.0–28.0)	29.8 (22.1–37.4)	23.4 (17.8–29.0)	29.2 (20.4–38.0)	36.5 (26.0–46.9)	22.9 (10.6–35.3)	0.149
Newsletters	20.4 (17.1–23.7)	31.2 (23.5–38.9)	26.1 (20.3–32.0)	17.0 (9.7–24.2)	17.6 (9.4–25.9)	33.3 (19.5–47.2)	0.009
Academic conferences	17.5 (14.4–20.6)	24.1 (17.0–31.3)	15.8 (10.9–20.6)	36.8 (27.5–46.1)	27.1 (17.4–36.7)	16.7 (5.7–27.6)	0.000
Face-to-face meetings with stakeholdersb	14.7 (11.8–17.6)	14.9 (8.9–20.8)	25.2 (19.5–31.0)	15.1 (8.2–22.0)	18.8 (10.3–27.3)	27.1 (14.0–40.1)	0.006

Note: Boldface indicates statistical significance (p<0.05).

^aPercentages and 95% CIs

^bOther commonly selected options not shown are: Other conferences, press releases, targeted mailings, social media (Facebook, Twitter), reports to funders and media interviews.

Table 2.

AORs For Research Evidence Use^a Among State Health Department Staff and Partners

Independent variables	Percentage of respondents	All (n=1,237) AORb (95% CI)
Administrative practices for evidence-based decision making (EBDM) support and use (strongly agree/agree)
My direct supervisor expects me to use EBDM (% strongly agreed/agreed)	58.5%	2.5 (1.9–3.2)
Performance is partially evaluated on EBDM use	28.8%	2.5 (2.1–2.9)
Work unit has access to current research evidence	56.4%	1.8 (1.4–2.3)
Informational resources (academic journals, guidelines, toolkits) are available to my work unit	51.3%	1.5 (1.2–1.9)
Information is widely shared in work unit so that everyone who makes decisions has access	48.7%	1.5 (1.2–1.8)
Work unit engages in a diverse external network of partners that share resources for EBDM	44.9%	1.6 (1.3–2.0)
Top leadership in my agency encourages use of EBDM	53.3%	1.8 (1.4–2.2)
EBDM practices sum score, percentage strongly agreed or agreed with
All 7 EBDM practices listed above	11.8%	4.3 (3.1–5.9)
5–6 EBDM practices	24.4%	2.2 (1.8–2.7)
3–4 EBDM practices	26.3%	1.5 (1.2–1.9)
0–2 EBDM practices	37.5%	1.0 (Reference)
Participant characteristics
Job position type
Program manager or coordinator	46.7%	1.0
Agency director or bureau head	18.7%	1.3 (1.1–1.7)
Specialist (epidemiologist, statistician, evaluator)	30.1%	0.8 (0.6–0.9)
Other position	4.5	1.4 (0.7–2.8)
At least master’s in any field	65.2%	1.9 (1.6–2.3)
Female	80.3%	1.6 (1.3–2.1)

Note: Boldface indicates statistical significance (p<0.05).

^aDependent variable = Research evidence use (highest tertile vs. other)

^bSeparate Generalized Estimating Equation (GEE) estimates for each EBDM practice and EBDM practices sum score, after adjusting for participant characteristics and clustering by state, and corresponding 95% CIs.

Commonly preferred methods for receiving research evidence varied significantly by agency type (Table 1). Webinars/workshops was the most frequently selected method to learn public health findings among those in state and local health departments, whereas academic journals was the top selection by those in universities and healthcare facilities (p<0.001).

The majority of participants reported often or always using research evidence to: select policies, programs, or other types of interventions (72.1%); justify the selection of interventions to funders, agency leadership, or external partners (69.9%); evaluate interventions (62.8%); write a grant application (60.6%); or plan or conduct a needs assessment (58.4%).

Table 1 shows correlates of the highest tertile of frequency of evidence use. Having at least a master’s degree in any field, being female, and working in leadership positions were associated with frequent research evidence use. Several EBDM administrative practices were associated with higher frequency of research evidence use. Findings were similar when stratified by state population or program area or accreditation status (data not shown). A direct relationship was observed, with the likelihood of frequent research evidence use increasing as the number of reported EBDM administrative practices increased.

DISCUSSION

Preferences for how to receive research evidence varied by agency type. Several modifiable and relatively low-cost EBDM practices such as supervisory expectations of EBDM use were associated with higher frequency of research evidence use in public health job functions among participants (regardless of organization type).

Preference to receive research evidence through academic journals was higher among those with access to full-text journal articles through university library subscriptions, whereas full-text access remained a barrier for others, as found previously.^5,6,14 Seminars or workshops was a top choice among staff from governmental agencies here and in a recent national study,⁵ and may relate to lack of journal access, time constraints, or information overload.⁷ E-mail alerts, policy briefs, and professional associations are examples of active dissemination methods preferred by many participants that can be tailored to various issues and settings.

Similar to Zardo and colleagues,⁸ participants in leadership or middle management were likely to use research evidence more frequently than program managers. Likewise, specialists used research evidence less often than program managers, suggesting a need for more frequent reference to evidence in daily planning positions and those overseeing the overall management of the programming process. The study also found having a master’s or doctoral degree in any field was positively associated with frequent research evidence use, also reported elsewhere.⁸ Dissimilarly,⁸ being female was positively associated with frequent evidence use, suggesting more information may be needed to understand how gender plays a role in research evidence use.

Several perceptions of organizational support for EBDM were found to be associated with more frequent use of research evidence in program planning, implementation, evaluation, and dissemination. Moreover, findings suggest that relatively low-cost administrative practices may increase the likelihood for more frequent research evidence use by staff. Consistent familiarity with the latest evidence-based approaches, especially within leadership teams, may create an environment for communication about systematic decisions to fund (or potentially retool or defund) various programs to implement. Individuals without advanced degrees in public health, and/or local health departments lacking staff capacity, may especially benefit from an organizational norm of valuing research evidence as part of EBDM planning and evaluation processes and from on-the-job trainings, supervisory expectations to use EBDM, and ready access to evidence sources to gain more skills, familiarity, and support for regular research evidence use.¹¹

Limitations

The sample involves staff working in non-communicable disease prevention and health promotion. Future research in other public health areas such as infectious disease and environmental health is needed. National-level estimates are needed to provide additional insight into research evidence use and preferences. As these data are cross-sectional, causality cannot be inferred.

CONCLUSIONS

Ongoing use of research evidence is key in building robust EBDM processes within governmental public health and partnering agencies. More tailoring of research evidence dissemination to reach partnering organizations is needed. Additional efforts to build organizational cultures and structures that support routine research evidence use (e.g., ongoing training) are needed, especially within partner organizations/health departments lacking staff with advanced public health training.