Abstract
Objectives
The purpose of this evaluation was to assess the effect of the online evidence-based cancer control (EBCC) training on improving the self-reported evidence-based decision-making (EBDM) skills in cancer control among Nebraska public health professionals.
Study design
Cross-sectional group comparison.
Methods
Previously developed EBDM measures were administered via online surveys to 201 public health professionals at baseline (comparison group) and 123 professionals who took part in the training. Respondents rated the importance of and their skill level in 18 EBCC skills. Differences were examined using analysis of variance models adjusted for gender, age, years at agency, and years in position, and stratified by respondent educational attainment.
Results
Among professionals without an advanced degree, training participants reported higher overall skill scores (P = .016) than the baseline non-participant group, primarily driven by differences in the partnerships and collaboration and evaluation domains. No differences in importance ratings were observed. Among professionals with advanced degrees, there were no differences in skill scores and small differences in importance scores in the expected direction (P < .05). Respondents at baseline rated the following facilitators for EBDM as important: expectations from agency leaders and community partners, high priority placed on EBDM by leadership, trainings, and positive feedback. They also reported using a variety of materials for making decisions about programs and policies, though few used individual scientific studies.
Conclusions
EBCC led to improved self-reported EBDM skills among public health professionals without an advanced degree, though a gap remained between the self-reported skills and the perceived importance of the skills. Further research on training content and modalities for professionals with higher educational attainment and baseline skill scores is needed.
Keywords: Cancer control, Evidence-based public health, Public health workforce, Online training
Introduction
Cancer and other chronic diseases account for the majority of deaths in the United States and abroad, and most share modifiable behavioral causes that include diet, physical activity, tobacco use, and alcohol consumption.1,2 Several evidence-based interventions have been identified for chronic disease prevention and control, and calls have been made to translate this knowledge into programs and policies.2,3 The process of doing so has been described in the evidence-based public health (EBPH) framework for public health practice, which defines evidence-based decision-making (EBDM) as the process of using the best available research evidence together with information regarding population priorities and available resources in choosing and prioritizing public health programs and policies.4 EBPH shares many similarities with the well-established concept of evidence-based medicine, however several differences between the two fields exist and necessitate separate approaches: type and volume of evidence, complexity of interventions, and heterogeneity of practitioners' training and disciplines.4
Despite the agreement about the importance of EBDM in cancer and chronic disease control, programs and policies are often not selected based on best existing evidence.4 Multiple barriers to utilizing EBDM in public health exist, and it is important to increase the skills of public health professionals in order to improve EBDM.4,5 Partially due to diversity of disciplines from which public health professionals originate, many are insufficiently trained in the science and practice of public health.6 Between one-fifth and one-third of the public health practitioners report having undergone formal public health training.7 Cancer control and other public health professionals report limitations in their organizations' capacity for using evidence-based practices even though they perceive that organizational leadership expects them to use these practices, and they have identified training as an important incentive for practicing EBDM.7–10 Cancer control practitioners from public health and other settings have previously identified several training needs around implementation of evidence-based cancer control (EBCC) programs.9
Several EBPH training programs for cancer control practitioners and other public health professionals exist,11–14 though only some of these have been evaluated and found effective in improving the knowledge, skills, or practice of the participants.12,13,15,16 Some of these training programs utilized technology,11,14 but their effectiveness was not evaluated. Distance learning has been emphasized as a promising training modality in Institute of Medicine's call for improving the capacity of the public health workforce.17 Although there is some preference among public health professionals for in-person training as it provides opportunity for interaction,18 online training and other distance learning technologies have the potential to reach public health professionals for whom in-person training is not feasible4 or to expand the scale of EBPH training where large groups of public health agencies could be combined into the same training system.18
The Prevention Research Center in St. Louis (PRC-StL) developed the online EBCC training aimed at increasing the adoption of evidence-based interventions to control obesity and cancer through promotion of physical activity and healthy eating. To the authors' knowledge, this online training is unique among cancer training programs in addressing a comprehensive set of competencies (deemed ‘skills’ hereafter). The online program was built around many of the core elements of an established EBPH training program15 and adapted with examples for obesity and cancer control. The online format of the training has the potential to reach a large number of public health professionals, and allows trainees to complete the training on their own time. Brownson et al. (2009)19 identified a set of skills for use in practitioner-focused training in EBCC, which were broken down by level of practitioner expertise and rated on priority. These skills serve as the foundation of the EBCC training. The EBCC training was completed by Nebraska public health professionals working in chronic disease prevention and control from February 2012 to June 2013. The purpose of this project was to evaluate the training for whether it increased the self-reported EBDM skills among the participants.
Methods
Sample
The baseline survey, which served as a comparison group, was distributed to the Nebraska Public Health Association Network (PHAN) membership between August and September 2011 by email and followed up with additional emails and phone calls. Of the 247 respondents at baseline, 46 were missing information regarding EBDM skills and were excluded. Analysis was carried out on 201 observations at baseline.
Following the baseline survey, EBCC training participants were recruited in partnership with the Nebraska PHAN, Nebraska Partners N Health, and Nebraska Cancer Coalition between February 2012 and June 2013. Training participants included Nebraska state and local health departments public health practitioners and their partners. The training was introduced at conference presentations, via meetings, personal phone calls, newsletters and through tailored emails. During this period, some Nebraska public health professionals also attended the PRC-StL's in-person EBPH training13,20 in August 2012, but they were excluded from the follow-up sample.
The follow-up survey was distributed by email to users of the online EBCC training. Of the 138 EBCC participants at follow-up, 15 were missing information regarding EBDM skills, and therefore 123 were included in the analysis.
The response rate for the baseline survey was 73%. The response rate for the follow-up survey could not be estimated precisely due to some loss of tracking data but was similarly higher than 70%.
This project was approved by the Human Research Protection Office of Washington University in St. Louis and all respondents gave informed consent to participate.
Online evidence-based cancer control training
The online EBCC training is based on 26 skills for training practitioners in EBCC.19 An initial set of EBDM skills was compiled based on existing competencies, trainings, and literature, and these were iteratively reviewed to remove redundancy, improve comprehensiveness, and tailor them to cancer control. These were prioritized and rated for expertise level in a card-sorting exercise among practitioners and trainers in cancer control. This development process identified a manageable set of cancer control skills19 which provided a foundation on which to build the practitioner-focused training program.
Two literature reviews were conducted to inform the development of the EBCC training. A curriculum review was conducted to determine which of the 26 skills were already covered in existing evidence-based courses in order to utilize high-quality material that had already been developed and to identify gaps. The curriculum review revealed that certain skills, especially in leadership, partnerships, policy-making, and grant-writing were not covered in existing training programs. In addition, a systematic review was conducted to asses the benefits and barriers of online training to individuals and organizations across five disciplines, as little evidence of online training exists in public health.21 The key findings and recommendations were used to guide the development of the EBCC training. These included the need to conduct formative research and evaluation, and to provide clear design, layout, concise content, interactivity, technical support, marketing and promotion, and incentives.21
The previously developed EBDM skills for cancer control19 were transformed into an innovative curriculum using adult learning principles and scenario-based learning around obesity and cancer. The EBCC training consists of six modules: introduction, policy, partnerships and collaboration, leadership, evaluation, and action planning. Each module contains a set of objectives and a Nebraska-specific example for cancer prevention (e.g. developing walking trails in Lincoln, Nebraska) that more fully engage participants with a scenario-based presentation. Following each module, learning is maximized by a set of questions that directs trainees into different teaching options based on whether they answered questions correctly or not. The modules enable practitioners to practice and strengthen the critical thinking skills necessary for EBDM. Instructional emphasis is placed on participants' understanding, recognition, and articulation of information associated with the skills while practicing simulated interactions that can be translated and applied to real-world situations. Participants received certificates after each module was completed and also had an option to notify their supervisor when they completed modules. To further incentivize EBCC participation, participants were entered into a drawing to win a $25 Amazon gift card after completing all modules.
To allow tailoring of the online training to the needs of Nebraska public health practitioners, rapid prototype testing of the training was conducted with guidance from members of the project's Nebraska Evidence-Based Advisory Committee which consisted of 10 practice partners who were selected based on their knowledge of cancer control, experience in public health practice, and expertise in training. Key aspects that were tested included ease of navigation, use of various media styles vs moving photo-realistic images, pace or style of activities, and use of a pause button that allowed participants to stop the training and resume when logging back on later.
Data collection
Anonymous online surveys were used at both data collection points. Previously developed measures were adapted in this evaluation to measure the EBDM skills of respondents.8 For each EBCC skill, respondents were asked to rate its importance and what their current skill level was using an 11-point Likert scale (from very unimportant or unskilled to very important or highly skilled). Table 3 shows the list of EBCC training skills and their domains.
Table 3.
Comparison of baseline and EBCC participant groups in the self-rated importance and skill level for evidence-based decision-making in cancer control, stratified by whether respondent holds an advanced degree.a
Domain | Skill | No advanced degree (n = 149) | Advanced degree (n = 122) | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||
Baseline | EBCC Training | EBCC vs baseline | Baseline | EBCC training | EBCC vs baseline | |||||||||
|
|
|
|
|
|
|||||||||
Adj. mean | SE | Adj. mean | SE | Difference | P-value | Adj. mean | SE | Adj. mean | SE | Difference | P-value | |||
Community-level planning | Community input | Importance | 9.84 | 0.14 | 9.98 | 0.16 | 0.14 | 0.507 | 10.20 | 0.10 | 10.32 | 0.17 | 0.11 | 0.566 |
Skill | 7.36 | 0.22 | 7.94 | 0.26 | 0.58 | 0.100 | 8.51 | 0.19 | 8.47 | 0.30 | −0.05 | 0.897 | ||
Community assessment | Importance | 9.94 | 0.14 | 10.06 | 0.16 | 0.12 | 0.563 | 10.18 | 0.09 | 10.57 | 0.15 | 0.39 | 0.033 | |
Skill | 7.16 | 0.23 | 7.99 | 0.27 | 0.83 | 0.022 | 8.18 | 0.19 | 8.44 | 0.31 | 0.25 | 0.495 | ||
Policy | Creating policy briefs | Importance | 9.46 | 0.17 | 9.86 | 0.19 | 0.40 | 0.125 | 9.64 | 0.13 | 10.12 | 0.21 | 0.48 | 0.063 |
Skill | 5.56 | 0.30 | 6.68 | 0.34 | 1.12 | 0.016 | 6.64 | 0.27 | 6.58 | 0.43 | −0.06 | 0.914 | ||
Transmitting evidence-based public health to policy makers | Importance | 9.93 | 0.13 | 10.09 | 0.15 | 0.17 | 0.419 | 10.20 | 0.11 | 10.32 | 0.18 | 0.13 | 0.552 | |
Skill | 5.75 | 0.29 | 6.57 | 0.34 | 0.82 | 0.072 | 6.99 | 0.25 | 6.75 | 0.39 | −0.24 | 0.608 | ||
Partnerships and collaboration | Partnerships at multiple levels | Importance | 10.33 | 0.12 | 10.18 | 0.14 | −0.15 | 0.409 | 10.41 | 0.09 | 10.63 | 0.13 | 0.22 | 0.178 |
Skill | 7.59 | 0.25 | 8.28 | 0.29 | 0.69 | 0.081 | 8.85 | 0.19 | 8.74 | 0.29 | −0.11 | 0.751 | ||
Collaborative partnerships | Importance | 10.25 | 0.11 | 10.16 | 0.13 | −0.09 | 0.600 | 10.14 | 0.11 | 10.53 | 0.18 | 0.38 | 0.072 | |
Skill | 7.02 | 0.26 | 7.90 | 0.30 | 0.87 | 0.032 | 8.54 | 0.20 | 8.65 | 0.32 | 0.11 | 0.771 | ||
Non-traditional partnerships | Importance | 9.83 | 0.15 | 9.88 | 0.18 | 0.04 | 0.853 | 9.94 | 0.12 | 10.30 | 0.19 | 0.37 | 0.114 | |
Skill | 6.56 | 0.28 | 7.57 | 0.32 | 1.01 | 0.021 | 7.93 | 0.23 | 8.13 | 0.37 | 0.20 | 0.651 | ||
Leadership | Leadership and evidence | Importance | 10.20 | 0.11 | 10.24 | 0.12 | 0.04 | 0.803 | 10.21 | 0.10 | 10.69 | 0.16 | 0.48 | 0.012 |
Skill | 7.27 | 0.24 | 7.99 | 0.27 | 0.72 | 0.050 | 8.57 | 0.20 | 8.04 | 0.31 | −0.53 | 0.155 | ||
Leadership at all levels | Importance | 10.23 | 0.12 | 10.19 | 0.13 | −0.04 | 0.842 | 10.24 | 0.11 | 10.47 | 0.17 | 0.24 | 0.240 | |
Skill | 7.35 | 0.25 | 8.01 | 0.29 | 0.66 | 0.091 | 8.68 | 0.19 | 8.22 | 0.30 | −0.46 | 0.200 | ||
Leadership and change | Importance | 10.07 | 0.12 | 10.25 | 0.13 | 0.19 | 0.299 | 10.17 | 0.11 | 10.54 | 0.17 | 0.38 | 0.064 | |
Skill | 7.41 | 0.24 | 8.08 | 0.28 | 0.66 | 0.075 | 8.66 | 0.20 | 7.99 | 0.31 | −0.68 | 0.075 | ||
Evaluation | Evaluation in 'plain English' | Importance | 10.29 | 0.11 | 10.47 | 0.13 | 0.18 | 0.302 | 10.38 | 0.08 | 10.73 | 0.13 | 0.35 | 0.027 |
Skill | 7.15 | 0.28 | 7.94 | 0.32 | 0.78 | 0.074 | 8.53 | 0.21 | 8.96 | 0.31 | 0.43 | 0.264 | ||
Qualitative evaluation | Importance | 9.78 | 0.15 | 9.72 | 0.17 | −0.06 | 0.797 | 10.05 | 0.12 | 10.35 | 0.18 | 0.30 | 0.183 | |
Skill | 5.56 | 0.29 | 6.63 | 0.33 | 1.07 | 0.017 | 7.76 | 0.22 | 8.00 | 0.34 | 0.25 | 0.555 | ||
Quantitative evaluation | Importance | 9.79 | 0.15 | 9.73 | 0.17 | −0.07 | 0.766 | 10.03 | 0.11 | 10.50 | 0.16 | 0.47 | 0.019 | |
Skill | 5.51 | 0.29 | 6.76 | 0.33 | 1.25 | 0.005 | 7.69 | 0.24 | 7.94 | 0.36 | 0.25 | 0.576 | ||
Evaluation designs | Importance | 9.57 | 0.17 | 9.45 | 0.19 | −0.12 | 0.628 | 9.69 | 0.14 | 10.10 | 0.21 | 0.42 | 0.110 | |
Skill | 5.09 | 0.30 | 6.22 | 0.34 | 1.13 | 0.014 | 7.13 | 0.27 | 6.84 | 0.42 | −0.29 | 0.569 | ||
Theory and analytic tools | Role of behavioral science theory | Importance | 9.55 | 0.18 | 9.34 | 0.21 | −0.22 | 0.439 | 9.88 | 0.14 | 9.98 | 0.22 | 0.10 | 0.710 |
Skill | 5.84 | 0.29 | 6.37 | 0.33 | 0.53 | 0.239 | 7.83 | 0.24 | 7.99 | 0.36 | 0.16 | 0.711 | ||
Evidence-based process | Developing an action plan for program or policy | Importance | 10.13 | 0.14 | 10.02 | 0.15 | −0.11 | 0.584 | 10.21 | 0.11 | 10.40 | 0.17 | 0.19 | 0.357 |
Skill | 6.88 | 0.27 | 7.81 | 0.30 | 0.93 | 0.024 | 8.50 | 0.20 | 8.39 | 0.31 | −0.12 | 0.749 | ||
Translating evidence-based interventions | Importance | 10.10 | 0.14 | 10.01 | 0.16 | −0.08 | 0.693 | 10.29 | 0.12 | 10.38 | 0.19 | 0.09 | 0.684 | |
Skill | 6.73 | 0.27 | 7.37 | 0.30 | 0.64 | 0.123 | 8.23 | 0.20 | 8.14 | 0.30 | −0.09 | 0.804 | ||
Developing a concise statement of the issue | Importance | 9.99 | 0.13 | 10.14 | 0.15 | 0.15 | 0.451 | 9.93 | 0.12 | 10.70 | 0.19 | 0.77 | 0.001 | |
Skill | 6.95 | 0.28 | 7.53 | 0.31 | 0.58 | 0.171 | 8.02 | 0.24 | 8.46 | 0.36 | 0.44 | 0.323 | ||
Overall score | Importance | 9.92 | 0.10 | 9.98 | 0.12 | 0.05 | 0.736 | 10.11 | 0.08 | 10.41 | 0.13 | 0.30 | 0.059 | |
Skill | 6.61 | 0.21 | 7.40 | 0.25 | 0.79 | 0.016 | 8.06 | 0.16 | 8.03 | 0.26 | −0.03 | 0.927 |
EBCC, evidence-based cancer control.
The models were adjusted for gender, age, years at agency, and years in position. Cases with missing education or covariate values were excluded.
In addition, the survey questions included several respondent characteristics: gender, age, organization type, years at organization, position, years in position, years in public health, and education level.
At baseline, respondents were also asked about the following: who expects them to use EBDM related to chronic disease program planning, which incentives to utilize EBDM are most encouraging, their decision-making power regarding chronic disease programs or policies, and what information and materials respondents use when making those decisions.
Data analysis
Descriptive characteristics were compared between the baseline group and those who participated in EBCC using Pearson's Chi-squared test for categorical variables and Wilcoxon Rank Sum test for continuous variables. To examine differences in the reported importance of individual skills and skill level following the EBCC training, analysis of variance models were used which included the following independent variables: group (baseline, EBCC participants), gender, age, years at agency, and years in position. Covariates were selected based on improvements in model fit, examined using the Bayesian information criterion, and association with the dependent variable. Since an interaction between having an advanced degree (master's or higher degree) and the group variable was significant for more than half of the outcome variables, the above analyses were stratified by the advanced degree variable.
Analyses were conducted using the Statistical Analysis Systems (SAS) statistical software package version 9.4 (SAS Institute, Care, NC, USA).
Results
The baseline group was similar to the EBCC group on most characteristics except for educational attainment where a significant difference was detected between the groups (Table 1). The majority of respondents were women (83.6% in baseline group, 90.5% in EBCC group). With the exception of the lowest category, the respondents were equally distributed across age categories. Most respondents came from a city or county health department. On average, respondents had worked in public health for 13 (baseline group) to 14 (EBCC group) years, spent 10 years at their organization, and been in their position for 6 (baseline group) to 7 (EBCC group) years. The respondents represented various positions in their organizations, though most were a program manager, administrator, or coordinator (24.9% in baseline group, 22.4% in EBCC group), held multiple positions (22.4% in baseline group, 19.0% in EBCC group), or held another position (18.9% in baseline group, 24.1% in EBCC group).
Table 1.
Characteristics of the public health professionals participating in the evaluation study at baseline and follow-up.
Characteristic | Baseline | EBCC participants | P-value | ||
---|---|---|---|---|---|
| |||||
N = 201 | N = 123 | ||||
|
|
||||
n | % | n | % | ||
Female | 168 | 83.6 | 105 | 90.5 | 0.085 |
Age | 0.093 | ||||
20–29 | 8 | 4.0 | 13 | 11.2 | |
30–39 | 46 | 22.9 | 20 | 17.2 | |
40–49 | 46 | 22.9 | 22 | 19.0 | |
50–59 | 51 | 25.4 | 34 | 29.3 | |
60 and over | 50 | 24.9 | 27 | 23.3 | |
Years in public health, mean (SD) | 13.30 (10.05) | 14.22 (11.58) | 0.819 | ||
Organization type | 0.467 | ||||
State health department | 32 | 15.9 | 25 | 21.6 | |
City or county health department | 93 | 46.3 | 49 | 42.2 | |
University | 36 | 17.9 | 16 | 13.8 | |
Other | 40 | 19.9 | 26 | 22.4 | |
Years at organization, mean (SD) | 9.93 (10.46) | 9.75 (9.94) | 0.969 | ||
Position | 0.053 | ||||
Program manager/administrator/coordinator | 50 | 24.9 | 26 | 22.4 | |
Health educator | 17 | 8.5 | 22 | 19.0 | |
Epidemiologist | 6 | 3.0 | 3 | 2.6 | |
Leadership | 12 | 6.0 | 2 | 1.7 | |
Community worker | 19 | 9.5 | 10 | 8.6 | |
Academic educator | 14 | 7.0 | 3 | 2.6 | |
Multiple positions selected | 45 | 22.4 | 22 | 19.0 | |
Other | 38 | 18.9 | 28 | 24.1 | |
Years in position, mean (SD) | 5.94 (5.54) | 7.16 (7.78) | 0.262 | ||
Education | 0.008 | ||||
RNRD | 18 | 9.0 | 6 | 5.4 | |
BSBA | 69 | 34.5 | 51 | 45.9 | |
Masters | 68 | 34.0 | 35 | 31.5 | |
Doctoral | 31 | 15.5 | 5 | 4.5 | |
Other | 14 | 7.0 | 14 | 12.6 |
EBCC, evidence-based cancer control.
Table 2 summarizes the facilitators to the use of EBDM in the baseline group. Respondents reported most frequently that health department leaders (74.7%) followed by community partners (62.4%) expect them to use EBDM related to chronic disease programming. The majority reported that the most encouraging incentives to use EBDM are trainings (64.1%), positive feedback or encouragement (60.0%), and EBDM being given a high priority by organization leaders (55.3%). Most respondents reported making (23.9%) or influencing (56.0%) decisions about chronic disease programs or policies, and among these the three most frequently used materials were health planning tools (e.g. MAPP or Healthy People 2010), funding guidance, and success stories and lessons learned from peers. Only 17.2% of respondents reported using individual scientific studies to make decisions.
Table 2.
Facilitators to the use of EBDM in the work of the public health professionals at baseline (N = 201).
n | % | |
---|---|---|
Person expecting practitioner to use EBDM related to cancer and chronic disease program planning1 | ||
Health department leaders | 127 | 74.7 |
Community partners | 106 | 62.4 |
Direct supervisor | 82 | 48.2 |
Co-workers | 70 | 41.2 |
Others | 32 | 18.8 |
Most encouraging incentive(s) to utilize EBDM1 | ||
Trainings | 109 | 64.1 |
Positive feedback or encouragement | 102 | 60.0 |
EBDM is given a high priority by leaders of my organization | 94 | 55.3 |
A performance evaluation that considers the use of EBDM | 63 | 37.1 |
Professional recognition | 63 | 37.1 |
Degree of decision-making about chronic disease programs or policies in respondent's job | ||
Influences decisions | 89 | 56.0 |
Makes decisions | 38 | 23.9 |
Has no influence over decisions | 32 | 20.1 |
Materials used for decision-making about chronic disease programs or policies1 | ||
Health planning tools (e.g. MAPP or Healthy People 2010) | 91 | 71.1 |
Funding guidance | 88 | 68.8 |
Success stories and lessons learned from peers | 84 | 65.6 |
Systematic reviews of the body of scientific literature | 74 | 57.8 |
Perspectives or priorities of organization leadership | 68 | 53.1 |
Reports (e.g. Institute of Medicine reports, Surgeon General reports) | 66 | 51.6 |
One or a few scientific studies | 22 | 17.2 |
EBCC, evidence-based cancer control; EBDM, evidence-based decision-making.
Multiple answer choices are possible.
Table 3 displays the education-stratified analyses of differences between the baseline and EBCC participant groups in how they rated the importance of and their skill in cancer control EBDM. Overall, respondents rated the importance of EBDM skills highly, on average near or above 10 on an 11-point scale. Respondents with an advanced degree had higher average skill scores than those without an advanced degree in both baseline and EBCC groups.
Among respondents without an advanced degree, the overall skill score and several individual skill scores were significantly higher in the EBCC participant group than the baseline group, though no differences in the importance scores were observed. The EBCC participants reported higher skill scores for two of the three skills in the partnerships and collaboration domain (collaborative and non-traditional partnerships); three of the four skills in the evaluation domain (qualitative and quantitative evaluation, and evaluation designs); and the community assessment, creating policy briefs, and developing an action plan for program or policy individual skills.
Among respondents with an advanced degree, no differences were observed in the overall importance or overall skill scores. Several individual importance scores were significantly higher in the EBCC participant group than the baseline group, though no differences in the individual skill scores were observed. The EBCC participants reported higher importance scores for the community assessment, leadership and evidence, evaluation in ‘plain English’, and quantitative evaluation individual skills.
Discussion
We evaluated the differences in self-reported EBDM skills among Nebraska public health professionals following the online EBCC training. Among those without an advanced degree, the training participants reported higher EBDM skills than the baseline group, particularly in the partnerships and collaboration and evaluation domains, though there were no differences in the already high perceived importance of EBDM skills. However, a gap remained after the training between the self-rated overall skills score (7.40) and the overall perceived importance of the skills (9.98). Among those with an advanced degree, no differences in EBDM skill and little differences in EBDM importance scores were observed. The respondents at baseline, over three-quarters of whom reported making or influencing decision-making about programs and policies, also rated several facilitators to the use of EBDM in their work as important. These include expectations from agency leaders and community partners, high priority placed on EBDM by leadership, trainings, and positive feedback regarding EBDM. Respondents at baseline used a variety of materials for decision-making about chronic disease programs or policies, though only a few used individual scientific studies.
Several previous evaluations of trainings in EBPH have shown effectiveness12,13,15,16 but unlike EBCC, these trainings relied on in-person delivery. An evaluation of a train-the-trainer approach for scaling up StL-PRC's EBPH course13 which was delivered primarily in-person and is very similar to the EBCC training, found overall improvement in EBDM skill availability in participating agencies and, similar to this evaluation's results, specifically improved the availability of evaluation designs, action planning, and communicating research to policy makers skills. In addition, findings regarding facilitators for the use of EBDM echo existing research, where health department staff report expectations and support from direct supervisors and agency leadership, and access to seminars and materials as important.22,23
EBDM training has been identified in this and other studies7,8 as an important facilitator to the practice of EBPH. Specifically, the improvement in participants' evaluation skills following the EBCC training fills a gap previously identified by public health professionals7,9 and emphasized in the Centers for Disease Control and Prevention guidance to public health agencies.24 In addition, public health agencies are increasingly expected to incorporate collaboration between diverse groups of stakeholders using coalitions and other partnerships in addressing cancer and other chronic disease.24,25 Training practitioners in how to put in place and sustain partnerships with researchers and with organizations traditionally considered outside of public health (e.g. transportation, planning) are key components in meeting these expectations.
It is encouraging that this project's findings among respondents without an advanced degree were comparable to evaluations of in-person training in EBPH. Online trainings can serve as more cost-effective alternatives or complements to in-person training and provide the opportunity to scale-up existing training across large geographic areas or agency systems.18 However, respondents who already had an advanced degree largely did not benefit from the EBCC training, potentially because higher skill scores were observed in this group at baseline, suggesting that a plateau exists for the EBCC training's impact. Given the differential in skills between those with and without formal education in public health, the online training could be used as a cost-effective option to bridge the skills gap between these two groups.
This is the first evaluation to the authors' knowledge that assesses the effect of a fully online training on public health professionals' EBDM skills. However, a few limitations must be noted. The outcomes in this project were measured using self-reported questionnaire responses, which may have introduced respondent bias into the evaluation where respondents may have particularly provided socially desirable answers following the EBCC training. In addition, the lack of a post-test comparison group in this project (comparison to baseline group was utilized) means that conclusions regarding causality in this evaluation cannot be made. Finally, it is possible that practitioners with higher interest and initial skill in EBDM were more likely to participate in the training, which could mean that some of the observed differences between groups could be due to selection bias.
Conclusions
This evaluation shows that the EBCC training led to improved self-reported EBDM skills related to cancer control among Nebraska public health professionals without an advanced degree, though the training did not improve the self-reported skills of respondents with an advanced degree. Online EBCC trainings are a potentially cost-effective and scalable way to improve public health professionals' capacity to translate knowledge regarding what is effective in cancer and chronic disease prevention and control into practice. Further research on advanced training content and modalities for professionals with higher baseline EBDM skill scores is needed to address the existing gap between perceived importance and skill level of practitioners in this group.
Acknowledgments
The authors are grateful for the valuable contributions made by their partners at the Nebraska State and Local Health Departments. Without their time, expertise, and guidance this project would have not been possible. The authors extend a special thanks to the following people who served on their Nebraska Evidence-Based Advisory Committee. They thank the Nebraska Department of Health and Human Services, specifically Kathy Ward, Dave Palm, Rita Parris, Paula Eurek, Dan Cillessen, Sue Medinger, June Ryan, Bruce Rowe, and Brian Coyle. The authors also thank Nebraska Partners N Health, specifically Betsey Heidrick and Julia Cañas. The authors are thankful to Lou Rowitz who provided guidance on the leadership module; Avatar Studios for their work on developing and testing their online interface and for filming and recording the EBCC training, and Talent Plus for providing narrators and actors.
Funding
This project was funded by the National Cancer Institute (5R25CA113443).
Footnotes
Ethical approval
This project was approved by the Human Research Protection Office of Washington University in St. Louis.
Competing interests
None declared.
References
- 1.Centers for Disease Control and Prevention. [June 30, 2015];Chronic diseases: the leading causes of death and disability in the United States. 2015 Available from: http://www.cdc.gov/chronicdisease/overview/
- 2.World Health Organization. Global action plan for the prevention and control of noncommunicable diseases 2013–2020. (in IRIS) Geneva: World Health Organization; 2013. [Google Scholar]
- 3.National Cancer Institute. Cancer Control P.L.A.N.E.T. 2015 May 18; Available from: https://ccplanet.cancer.gov/
- 4.Brownson RC, Baker EA, Leet TL, Gillespie KN, True WR. Evidence-based public health. Oxford: Oxford University Press; 2010. [Google Scholar]
- 5.Jacobs JA, Dodson EA, Baker EA, Deshpande AD, Brownson RC. Barriers to evidence-based decision making in public health: a National Survey of Chronic Disease Practitioners. Public Health Rep. 2010;125:736–42. doi: 10.1177/003335491012500516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Koo D, Miner K. Outcome-based workforce development and education in public health. Annu Rev Public Health. 2010;31:253–69. doi: 10.1146/annurev.publhealth.012809.103705. 1 p following 69. [DOI] [PubMed] [Google Scholar]
- 7.Jacob RR, Baker EA, Allen P, Dodson EA, Duggan K, Fields R, et al. Training needs and supports for evidence-based decision making among the public health workforce in the United States. BMC Health Serv Res. 2014:14. doi: 10.1186/s12913-014-0564-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jacobs JA, Clayton PF, Dove C, Funchess T, Jones E, Perveen G, et al. A survey tool for measuring evidence-based decision making capacity in public health agencies. BMC Health Serv Res. 2012:12. doi: 10.1186/1472-6963-12-57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hannon PA, Fernandez ME, Williams RS, Mullen PD, Escoffery C, Kreuter MW, et al. Cancer control planners' perceptions and use of evidence-based programs. J Public Health Man. 2010;16:E1–8. doi: 10.1097/PHH.0b013e3181b3a3b1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Steele CB, Rose JM, Townsend JS, Fonseka J, Richardson LC, Chovnick G. Comprehensive cancer control partners' use of and attitudes about evidence-based practices. Prev Chronic Dis. 2015;12 doi: 10.5888/pcd12.150095. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brownson RC, Ballew P, Brown KL, Elliott MB, Haire-Joshu D, Heath GW, et al. The effect of disseminating evidence-based interventions that promote physical activity to health departments. Am J Public Health. 2007;97:1900–7. doi: 10.2105/AJPH.2006.090399. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Maylahn C, Born C, Hammer M, Waltz EC. Strengthening epidemiologic competencies among local health professionals in New York: teaching evidence-based public health. Public Health Rep. 2008;123:35–43. doi: 10.1177/00333549081230S110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jacobs JA, Duggan K, Erwin P, Smith C, Borawski E, Compton J, et al. Capacity building for evidence-based decision making in local health departments: scaling up an effective training approach. Implement Sci. 2014;9 doi: 10.1186/s13012-014-0124-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kenefick HW, Ravid S, MacVarish K, Tsoi J, Weill K, Faye E, et al. On your time: online training for the public health workforce. Health Promot Pract. 2014;15:48S–55S. doi: 10.1177/1524839913509270. [DOI] [PubMed] [Google Scholar]
- 15.Dreisinger M, Leet TL, Baker EA, Gillespie KN, Haas B, Brownson RC. Improving the public health workforce: evaluation of a training course to enhance evidence-based decision making. J Public Health Manag Pract. 2008;14:138–43. doi: 10.1097/01.PHH.0000311891.73078.50. [DOI] [PubMed] [Google Scholar]
- 16.Gibbert WS, Keating SM, Jacobs JA, Dodson E, Baker E, Diem G, et al. Training the workforce in evidence-based public health: an evaluation of impact among US and international practitioners. Prev Chronic Dis. 2013;10:E148. doi: 10.5888/pcd10.130120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Institute of Medicine (U.S.) Committee on assuring the health of the public in the 21st century. The future of the public's health in the 21st century. Washington, D.C.: National Academies Press; 2003. [Google Scholar]
- 18.Kreitner S, Leet TL, Baker EA, Maylahn C, Brownson RC. Assessing the competencies and training needs for public health professionals managing chronic disease prevention programs. J Public Health Manag Pract. 2003;9:284–90. doi: 10.1097/00124784-200307000-00006. [DOI] [PubMed] [Google Scholar]
- 19.Brownson RC, Ballew P, Kittur ND, Elliott MB, Haire-Joshu D, Krebill H, et al. Developing competencies for training practitioners in evidence-based cancer control. J Cancer Educ. 2009;24:186–93. doi: 10.1080/08858190902876395. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Yarber L, Brownson CA, Jacob RR, Baker EA, Jones E, Baumann C, et al. Evaluating a train-the-trainer approach for improving capacity for evidence-based decision making in public health. BMC Health Serv Res. 2015;15:547. doi: 10.1186/s12913-015-1224-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Ballew P, Castro S, Claus J, Kittur N, Brennan L, Brownson RC. Developing web-based training for public health practitioners: what can we learn from a review of five disciplines? Health Educ Res. 2013;28:276–87. doi: 10.1093/her/cys098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jacob RR, Allen P, Ahrendt LJ, Brownson RC. Understanding how state and local public health workers and partners learn about and use research evidence. Am J Prev Med. 2017 doi: 10.1016/j.amepre.2016.10.010. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fields RP, Stamatakis KA, Duggan K, Brownson RC. Importance of scientific resources among local public health practitioners. Am J Public Health. 2015;105(Suppl. 2):S288–94. doi: 10.2105/AJPH.2014.302323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Centers for Disease Control and Prevention. [September 6, 2016];CDC-RFA-DP13-1305: state public health actions to prevent and control diabetes, heart disease, obesity and associated risk factors and promote school health. 2013 Available from: http://www.grants.gov/web/grants/search-grants.html.
- 25.Centers for Disease Control and Prevention. [September 6, 2016];National Comprehensive Cancer Control Program (NCCCP) 2016 Available from: http://www.cdc.gov/cancer/ncccp/index.htm.