Training Employers to Implement Health Promotion Programs: Results From the CDC Work@Health^® Program

Abstract

Purpose:

Centers for Disease Control and Prevention (CDC) initiated the Work@Health Program to teach employers how to improve worker health using evidence-based strategies. Program goals included (1) determining the best way(s) to deliver employer training, (2) increasing employers’ knowledge of workplace health promotion (WHP), and (3) increasing the number of evidence-based WHP interventions at employers’ worksites. This study is one of the few to examine the effectiveness of a program designed to train employers how to implement WHP programs.

Design:

Pre- and posttest design.

Setting:

Training via 1 of 3 formats hands-on, online, or blended.

Participants:

Two hundred six individual participants from 173 employers of all sizes.

Intervention:

Eight-module training curriculum to guide participants through building an evidence-based WHP program, followed by 6 to 10 months of technical assistance.

Measures:

The CDC Worksite Health ScoreCard and knowledge, attitudes, and behavior survey.

Analysis:

Descriptive statistics, paired t tests, and mixed linear models.

Results:

Participants’ posttraining mean knowledge scores were significantly greater than the pretraining scores (61.1 vs 53.2, P < .001). A year after training, employers had significantly increased the number of evidence-based interventions in place (47.7 vs 35.5, P < .001). Employers’ improvements did not significantly differ among the 3 training delivery formats.

Conclusion:

The Work@Health Program provided employers with knowledge to implement WHP interventions. The training and technical assistance provided structure, practical guidance, and tools to assess needs and select, implement, and evaluate interventions.

Purpose

Recent workplace health promotion (WHP) research suggests that successful implementation of evidence-based strategies can improve employees’ risk factors for chronic disease.^1–4 Much of the published research about WHP programs is based on mid- or large-sized companies (≥250 employees), making it difficult to generalize to small employers who account for most US worksites.^5–10

Evidence about positive outcomes from WHP programs is not meaningful to many employers because they lack the knowledge or resources to implement programs.^2,11 Researchers who have studied the implementation of health promotion interventions in mid-sized worksites (100–999 employees) advised that companies need assistance selecting appropriate evidence-based interventions.⁵ However, there are few employer training opportunities about implementing workplace health programs; most training is geared toward wellness specialists.^12–15 Little information is available about strategies to train employers who do not employ wellness professionals—a particular issue for smaller employers.^16–19 In response, the Centers for Disease Control and Prevention (CDC) initiated the Work@Health Program to teach employers how to improve worker health, using evidence-based strategies focused on chronic health conditions. The program’s goals included (1) determining the best way(s) to deliver training to employers; (2) increasing employers’ knowledge of workplace health concepts, tools, and resources to support the implementation and evaluation of interventions; and (3) increasing the number of evidence-based workplace health interventions at participating employers’ worksites. This study of the first year of the Work@Health Program (2014–2015) is one of a few to examine the effectiveness of a program designed to train employers how to implement WHP programs. This study had 3 research questions. First, did individuals who participated in the Work@Health training increase their knowledge about WHP and their confidence and motivation to implement WHP programs at their own worksites? Second, would employers increase the number of evidence-based WHP strategies they had in place participating in the training and technical assistance (TA) activities? Third, would changes in knowledge, attitudes, or the number of WHP strategies in place differ based on whether the training was delivered in person or online? We hypothesized that after the training, (1) participants would exhibit improved knowledge of core concepts and increased confidence and motivation for implementing a program and (2) employers would have more evidence-based strategies in place than before the training. We did not hypothesize about how the method of training delivery might affect the outcomes.

Methods

Design

We used a pre- and posttest evaluation design to assess changes in participants’ knowledge and attitudes related to WHP and changes in the number of evidence-based interventions employers had in place.

Sample

Employers applied to Work@Health through an online application with questions about worksite size, industry, level of leadership commitment, and motivations. Eligibility criteria helped ensure that participating employers would have the capacity to make meaningful changes. Eligible employers needed:

• to be based in the United States;

• to be operational for at least 1 year;

• to offer health insurance to employees: to make evidence-based interventions related to disease management and preventive services feasible;

• at least 20 employees: to give employers the opportunity to use aggregate employee-level data; and

• Internet access: for access to online training modules and interactive components.

The application consisted of 19 questions to address the eligibility criteria as well as employer demographics related to industry size and sector, readiness to take action to improve workplace health programs, and level of commitment to fully participate in all program activities. Potential training participants were driven to the online application system using a variety of communication channels including social media; email; mass media, specifically radio; and direct outreach via webinars and in-person presentations and meetings, such as through a local Chamber of Commerce.

RTI International’s Institutional Review Board reviewed the study and deemed it exempt from institutional review because it was categorized as a program evaluation. Participants provided informed consent prior to providing responses to each data collection instrument.

A total of 402 employers applied, 200 employers were eligible and enrolled, and 173 completed the training in 2014. The participating employers were fairly evenly distributed across size categories and employment sectors. Table 1 displays the number of employers by training location, size, and sector. There were 206 individual training participants because employers were permitted to enroll 1 or 2 participants in the training and 33 of the employers enrolled 2 individuals.

Table 1.

Work@Health Employer Profile, 2014 to 2015.

Characteristic	Total No. Applicants	No. Ineligible or Not Enrolled	No. Enrolled But Did Not Complete Training	No. Completed Training	No. Completed Follow-Up Health ScoreCard (2015)
Training location
Atlanta	50	17	4	29	10
Baltimore	75	17	9	49	20
Chicago	78	33	7	38	20
Oakland	42	14	3	25	7
Philadelphia	19	11	2	6	0
Raleigh	60	39	1	20	4
Virtual	19	12	1	6	4
Other	59	59	0	0	0
Employer size
1–99 employees	124	60	4	60	21
100–249 employees	91	39	9	43	16
250–749 employees	79	36	7	36	17
750+ employees	97	57	6	34	11
Missing	11	10	1	0	0
Employer sector
Government	92	49	6	37	18
Nonprofit/education	156	73	11	72	25
Private sector	139	73	7	59	22
Other/missing	15	7	3	5	0
All organizations	402	202	27	173	65

Many of the 173 participating employers were new to implementing WHP programs, with 47.1% having programs in place for less than 2 years. The majority of employers reported being at least somewhat ready to take action to address employee health, with 57.3% reporting being somewhat or almost ready and 40.4% being completely ready. Only 2.4% reported being not quite ready or not ready at all.

Measures

The CDC Worksite Health ScoreCard.

The CDC Worksite Health ScoreCard is a validated²⁰ web-based instrument with 125 items assessing the presence of evidence-based health promotion strategies across 15 health topics (eg, organizational supports, nutrition, physical activity) and a community resource module.²¹ All items refer to the previous 12 months and answered with a “yes” or “no” response. Each employer completed 1 ScoreCard online prior to starting the training and 12 and 15 months after training.

Organizational assessment survey.

The team developed this 22-item, organization-level instrument to assess factors motivating employers to implement workplace health programs and implementation barriers. Employers completed the organizational assessment online prior to training and again 12 to 15 months after training.

Knowledge, attitudes, and behavior survey.

The team developed this 35-item instrument to capture changes in participants’ knowledge of WHP concepts in the curriculum as well as motivation, confidence, and other attitudes related to implementing a program. Participants completed the survey online prior to training and 12 months after the training.

Intervention

The Work@Health curriculum.

To inform the training and TA materials, CDC and its partners (the team) conducted formative research including an environmental scan of existing workplace health training programs; a literature review of health promotion practices and barriers at small-and mid-sized worksites; a training needs assessment of employers; and key informant interviews with WHP experts. The team designed a core curriculum and TA strategy for Work@Health with the guidance of several national experts in WHP and instructional designers specializing in adult learning.

The curriculum content, shown in Figure 1, consisted of 8 modules designed to guide the participant through the logical progression of building an evidence-based WHP program. The content aligned with CDC’s workplace health model, which includes 4 steps: assessment, planning, implementation, and evaluation, consistent with essential public health services and functions.¹⁹ Each module included applied exercises, opportunities for peer discussion (in person or online), and breaks to check for understanding. To anchor the training content with a real-world example, the materials described a fictional employer undertaking the activities from each module in an evolving case study. The training highlighted how to make data-driven decisions, build the infrastructure needed to implement interventions, and conduct continuous quality assessments to gauge progress and make improvements. The training materials also provided extensive links to tools and resources, directing participants toward best practices with an emphasis on physical activity, nutrition, and tobacco cessation interventions.

Figure 1.

The Work@Health Program Training Curriculum Components.

Centers for Disease Control and Prevention designed the implementation of Work@Health to deliver the curriculum using 3 training models: hands-on, online, and a blended model, which combined the hands-on and online models. The differences between the 3 models are shown in Figure 2.

Figure 2.

Work@Health Program Training Models.

Delivering the Work@Health Program.

The team conducted free training in 6 regional locations in 2014: Baltimore, Maryland; Oakland, California; Atlanta, Georgia; Chicago, Illinois; Raleigh, North Carolina; and Philadelphia, Pennsylvania. Participants were responsible for their own travel, and their preferences were taken into account to minimize travel costs and time. Employers whose participation was contingent upon being assigned to the online model were assigned to that model. Otherwise, the team randomly assigned employers to the 3 models. There were 106 trainees assigned to the hands-on model, 56 to the online model, and 63 to the blended model. Most, but not all, of the trainees assigned to a model participated in and completed the training. An additional group, referred to as the virtual employers, participated in the online model and were not associated with any of the training sites.

Participants in the online and blended models were given access to the web-based training platform after they completed required baseline surveys. These participants had approximately 3 weeks to complete the modules (all 8 for the online model participants and modules 1 to 6 for the blended model participants). All the participants were assigned a Work@Health coach who answered technical and logistical questions about the program and prompted them to complete the modules on time.

A certified Work@Health trainer paired with a workplace health subject matter expert delivered the in-person trainings. The training sessions had between 7 and 24 participants. The sessions for the hands-on model participants lasted approximately 8 hours, and the in-person sessions for the blended model participants lasted approximately 6 hours. This included a review of the online modules and primarily covered modules 7 and 8. Participants were encouraged to ask questions and share their own experiences.

After the training, participants received 6 to 10 months of free TA in the form of phone and online coaching, live and recorded webinars, and online interactions with peers (eg, sharing strategies for overcoming barriers). During the TA period, participants were expected to complete milestones associated with assessing, planning, implementing, and evaluating their new workplace health interventions.

Analysis

We calculated descriptive statistics summarizing responses from data collection instruments. We used SAS (version 14.1) statistical software (SAS Institute Inc., Cary, North Carolina) to conduct paired t tests to compare participants’ knowledge and attitudes and employers’ ScoreCards from 2014 and 2015. One of the objectives of the Work@Health evaluation was to compare the effectiveness of each of the 3 training methods in reaching the program’s goals. We used SAS software to test for differences between training methods using mixed linear models. The models controlled for baseline scores of the outcome constructs examined and included a covariate for employer to minimize the effects of employer differences.

Results

Participant-Level Changes in Knowledge and Attitudes

Of the 206 trainees who completed training in 2014, 179 (87%) participated in the follow-up evaluation by completing the knowledge, attitudes, and behavior (KAB) survey again in 2015. Key elements of participant knowledge and attitudes assessed are presented in Table 2. The mean knowledge quiz score used to assess participants’ overall knowledge about WHPs significantly increased from 2014 to 2015. Participants from each training model and each employer size category had significantly more correct answers in 2015 compared to 2014. Statistically significant knowledge score improvements were seen in all of the training modules, with the exception of Making the Business Case. The greatest improvements in the mean number of correct responses between 2014 and 2015 were seen in Planning and Designing Your Program (15% increase), Implementing and Sustaining Your Program (14.8% increase), and Evaluating Your Program (10.3% increase).

Table 2.

Changes in Participant Knowledge, Motivation, and Confidence, 2014 and 2015, by Training Model and Employer Size.^a

Participant Training Model/Employer Size	No. Correct Knowledge Quiz Answers (88-pt scale)				How Motivated to Implement/Enhance a Worksite Health Program?b				How Confident Are You in Your Ability to Start/Expand a Worksite Health Program?b
	2014	2015	P Valuec	N	2014	2015	P Valuec	Nd	2014	2015	P Valuec	Nd
Overall	53.2	61.1	<.001	179	4.5	4.2	<.001	177	4.0	3.9	.138	177
Blended	51.7	62.3	<.001	48	4.6	4.3	.002	48	4.0	4.0	.999	48
Hands-on	53.1	60.7	<.001	75	4.4	4.1	.014	74	4.0	3.7	.021	74
Online	54.5	60.5	<.001	56	4.6	4.3	.001	55	4.0	4.0	.999	55
1–99 employees	52.1	60.3	<.001	54	4.4	4.2	.014	53	3.8	3.8	.780	53
100–249 employees	54.1	61.2	<.001	51	4.4	4.0	<.001	50	4.0	3.8	.070	51
250–749 employees	52.5	61.5	<.001	35	4.4	4.4	.999	35	4.1	4.1	.831	35
750+ employees	54.1	61.6	<.001	39	4.7	4.3	<.001	39	4.1	3.9	.303	38

^aN = 179.

^bMeans in 2014 and 2015 using 5-point scales: 1 = not at all motivated/confident to 5 = extremely motivated/confident.

^cMeans were compared using paired t tests.

^dNo response from 2 participants.

Participants’ motivation to implement or enhance a WHP program significantly decreased between 2014 and 2015. Overall participants’ confidence in their ability to start or expand a WHP decreased slightly, but the change was not significant. Participants in the hands-on model reported a significant decrease in mean confidence, while the means for the other 2 models remained consistent from 2014 and 2015. Although participants did not report increased confidence levels, they did perceive themselves as being more proficient in developing a worksite health program in 2015 than they did in 2014. In 2014, only 10.7% indicated that they “fully understand how to develop a worksite health program, and can implement the program in my place of employment” compared to 54.8% in 2015. In 2014, 47.2% indicated that they were “just beginning to understand” or “had no knowledge of” how to develop a worksite health program; in 2015, this percentage decreased to 8.9%. The vast majority (91.1%) of participants reported either fully or partially understanding how to develop and implement programs at follow-up (compared to 52.8% at baseline).

We used mixed linear models with least squares means to test for differences between training methods (eg, mean change of blended vs mean change of hands-on; blended vs online; hands-on vs online), with regard to changes in knowledge, motivation, and confidence. The models included change scores as well as a covariate for employer to account for confounding factors associated with employer characteristics. There were no significant differences between the mean change scores associated with any of the training methods for knowledge, motivation, or confidence. Blended model participants had the largest gain in the mean number of correct responses on the knowledge assessment.

Employer-Level Changes

Of the 173 participating employers, 65 (37.6%) who completed the ScoreCard in 2014 completed it again in 2015. The 2014 baseline ScoreCard scores of the 65 employers used in these analyses did not differ significantly from the baseline scores of the 108 employers who did not complete the assessment in 2015. Table 1 shows the training locations, sizes, and sectors of the 65 employers. These employers’ overall ScoreCard scores significantly improved by an average of 36.9 points (from 126.1 to 163.0 of a total 264 possible points). On average, employers implemented significantly more evidence-based interventions of all types in 2015 compared to 2014 (Table 3). Across health categories, on average, the number of programs (ie, opportunities employers made available to employees to begin, change, or maintain health behaviors) in place increased from 35.5 to 47.7. The number of policies (ie, formal or informal statements designed to protect or promote employee health), environmental supports (ie, physical or structural elements to improve employee health), and benefits (ie, aspects of the employer’s overall compensation package) also increased. The greatest number of interventions added between 2014 and 2015 was seen in the area of organizational supports (average increase of 3.4), followed by tobacco control (average increase of 1.4).

Table 3.

Mean Number of Interventions Employers Had in Place in 2014 Compared to 2015, by Health Topic and by Intervention Type.^a

Intervention	Number of Possible Interventions	2014		2015		P Valueb
		Mean	SD	Mean	SD
Health topic
Organizational supports	18	9.0	4.4	12.4	4.1	<.001
Tobacco	10	5.8	2.7	7.2	2.2	<.001
Nutrition	13	5.0	3.1	5.7	2.7	.051
Lactation support	6	2.5	1.7	3.3	1.6	<.001
Physical activity	9	3.9	2.4	4.9	2.3	.002
Weight management	5	2.2	1.6	3.0	1.7	<.001
Stress management	6	2.6	1.6	3.7	1.6	<.001
Depression	7	2.4	2.2	3.4	2.1	<.001
High blood pressure	7	3.3	2.1	4.4	1.9	<.001
High cholesterol	6	2.7	2.0	3.6	2.0	<.001
Diabetes	6	2.9	2.0	4.0	1.9	<.001
Signs and symptoms of heart attack and stroke	4	1.2	1.4	2.0	1.5	<.001
Emergency response to heart attack and stroke	9	4.5	3.5	5.2	3.3	.022
Occupational health and safety	11	6.7	3.0	7.8	3.1	.004
Vaccine preventable diseases	6	4.7	1.4	5.0	1.3	.083
Intervention type
Program	76	35.5	16.4	47.7	16.9	<.001
Policy	18	8.5	3.5	10.2	3.4	<.001
Environmental support	18	6.6	3.1	8.2	3.2	<.001
Benefit	11	8.6	2.2	9.5	2.4	.004

Abbreviation: SD, standard deviation.

^aN = 65.

^b2014 and 2015 means were compared using paired t tests.

A key evaluation question for the Work@Health Program was whether 1 training model was associated with the implementation of more workplace health interventions than other training models. Using ScoreCard overall and module change scores as the outcome variables, we used mixed linear models with least squares means to test for differences between training methods. Again, the models included the change scores as well as a covariate for employer. At the overall ScoreCard level, there were no significant differences between the mean change scores associated with any of the training models. At the ScoreCard module level, the mean change scores differed significantly for 2 of the 15 health modules. Compared to employers in the blended model, employers in the online model had significantly greater increases in mean scores in organizational supports, which included interventions such as conducting a needs and interests assessment for planning health promotion activities and having an annual budget dedicated to health promotion programs. Compared to employers in the hands-on model, employers in the online model had significantly greater increases in mean scores on the occupational health and safety module, which includes interventions like encouraging reporting of injuries and near misses and having a written injury and illness prevention program.

Perceived barriers to implementation.

The organizational assessment revealed factors that employers perceived as significant barriers to health program implementation. Of the 173 employers who completed the training, 34 (19.7%) completed the follow-up organizational assessment. Among employers with both 2014 and 2015 organizational assessment data, the factors most commonly perceived as significant barriers in 2014 were lack of employee time to participate (56.3%), lack of funding (43.8%), lack of interest among employees (40.6%), lack of staff and lack of space (each 31.3%), and a dispersed workforce, low management/supervisory support, and program is difficult to administer (each 25%). Two barriers had lower percentages of employers who perceived them as significant in 2015: lack of interest among employees (24.1%) and difficulty in administration (17.2%).

Discussion

The Work@Health Program team developed a curriculum to train employers on best practices for planning, implementing, and evaluating WHP programs. The team recruited employers from around the United States to participate in 1 of the 3 training models. The project team delivered in-person training at 6 locations and developed an online learning platform to deliver online portions of training and posttraining technical assistance.

Our hypothesis about training participant improvements was partially supported. Participants significantly increased their knowledge quiz scores after participating in the program, with the largest score increases in topics focused on how to implement a program. Participants were more likely to characterize their level of proficiency in developing and implementing a WHP at a high level after participating in the training compared to prior to the training. Although participants did not report having higher motivation or confidence in their ability to implement or enhance a program after participating in the program, on average, participants rated themselves as “very” to “extremely” motivated and “somewhat” to “very” confident before they began the Work@Health program. They had little room to increase motivation and confidence, and we speculate that the comprehensiveness of the curriculum may have been somewhat overwhelming, leading to a slight decrease in some participants’ confidence levels.

Over the course of 12 to 15 months, participants from all 3 training models were able to return to their worksites and significantly increase the number of evidence-based health promotion interventions they had in place. Employers reported the largest increases in the number of implemented organizational supports, which are foundational to a sustainable program. In 2015, 78% indicated that their organization demonstrated organizational commitment and support of worksite health promotion at all levels of management, compared to 58% in 2014, and 69% had an annual budget/dedicated funding for health promotion in 2015, compared to 42% in 2014. More employers designed effective communications, with 63% tailoring some health promotion programs and education materials to various segments of the workforce, compared to 29% in 2014. More of the employers were also evaluating their programs after participating (68% in 2015 compared to 40% in 2014).

The first-year evaluation of the Work@Health program compared 3 different models for delivering the same curriculum. Neither the participant-level knowledge gains nor the organization-level changes employers made, in terms of implementing evidence-based strategies, differed meaningfully across the 3 models. This finding suggests that using the most cost-effective model may be the most practical approach to reach the largest number of employers. As more small- and mid-sized employers recognize the potential productivity benefits associated with supporting employee health,^22,23 they will need training support.

Serious study limitations prevent us from concluding that participation in the training program caused employers to implement more health promotion interventions. These limitations include the absence of a control group, nonrandom assignment of employers to training models, and high employer attrition. There was no control group of employers because the primary objective of the Work@Health Program was to demonstrate that the CDC could develop an evidence-based workplace health training curriculum, recruit employers, and deliver training using the 3 different models. We cannot be certain how much of the participating employers’ ScoreCard change was due to the training and how much may have been due to external factors or the assessment process itself.

For practical reasons, not all employers were randomly assigned to training models. Although the project team made an effort to randomly assign employers when possible, they gave employers the option to select the online model if that was the only way they could participate. Although we found no significant differences in follow-up scores between the training models, the lack of random assignment does not allow us to make this conclusion.

More than half of the employers who completed the training did not complete the follow-up ScoreCard, thus we don’t know if they made similar increases in the number of interventions in place. Although there were no significant ScoreCard differences at baseline between participants who completed the follow-up and those who did not, it is possible that those who failed to complete the follow-up made less progress. On average, employers completed 2.2 of 4 potential milestones meaning that they completed the assessment and planning milestones but did not proceed to complete the implementation and evaluation milestones. To try and understand the cause of attrition, we analyzed the correlations between various factors motivating program implementation and the number of milestones completed; none of the correlations were significant. Future research is needed to better understand the participants’ barriers to program completion. Those employers lost to follow-up who could be contacted for exit interviews cited changes in employer leadership; changes in trainee job roles/responsibilities; and time and resource constraints due to higher priority issues as the main reasons for dropping out.

There are additional limitations associated with the data collection instruments and methods. The KAB instrument was developed for this study by the project team, and the knowledge quiz items had face validity but were not tested for construct validity. Confidence and motivation were each measured with a single item developed by the team and not validated. Another potential limitation was the self-reported ScoreCard. There was no confirmation or auditing of the interventions the employers reported having in place.

Overall, the Work@Health program was shown to be effective in providing employers with the knowledge needed to create and/or enhance WHP interventions to support the health and well-being of employees. The training and technical assistance provided a timeline and structure along with practical guidance and tools to build support; assess needs; and select, implement, and evaluate interventions. The success of the program is particularly notable among the smaller employers, among whom little WHP research had previously been conducted.

What is already known on this topic?

Although most employers offer some type of wellness programming for employees, small and midsize employers have limited resources available to select and implement interventions most appropriate for their employees. While there is some debate in the literature about the effectiveness of WHP programs, employers have the best chance for positive outcomes if they implement well-designed, evidence-based programs. However, there is a scarcity of evaluated professional training programs to help small and midsize employers do this.

What does this article add?

This article contributes to the body of knowledge of how to design, implement, and evaluate a professional adult training program that effectively improves knowledge, attitudes, and behaviors with applied knowledge in taking action to improve practice (in this case building/enhancing evidence-based workplace health programs).

What are the implications for health promotion practice or research?

Employers hoping to implement or enhance their own health promotion programs may use the Work@Health Program to increase their knowledge of the processes and resources available to assess, plan, implement, and evaluate a wellness program built upon evidence-based intervention strategies. Empowering employers with this structure and process can make them more informed and effective consumers of health promotion services and programs.