Worksite wellness program implementation: a model of translational effectiveness

ABSTRACT

Occupational health promotion programs with documented efficacy have not penetrated worksites. Establishing an implementation model would allow focusing on mediating aspects to enhance installation and use of evidence-based occupational wellness interventions. The purpose of the study was to implement an established wellness program in fire departments and define predictors of program exposure/dose to outcomes to define a cross-sectional model of translational effectiveness. The study is a prospective observational study among 12 NW fire departments. Data were collected before and following installation, and findings were used to conduct mediation analysis and develop a translational effectiveness model. Worker age was examined for its impact. Leadership, scheduling/competing demands, and tailoring were confirmed as model components, while organizational climate was not a factor. The established model fit data well (χ²(9) = 25.57, CFI = 0.99, RMSEA = 0.05, SRMR = 0.03). Older firefighters, nearing retirement, appeared to have influences that both enhanced and hindered participation. Findings can inform implementation of worksite wellness in fire departments, and the prioritized influences and translational model can be validated and manipulated in these and other settings to more efficiently move health promotion science to service.

INTRODUCTION

Worksites have the potential to be effective vehicles for health promotion. Interventions can be delivered, social norms altered, peer support recruited, and environments restructured to benefit both workers and their employers [1]. However, despite studies documenting reduced healthcare costs and improved employee productivity, evidence-based worksite programs often are not used [2]. Partially as a consequence, a national initiative is underway to implement and expand evidence-based health promotion [3]. Establishing a model for occupational wellness program implementation could further assist worksites in their effective use.

Most implementation research on the uptake of evidence-based interventions focuses on programs to alter providers’ care patterns [4], introduce new school curricula [5], or modify practices in community settings [6]. Those models may differ from worksite dissemination where adoption and implementation is by an organization, and employees are asked to change their own health behaviors. No published study has assessed what factors mediate translation of a worksite health promotion program. In addition, few studies have examined the influence of employee age or pre-retirement status on adoption of occupational wellness activities.

The investigation to guide new insight for translational effectiveness (IGNITE) trial

The IGNITE study was funded through the American Reinvestment and Recovery Act Challenge Topic ‘Pathways for Translational Research’ to define and prioritize determinants that enable and hinder the uptake and use of evidence-based health interventions in well-defined settings. IGNITE’s aim was to define a model for worksite wellness program translation among moderately-sized fire departments [7].

Despite perceptions to the contrary, firefighters’ behavioral profile is comparable to other workers, including unhealthy diets and lack of regular physical activity [8]. Their episodic intense work, combined with those health risks, contribute to myocardial infarctions being the leading cause of on duty death [9]. Healthy lifestyles are of additional importance because firefighters’ risk of cancer is increased [10].

The promoting healthy lifestyles (PHLAME) wellness/injury reduction program was developed, shown efficacious, and beta-tested with funding from the National Institutes of Health [11–13]. The PHLAME curriculum offers 12 45-min interactive sessions to be completed once per week over approximately 4 months. Its team-centered, peer-led format is a natural fit for firefighters’ work structure. Typically smaller departments have three stable shifts, with each shift working 24 h followed by 48 h off duty. Stations are staffed by four to eight firefighters, and each station shift can participate as a PHLAME team, inserting sessions into their usual activities. To enhance fidelity and ease of use, sessions are explicitly scripted with a leader’s manual and workbooks for other firefighters. PHLAME is listed on the Cancer Control P.L.A.N.E.T. evidence-based website for both promoting healthy nutrition and enhancing physical activity (http://cancercontrolplanet.cancer.gov/).

Non-comparability among businesses and turbulence within and across sites makes worksites problematic study environments [14]. Although fire departments differ in size, location, revenue sources, and competing economic demands, the fire service has advantages for examining translation. Most departments share a uniform hierarchical structure, and trajectories of employment also are similar, with firefighters generally beginning to work in their twenties and continuing until eligible for retirement and pensions in their fifties. Accordingly, studying PHLAME implementation among departments provided variability to evaluate fidelity to a hypothesized translational model and stability for longitudinal assessment.

METHODS

Study design

Observational Study

The IGNITE trial was designed as a prospective observational cohort study [15]. Data collection instruments assessed theoretical dimensions shared by four large reviews of the implementation literature [16–19]. Some potential determinants, such as financial incentives and training/technical support were not options or uniformly applied across sites, respectively, and not included in the model. The mediators for the second phase of the model (program exposure/dose to outcomes) were assessed in the original PHLAME efficacy trial mediation analysis [20], and we focused on a simplified model with program exposure leading directly to participant outcomes. Different terms have been used to describe the process of how a program is used [21]. We chose to call the proximal variable between implementation constructs and outcomes “exposure.” In some cases, that term has related only to participant awareness [22]. However, as we only had self-reported measures of fidelity and session duration, we used the more general term, with a meaning more akin to “implementation effectiveness,” which concerns participants’ aggregate use [23]. To enhance understanding and explore features not captured by this model, a parallel qualitative analysis used group interviews [24]. Study method and reported findings adhere to components of the STROBE Statement for observational studies [25]. The institutional review board of the Oregon Health & Science University approved the study in August of 2009.

Site recruitment and adoption

Information about the study was mailed to all (n = 70) medium-sized fire departments (40–140 career firefighters) in Oregon and Washington, and 19 agreed to participate. Twelve sites were selected to provide a spectrum of contextual variables. This number was predicted as sufficient to detect effects where paths to or from purported mediators were medium or larger. Each selected site provided assurance that its firefighters were informed about the program and a tentative agreement that the majority would voluntarily participate. Additional details concerning recruitment and the decision to adopt are described separately [26].

Data collection

Reflecting the three 24-h shift structure, each site was assessed over 3 days in late spring 2010, during which consents were obtained, surveys completed, limited physiological data acquired (height, weight), and group interviews conducted. Following data collection, PHLAME materials were distributed, and shifts were oriented to conducting the program. Technical support was available to all sites throughout the study. Approximately 6 months later, the initial data gathering activities were repeated during a second round of 3-day visits. Follow-up assessments included program exposure indexed as the participant self-reported session completion, using cued recall of session content; the survey and group interviews included additional items relating to the program’s characteristics and use.

Survey data were used to identify firefighter demographics and dietary indices [27]. Firefighters’ self-reported general health was measured with the ‘General Health’ subscale of the SF-36 instrument [28]. Additional items were selected from existing instruments to assess components of the hypothesized translational model, including organizational climate [29, 30], leadership characteristics [31], and items relating to program scheduling/competing demands and tailoring. The survey construct items were answered using a seven-point agreement scale. Individual items for the constructs are shown in Table 1.

Table 1.

Survey constructs, items and reliability

Construct and items	Cronbach’s α	Mean (SD)a
Organizational Climate	0.70	4.2 (1.2)
• Our department is progressive in adopting new activities
• Our department is one where everyone has a voice in decision making
• Mutual trust and cooperation among our department members is strong
Leadership	0.82	4.8 (1.4)
• Our department advocates for the improvement of firefighter health
• We have a strong network of like minded people who can make things happen
• Our department is at the forefront of using new technology
• We have people in our department who can make things happen
• Management and firefighters work well together as a team
Scheduling/Competing Demands	0.73	5.0 (1.2)
• Things were too hectic to get PHLAME sessions done
• PHLAME sessions were a priority in our department
• We found a way to schedule the PHLAME sessions
• It was difficult to find a consistent time to do the PHLAME sessions and stay on track
Tailoring	0.71	3.9 (1.2)
• Our department modified PHLAME to fit our own needs
• Our team made a few changes in the scheduling of sessions to make PHLAME work for us
• Being able to tailor the PHLAME program helped us get it done

^aConstruct scores were sum of the individual items divided by the number of items

Following individual measurements and survey completion, participating firefighters were engaged in a group interview. Experienced interviewers used a semi-structured guide based on the hypothesized translational model’s dimensions [32]. The interviews were audiotaped, transcribed, reviewed for accuracy, redacted of names, and assembled by site for qualitative analysis.

Data analysis

Quantitative data

Factor analysis and reliability assessment was used to confirm constructs, and internal consistency of the measures is shown in Table 1. Standard descriptive statistics were used to examine baseline characteristics of departments.

Mediation analysis was used to examine the hypothesized relationship among constructs and understand the underlying mechanism of implementation [33] (Fig. 1). As shown in Fig. 2, a simple mediated effect occurs when a predictor variable changes a mediator (a path) and that mediator changes the outcome (b path). The mediated effect is the product of a and b paths, ab, which estimates the part of the effect of the predictor variable on the outcome variable transmitted through the mediator. The direct path, c’, is the relation of the predictor to outcome variable that is not through the mediator. The mediation model was estimated with Mplus 5.2 computer program [34]. Exposure and mediator variables were assessed at the individual level, and although multilevel analyses were not performed due to the limited number of departments, the clustering of individuals within departments was controlled to yield accurate assessment of inference [35]. The baseline outcome variables were included as covariates to increase power to detect effects and to adjust for change over time. Maximum likelihood estimation was employed to include all available data. The statistical significance of mediated effects was computed by the PRODCLIN program, which provides more accurate mediated effects confidence limits since the distribution of the product of two normally distributed variables is not normal [36].

Fig 1.

Hypothesized mediation model of worksite wellness implementation

Fig 2.

Diagram of mediator model

Qualitative data

The interview transcripts were read and re-read, and organized in a systematic search for patterns and underlying meanings, leading to construction of interpretations [37], using an adaptation of a constant-comparative method [38], descriptive matrices of themes were constructed and substantiated with illustrative quotations. Reflecting this special issue’s theme, follow-up transcripts underwent a second analysis involving a search for terms related to employee age (i.e., age, years, young, old, generation, and retire), then scrutiny of the context of each for meaning. Due to limitations in length, only the age-related qualitative findings are included here. Presented quotations are as stated, with wording altered only as needed to preserve clarity outside of the original context.

RESULTS

Sites and participants

A description of the sites is presented in Table 2. Participation rates at enrollment and follow-up were comparable across departments. In general, study participation rates were high, with most more than 80%. Non-participation generally was due to scheduling issues, with firefighters not working at the times of initial and follow-up visits.

Table 2.

Description of Fire Departments, participation rates and program exposure

Department	Number of firefighters	Number of stations	State	Locationa	Firefighter Age (mean years [SD])	Overall participation (%)	Baseline with follow-up (%)	Exposure (sessions completed [mean (SD)])
A	38	2	WA	Rural	35.9 (8.9)	82	76	8.5 (2.6)
B	68	4	OR	Rural	37.6 (8)	84	56	3.5 (2.9)
C	54	3	WA	Urban	41.5 (8.2)	69	56	6.0 (3.3)
D	89	7	WA	Urban	37.1 (8.6)	95	43	1.4 (2.6)
E	50	4	OR	Urban	40.5 (7.7)	88	62	4.4 (4.1)
F	44	3	WA	Urban	39.3 (7.5)	94	53	9.5 (3.0)
G	40	3	OR	Suburban	38.1 (7.9)	97	68	6.9 (4.4)
H	55	3	WA	Suburban	40.4 (8.4)	82	53	8.0 (3.6)
I	100	6	WA	Urban	40.6 (8.8)	61	62	2.0 (1.9)
J	140	8	WA	Urban	43.2 (8.6)	72	58	2.4 (2.3)
K	62	4	WA	Urban	41.2 (8.9)	73	66	2.1 (1.8)
L	34	2	WA	Suburban	42.9 (10.2)	8	82	5.6 (2.9)

^aDesignation based on rural–urban continuum codes with surveillance epidemiology and end result codes that use population size of their metro area, and nonmetropolitan (nonmetro) counties by degree of urbanization and adjacency to a metro area or areas (http://seer.cancer.gov/seerstat/variables/countyattribs/ruralurban.html)

Outcome measures assessed were daily servings of fruits and vegetables, calculated body mass index (BMI), and health quality. The pre to post means (SD) for all participants were fruits and vegetable intake from 6.7 (4.7) to 8.0 (5.6) servings per day, BMI from 29.0 (4.4) to 28.7 (4.2) kg/m², and health quality from 4.5 (1.3) to 4.7 (1.3), respectively. Baseline equivalence among sites was tested for all outcome variables, and departments were equivalent on all but fruit and vegetable intake. Pre-test measures of outcome variables were included in the quantitative analyses to control for baseline differences.

Mediation model

The model of implementation to exposure and exposure to outcomes had four predictor variables (organizational climate, leadership, scheduling issues, and tailoring), one mediator (exposure defined as the average numbers of PHLAME sessions completed for that department), and three outcome variables (daily fruit and vegetable intake, BMI, and health quality) (Fig. 3). This model fit the data well (, CFI = 0.99, RMSEA = 0.05, SRMR = 0.03). Table 3 provides the path estimates, mediated effect estimates, and corresponding standard errors for the model. The cross sectional model lacks randomization of units to exposure level of PHLAME and is a description of relations among variables rather than an inferential model.

Fig 3.

Full mediation model. All outcome variables were predicted by their corresponding pre-test score. Model fit the data well (, CFI = .99, RMSEA = .05, SRMR = .03). Unstandardized path estimates and standard errors are shown. Predictors were allowed to correlate. Outcomes were allowed to correlate. Paths that were statistically significant at are depicted in bold. **p < 0.01, *p < 0.05, †p = 0.055

Table 3.

Indirect and direct unstandardized path estimates in the mediation model

Construct	Exposure (mediator)	Fruit and vegetable intake (outcome)		Body mass index (outcome)		Health quality (outcome)
	B (SE)	B (SE)	Indirect	B (SE)	Indirect	B (SE)	Indirect
Organizational climate	−0.15 (.20)	0.17 (0.14)	−0.03 (0.04)	0.05 (0.04)	0.01 (0.01)	0.02 (0.05)	−0.01 (0.01)
Leadership	0.93 (0.21)*	−0.07 (0.20)	0.17 (0.08)**	0.04 (0.05)	−0.04 (0.01)*	0.02 (0.05)	0.03 (0.01)**
Scheduling/competing cemands	−0.74 (0.19)*	0.03 (0.24)	−0.13 (0.07)**	−0.06 (0.04)	0.03 (0.01)**	0.02 (0.07)	−0.02 (0.01)**
Tailoring	0.88 (0.21)*	0.04 (0.27)	0.16 (0.08)**	0.00 (0.06)	−0.03 (0.01)**	0.0 (0.07)	0.03 (0.02)**
Exposure	–	0.18 (0.08) **	–	−0.04 (0.01)*	–	0.03 (0.02)***	–

Baseline measures of outcome variables were included as covariates

*p < 0.01; **p < 0.05; ***p = 0.055

Three implementation effectiveness measures had significant mediation effects through exposure to outcome variables, while one did not have a significant effect. First, leadership, scheduling issues, and tailoring had significant mediated effects through program exposure on fruit and vegetable intake (ab = 0.17, 95% CI [0.02, 0.35]; ab = −0.13, 95% CI [−0.29, −0.02]; ab = 0.16, 95% CI [0.02, 0.34], respectively). Second, mediated effects of the leadership, scheduling issues, and tailoring through program exposure on general health were significant (ab = 0.03, 95% CI [0.001, 0.07]; ab = −0.002, 95% CI [−0.05, −0.001]; ab = 0.03, 95% CI [0.001, 0.06], respectively). Third, mediated effects of the leadership, scheduling issues, and tailoring through program exposure on BMI also were significant (ab = −0.04, 95% CI [−0.07, −0.01]; ab = 0.03, 95% CI [0.01, 0.06]; ab = −0.03, 95% CI [−0.06, −0.01], respectively). However, mediated effects of organizational climate through program exposure on fruit and vegetable intake, BMI, and health quality were not statistically significant (ab = −0.03, 95% CI [−0.11, 0.04]; ab = 0.01, 95% CI [−0.01, 0.02]; ab = −0.01, 95% CI [−0.02, 0.01], respectively).

Age was included as a predictor, but it did not significantly contribute to the model. Additionally, age did not moderate the relations in the model, which was tested by using a grouping variable of age less than or equal to 40. Therefore, results from those models are not reported.

Qualitative findings concerning age analyses

Although age was not a significant factor in the mediation model, older firefighters did express awareness of their declining physical abilities to perform their jobs, their increased need for rest and recovery, and their increasing susceptibility to injury. One commented, “People are going to get older in the fire service, and you’re not going to be able to perform like you do when you’re young.” Looming retirements after “long careers, maybe 25–30 years, in a physically demanding and abusive environment” intensified their concerns, “I get statistics about the cops, you know, saying they die 6 or 7 years after they retire on average. Now, what about the firefighters?” “Then, having the family history, you know, heart problems and cancer—it just drives it home that much more. As I’m closer to retirement, it’s a big deal.”

Older firefighters also appeared to recognize that “We’re getting to that age where we need to pay a little more attention.” However, perhaps relating to lack of an overall age effect, increased awareness was rarely sufficient for change. To the contrary, long established habits appeared more difficult to alter, “that habit. It’s hard to break, you know, at our ages—to suddenly [start eating] a lot of fruit.” Rather, interview data strongly suggested that the older the firefighter, the more momentous was the effort needed to overcome entrenched habit. A few justified their inertia, one saying, “The doctors’ recommendations on foods change a lot. So, in ten years, sugar might be good for you.”

DISCUSSION

We defined a mediation model of worksite wellness program implementation among medium-sized fire departments. Departmental leadership, scheduling issues, and tailoring related to greater number of PHLAME sessions, which in turn related to improved outcomes. Although implementation has been examined using other methods, such as qualitative single site studies, cross-site meta-analysis, case studies, and ethnographic means [39–41], we believe that this is the first application of mediation analysis to understanding implementation effectiveness of an occupational health promotion intervention. The findings provide insight into which factors should be targeted to make the process more effective and which activities may be less influential.

The three identified significant predictors are each aspects that, when studied in isolation, are related to implementation [16–19]. For leadership, we sought to use items drawn from existing instruments [31] and also capture collaboration between departmental leaders and employees in advancing wellness. Similar to organizational readiness for health promotion [42], our leadership dimension included that component. Other potential leadership aspects, such as awareness of links between health and worker productivity and business alignment with health promotion objectives, were omitted as they seemed integral to career firefighting. The scheduling construct might be viewed as a dimension of organizational capacity for change [42], and it also included a measure of turbulence or unanticipated events, which have been found a factor in implementation in other settings [43]. Previous studies have identified adaptability/compatibility or contextual appropriateness as innovation features relating to implementation [44, 45]. The latter was addressed in the development and beta testing of the program [12], and the tailoring construct items primarily related to adaptability. Rather than an implementation failure, some degree of tailoring may be inevitable [46] and even improve program outcomes [47].

Organizational climate was not an influential factor in our implementation model. This worksite dimension originally was conceived as perceptions of behaviors that are expected, supported, and rewarded by the organization [48]. In theoretical discussions of implementation, it has been viewed as a critical aspect [23]. However, two factors may have reduced the impact of this component. First, this dimension may be relatively fixed among fire departments, and secondly was that its items did not relate specifically to the innovation [28].

Worker age did not appear to be a factor in this implementation model. However, the qualitative comments indicated worker age had competing influences that may have resulted in minimal net impact. In limited studies, the transition to retirement was a factor relating to improved health habits [49]. However, our findings indicated that those influences were counterbalanced by older firefighters’ difficulty in changing long established habits. Retirement is a major life transition [50], and good health is a prerequisite for quality of retirement life [51]. In the future, targeting implementation efforts for older workers might enhance their participation and outcomes.

There are limitations to our results, including the possible existence of alternative models that may fit the data as well or better. Establishing mechanisms is challenging when neither the independent variable nor the mediating variable is randomized. Future studies, where components are directly manipulated and outcomes subsequently assessed could provide stronger evidence for the causal links. In addition, the fire service is a unique occupation, which could limit model generalizability [52]. Finally, the term Hawthorne effect to denote altered behaviors because of being studied originated in a worksite investigation [53], and the study process itself may have influenced findings.

Our project was funded to advance the wellness of first responders, and the model will inform future efforts to efficiently implement PHLAME. In addition, the prioritized influences and translational model can be validated and manipulated in these and other settings to more efficiently move health promotion science to service.