Return on Investment of a Work–Family Intervention

Abstract

Objective:

To estimate the return on investment (ROI) of a workplace initiative to reduce work–family conflict in a group-randomized 18-month field experiment in an information technology firm in the United States.

Methods:

Intervention resources were micro-costed; benefits included medical costs, productivity (presenteeism), and turnover. Regression models were used to estimate the ROI, and cluster-robust bootstrap was used to calculate its confidence interval.

Results:

For each participant, model-adjusted costs of the intervention were $690 and company savings were $1850 (2011 prices). The ROI was 1.68 (95% confidence interval, −8.85 to 9.47) and was robust in sensitivity analyses.

Conclusion:

The positive ROI indicates that employers’ investment in an intervention to reduce work–family conflict can enhance their business. Although this was the first study to present a confidence interval for the ROI, results are comparable with the literature.

Work–family conflict (WFC) occurs when workers struggle to manage both their work and personal obligations.^1–3 Work–family conflict has been linked to lower family, marital, life, and job satisfaction⁴; to higher work stress, turnover intentions,⁵ and absenteeism⁴; and to lower organizational commitment and productivity.^3,4,6–9 As a result, work–family initiatives intended to support employees’ lives outside of work and reduce WFC have been implemented increasingly in US workplaces.^10,11 Nevertheless, the effects of these initiatives on WFC are not well known as most were either not developed on empirically based principles and/or not systematically evaluated. Moreover, few studies have utilized scientifically rigorous designs, such as longitudinal data collection or randomization into intervention and comparison groups, to test the effects of workplace policies and practices on WFC and its correlates.⁸

To address this scientific gap, the National Institutes of Health and the Centers for Disease Control and Prevention established the Work, Family, and Health Network (“network”) in 2005.¹² The goal of the network is to advance the field of workplace psychosocial interventions by targeting WFC to improve the health and well-being of workers and their families. The network designed an intervention targeting three elements: (1) enhancing employees’ control over their work time, (2) increasing supervisor support for employees to manage work and family responsibilities, and (3) reorienting the culture toward results instead of time spent in meetings or at the office. The network assessed the efficacy of this intervention via a group randomized field experiment in the information technology division of a large Fortune 500 company (referred to as “Tomo”) in the United States.^13,14

Previous research suggests that positive changes in employee stress, including reduced WFC, may benefit employers financially.⁸ Improved health and well-being may affect health care costs directly; furthermore, there is a significant amount of evidence linking health risks (eg, stress) to indirect business costs in the form of absenteeism, workers’ compensation costs, and decreased work performance (presenteeism).^15–19 Strong associations exist between employee health and work performance.^18,20,21 Furthermore, some uncontrolled studies have shown positive associations between improvement in health status and work performance.^22,23 The most common measures of workplace health-related productivity include absenteeism, presenteeism, and employee turnover/replacement.²⁴

Accurate estimates of the economic return of workplace initiatives to reduce WFC are critical to establishing such initiatives as strategic expenditures. A useful way for communicating the financial ramifications of a given initiative is the return on investment (ROI), a metric often used in business administration that compares an initiative’s costs to its financial benefits.²⁵ No study to date has estimated the ROI to the workplace of implementing an initiative to reduce WFC. This article estimates the ROI of the network intervention, named STAR (support, transform, achieve, results), as implemented in Tomo. Previous workplace ROI studies focus mainly on wellness programs for weight loss and fitness, and few of them used scientifically rigorous designs.^4,26–28 To our knowledge, no study in the workplace literature has assessed the statistical significance of the ROI estimates they present. In this study, the cost of STAR is compared with the aggregate change in the following three organizational costs: presenteeism, voluntary termination of employment, and health care utilization. This study relies on the group-randomized design of the underlying network field experiment¹³ to establish whether investing in STAR is worthwhile from a company perspective, and is the first study to calculate confidence intervals for ROI estimates.

METHODS

Intervention

STAR encompassed three components designed to support and build on each other: participatory training sessions, computer-based training (CBT), and behavioral self-monitoring.¹⁴ Face-to-face participatory sessions were delivered to employees and managers by external consultants. Overall, six sessions were delivered over a 4-month period, four for managers and employees together (to learn intervention concepts that focus on results and not location and timing of work)^29,30 and two for managers only (to reinforce intervention support and cultural changes). Participatory sessions were linked with two outside activities for all employees to strengthen learning by self-monitoring as they put the new concepts into practice. Managers participated in one CBT session designed to increase family-supportive behaviors that are known to affect outcomes, such as employee health, absenteeism, turnover, and job satisfaction.³¹ Examples of family-supportive behaviors include helping an employee find a replacement if absent, asking how employees are doing, communicating genuine concern about employees’ work/life challenges, and showing that managers value involvement in nonwork life.¹⁴ Computer-based training was designed in software built on behavioral principles of instructional design³² and assessed understanding using pre- and posttests and intermittent quizzes. Transfer of CBT knowledge to worksites was supported with behavioral self-monitoring designed by the research team to help managers set goals and increase support for their employees.^33,34 Managers completed two 2-week behavioral self-monitoring trials using iPod Touch devices to observe and record supportive behaviors.³⁵ The trials involved goal setting, daily self-monitoring and tracking of family and performance supportive behaviors, and individual and group feedback. Intervention materials are available for download at www.workfamilyhealthnet-work.org. Methods, measures, and study design are described in more detail in Bray et al.¹³

Design

Our ROI analysis builds on the group-randomized multisite controlled experimental study of STAR in Tomo. Researchers aggregated existing “workgroups” (teams of employees and supervisors) to create study groups to serve as the unit of randomization. There were 56 study groups, with some comprising large teams of workers reporting to the same supervisor, whereas others included multiple teams reporting to the same senior leader or working closely together on the same project. Adaptive randomization ensured balance on job function, leadership, and size of the study group. For details on study design, see Bray et al¹³ and Kelly et al.³⁶

Study groups were randomized to intervention or control/usual practice. In the intervention arm, a multicomponent process to reduce WFC was delivered as described above. Primary evaluation data for this study were collected using computer-assisted personal interviews (CAPIs) lasting about 60 minutes with employees and their supervisors (“employees”) in the intervention and control groups at baseline, 6, 12, and 18 months. Corporate administrative data were available to the research team on an ongoing basis. Data collectors were blind to the experimental condition of the participants, who consented to each component of data collection.

The ROI is calculated as the ratio of the difference between the benefits, defined as changes in organizational costs that result from the intervention, and costs of the intervention to the costs of the intervention [ROI=(Benefits − Costs)/Costs]. The primary variables used to calculate the ROI were differences between the intervention and control groups in (1) intervention costs, (2) productivity represented by presenteeism (ie, being present at work but working at a reduced capacity), (3) health care utilization, and (4) voluntary termination (“turnover”).

Sample

A total of 1427 employees were eligible to participate in STAR and the workplace data collection using CAPI. To be eligible for CAPI survey data collection, employees had to be located in the two cities where data collection occurred and had to be classified as employees, rather than independent contractors, of the company. Of those eligible, 73.1% completed the baseline survey (n=1044), and of these 94.5% (n=987) completed any of the follow-up surveys. Employees with only baseline data (n=57) were excluded from the analysis.

The ROI was conducted on the sample of respondents who completed both baseline and at least one follow-up surveys with the following exclusions. Seventeen employees who were randomized to the intervention condition but never invited to participate in any STAR sessions because of an error on the part of the research staff were excluded. In addition, eight employees who because of restructuring began reporting to a supervisor already going through STAR, were excluded because they were not randomized to either arm. Fourteen employees were excluded because they were missing data required to calculate intervention benefits. Finally, two employees who were lacking demographic information were excluded. The resulting analytic sample consists of 946 employees (473 STAR and 473 in control condition). Figure 1 confirms that response rates are similar for employees in intervention and control conditions and that all study groups identified as eligible for the study were randomized and had at least some employees who participated in the survey.

FIGURE 1.

Study design, sample formation, and response rates.

MEASURES

Intervention Costs

Intervention costs were estimated using a micro-costing approach (ie, data were collected on the exact number and type of each resource consumed). Costs were measured from the perspective of the implementing organization and excluded research-related costs.

Activities required to implement STAR in Tomo were separated into the following: customization (ie, adapting STAR to the context of the organization’s existing policies and regulations, eg, Tomo documents review and tailoring of computer training material); start-up (ie, activities required to begin the intervention, eg, scheduling participatory face-to-face training sessions); and implementation (ie, activities performed as part of delivering the intervention, eg, employees participation in training sessions). Costs were separated into labor (eg, time spent by company staff in intervention-related activities) and nonlabor (eg, contracted services, materials, and space related to the intervention). For customization and start-up activities, the study used semi-structured interviews to assess staff time spent on each activity; for implementation activities, individual-level attendance for all employees and managers were captured in intervention rosters and matched with records on the time spent on each session. Data were gathered on space and materials associated with each activity. See Barbosa et al³⁷ for complete cost study details.

Intervention Benefits

Intervention benefits related to lowering WFC included reductions in self-reported presenteeism, reductions in employee medical care costs, and lower turnover. The three benefits are consistent with the study of Nicholson et al,³⁸ which assumes that a workplace health improvement program provides four potential benefits to employers: better productivity (reflected in lower presenteeism), lower medical expenditures, lower turnover, and fewer absences. This study did not collect data on workdays missed because of a health condition to measure absenteeism. Nevertheless, we deal with this issue in the sensitivity analysis by valuing unused paid time off (PTO), as explained below.

The baseline, 6-, 12- and 18-month postbaseline CAPI surveys collected self-reported measures of workplace outcomes, physical and mental health outcomes, demographic information, and family relationships. Specific measures relevant to the ROI included presenteeism, health care utilization, sex, age, and education. Corporate administrative data were collected on subjects who consented to administrative data collection on an ongoing basis and were merged with CAPI survey data. Administrative data provided information on voluntary termination.

The CAPI survey included the presenteeism questions of the World Health Organization’s Heath and Work Performance Questionnaire (HPQ).^20,39 To measure presenteeism, the HPQ 10-point self-rating scale was used, with 10 representing the best work performance and zero representing the worse job performance anyone could have. We took an approach used in previous studies in which the self-rating represented the proportion of work time that the employee was optimally engaged in his or her job; employees were assumed to be working at 50% capacity for the remaining time.^19,40–43 The self-rating, which represented the past 4 weeks, was scaled to the 6-month CAPI data collection period. Ideally, we would have had access to administrative data on job performance, instead of relying on a self-reported measure of productivity. Nevertheless, scores on global questions about the ability to perform one’s job have been shown to be significantly correlated with business performance scores.³⁹ A review of health-related productivity measurement instruments rated the HPQ very high as a performance indicator, particularly for presenteeism.²⁴ The instrument has excellent reliability, validity, sensitivity to change, and high external validity.^20,44

Intervention benefits related to health care utilization used survey questions about the frequency of health care utilization (hospital nights, emergency department or urgent care visits, and other outpatient visits) over the previous 6 months and whether health care use was related to substance use or mental health. Self-reported health care use is prone to recall bias,⁴⁵ and ideally we would have had access to claims data. Nevertheless, health care utilization questions were taken from the Economic Form 90,⁴⁶ which has shown sensitivity to changes in health care use. Because we did not have an estimate of the number of employees covered by Tomo’s health insurance plan, we used the estimate of a census report that shows that 76.8% individuals with management and professional roles participate in their employers’ health insurance.⁴⁷

ANALYSIS

Dollar Valuation of Intervention Costs and Benefits

Intervention resource use was valued in monetary terms by multiplying the quantity of resources by their unit cost. Staff time spent on STAR activities was valued with employees’ compensation defined as salary costs, available in administrative data, loaded with Tomo’s fringe rate of 30%. Space was valued using the average yearly lease rate, and other nonlabor resources were valued at their market price.³⁷

Presenteeism and voluntary termination were monetized using employees’ compensation. Presenteeism costs were assumed to be half the employee’s hourly compensation for every hour in which the employee was working at 50% capacity. Voluntary termination costs were assumed to be 1.5 times the employee’s total annual compensation, on the basis of Ramall,⁴⁸ who estimates the cost of voluntary termination for companies to be “a minimum of one year’s pay and benefits, or a maximum of two years’ pay and benefits (p. 52).” Other studies have also shown that in most instances the firm’s cost of missed work or productivity loss exceeded the wage rate by a “multiplier” greater than one, depending on the ease of finding an appropriate replacement worker.^49,50 Termination costs were applied to the next CAPI interview that would have occurred had the employee not left Tomo.

Employee health care utilization was valued using unit costs from the literature. Costs of emergency department episodes ($869 in 2011 dollars), outpatient visits ($155), and days spent in the hospital ($1746) were taken from French and Martin.⁵¹ Costs of outpatient visits, specifically related to alcohol, drug use, or mental health ($178), were taken from Roebuck and colleagues’ estimate of the cost of outpatient substance abuse treatment.⁵² Total health care costs were calculated as the sum of all health care utilization costs for that wave.

ROI Estimation

We applied a 3% annual discount rate to costs and benefits accrued in the study’s second year to account for the changing value of a dollar over time and the fact that costs and benefits occurring immediately are valued more highly than those occurring in the future.^53,54 Intervention benefits at 6, 12, and 18 months after intake were summed, yielding a single postintervention measure of benefit for each employee. To calculate the ROI, separate regressions of intervention costs and benefits were estimated. Some individuals assigned to the control condition participated in the intervention, usually because of a postrandomization reassignment to another workgroup within the company; however, they were classified as part of the control group because we conducted an intention-to-treat analysis. Thus, it was necessary to model intervention costs rather than using the average of the treatment group. Intervention costs were regressed on the treatment indicator, sex, age, race, an indicator of college completion, and a set of randomization covariates. Randomization covariates included indicators of the function of the study group, the number of employees in the study group, and the vice president to which the study group reports.¹³ Intervention benefits were regressed on the same covariates, as well as a baseline measure of organizational costs, the number of data collection waves missed not due to voluntary termination, and calendar quarter indicators. Intervention costs were estimated using ordinary least squares regression. Intervention benefits were estimated using marginal effects from Gamma GLM regression with a log-link to account for the strong positive skew in the data, which was not present in the intervention costs.⁵⁵ Both models were estimated with cluster-robust standard errors with the study groups specified as the unit of clustering.

The ROI was calculated as the difference between the intervention effects on organizational benefits and intervention costs, divided by the effect on intervention costs, where intervention effects were represented by the marginal effects on the treatment indicator in the models described above. The confidence interval around the ROI was estimated by sampling clusters (study groups) with replacement and assigning a new cluster identifier to each cluster selected. The main regression models were estimated for each bootstrap replicate, resulting in an estimate of the ROI for each replicate. The 95% confidence interval was the 2.5 and 97.5 percentiles of the distribution of ROI estimates.⁵⁶

Sensitivity Analyses

We analyzed the impact of assumptions and uncertainty in key parameters on the estimated ROI through a series of five sensitivity analyses:

1. As a proxy to absenteeism, we included hours of PTO taken as an additional intervention benefit. We did not have a true measure of absenteeism because our survey did not include the HPQ questions on workdays missed because of health reasons. The measure of PTO reported in administrative data included both sick and personal days. Paid time off used was valued with employees’ compensation. There were 253 employees (27%), with missing data on PTO for one or more observations. We imputed missing values using regression-based conditional mean imputation. Variables were regressed on age, sex, and job title. To incorporate uncertainty, missing values were imputed as the predicted value plus a random draw from the model implied error distribution.⁵⁷

2. We added bonuses gained to employees’ total compensation. Bonuses were not part of employees’ compensation in the main analysis because there were 108 employees (11%), with missing data on bonuses for one or more observations. Values were imputed using the same approach as above.

3. We included the 14 employees with missing data on variables required to calculate intervention benefits. Values were imputed using the same approach as above.

4. We applied a discount rate of 0% and 6%.

5. We adjusted the valuation of intervention benefits upward or downward, one at a time. Presenteeism costs were varied by changing the cost of work time for which self-reported productivity was not optimal by 20%; health care costs were adjusted by varying the unit costs applied by 20%; and turnover costs were varied by changing the wage rate multiplier by 33%.

All analyses were conducted using Stata 13 (StataCorp, College Station, TX). All costs are presented in 2011 US dollars.

RESULTS

Table 1 presents the baseline characteristics of the analytic sample for the control and STAR intervention groups. The typical subject is a white, male, college graduate between 45 and 46 years old. The imbalance between groups on sociodemographic characteristics supports covariate-adjusted ROI estimation. Approximately 20% of the subjects managed other employees.

TABLE 1.

Descriptive Statistics at Baseline

	Control (N = 473) Means (Standard Deviations)a	Intervention (N = 473) Means (Standard Deviations)a
Sociodemographic factors
Male	0.64 (0.48)	0.59 (0.49)
Age*	45.23 (8.77)	46.47 (8.87)
Race
White*	0.67 (0.47)	0.71 (0.45)
Indian*	0.19 (0.39)	0.12 (0.33)
Other*	0.14 (0.35)	0.16 (0.37)
College graduate*	0.81 (0.39)	0.76 (0.43)
Employee characteristics
Costs to employer at baselineb,c	6,005.43 (4,071.46)	5,838.93 (3,726.39)
Manager	0.21 (0.40)	0.22 (0.42)
Salaryb	92,145.48 (19,117.17)	90,850.93 (19,684.39)
Bonusb,d	12,937.45 (4,487.07)	12,810.13 (5,063.67)
Worked per week, past 4 wks, hrs	45.39 (9.18)	45.66 (9.59)
Overall performance self-rating, past 4 wkse	8.18 (1.13)	8.25 (1.01)
PTO taken, past 6 mos, hrs	98.77 (22.51)	100.87 (22.54)
Healthcare utilizationf
Nights in hospital, past 6 mos	0.07 (0.54)	0.09 (0.55)
Visits to emergency department/urgent care facility, past 6 mos	0.10 (0.36)	0.12 (0.38)
Visits to outpatient provider, past 6 mos	1.49 (4.23)	1.60 (3.66)
Instances of outpatient substance abuse or mental health care, past 6 mos	0.25 (1.68)	0.32 (2.01)

^*P < 0.05; tests are independent sample t tests or chi-square tests (race).

^aBased on respondent self-report on 6 months before baseline interview.

^bExpressed in 2011 US dollars.

^cIncludes costs incurred by employer based on presenteeism and health care utilization.

^dMissing values imputed on 11% of participants.

^eUsing a scale of 0 to 10, how would you rate your overall performance on the days you worked during the past 4 weeks?.

^fUnconditional averages.

PTO, paid time off.

Table 2 presents unadjusted average intervention costs and benefits by group. The average cost of the intervention was $707 among employees randomly assigned to the intervention. Because some employees shifted between workgroups after treatment assignment, the average intervention cost among the control group was $15. In the 6 months before the intervention, the firm incurred an average of nearly $6000 per employee in presenteeism and health care. Costs incurred by the firm on the three organizational outcomes averaged $29,952 for the control group and $25,326 for the STAR group in the 18 months after the baseline interviews. Differences between pre- and postintervention costs within each group are explained by the omission of turnover costs in the preintervention period and the difference in the periods covered (6 months vs 18 months). The values in Table 2 do not account for some subjects leaving the study before the end of the data collection period.

TABLE 2.

Unadjusted Means of Intervention Costs and Benefits by Treatment Group^a

	Control (N = 480) Means (Standard Deviations)b	Intervention (N = 480) Means (Standard Deviations)b
Costs of intervention*	14.95 (85.46)	707.48 (259.93)
All organizational cost domains
Preintervention	6,005.43 (4,071.46)	5,838.93 (3,726.39)
Postintervention	29,952.04 (44,884.53)	25,326.43 (37,033.32)
Presenteeism costs
Preintervention	5,666.60 (3,937.19)	5,443.60 (3,533.93)
Postintervention	15,308.44 (9,782.57)	14,804.69 (9,131.45)
Health care costs
Preintervention	338.83 (958.20)	395.33 (951.60)
Postintervention	803.99 (1,667.80)	922.81 (3,134.20)
Costs because of voluntary termination
Postintervention	13,839.61 (46,719.20)	9,598.93 (38,708.40)

^*P < 0.001—tests are independent sample t tests.

^aValues are total costs based on employee self-report in the preintervention (6 months before the intervention) period, or in the postintervention period (18 months after the start of the intervention, collected at 6, 12, and 18 months postbaseline) and are discounted at a 3% annual rate. Turnover is not included in the preintervention period.

^bExpressed in US dollars, 2011 prices.

In the preintervention period, presenteeism costs were considerably higher than health care utilization costs. Control subjects had higher presenteeism costs and lower health care utilization costs, but none of the differences were statistically significant.

Similar to the preintervention period, in the postintervention period, control subjects had higher presenteeism costs and lower health care utilization costs. Control subjects had considerably higher postintervention voluntary termination costs (approximately $4200 higher). Nevertheless, because voluntary termination was a rare event⁵ with a high cost, the standard deviation was quite high and the difference was not statistically significant.

Table 3 presents marginal effects from the regressions of intervention costs and benefits. Model-adjusted average intervention benefits in the postintervention period were $28,497 for the control and $26,647 for the treatment group. Adjusted average intervention costs were $16 and $706, respectively. Differences in these values, which are equal to the marginal effects on the treatment indicators, yield an estimated ROI of 1.68 (95% confidence interval, −8.85 to 9.47).

TABLE 3.

Multivariate Models of Intervention Costs and Benefits^a

Covariate	Organizational Costsb	Intervention Costsc
STAR participant	−1,850.20 (2,547.46)	690.13*** (14.41)
Baseline costs to employer	2.15*** (0.356)
Male	1,922.33 (3,177.68)	−18.99 (12.30)
Age	−705.35*** (176.98)	3.45*** (0.99)
Race
Indian	7,650.25 (4,975.05)	−15.58 (20.32)
Nonwhite, non-Indian	−3,667.41 (3,503.73)	−32.08 (17.12)
College graduate	339.52 (2,656.79)	12.89 (17.76)
Constant		−84.54 (62.25)
Observations	946	946
Adjusted meansd
Control	28,497.02 (2,158.88)	16.15 (6.67)
STAR	26,646.81 (1,582.61)	706.28 (12.18)
ROIe	1.68 (−8.85 to 9.47)

^*P < 0.001.

^aDependent variables expressed in US dollars, 2011 prices.

^bMarginal effects from Gamma GLM regression with cluster-robust standard errors in parentheses. Model also controls for number of missed data collection waves, calendar quarters, and randomization factors.

^cOLS regression with cluster-robust standard errors in parentheses. Model also controls for randomization factors.

^dAdjusted means (standard errors) computed using recycled predictions.

^eROI calculated as ΔE − ΔΨΔΨ where ΔE is the marginal effect of STAR on organizational costs and ΔC is the marginal effect of STAR on intervention costs. The confidence interval (in parentheses) was calculated from the empirical distribution of ROI point estimates using a cluster-robust bootstrap routine with 1000 replications.⁵⁶

ROI, return on investment; STAR, support, transform, achieve, results.

Table 4 presents the results of the sensitivity analyses. Including PTO taken as a benefit along with presenteeism, health care utilization, and turnover decreased the ROI to 1.24. Including the employee bonuses in the total compensation figure raised the average hourly compensation from $57 to $63 and increased the ROI to 2.00. Reintroducing 14 subjects with estimated benefits calculated using imputed values also increased the ROI to 2.02. Removing or doubling the discount rate had a minimal impact on the ROI, increasing or decreasing the estimate by 0.06, respectively.

TABLE 4.

Sensitivity Analyses^a

		Adjusted Meansb
Sensitivity Analysis	N	Control	STAR	ROIc
Include hours of paid time off taken	946	43,735.57	42,190.34	1.24
Include employee bonus in total compensation	946	32,169.69	30,098.49	2.00
Include 14 subjects with missing data on benefits	960	28,580.65	26,498.97	2.02
Change discount rate to 0%	946	29,042.93	27,149.32	1.74
Change discount rate to 6%	946	27,981.94	26,172.67	1.62
Increase costs of presenteeism by 20%	946	31,545.30	29,708.14	1.66
Decrease costs of presenteeism by 20%	946	25,469.02	23,570.17	1.75
Increase costs of health care utilization by 20%	946	28,673.27	26,827.92	1.67
Decrease costs of health care utilization by 20%	946	28,327.54	26,469.26	1.69
Increase costs of turnover to two times total compensation	946	32,653.15	30,099.41	2.70
Decrease costs of turnover to 1 time total compensation	946	24,365.79	23,151.75	0.76

^aResults from regression models with adjusted dependent variables. Dependent variables are expressed in US dollars (2011 prices).

^bAdjusted means of organizational costs computed using recycled predictions.

^cROI calculated as $\frac{\mathrm{\Delta }E-\mathrm{\Delta }C}{\mathrm{\Delta }C}$, where ΔE is the marginal effect of STAR on organizational costs and ΔC is the marginal effect of STAR on intervention costs.

ROI, return on investment; STAR, support, transform, achieve, results.

The organizational outcome with the highest impact on the ROI was voluntary termination. When voluntary termination costs were increased to two times total annual compensation, the ROI increased to 2.70. When they were decreased to 1 time total annual compensation, the ROI decreased to 0.76. Changing presenteeism or health care utilization costs by 20% in either direction had a very small impact on the ROI estimate.

DISCUSSION

This article presented the ROI of a multicomponent intervention delivered in a group-randomized, 18-month, field experiment designed to reduce employees’ WFC and improve the health and well-being of workers and their families. The main analysis accounted for differences between trial arms in intervention costs and three benefits: presenteeism, voluntary termination, and health care utilization. Consistent with economic theory, the first two organizational outcomes were monetized using employees’ wage.⁵⁸ To increase the accuracy of employers’ “value,” we also included the cost of fringe benefits and bonuses (in sensitivity analysis) as a measure of total compensation. To test the sensitivity of our results to model assumptions and address the uncertainty in some parameters, we conducted several sensitivity analyses. We estimated the 95% confidence interval of the ROI, accounting for the cluster-randomized design of the study.

After adjusting for baseline differences, the intervention led to company savings of $1850 per participant, over an 18-month period. The overall ROI was 1.68 (95% confidence interval, −8.85 to 9.47), indicating that on average organizational costs fell by $1.68 for every $1.00 spent on STAR. We found that the intervention benefit with the highest impact on the ROI was voluntary termination followed by presenteeism and health care utilization. Voluntary termination has the highest monetary valuation and, despite being a rare event, the impact on the overall ROI is substantial. Including PTO taken as a proxy to absenteeism resulted in a lower ROI. This might be explained by a higher number of personal days, not sick days, taken by the intervention group. Nevertheless, the data did not support disentangling the two. Our main analysis adopted a conservative approach whereby we excluded individuals with missing values in intervention benefits and did not include bonus in employees’ total compensation. Relaxing each of those assumptions resulted in a slightly higher estimate of the ROI.

This study advances the field of economic evaluations of workplace interventions, in general, by presenting the confidence interval around the ROI estimated in the context of a clustered experimental study and, in particular, by providing the first estimate of the ROI of a workplace intervention to reduce WFC. Systematic reviews of the financial return of worksite health promotion programs show that few, if any, studies conduct sensitivity analyses or report the confidence intervals of their financial return estimates.^4,59

Although no previous study reports the ROI of initiatives to reduce WFC, our results can be put into the context of previous ROI studies of other health-related interventions implemented in the workplace. As previously noted,³⁷ most studies analyzing the ROI of a workplace intervention have not estimated the cost of the intervention with the rigor that this study does. The adjusted cost of STAR was $690 per participant, which is about four times more than the median costs of workplace interventions described in the literature, and may explain the lower ROI we found.⁴ The reason for this is twofold. First, our analysis of intervention costs included costs beyond implementation costs, such as start-up, customization, space, and employee time,³⁷ which have not been taken into account in previous studies. Second, STAR was more resource intensive than other workplace interventions. A recent meta-analysis of the literature on costs and savings associated with employer-based wellness promotion policies found that the majority of the programs focused on weight loss and fitness. These programs entailed a combination of self-help education materials, individual counseling with health care professionals, or on-site group activities led by trained personnel.²⁸ Such interventions do not typically require the extensive employee and resource involvement that STAR did.

Despite the differences in the programs analyzed and methods applied, our ROI estimate is within the range of ROI estimates of workplace interventions in the literature. Baicker et al,²⁸ focusing only on health care costs and absenteeism, reported separate ROIs of 3.27 for health care costs and 2.73 for absenteeism. van Dongen et al⁴ conducted a systematic review of 13 nonrandomized studies (NRSs) and 4 randomized controlled trials (RCTs) on the financial return of worksite health promotion programs aimed at improving nutrition and/or increasing physical activity. The review reported that the ROI for absenteeism was 3.25 for NRSs and −0.49 for RCTs, the ROI for health care was 0.95 for NRSs and −1.12 for RCT, and for both absenteeism and health care the ROI was 3.87 for NRSs and −0.92 for RCTs. The authors called for more RCTs of workplace interventions and expressed concern that the cost savings and high ROI of the NRSs reviewed were likely the result of selection bias. The authors also suggested that longer follow-up duration of the NRSs, combined with the potential for gradually accumulating benefits, may have contributed to larger ROIs than in the RCTs. In contrast to this study, which included three organizational outcomes, the studies reviewed only included absenteeism and/or medical benefits, neglecting other types of financial benefits such as reduced presenteeism and turnover. In this study, unadjusted health care costs after STAR were slightly higher in the intervention group. An increase in health care utilization after STAR might be related to both greater schedule control (a primary target of the intervention) that facilitates health care appointments and the training message that managers and coworkers support employees’ pursuit of their health and their personal goals. This finding is consistent with previous workplace studies.^4,28

STAR yielded a positive ROI, showing that employer costs of the intervention were more than offset by cost savings, a result that was robust in sensitivity analyses. Although this result was not statistically significant by common standards, the robustness of the positive ROI estimate to a variety of sensitivity analyses increased confidence that STAR yielded a positive ROI. Moreover, no other study has presented confidence intervals around an ROI point estimate, so we cannot compare the significance of our results to the significance of ROIs previously reported, regardless of their magnitude. Despite the statistical insignificance, our ROI estimate is economically meaningful by most financial assessment standards. Furthermore, the nonsignificant ROI estimate can partly be attributed to the study being powered to assess the impact of the intervention on WFC (for which it was significant³⁶) rather than for organizational outcomes. The study was powered to ensure the identification of the effect of the intervention on WFC at α=0.05, with power of 0.8. For this reason, expecting not just a positive but also a significant ROI for STAR may be too high a threshold.

Five main limitations should be pointed out. First, this study may have understated the true ROI because health benefits are likely to accumulate gradually and a longer follow-up period might have captured a greater extent of intervention benefits as a result of a reduction in WFC. Similarly, we may overstate the true costs of STAR by including start-up and customization costs. If these costs are greater in the first worksite to implement STAR, which seems likely, then we have overstated the costs that would be incurred by worksites alike that implement STAR in the future. Second, health care and presenteeism outcomes were self-reported, and unit costs were derived from the literature. Two superior sources of health care cost data–-medical claims or industry-specific averages–-were unavailable. Third, we relied on self-reported productivity data. It is noteworthy, though, that previous longitudinal research has consistently shown that the HPQ self-reported productivity measure significantly predicts administrative records of work performance.^20,39 Fourth, despite the randomized nature of this study, there was some imbalance in demographic characteristics (age, race, and education). Although it is possible that our sample inclusion criteria created the imbalance, we found no evidence that selection criteria differentially affected one condition over another at baseline. In another analysis, we found that the intervention reduced voluntary turnover (Moen et al⁵), which is consistent with our results here, but this would not explain baseline differences across the study conditions. Thus, although we used regression models to adjust outcomes for the imbalance, some selection bias may be present. Finally, we did not have data on absenteeism, which was excluded from the main analysis. We included PTO used as a proxy to absenteeism in the sensitivity analysis.

The RCT design of the study attempted to ensure internal validity, not external validity, which might hamper the generalizability of our results. Tomo was characterized by a high-salary, highly educated workforce, which might be in line with many other information technology companies but not the general workforce. The extent to which our results can be translated to other industries depends on the similarity of the workplace environment, specific industry characteristics, and on whether the same implementation strategy is and can be followed. To partly address concerns that findings are not generalizable beyond a single industry or type of workforce, future work by the Work, Family, and Health Network will examine the ROI of a WFC intervention in a nursing care facility. A salaried professional workforce like Tomo may have different underlying health risks and may respond to the intervention differently than a predominantly low-wage workforce in a nursing care facility.

This study shows that employers’ investment in an intervention to reduce WFC can enhance their business. It provides further evidence to policy makers who are interested in aligning the social welfare objective of improving workers’ health and well-being with the need to make the business case to employers. This is particularly relevant in the current climate of health care reform in the United States where the Affordable Care Act contains provisions that encourage employers to adopt health promotion and risk reduction programs.