Validation of a New Metric for Assessing the Integration of Health Protection and Health Promotion in a Sample of Small and Medium Size Employer Groups

Abstract

Objective

To conduct validation analyses for a new measure of the integration of worksite health protection and health promotion approaches developed in earlier research.

Methods

A survey of small to medium size employers located in the United States was conducted between October 2013 and March 2014 (N=111). Cronbach’s Alpha coefficient was used to assess reliability and Pearson correlation coefficients were used to assess convergent validity.

Results

The Integration Score was positively associated with the measures of occupational safety and health and health promotion activities/policies–supporting its convergent validity (Pearson correlation coefficients of 0.32–0.47). Cronbach’s alpha coefficient was 0.94, indicating excellent reliability.

Conclusions

The Integration Score appears to be a promising tool for assessing integration of health promotion and health protection. Further work is needed to test its dimensionality and validate its use in other samples.

Introduction

The prospect that using an integrated approach to protection and promotion of worker health could improve worker health and wellbeing and prevent injury and illness is the primary motivation for the National Institute for Occupational Safety and Health’s Total Worker Health™ initiative. (1–3) Sorensen, et al., define an integrated approach as a “strategic and operational coordination of policies, programs and practices designed to simultaneously prevent work-related injuries and illnesses and enhance overall workforce health and well-being.” (4) The evaluation of this approach necessitates the assessment of both occupational safety and health (health protection) and health promotion programs, practices, and policies, as well as the level of integration.

While many studies have endeavored to measure the effectiveness of specific occupational safety and health interventions and health promotion programs separately, very few measure these efforts concurrently. (5–8) Even less work has attempted to measure integration directly, especially within small to medium sized businesses. One study that used a random sample of Massachusetts employers found that 28% of respondents reported coordinating occupational safety and health and health promotion efforts (always or often).(9) However, as the authors noted, it was unclear how respondents might have understood the extent of coordination.(9)

While definitions of integration have been specified in the literature and several studies have begun to test integrated approaches, no common practical measure of integration has been validated particularly for small to medium-sized businesses.(4) A validated measure of integration would improve dialogue among researchers, facilitate the research-to-practice process and define optimal best practice. (4) This study uses a measure of integration of health protection and health promotion developed in earlier work and assesses its reliability and convergent validity in a sample of small to medium sized firms, defined as (<750 employees). (4) The vast majority of firms in the U.S. have less than 750 employees, and these firms are also less likely to provide certain benefits to their employees—increasing the value of a tool to measure integration (7, 10). As described in Sorensen et al. 2013, the measure was created using a modified Delphi process. (4)

In this paper, we evaluate the new measure’s internal consistency and its convergent validity with respect to measures of the extent and capacity of health protection and health promotion. The number of activities/programs and policies (such as a way for employees to report safety issues or a written no smoking policy) is calculated separately for health protection and health promotion. Capacity is measured with questions on whether the firm had a dedicated budget, dedicated staff person, and a committee, calculated separately for health protection and health promotion. We hypothesize that: (1) organizations with more extensive health protection and health promotion programming would have higher integration scores, and (2) organizations with higher health protection capacity and health promotion capacity would have higher integration scores.

Methods

Study Design

The web-based survey of small to medium employers (<750 employees) used in this study was conducted between September 2013 and March 2014. The survey took participants approximately 15 minutes to complete, and respondents were offered a $25 Amazon gift card as a participation incentive. Lists of human resource directors/managers obtained from an insurance brokerage firm in Minnesota and the Minnesota Chamber of Commerce comprised the survey sample of respondents. The survey was given to 400 small- and medium-size businesses with up to 3 additional electronic and phone attempts to reach non-respondents; subsequently surveys were sent in the mail to the non-respondents one time. Detailed information on the design and conduct of the survey can be found in McLellan et al. (11) Approval was obtained from the IRB at the Harvard T. H. Chan School of Public Health.

Quantitative Variables

Integration of Health Protection and Health Promotion

The measure of integration of health protection and health promotion has several domains as described by Sorensen et al. (4) These domains include both theoretically and practically relevant aspects of integration: organizational leadership and commitment; coordination of worksite efforts to protect and promote worker health, safety, and wellbeing; supportive organizational policies and practices; accountability and training; ‘management and employee engagement’; benefits and incentives to support health protection and health promotion; integrated evaluation and surveillance; and comprehensive program content. The full measure of integration is available in in Table 1 and Sorensen et al. (4) One question from the original measure, whether workers are actively engaged in planning and implementing health promotion and occupational safety and health programs and policies, was inadvertently omitted from the survey. Additional questions on the survey not related to the measure of integration were informed by qualitative interviews with HR Directors and Safety Managers and adapted from prior surveys conducted to determine the characteristics of health protection and health promotion programs: including the Massachusetts Department of Public Health survey, the 2004 National Worksite Health Promotion Survey, and an adaptation of an Occupational Safety and Health Administration 1995 Program Evaluation Profile used in previous work. (4, 9, 12–14)

Table 1.

Responses to Questions¹ about the Extent of Integration between Health Protection and Health promotion Programs/Policies among Participants (n=111).

Question Construct	Percent2
	Absent	Partially   Adopted	Fully   Achieved
Organizational leadership and commitment
(1) Top management expresses its commitment to a culture   of health and an environment that supports employee   health	17.1%	69.4%	13.5%
(2) Both worker and worksite health are included as part of   the organization’s mission	57.7%	31.5%	10.8%
(3) Senior leadership allocates adequate human and fiscal   resources to implement programs to promote and protect   worker health	33.3%	51.4%	15.3%
Coordination between health protection & promotion
(4) Decision making about policies, programs, and practices   related to worker health is coordinated across departments,   including those responsible for occupational safety & health   and those responsible for worksite wellness	28.8%	56.8%	14.4%
(5) Processes are in place to coordinate and leverage   interdepartmental budgets allocated toward both worksite   wellness and occupational safety and health.	58.6%	34.2%	7.2%
(6) Efforts to promote and protect worker health include   both policies about the work organization and environment   and education and programs for individual workers	33.3%	47.8%	18.9%
Processes for accountability & training
(7) Program managers responsible for worksite wellness and   OSH are trained to coordinate and implement programs,   practices and policies for both worksite wellness and   occupational safety and health	43.2%	46.9%	9.9%
(8) Operation managers are trained to ensure employee   health through coordination with and support for   occupational safety and health and worksite wellness	44.1%	39.6%	16.2%
(9) Job descriptions for staff responsible for worksite   wellness and occupational health and safety include roles   and responsibilities that require interdepartmental   collaboration and coordination of worksite wellness and   occupational safety and health programs, policies, and   practices	56.8%	27.9%	15.3%
(10) Performance metrics for those responsible for worksite   wellness and occupational safety and health include success   with interdepartmental collaboration and coordination of   worksite wellness and occupational safety and health   programs, policies, and practices	64.0%	32.4%	3.6%
(11) Professional development strategies include training   and setting goals at performance reviews related to   interdepartmental collaboration and coordination of   worksite wellness and occupational safety and health   programs, policies, and practices	63.1%	34.2%	2.7%
(12) Worksite wellness and occupational safety and health   vendors have the experience and expertise to coordinate   with and/or deliver approaches that support the   coordination and collaboration of workplace wellness and   occupational safety and health efforts	45.1%	38.7%	16.2%
Coordinated management & employee engagement strategies
(13) Both managers and employees are engaged in decision-   making about priorities for coordinated worksite wellness   and occupational safety and health programs, policies, and   practices	39.6%	46.0%	14.4%
(14) Joint worker-management committees addressing   worker and worksite health reflect both worksite wellness   and occupational safety and health.	48.7%	38.7%	12.6%
Benefits & Incentives to support workplace health promotion and   protection
(15) Incentives are offered to employees to complete   activities to stay healthy (e.g. attend a training on   health/safety), reduce their high risk behavior (e.g. quit   smoking), and/or practice healthy lifestyles (e.g. gym   membership discounts)	46.9%	33.3%	19.8%
(16) Incentives are offered to managers who protect and   promote health (e.g. accomplish health and safety in their   departments and encourage reporting of hazards, illnesses,   and injuries, and near misses; lead and encourage their   employees in health promotion and protection efforts)	84.7%	10.8%	4.5%
(17) Workplace benefits exist that address health, safety, and   well-being (e.g. health care coverage, flex-time, paid sick   leave, screening and prevention coverage, wellness   opportunities)	17.1%	45.1%	37.8%
Integrated evaluation & surveillance
(18) The effects of worksite wellness and occupational   safety and health programs are monitored jointly	57.7%	36.9%	5.4%
(19) Data related to employee health outcomes are   integrated within a coordinated system	76.6%	18.0%	5.4%
(20) High-level indicator reports (e.g., “dashboards”) on   integrated programs are presented to upper level   management on a regular basis, while protecting employee   confidentiality	74.8%	18.9%	6.3%
Comprehensive program content
(21) The content of educational programs, such as classes,   online courses or webinars, or toolbox talks, addresses   potential additive or synergistic risks posed by exposures on   the job and risk-related behaviors	57.7%	32.4%	9.9%
(22) The content of educational programs, such as classes,   online courses or webinars, or toolbox talks, acknowledges   the impact of job experiences and the work environment   on successful health behavior change	54.1%	37.8%	8.1%

Notes: Totals may not sum to exactly 100% because of rounding. This table contains the frequencies of response to questions measuring the level and dimensions of the integration between occupational safety and health programs and policies and health promotion programs and policies. This web-based survey of small to medium employers (<750 employees) was conducted between September 2013 and March 2014.

Number of Activities/Policies

The number of health protection activities/policies is measured by summing responses to the following questions about the following topics: existence of system/program, updated on regular basis, written statement, management sets goals, managers/supervisors directly accountable, way for employees to report safety issues, feedback to employees who report, hourly employees provided with training, orientation process, and training for supervisors. The number of health promotion programs/policies is measured by summing responses to questions that addressed the following topics: written policies for no smoking, alcohol, drugs, employee counseling, seat belt use at work, prohibiting firearms at work, fitness breaks, and healthy food options; programs for health screenings, health risk assessment, physical activity, on-site education, individual coaching, and whether an employee assistance program exists; and environment including existence of a cafeteria, labeling food items for healthy choices, vending machines, special promotions to encourage health food choices, and onsite showers.

Capacity

Capacity for health promotion is measured with questions on whether the firm had a dedicated budget, dedicated staff person, and a committee for health promotion. These measures have been used in previous work. (11, 15) An analogous set of questions is used to measure capacity for health protection, because there were no existing measures for assessing capacity for health protection.

Statistical methods

In order to develop a numeric score for the measure of integration, the answers to each question in the measure are coded so that higher numerical scores indicated greater integration between health protection and health promotion. Answers of “absent” are scored as 0, answers of “partially adopted” are scored as 1, and answers of “fully adopted” are scored as 2. Adding together the responses for each question in the measure yields a theoretical range of 0 to 44 and will hereafter be referred to as the Integration score.

In these analyses we consider the Integration score as having a single dimension and therefore characterize its reliability, or internal consistency, with Cronbach’s Alpha coefficient for all questions on the survey. Convergent validity is assessed by associations between the Integration score and concepts hypothesized to be associated with it. These include the number of health protection activities and policies, the number of health promotion activities and policies, the health protection and health promotion capacity (i.e. committee, budget, dedicated staff for health protection and health promotion). The numbers of programs/policies and the measure of capacity are each the sums of the relevant items. We use Pearson Correlation Coefficients to characterize these associations, with Bonferroni corrections to account for multiple comparisons. All analyses were conducted using STATA 13.1. (16)

Results

Participants

The survey was distributed to 400 companies. The total response rate was 117 of 400 or 29%, but, after excluding respondents who did not answer the questions about integration our final number of observations was 111. The average number of employees at enterprises used in the analysis was 163 (SD 149, Range [7, 735]). Companies were in industries such as professional, scientific, and technical services; healthcare and social assistance, manufacturing, among many others. Full details are available in McLellan et al. (11)

Descriptive Statistics

Survey respondents were asked whether the measures of integration were absent, partially adopted, or fully adopted in their organization. Question text and summary statistics for these responses are given in Table 1. The indicator of integration with the least implementation was for “incentives are offered to managers who protect and promote health,” and the indicator with highest level of full implementation was “the content of education programs … acknowledges the impact of job experiences.”

Figure 1 shows the distribution of the Integration score. There was a large range of actual scores, with most of the sample falling below half of the theoretical range. The average score in the sample was 13.66 with a standard deviation of 9.59.

Figure 1.

Response Distribution for Integration Score (N=111).

Notes. The index was calculated as the sum of responses for relevant questions with Absent =0, Partially Achieved =1 and Fully Achieved =2. The index’s theoretical range is [0, 44].

The average number of health protection programs and policies was 8.16 (SD 2.46) in the sample. The average number of health promotion programs/policies in the sample was 9.15 (SD 2.99). The number of health promotion programs/policies ranged from 3 to 18 in the sample. The average capacity of health protection was 1.64 (SD 1.03) and health promotion was 0.93 (SD 1.16).

Main Results: Reliability & Convergent Validity

Cronbach’s alpha for the Integration score was 0.94. The average inter-item correlations ranged from 0.17 to 0.19. The Pearson correlation coefficients for the Integration score and the number of occupational safety & health and health promotion programs/policies are given in Table 2. The Integration score was positively correlated with the numbers of programs and policies for both health protection and health promotion; the correlation coefficients were 0.40 and 0.45, respectively. The parallel results for Pearson correlation coefficients between the integration score and the measured health protection and health promotion capacities are given in Table 3. The indicators of integration score was positively correlated with both measures; the correlation coefficient was 0.47 with respect to health protection capacity and was 0.32 with respect to health promotion capacity.

Table 2.

Pearson Correlation Coefficients¹ for Integration Score, Number of Health Protection Programs/Policies, and Number of Health Promotion Programs/Policies

	Integration score	Number of Health Protection Programs/Policies	Number of Health promotion Programs/Policies
Integration score	1
Number of Health Protection Programs/Policies	0.40**	1
Number of Health promotion Programs/Policies	0.45**	0.10	1

Notes:

¹Bonferroni correction for multiple comparisons.

^**Indicates significance at the 1% level.

Table 3.

Pearson Correlation Coefficients for Integration Score, Health Protection Capacity, and Health promotion Capacity

	Integration score	Health Protection Capacity	Health promotion Capacity
Integration score	1
Health Protection Capacity	0.47**	1
Health promotion Capacity	0.32**	0.20	1

Notes:

¹Bonferroni correction for multiple comparisons.

^**Indicates significance at the 1% level.

Discussion

These analyses illustrate the reliability and convergent validity of a set of questions designed to measure the integration of health protection and health promotion programs and policies, the Integration score, in a sample of small to medium size employers. The Cronbach’s alpha was high—indicating internal reliability and that is might be possible to reduce the measure’s length while maintaining internal reliability. (17) The Pearson correlation coefficients indicated moderate correlation between related concepts and were statistically significant at the 5% level, as expected. The range of Integration scores indicates considerable variation in adoption of integrated approaches as well as the potential for greater integration even among high-scoring organizations. The variation found in this sample of small and medium sized employers provides evidence that integration occurs in a variety of ways that differ across organizations.

Being able to measure the integration of health protection and health promotion programs accurately and succinctly will improve future research seeking to validate the concepts underlying Total Worker Health™. The underlying motivation for studying the integration of health protection and health promotion is driven by suggestive evidence that integrated programs are more effective and have higher participation rates. (13, 18–22)

Accurate and practical measurement of the concept of integration will allow a standardized definition of integration, as well as increase the feasibility multi-employers surveys. Using a score with separate items for potential areas of integration will allow future analyses of the components of integration as well as the variability in the extent of integration. Because of its ease of administration, the indicators of integration measure is a practical assessment tool to use for organizations that desire to assess or track progress toward the integration of health protection and health promotion programs and policies. A useful feature of the measure is that it may be administered by interview or in a written format. The index questions themselves may suggest to organizations potential changes that would lead to greater use of integrated approaches. Examining the extent to which organizations use the indicators of integration to guide the development of programs and policies would be a beneficial avenue of future work.

The sample’s limited geography is the major limitation of these analyses. It is possible that the convergent validity and reliability properties of the index might vary in other samples. Because of limitations in the scope of the survey, no analyses were possible to determine the discriminant validity of the measure.

From the research perspective, these analyses are a useful first step that will inform future research on integrated approaches by allowing for standardized measurement of integration. Ongoing qualitative research will add depth to our understanding of how integrated programs and policies are viewed by small-to-medium sized businesses and how interventions to increase integration are engaged. Additional work to test the measure in larger samples and other geographic regions will increase the scale’s applicability to measure integration in different contexts.