Validation of the Workplace Integrated Safety and Health (WISH) assessment in a sample of nursing homes using Item Response Theory (IRT) methods

Abstract

Objectives

To validate and test the dimensionality of six constructs from the Workplace Integrated Safety and Health (WISH) assessment, an instrument that assesses the extent to which organisations implement integrated systems approaches for protecting and promoting worker health, safety and well-being, in a sample of nursing homes in the USA.

Design

Validation of an assessment scale using data from a cross-sectional survey.

Setting

Nursing homes certified by the Centers for Medicaid and Medicare services in three states of the USA: Ohio, California and Massachusetts.

Participants

569 directors of nursing from nursing homes serving adults and with more than 30 beds participated in the study.

Results

Graded response Item Response Theory (IRT) models showed that five out of six constructs were unidimensional based on balanced interpretation of model fit statistics—M2 or C2 with p value >0.05, Comparative Fit Index >0.95, lower bound of the root mean squared error of approximation 90% CI <0.06 and standardised root mean square residual <0.08. Overall measure and construct reliability ranged from acceptable to good. Category boundary location parameters indicated that items were most informative for respondents in lower range of latent scores (ie, β₁, β₂, β₃ typically below 0). A few items were recommended to be dropped from future administrations of the instrument based on empirical and substantive interpretation.

Conclusions

The WISH instrument has utility to understand to what extent organisations integrate protection and promotion of worker health, safety and well-being; however, it is most informative in organisations that present lower scores.

Strengths and limitations of this study.

• The sample was large and heterogeneous: Validation of the Workplace Integrated Health and Safety (WISH) assessment used survey data from 569 nursing homes in three states of the USA: Ohio, California and Massachusetts.

• The respondents were knowledgeable: Directors of nursing responded on behalf of nursing homes as previous studies showed that they are more knowledgeable about policies, practices and programmes than any other manager; directors of nursing had the option to pass the survey to a health and safety representative at the facility if they judged them to be more knowledgeable about the survey topic.

• The methods for selecting items was rigorous: The study used graded response Item Response Theory modelling to identify items for unidimensional WISH domains based on discrimination parameters, and information curves to assess conditional reliability along the latent scores.

• The generalisability of the measure is limited by the study sample which included nursing homes with at least 30 beds located in the USA.

Background

Increasing evidence supports the integration of protection and promotion of workers’ health, safety and well-being as best practices to achieve a healthy workplace.^1–3 Accordingly, the National Institute for Occupational Health and Safety, the agency focused on worker safety and health from the US Centers for Disease and Control Prevention, developed the Total Worker Health (TWH) Initiative that has as a goal to promote worker well-being by focusing on ‘programmes, policies and practices that integrate protection from work-related safety and health hazards with promotion of injury and illness prevention efforts to advance worker well-being.’⁴ Understanding the scope of TWH approaches, the extent to which they are implemented and how they relate with workers’ health, safety and well-being is essential for employers and for occupational health and safety organisations in setting priorities when developing interventions, policies and programmes.

Investigators from the Harvard T.H. Chan School of Public Health Center for Work, Health and Well-being (CWHW) have conducted research on the causal pathways through which policies, programmes and practices impact workers’ health, safety and well-being outcomes.2 3 5 6 Evidence from this research informed the development of a conceptual model that serves as a framework to conduct research and workplace interventions using TWH approaches.³ The conceptual model underscores the central role of working conditions and their impact on worker (health and well-being) and enterprise outcomes such as turnover and healthcare costs and the role that policies, programmes and practices have in shaping these working conditions.³ This conceptual model, related literature, and an expert panel guided the development of two measures to assess the extent to which organisations implement evidence-based integrated systems approaches for protecting and promoting worker health, safety and well-being: the Workplace Integrated Safety and Health (WISH) assessment⁷ and, its predecessor, the Indicators of Integration.²

The Indicators of Integration were validated in a sample of small-sized to medium-sized employers^{8 9} and a sample of the Department of Veterans Affairs (VA) administrative parents in the USA showing satisfactory convergent validity and reliability.^{8 9} However, the Indicators of Integration were primarily designed to measure the integration of health protection and health promotion. The WISH is more comprehensive and measures additional policies, programmes and practices that affect worker safety, health and well-being.⁷ It consists of six constructs that were identified as central in the integration of protection and promotion of workers’ health, safety and well-being: Leadership Commitment; Participation; Policies, Programmes and Practices that foster positive working conditions; Comprehensive and Collaborative Strategies; Adherence to federal and state regulations and ethical norms; and Data-driven Change. The WISH tool aims to assess the extent to which organisations adhere to best practice recommendations in terms of an integrated systems approach of programmes, policies and practices that address protection and promotion of health, safety and well-being.

This study aims to validate each domain comprising the WISH instrument in the nursing home industry. Specifically, we test each domain in the WISH assessment instrument as a unidimensional latent construct, estimate discrimination and location parameters, and inspect information curves to assess the conditional reliability and utility of each scale along the respective latent scores. The nursing home industry is of interest as its main goal is to provide high-quality patient care to an increasing ageing population. However, in the USA, the industry is sustained by a low-wage labour force and under-represented minorities and is largely dependent on payments from Medicaid and Medicare,^{10 11} which are federal health programmes for low-income individuals and individuals over age 65. Evidence from research on nursing home facilities shows that placing workers’ health as secondary to patient care may have deleterious effects on quality of patient care by increasing employee turnover.^{10 12} Measuring the implementation of integrated approaches to protect and promote workers’ health, safety and well-being in a nursing home population could help us understand how TWH approaches are associated with workers’, patients’ and enterprise outcomes.

Methods

The present study used data from the Enterprise Outcomes (EO) Study, a project from the Harvard T.H. Chan School of Public Health CWHW designed to assess TWH implementation in nursing homes and its relationship with worker and enterprise outcomes. The study uses a cross-sectional survey of directors of nursing (DONs) in nursing homes across three states in the USA: Ohio (OH), California (CA) and Massachusetts (MA). The survey included the WISH assessment and additional related questions.

Instrument

The items comprising the WISH constructs were developed through an extensive literature review and modified Delphi approach to ensure content validity, and cognitive pretesting with nursing home directors to ensure suitability, comprehension, information retrieval and response mapping.⁷ Methods on the approach are detailed elsewhere.⁷ The WISH assessment consists of six constructs ranging from 4 to 14 items with closed-ended questions and four-point ordinal response scale (not at all, somewhat, mostly, completely/not at all, some of the time, most of the time, all of the time) (table 1).

Table 1.

Domains and their definitions of the Workplace Integrated Safety and Health (WISH) assessment

Domain name	Definition	Number of questions
Leadership Commitment	Leadership makes worker safety, health and well-being a clear priority for the entire organisation. They drive accountability and provide the necessary resources and environment to create positive working conditions.	6
Participation	Stakeholders at every level of an organisation, including labour unions or other worker organisations if present, help plan and carry out efforts to protect and promote worker safety and health.	5
Policies, Programmes and Practices focused on positive working conditions	The organisation enhances worker safety, health and well-being with policies and practices that improve working conditions.	14
Comprehensive and Collaborative Strategies	Employees from across the organisation work together to develop comprehensive health and safety initiatives.	6
Adherence	The organisation adheres to federal and state regulations, as well as ethical norms, that advance worker safety, health and well-being.	5
Data-driven Change	Regular evaluation guides an organisation’s priority setting, decision-making and continuous improvement of worker safety, health and well-being initiatives.	4

Survey administration

All 2388 nursing homes in OH, CA and MA with at least 30 beds, serving adults and listed in Nursing Home Compare as accepting Medicare or Medicaid on 23 August 2018, were invited to participate in the survey (Nursing Home Compare (https://www.medicare.gov/nursinghomecompare). The states of OH, CA and MA were chosen for the survey as part of a larger project that links nursing home data on injuries from state Bureaus of Worker’s Compensation. DONs were initially invited to respond to the survey based on previous evidence indicating that nursing home directors are more knowledgeable about workplace policies, programmes and practices than any other managers in the organisation.¹⁰ Their knowledge is due to their involvement in the development and implementation of nursing home policies and procedures¹³ and exposure to the clinical operations of nursing homes.¹⁴ However, DONs were instructed to seek input or pass the survey to an appropriate health and safety representative for their nursing home, if needed. Data were collected via a sequential mixed mode survey, where a web survey protocol was implemented followed by a mail survey among non-respondents. An initial invitation was sent to DONs via email at each nursing home. The invitation explained the purpose of the survey, that the survey was voluntary and confidential, and that the survey was intended to be completed by the DON or a health and safety representative if they choose to pass it along. Up to three reminder emails were sent to non-respondents, followed by a mailed paper survey. There were some DONs without working email addresses—these individuals only received paper copies. Five hundred sixty-nine surveys were returned for a 24% response rate. A low response rate may be indicative of non-response bias; however, a previous survey study with Chiefs of Medicine showed that answers from a first survey wave with 29.7% response rate did not represent any statistical significant differences from answers collected during the first to the fifth survey waves that achieved an 85% response rate.¹⁵ The only parameter that changed was precision which increased as respondents accumulated.¹⁵ In other analyses (not shown), we used nursing home characteristics, such as for-profit status, state, number of beds, Medicare quality rating, Medicare staffing rating, Medicare health inspection rating, survey wave and rurality, to predict whether a nursing home responded to the survey.¹⁶

Statistical analysis

Data were initially characterised by frequencies to identify items with severely skewed distributions. We collapsed response categories with an adjacent category if fewer than 10% of respondents selected it. We analysed each domain of the WISH separately because they represent different latent constructs. We chose IRT analysis because it characterises the relationship between items and their respective constructs on several important parameters—discrimination (ie, strength of relationship of item with construct), category location (ie, location along the construct score where), and their information or reliability (ie, their contribution to measurement precision).

For each domain, we fitted unidimensional graded response models to select items based on empirical information (discrimination and local dependence) and, importantly, substantive interpretation to ensure content validity. Items were flagged for discussion if they had small discrimination parameters relative to the rest of the items in their domains or exhibited local dependence. Local dependence was assessed using the G² statistic, with items flagged if the p value was below Bonferroni-corrected alpha of 0.05. When local dependence was identified in a pair of items, the item with greater discrimination was retained unless the other item better ensured content validity. Likewise, items with smaller discrimination relative to other items were retained if their elimination compromised content validity. Substantive interpretation was conducted by the study team with input from other investigators at the Center for Work, Health, and Well-being.¹⁷

We examined measurement precision—which in IRT is summarised with information curves where higher numbers indicate better precision or, equivalently, greater information—for each construct across the range of their latent scores. We also present the empirical reliability as a measure of the overall reliability of the latent scores.

Limited information goodness-of-fit χ² statistic (M2 or C2 depending on model df), root mean square error of approximation (RMSEA), standardised root mean square residuals (SRMSR) and Comparative Fit Index (CFI)¹⁸ were used to evaluate fit of unidimensional model for each construct. These fit indices evaluate models relative to a saturated model (M2 and C2), improvement relative to a null model (CFI), as well as absolute measures of fit (RMSEA and SRMSR). We use Hu and Bentler’s (1999) threshold recommendations to consider good model fit—lower bound of the 90% CI for RMSEA (RMSEA LB)<0.06, SRMS <0.08 and CFI>0.95. Item fit was assessed by the signed χ² test and by interpreting empirical plots.¹⁹ The analyses were conducted using the mirt package in R, using marginal maximum likelihood via Expectation Maximisation estimation²⁰ (R Core Team, 2019).

Patient or public involvement

No patients were involved in this study.

Results

Sample

Table 2 depicts the characteristics of the participating nursing homes. Most surveys were completed by the DON (n=480, 84.4%)%), 7.6% (n=43) reported a title other than DON (13 were administrators and 11 were assistant/interim DONs). Response rates did not differ significantly between profit and non-for-profit nursing homes or based on the mode (paper/email) of the survey.¹⁶ The majority of nursing homes in OH and CA had 51–100 beds with plurality in MA having 101–150 beds, and the majority in all states being for-profit.

Table 2.

Nursing home characteristics

Characteristic	California, N=216*	Massachusetts, N=119*	Ohio, N=234*	Overall, N=569
Number of beds
30–49	28 (13.0%)	4 (3.4%)	23 (9.8%)	55 (9.7%)
50–99	118 (54.6%)	34 (28.6%)	147 (62.8%)	299 (52.5%)
100–149	45 (20.8%)	52 (43.7%)	41 (17.5%)	138 (24.3%)
150–199	19 (8.8%)	22 (18.5%)	17 (7.3%)	58 (10.2%)
200 or more	6 (2.8%)	7 (5.9%)	6 (2.6%)	19 (3.3%)
Type of nursing home
For profit	173 (80.1%)	78 (65.5%)	177 (75.6%)	428 (75.2%)
Not-for-profit	43 (19.9%)	41 (34.5%)	57 (24.4%)	141 (24.8%)
Title of respondent
Director of nursing	187 (93.0%)	96 (88.9%)	197 (93.4%)	480 (92.3%)
Other	14 (7.0%)	12 (11.1%)	14 (6.6%)	40 (7.7%)
Unknown	15	11	23	49

*Statistics presented: n (%).

Table 3 shows the distribution of responses by category for each WISH item retained in the final scales. All variables were skewed towards selection of Mostly/Most of the time and Complete/All of the time. ‘Not at all’ consistently had less than 10% selection, with the exception of two items (Participation q2 and Policies, Programmes and Practices q11), thus all variables collapsed ‘not at all’ with ‘somewhat’ or ‘some of the time’, as applicable, resulting in a three-point analytical scale. Online supplemental appendix table 1 includes items that were deleted due to lower discrimination.

Table 3.

Final items in the Workplace Integrated Safety and Health (WISH) assessment response distribution

Leadership Commitment	Not at all	Some of the time	Most of the time	All of the time
1. (q1a) The company’s leadership, such as senior leaders and middle managers, communicate their commitment to a work environment that supports employee safety, health and well-being.	0.5	9.9	36.5	53.1
2. (q1b) The organisation allocates enough resources, such as enough workers and money to implement policies or programmes to protect and promote worker safety and health.	2.6	18.9	40.4	38.1
3. (q1d) Worker health and safety are part of the organisation’s mission, vision or business objectives.	1.6	7.1	24.8	66.5
4. (q1f) The importance of health and safety is consistently reflected in actions across all levels of the organisation, both formally and informally.	1.9	13.3	33.3	51.5
Participation	Not at all	Some of the time	Most of the time	All of the time
1. (q2a) Managers and employees work together in planning, implementing and evaluating comprehensive safety and health programmes, policies and practices for employees.	2.7	17.4	47.5	32.4
2. (q2b) This company has a joint worker-management committee that addresses efforts to protect and promote worker safety and health.	12.3	13.9	33.6	40.2
3. (q2c) In this organisation, managers across all levels consistently seek employee involvement and feedback in decision-making.	2.9	22.4	42.5	32.2
4. (q2d) Employees are encouraged to voice concerns about working conditions without fear of retaliation.	1.4	6.4	22.5	69.7
Policies, Programmes and Practices	Not at all	Some of the time	Most of the time	All of the time
1. (q3a) The workplace is routinely evaluated by staff trained to identify potential health and safety hazards.	2.6	11.0	39.3	47.1
2. (q3c) Supervisors are responsible for correcting unsafe working conditions on their units.	0.9	8.8	39.8	50.5
3. (q3e) Organisational policies or programmes are in place to support employees when they are dealing with personal or family issues.	4.7	18.3	31.4	45.6
4. (q3f) Leadership, such as supervisors and managers, make sure that workers are able to take their entitled breaks during work (eg, meal breaks).	0.9	7.3	33.3	58.5
5. (q3g) Supervisors and managers make sure workers are able to take their earned times away from work such as sick time, vacation and parental leave.	1.5	4.8	32.1	61.6
6. (q3i) This organisation has trainings for workers and managers across all levels to prevent harm to employees from abuse, harassment, discrimination and violence.	1.7	7.4	15.8	75.1
7. (q3j) This workplace provides support to employees who are returning to work after time off due to work-related health conditions.	1.9	8.2	23.0	66.9
8. (q3k) This workplace provides support to employees who are returning to work after time off due to non-work-related health conditions.	3.0	12.5	28.8	55.7
9. (q3l) This organisation takes proactive measures to make sure that the employee’s workload is reasonable, for example, that employees can usually complete their assigned job tasks within their shift.	3.5	10.3	41.4	44.8
10. (q3m) Employees have the resources such as equipment and training to do their jobs safely and well.	0.9	7.5	33.2	58.4
11. (q5e) The wages for the lowest-paid employees in this organisation seem to be enough to cover basic living expenses such as housing and food.	11.7	21.7	30.7	35.9
Comprehensive and Collaborative Strategies	Not at all	Some of the time	Most of the time	All of the time
1. (q4c) This company coordinates policies, programmes and practices for worker health, safety and well-being across departments.	1.9	10.9	31.7	55.5
2. (q4d) Managers are held accountable for implementing best practices to protect worker safety, health and well-being, for example, through their performance reviews.	2.2	12.2	33.0	52.6
3. (q4e) Managers are given resources, such as equipment and trainings, for implementing best practices to protect and promote worker safety, health and well-being.	3.5	14.1	33.1	49.3
4. (q4f) This company prioritises protection and promotion of worker safety and health when selecting vendors and subcontractors.	3.7	12.5	34.2	49.6
Adherence	Not at all	Some of the time	Most of the time	All of the time
1. (q5a) This organisation complies with standards for legal conduct.	0.4	2.6	13.1	83.9
2. (q5b) In this organisation, people show sincere respect for others’ ideas, values, beliefs.	1.1	9.6	31.9	57.4
3. (q5c) This workplace complies with regulations aimed at eliminating or minimising potential exposures to recognised hazards.	0.2	4.1	19.4	76.3
4. (q5d) This company ensures that safeguards regarding worker confidentiality, privacy and non-retaliation protections are followed.	1.1	4.3	19.0	75.6
Data-driven Change (included here for reference)	Not at all	Some of the time	Most of the time	All of the time
1. (q6a) The effects of policies and programmes to promote worker safety and health are measured using data from multiple sources, such as injury data, employee feedback and absence records.	4.3	13.7	34.0	47.9
2. (q6b) Data from multiple sources on health, safety and well-being are integrated and presented to leadership on a regular basis.	7.1	16.9	33.3	42.7
3. (q6c) Evaluations of policies, programmes and practices to protect and promote worker health are used to improve future efforts.	4.7	13.4	36.0	45.9
4. (q6d) Integrated data on employee safety and health outcomes are coordinated across all relevant departments.	5.7	15.4	34.1	44.8

Number in parentheses is the number of the original WISH assessment tool before IRT analysis.

IRT, Item Response Theory.

Table 4 shows model fit statistics, discrimination (α) and category boundary location (β_i) parameters for each item by domain. Leadership Commitment, Comprehensive and Collaborative Strategies, and Adherence had good fitting unidimensional models indicating it fits the data as well as a saturated model (C2 or M2), improvement from a null model (CFI >0.95), deviation of the model implied correlation matrix (SRMSR <0.08), and absolute fit (lower bound of RMSEA 90% CI (RMSEA LB)<0.06). Participation showed good fit relative to a null model (CFI=0.99), deviation of the model implied correlation matrix (SRMSR=0.03) and the RMSEA (RMSEA=0.10, RMSEA LB=0.06), but indicated some misfit relative to a saturated model (M2(df=2)=14.2). Policies, Programmes and Practices showed acceptable fit of the model implied correlation matrix (SRMSR=0.05) and RMSEA (RMSEA=0.08, RMSEA LB=0.06), but showed some lack of fit relative to a null model (CFI=0.93) and saturated model (M2(df=22)=84.93). Adherence had good fit of the model implied correlation matrix, RMSEA (RMSEA=0.07, RMSEA LB=0.02) and relative to a null model (CFI=1.0), but some misfit relative to a saturated model (C2=6.640, df=2). No reasonably fitting model could be identified for the dimension Data-driven Change, thus it is not presented further.

Table 4.

Discrimination and category location parameters by domain

	α	β1	β2	β3
Leadership (M2df=2=3.15, RMSEA=0.03, LB=0; SRMSR=0.01; CFI=1.0)
q1a	3.157	−3.001	−1.444	−0.087
q1b	2.548	−2.394	−0.947	0.370
q1d	3.625	−2.437	−1.529	−0.469
q1f	4.068	−2.256	−1.122	−0.037
Participation (M2df=2=14.2, RMSEA=0.10, LB=0.06; SRMSR=0.03; CFI=0.99)
q2a	2.957	−2.274	−0.955	0.528
q2b	2.357	−1.463	−0.797	0.304
q2c	3.311	−2.179	−0.740	0.514
q2d	2.099	−2.948	−1.818	−0.639
Policies (M2df=22=84.9, RMSEA=0.07, LB=0.06; SRMSR=0.05; CFI=0.93)
q3a	1.790	−2.762	−1.505	0.104
q3c	1.391	−3.985	−2.070	−0.026
q3e	1.431	−2.691	−1.166	0.147
q3f	2.106	−3.158	−1.770	−0.274
q3g	2.590	−2.689	−1.834	−0.355
q3i	2.517	−2.629	−1.631	−0.838
q3j	4.341	−2.268	−1.345	−0.470
q3k	3.466	−2.145	−1.135	−0.171
q3l	3.208	−2.097	−1.244	0.138
q3m	2.677	−2.836	−1.654	−0.253
q5e	1.190	−2.153	−0.795	0.586
Comprehensive and Collaborative Strategies (M2df=2=3.13; RMSEA=0.03; SRMSR=0.01; CFI=1.0)
q4c	4.140	−2.259	−1.241	−0.169
q4d	3.302	−2.266	−1.182	−0.087
q4e	4.843	−1.933	−0.996	0.004
q4f	3.667	−2.000	−1.106	−0.002
Adherence (M2df=2=6.6; RMSEA=0.07, LB=0.014, SRMSR=0.03; CFI=1.0)
q5a	3.480	−2.982	−2.084	−1.130
q5b	2.962	−2.646	−1.457	−0.222
q5c	5.104	−3.033	−1.790	−0.789
q5d	4.289	−2.416	−1.726	−0.774

α, discrimination; β1, β2, β3, category boundary locations; CFI, Comparative Fit Index; M2, limited information goodness of fit χ²-statistic; RMSEA, root mean square error of approximation; SRMSR, standardised root mean square residuals.

Discrimination parameters (α) were large for all constructs, indicating each item is sensitive to differences on the latent scores for each construct. These discrimination parameters ranged from 2.55 to 4.07 for Leadership, 2.1 to 3.31 for Participation, 1.19 to 4.34 for Policies, Programmes and Practices, 3.3 to 4.84 for Comprehensive and Collaborative Strategies, and 2.96 to 5.1 for Adherence. Category boundary location parameters (β_1, β_2, β₃) were mostly in the negative range of the latent scores, indicating most items are most informative for nursing homes scoring lower on the constructs.

Figure 1 shows the test information curve for each domain. These curves show the amount of information (or measurement precision) in the sets of items at different scores of the latent construct (locations). Each domain has limited range on the latent trait within which it has high information. The peak of the information curve was above 15 for Policies, Programmes and Practices, and Adherence, and above 10 for Leadership and Comprehensive and Collaborative Strategies. The sets of items measuring these constructs were most informative (thus providing the most reliable measurement) among nursing homes with lower scores on the respective latent traits, typically in the range of −2 to 0. The range for Participation was more informative in the range of −1 to 1, with its peak at 8. The empirical reliability of the scores on the latent traits was 0.81 for Leadership, 0.82 for Participation, 0.87 for Policies, Programmes and Practices, 0.82 for Comprehensive and Collaborative Strategies, and 0.70 for Adherence.

Figure 1.

Test information curves by domain of the Workplace Integrated Safety and Health (WISH) assessment.

Discussion

This paper aimed to validate the WISH assessment by examining the dimensionality of its six constructs based on a survey conducted with DONs in nursing homes in three states in the USA. We identified unidimensional scales for Leadership Commitment, Participation, Policies, Programmes and Practices, Comprehensive and Collaborative Strategies, and Adherence, with acceptable fit on most or all fit measures. Empirical reliability suggested overall acceptable to good reliability among all constructs. Additional analyses (category boundary locations and test information curves) revealed that most items, and thus the scales, were most informative among nursing homes with low levels working conditions that support safety, health and well-being. The WISH can further our understanding of best practices for an integrated systems approach to protect and promote workers’ health, safety and well-being.

The WISH has several strengths that make it well suited to different applications. Due to the large discrimination parameters, reliable estimates of the respective constructs can be estimated with relatively few items. This makes the assessment particularly well suited for studies covering the WISH and additional topics, reducing participation time.

There were two domains that would benefit from further study. Although all domains exhibited some ceiling effect, Adherence’s was pronounced as 53% of nursing homes had a maximum total score. Future work on the Adherence domain should seek to develop items that discriminate and are informative among higher scoring nursing homes. Additionally, we could not identify a reasonable and good fitting model for Data-driven Change. At present, we cannot claim these items measure a latent data-driven change construct. However, the individual items may still be of interest to researchers for their individual measurement potential. They can be directly interpreted on an item-by-item basis. For example, the item ‘Integrated data on employee safety and health outcomes are coordinated across all relevant departments’ can be used and interpreted as a measure of integrated data coordination without any claim that it is a manifestation of data-driven change. Indeed, these items (table 3) have been thoroughly pretested previously.⁷

The study has limitations regarding the nature of the sample. The sample only included nursing homes with at least 30 beds or more, from three states. The survey was sent to DONs; 84.4% of the respondents were indeed DONs, 7.4% did not report a title, and 7.6% reported a different, but often similar, title. We assume that the individuals answering the survey had requisite knowledge to complete the task, particularly for non-DOs respondents. Responses from nursing homes with a DON respondent did not differ significantly from nursing homes with a different type of respondent.¹⁶ After controlling for for-profit status, state, number of beds, Medicare quality rating, Medicare staffing rating, Medicare health inspection rating, survey wave and rurality, we did not find any statistically significant predictors of whether a nursing home responded.¹⁶ The WISH should be validated in additional samples for robustness to geographic location and setting. Our findings should be regarded as a first version of these scales, to be expanded on and modified in future studies. This should, as the body of research grows, result in scales that are generalisable with respect to type of establishment studied.

Despite ceiling effects on some domains, the WISH instrument has utility for many research and applied settings. For example, research among organisations with low performance on Adherence can get reliable estimates of the respective latent scores. Organisations such as these are often of most concern for researchers and practitioners trying to improve workplace safety, health and well-being. As such, an intervention targeting workplace safety, health and well-being can still detect meaningful changes and non-intervention longitudinal studies can similarly observe changes over time. Moreover, the tools can have tremendous utility in applied work as a screening instrument to identify low-performing organisations that need intervention or further study. Findings from this validation study indicate that the WISH assessment is adequate to measure the extent to which organisations adhere to best practices related to integrated systems approaches that protect and promote workers’ health, safety and well-being. Measuring the domains of the WISH can help organisations and researchers in their prioritisation and decision-making to improve workplace conditions, using resources to affect the areas with the worst performance. Using the WISH tool at baseline with follow-up assessments has the potential to provide organisations a way to identify improvements or setbacks in the organisation in relation to recommended best practices. The WISH tool may also complement surveys directed to workers which can inform managers and researchers alike of any discordances between organisational policies, practices and programmes and workers’ perceptions of these.

Data availability statement

Data are available upon reasonable request and by e-mail to: jwilliams13@kumc.edu.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study received ethics approval from the Harvard T.H. Chan School of Public Health (IRB file number 18-1245).