Development of an early warning resilience survey for healthcare organizations

Abstract

Objective

To design and validate a brief set of measures identifying staff and work areas exhibiting low levels of resilience within healthcare organizations.

Data sources/Study design

Primary data were gathered via survey administration between April and August of 2016 from 33,622 respondents across 123 facilities. These surveys included pilot items designed to measure resilience and were administered to all employees alongside employee engagement surveys.

Data collection/Extraction methods

Following the data collection period for the pilot survey, data from all organizations were integrated into a single analytical dataset. Factor analyses were used to determine the underlying constructs of healthcare worker resilience. Cronbach's alpha and correlation analyses tested the internal consistency and validity of the instrument.

Principal findings

A brief set consisting of eight items was identified as a psychometrically validated measure of resilience. This measure consists of two subscales, Activation and Decompression. These measures exist independent of employee engagement, indicating an empirical distinction between the two concepts. Resilience was found to predict 38% of variance in engagement scores.

Conclusions

An eight-item instrument can accurately measure resilience to identify burnout risk and serve as a predictor of other workforce outcomes such as engagement.

Psychology; Health profession; Resilience; Burnout; Engagement; Survey design; Psychometrics

1. Introduction

Healthcare workers experience occupationally-induced stress as a result of challenging or traumatic experiences in their daily line of duty (Jackson et al., 2007; Portoghese et al., 2014). Institutional elements such as work policies, communication, or decision-making at the team or organizational level can contribute to this stress. Previous studies have linked occupationally-induced stress to high rates of depression and anxiety, burnout, and compassion fatigue (Maslach et al., 2001; Rees et al., 2015; Thorsteinsson et al., 2014). These negative impacts can extend beyond the individual level and impact employee performance, the quality of patient care, patient safety and the likelihood of patients adhering to care plans (Cooke et al., 2013; Rees et al., 2015; Salyers et al., 2017; Van den Broeck et al., 2008; Williams et al., 2006). Organizations have the potential to improve patient care by identifying the elements that help mitigate this stress and/or help engage employees to resist these stressors.

1.1. Measuring resilience

Resilience is a multi-dimensional concept that reflects the ability of an individual, community or organization to adapt and move on in a positive way from stressful or adverse events (Cimarello et al., 2016; Connor and Davidson, 2003; Rees et al., 2015; Tugade and Fredrickson, 2004; Wu et al., 2017). Resilience is made up of concepts such as personal hardiness, goal orientation, adaptability, endurance, and the ability recover and recharge (Jackson et al., 2007; Judkins et al., 2005; Kobasa, 1979; Lyons, 1991; Maslach et al., 2001; Rutter, 1985; Sonnentag and Fritz, 2007; Sonnentag and Kruel, 2006). Resilience measures occur at individual, team, and organizational levels (Jeffcott et al., 2009) because social vulnerabilities inherent in interactions, institutions and systems can affect individual or overall community resilience (Achour and Price, 2010; Cimarello et al., 2016; McAllister and McKinnon, 2009).

While multiple resilience measures exist, many of these instruments differ in how they define and target resilience in a population. Some scales, such as the Connor-Davidson Resilience Scale (CD-RISC) (Connor and Davidson, 2003), Resilience Scale (RS) (Wagnild and Young, 1993), and Resilience Scale for Adults (RSA) (Friborg et al., 2003), measure personal and protective characteristics and coping styles that promote resilience. Others, such as the Ego Resilience Scale (Block and Kremen, 1996), focus more on an individual's ability to adapt to changing circumstances. The Brief Resilience Scale (BRS) (Smith et al., 2008) focuses solely on the ability to bounce back from stressful events, while the Scale of Protective Factors (Ponce-Garcia et al., 2015) focuses on factors that serve as a buffer to reduce the impact of stress in people who have experienced traumatic events.

Choosing an appropriate scale to measure resilience depends not only on the theoretical aim of the study but also the target population. Cultural background, socio-demographic characteristics, living conditions, work area, and education level can influence the factorial framework of the resilience construct, making a single scale difficult to generalize across populations (Connor and Davidson, 2003; Lamond et al., 2008; Wu et al., 2017). Some studies address this issue by modifying established scales and removing factors to better represent the target population (Lamond et al., 2008; Wu et al., 2017). Other measures are developed to target a specific population (Hurtes and Lawrence, 2001).

To the authors' knowledge, no current resilience instruments specifically target healthcare employees. Yet studies have shown healthcare workers face increasing risk of burnout due to the stress associated with heavy workloads, challenging work conditions, work-family conflicts, and the frequent spillover of work into personal time (Houkes et al., 2011; Montgomery et al., 2006; Shanafelt et al., 2015). As healthcare worker burnout can affect not only job performance and satisfaction but also the quality of patient care (Humphries et al., 2014), a need exists for a concise and targeted measure of resilience designed specifically for healthcare organizations that furthers the understanding of resilience as an institutional element as well as an individual trait.

The purpose of our study was to determine whether a short resilience survey could serve as an early warning indicator of low levels of resilience among healthcare workers. Our objectives were to (i) examine the performance and reliability of a Short Resilience Survey (SRS) for healthcare settings, and (ii) measure the relationship between our resilience instrument and employee engagement. We hypothesized that (i) the SRS items would load as two distinct factors, one factor being related to the ability to recover and recharge from work and the other being related to personal characteristics such as motivation and acknowledgement of the intrinsic value of the work. We further hypothesized that (ii) employee engagement would increase with increasing resilience, since employees at risk of burning out are presumably also less likely to be engaged. Our results can provide future researchers and healthcare organizations with a valid and reliable measure of employee resilience that is applicable across clinical and non-clinical roles within the healthcare industry.

2. Methods

The content of the SRS was drawn from a number of sources following a review of contemporary burnout and resilience literature. An expert panel synthesized the reviewed literature to generate a 14-item multidimensional pilot instrument of resilience tailored specifically to healthcare workers (Table 1). Concepts that have been previously linked to established measures of resilience and burnout (e.g. emotional exhaustion, depersonalization, personal accomplishment, strong relationships and meaningful work) were used for question creation and inclusion (Garmezy, 1971; Giordano, 1997; Jackson et al., 2007; Tugade and Fredrickson, 2004; Tusaie and Dyer, 2004). From Tugade and Fredrickson (2004), items assessing the ability to effectively rebound from stressful encounters were included. From the CD-RISC (Connor and Davidson, 2003), items reflecting personal competence, coping/recovery from negative events, and sense of purpose were included. From the Maslach Burnout Inventory (MBI) (Maslach et al., 2001), items designed to measure personal accomplishment and cynicism were included.

Table 1.

Pilot resilience items test question set.

	Item Wording
1	I rarely lose sleep over work issues.
2	I am able to free my mind from work when I am away from it.
3	I can enjoy my personal time without focusing on work matters.
4	I am able to disconnect from work communications during my free time (emails/phone etc.)
5	I care for all patients/clients equally even when it is difficult
6	I see every patient/client as an individual person with specific needs
7	This job rarely makes me feel detached.
8	At the beginning of a work day I am emotionally ready for whatever comes my way.
9	At the end of a work day I have enough energy to pursue non-work activities.
10	My coworkers help alleviate the stress of my daily work.
11	The work I do makes a real difference.
12	My work gives me a feeling of accomplishment.
13	My work is meaningful.
14	I rarely experience burnout from my work.

The SRS also included items designed to be more applicable in a modern healthcare setting, such as being able to focus on non-work activities after the work day and finding intrinsic meaning in daily work activities (Jackson et al., 2007; Robertson et al., 2016; Sonnentag and Kruel, 2006; Tugade and Fredrickson, 2004). Being able to disconnect from occupationally-induced fatigue after work hours shows that job-related stress does not constantly permeate life outside of work (Jackson et al., 2007; Sonnentag and Kruel, 2006). Likewise, positive work attitudes display investment in one's job by internalizing the human interaction components of the work while also deriving a sense of intrinsic motivation from it (Robertson et al., 2016; Tugade and Fredrickson, 2004). The SRS was designed to be as concise as possible to help identify low levels of resilience across an organization while not substantially increasing current healthcare survey lengths.

The 14-item pilot survey (Table 1) was administered to 11 participating organizations across 13 states (CA, GA, IL, LA, MD, MI, MN, NC, NY, OK, PA, VA and WV). One hundred and twenty three facilities were represented, including: acute care hospitals, medical practice groups, outpatient surgery centers, children's hospitals, home health organizations as well as other ancillary services. Survey administration occurred between April and August of 2016. Census sampling ensured all relevant employee types were included in the sample and allowed for testing across clinical and non-clinical staff (Table 2). All 14 items were presented to respondents in the same scale, with response categories of Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree, and Not Applicable (1 = Strongly Disagree, 5 = Strongly Agree). Surveys were administered through the Press Ganey eSurvey system whereby each respondent was given a unique passcode and allowed to complete only one survey.

Table 2.

Number and percentage of respondents by job category and demographic classification.

		Clinical	%	Non-Clinical	%	Provider	%	Missing
Total Respondents		13910	100.0	16558	100.0	912	100.0	3154
Sex	Female	12069	86.8	13181	79.6	419	45.9	3192
	Male	1805	13.0	3375	20.4	464	50.9
Patient Care (%)	<50	469	3.4	5626	34.0	*	*	3532
	>50	6552	47.1	1270	7.7	576	63.2
	N/A	6525	46.9	9648	58.3	336	36.8
Length of Service (years)	>1	2038	14.7	2076	12.5	65	7.1	3509
	1–2	3493	25.1	3518	21.2	126	13.8
	3–5	2432	17.5	2903	17.5	128	14.0
	6–10	2445	17.6	3380	20.4	137	15.0
	11–15	1284	9.2	2048	12.4	66	7.2
	16–20	651	4.7	1050	6.3	23	2.5
	20–25	447	3.2	611	3.7	20	2.2
	26+	767	5.5	970	5.9	11	1.2

Provider = Physicians + Advanced Practice Providers. Missing = demographic question left unanswered.

The SRS was administered concurrently with the Press Ganey Employee and Press Ganey Physician Engagement surveys. These two surveys were initially developed in 2001 following an extensive literature review and discussions with healthcare employee and physician focus groups. The Press Ganey Employee survey was updated in 2014, drawing respondent-level data from over 300 healthcare organizations (Press Ganey Associates, 2014). The Press Ganey Physician Engagement survey was updated in 2016 and was based on physician respondents from 154 hospital sites and 241 clinic sites in calendar year 2015 (Press Ganey Associates, 2017a,b). The Press Ganey Employee instrument includes six dimensions: engagement, organization quality patient care and safety, organization communication & effectiveness, manager leadership and employee involvement, employee individual work and employee work unit relations. The Press Ganey Physician Engagement instrument includes four dimensions: engagement, physician alignment with the organization, organization quality, care, communication and effectiveness, and staff relations.

Both questionnaires include employee demographic information along with positively-framed scaled items that explore the respondent's evaluation of a single behavior-based workplace characteristic, feature, policy or psychological construct (Press Ganey Associates, 2014). Inclusion of only positively-framed items was intentional, as comparisons of positive and alternating versions of the same scale have shown no difference in response biases (Sauro and Lewis, 2011). Avoiding alternating versions of the same item also reduces the possibility of mistaken answers, miscoding, distortion of the factor structure and interpretational problems (Krosnick and Presser, 2010; Sauro and Lewis, 2011). All scaled items use a 5-point Likert scale, and both questionnaires were validated via test-retest.

2.1. Analytical strategy

Data were analyzed using SPSS 22 statistical software (IBM, Armonk, NY). Descriptive statistics were used to describe the characteristics of the study sample. All 14 resilience items were included in a factor analysis. An additional set of six items that make up the employee engagement dimension in both the Press Ganey Employee and Physician Engagement Survey were also included to examine the level of conceptual overlap between resilience and engagement and ensure that they were distinctly different constructs. The engagement dimension used in these surveys is based on Meyer and Allen's Affective Commitment construct (Jaros, 2007), and includes concepts such as attachment, loyalty, pride and recommendation as a place of work or to receive care.

The factor analysis was used to determine the factor loadings of each question as well as each question's item-to-scale relationship. This was done to ensure that items were highly related to singular factors and did not load across multiple factors. Promax rotation was used in this analysis, allowing the factors to correlate. Promax rotation is often preferred in psychological research with large sample sizes (e.g., Fabrigar et al., 1999). Questions were included in the final model if their factor loadings were higher than 0.70 (Ford et al., 1986; Hair et al., 2010) and they loaded on a singular factor. Items with weak loadings and those that were cross-loaded were dropped from the model (Tabachnick and Fidell, 2013), resulting in the retention of a total of eight resilience items (Table 3). Inclusion or exclusion of questions was based on their performance across the entire sample (N = 33,622) as well as across various employee types (physicians and advanced practice providers (N = 912), clinical staff (N = 13,910), and non-clinical staff (N = 16,558)). Given that the quality of care received in a hospital setting is dependent on those performing a variety of different job functions, the survey developers decided that the final instrument should be applicable to all types of personnel involved. Results of reliability and validity testing were nearly identical for total sample as well as the three employee subgroups.

Table 3.

Final resilience model question set.

Factor	Item Wording
Decompression
	I can enjoy my personal time without focusing on work matters.
	I am able to disconnect from work communications during my free time (emails/phone, etc.).
	I rarely lose sleep over work issues.
	I am able to free my mind from work when I am away from it.
Activation
	I see every patient/client as an individual person with specific needs.
	I care for all patients/clients equally even when it is difficult.
	My work is meaningful.
	The work I do makes a real difference.

Internal consistency was tested using Cronbach's Alpha. A Cronbach's Alpha of 0.70 is the typical standard for reliability measures (Nunnaly and Bernstein, 1994), and this standard was adopted for the present research. Following the factor analysis, a multiple linear regression was used to estimate the impact the resilience scale had on the performance of engagement. To conduct this analysis, respondent-level data for relevant items were averaged to create composite-level scores for each of the three identified factors. The model was run using default settings and checked to ensure it met all the assumptions of the multiple regression model.

Properties of convergent and discriminant validity were also assessed using correlation analyses. Corrected item-total correlation coefficients were estimated between each item and its own scale as an indicator of convergent validity. A minimum correlation of 0.60 was established as part of this assessment (Hair et al., 2010). These results were then compared to correlations between items and items outside of their own scale to assess discriminant validity.

3. Results

A total of 33,622 surveys were completed in the Press Ganey Engagement Survey Portal. Response rates ranged from 43% to 86% with an average response rate of 66%. Descriptive data for sample groups within the pilot survey are presented in Table 2.

Three factors accounted for 75.23% of the variance in the resilience and engagement items. The first factor (48.92% of the variance) consisted of the six engagement indicators, representing such things as job satisfaction as well as pride in, and loyalty to, the employing organization. The second factor (15.41% of the variance) consisted of four items and was termed Decompression. This factor reflected a focus on the respondent's ability to disconnect from work issues, enjoy personal time, and “recharge” while outside of work. The third factor (10.89% of the variance) also consisted of four items and was termed Activation. This factor reflected the ability of respondents to find meaning in their work and continue to approach patients/clients as individuals. Factor loadings from the initial analysis provided support for the notion that indicators of resilience are constructs that are independent of overall employee engagement metrics (Table 4).

Table 4.

Factor loadings for the resilience and employee engagement scales.

	Decompression	Activation	Engagement
Factor 1: Decompression
I can enjoy my personal time without focusing on work matters.	.938	.299	.414
I am able to disconnect from work communications during my free time (emails/phone, etc.).	.847	.281	.323
I rarely lose sleep over work issues.	.858	.276	.461
I am able to free my mind from work when I am away from it.	.937	.270	.399
Factor 2: Activation
I see every patient/client as an individual person with specific needs.	.230	.865	.350
I care for all patients/clients equally even when it is difficult.	.273	.842	.344
My work is meaningful.	.275	.771	.578
The work I do makes a real difference.	.291	.769	.568
Factor 3: Engagement
I am proud to tell people I work for this organization.	.367	.483	.887
I would recommend this organization to family and friends who need care.	.331	.455	.845
I would recommend this organization as a good place to work.	.424	.437	.919
Overall, I am a satisfied employee.	.467	.427	.885
I would like to be working at this organization three years from now.	.334	.426	.842
I would stay with this organization if offered a similar job elsewhere.	.369	.377	.830

Factor loadings >0.70 are highlighted in bold.

Within the resilience measures, the Decompression and Activation scales had Cronbach's Alpha scores of 0.917 and 0.829, respectively. The employee engagement scale had a Cronbach's alpha score of 0.934 (Table 5). Cronbach's alpha also remained above the established standard of 0.70 when subgroups were analyzed (Table 5). The average item-total correlations for the Decompression, Activation, and Engagement scales exceeded the 0.60 threshold. Correlations between items and items outside of their respective scales were weaker than correlations between items and their own scales (Table 5).

Table 5.

Item-total and non-scale item correlations for resilience and engagement composites.

Category	Factor	Avg Corrected Item-Total Correlations	Range of Corrected Item-Total Correlations	Avg Non-Scale Item Correlations	Range of Non-Scale Item Correlations	Alpha	Range of Alpha if Item Deleted
RN	Decompression	.805	.705–.876	.347	.119–.637	.913	.863–.920
	Activation	.633	.554–.681	.316	.145–.585	.808	.732–.793
	Engagement	.820	.783–.884	.498	.253–.769	.939	.920–.933
Provider	Decompression	.803	.717–.864	.303	.103–.528	.913	.865–.915
	Activation	.650	.574–.703	.333	.051–.621	.821	.746–.806
	Engagement	.817	.743–.870	.478	.165–.769	.937	.918–.934
Clinical	Decompression	.800	.750–.874	.331	.176–.599	.911	.859–.914
	Activation	.645	.585–.681	.319	.137–.582	.818	.752–.796
	Engagement	.815	.774–.883	.502	.258–.762	.937	.917–.931
Non-Clinical	Decompression	.821	.749–.888	.289	.112–.607	.922	.876–.922
	Activation	.697	.660–.723	.371	.183–.604	.850	.795–.822
	Engagement	.804	.747–.878	.508	.212–.779	.932	.909–.928
Total	Decompression	.811	.731–.880	.310	.165–.580	.917	.869–.918
	Activation	.663	.609–.696	.343	.160–.595	.829	.766–.804
	Engagement	.808	.761–.877	.507	.252–.760	.934	.913–.929

RN = Registered Nurses, Provider = Physicians + Advanced Practice Providers, Total = All Respondents.

A multiple linear regression predicting employee engagement based on Decompression and Activation was statistically significant (R² = 0.38, F _{(2, 29330)} = 8981.92, p < 0.001). Plots of standardized residuals showed that the data met assumptions of homoscedasticity, linearity and normality. Assumptions of collinearity (Decompression, Tolerance = 0.89, VIF = 1.12; Activation, Tolerance = 0.89, VIF = 1.12), and independence (Durbin-Watson = 1.59) were also met. In this model, employee engagement was equal to 0.245 (Decompression) + 0.673(Activation) + Constant. Thus, for every one point increase on the Decompression scale, engagement scores increased by 0.245 and for every one point increase on the Activation scale, engagement scores increased by 0.673. Overall, resilience predicted 38% of variability in employee engagement in the model.

4. Discussion

The present study tested a Short Resilience Survey (SRS) on 33,622 participants that work in the healthcare industry to determine whether this survey could serve as an early warning indicator of low levels of resilience among healthcare workers. Healthcare workers were specifically targeted because they face unique and challenging work environments and work-related stressors (Shanafelt et al., 2012). At eight items, the SRS is shorter than many established scales of resilience or burnout (Carver, 1998; Connor and Davidson, 2003; Maslach et al., 2001; McLarnon and Rothstein, 2013). The SRS demonstrates good internal consistency, reliability and convergent validity for the overall sample as well as among different job categories. The factorial structure of the final eight-item model consists of two factors reflecting an individual's ability to recover/recharge (Decompression) and one's self-actuation (Activation). These two factors were predictive of employee engagement, which increased with increasing resilience scores.

The average response rate for the SRS (66%) is consistent with other internal workforce surveys (Bluth et al., 2017; Singer et al., 2007). Although a greater proportion of the respondents were female than male, this is consistent with the gender composition of the Press Ganey Employee dataset (N = 1.2 million respondents) and is also reflective of the healthcare sector as a whole, where women make up roughly 75% of the workforce (U.S. Bureau of Labor Statistics, 2018). The large sample size used to test the SRS, and the similar proportion of males to females in the SRS compared with census data, indicate the present survey adequately represents the healthcare workforce population.

4.1. Dimensions measured by the SRS

The first factor, Decompression, explained the majority of the variance in the SRS and reflects an individual's ability to disconnect from occupationally-induced fatigue in order to recover from stressful job events. Previous studies have used a similar construct to measure one's ability to refocus and recharge in a work environment (Jackson et al., 2007; Sonnentag and Fritz, 2007; Sonnentag and Kruel, 2006). The SRS construct differs from these studies because Decompression also incorporates other sources of stress for healthcare providers, such as constant connection to technology (e.g. smartphone constantly keeping individuals in contact via both phone and email), that are not directly measured by current resilience tools. The SRS Decompression items do not indicate that a respondent is ambivalent about work, rather that they are able to take a break when necessary so that job-related stress does not constantly permeate life outside of work.

Factor 2, Activation, reflects the ability to find meaning in one's work as well as to derive intrinsic motivation from it. This dimension is important for organizational stability, as poor motivation and morale among healthcare workers can undermine service quality and promote workforce attrition (Janssen et al., 1999; Weldegebriel et al., 2016). The Activation measure shows the employee's commitment and ability to approach patients/clients as individuals, and incorporates both Personal Accomplishment and Cynicism from the MBI (Maslach et al., 2001). This metric is consistent with other studies that indicate that concepts such as self-direction, motivation, and the ability to find positive meaning in adverse situations are positively linked to resilience (Connor and Davidson, 2003; Friborg et al., 2003; Robertson et al., 2016; Tugade and Fredrickson, 2004; Wagnild and Young, 1993).

4.2. Applications for the SRS

The SRS can serve as a tool to promote rapid screening for low levels of resilience in individuals subjected to high-stress occupations in the healthcare sector. The brevity of the SRS allows it to easily be appended to existing workplace surveys without significantly increasing survey length. While the SRS does not provide insight into the resilience process or an individual's biological or physiological predisposition for resilience, its use as a screening tool could be included in future work aimed at developing intervention techniques to foster resilience among healthcare workers. Such interventions are necessary in the healthcare sector, where difficult emotional situations and spillover of work into personal life can significantly impact on work functioning (Montgomery et al., 2006). The overall resilience score and two factor structure of the SRS provides healthcare leaders and managers with an initial means to identify and drill down into areas causing stress for employees, such as work-life imbalance or depersonalization and lack of motivation. Identifying areas of low or high Resilience, Activation or Decompression in the workplace can serve as a starting point to develop interventions and strategies at individual and administrative levels to promote emotional stability, self-efficacy and stress coping mechanisms.

4.3. Limitations of the study

There are limitations of the present study that should form the basis of future research. The SRS is a rapid screening tool for resilience among healthcare workers. As such, it does not address the resilience process or provide information about protective mechanisms or resource availability that promote individual resilience. The SRS has also not been validated against an established third party scale such as the RS or CD-RISC, and the information presented in this study only describes how an individual felt at the time of the measurement. Future longitudinal studies are needed to determine the temporal consistency and predictive ability of the instrument among specific work groups, by employee age, or between generational groups. Longitudinal studies will also provide an opportunity to measure differences in resilience tied to region, specialty, or facility type.

5. Conclusion

The Short Resilience Survey (SRS) provides a valid and consistent screening technique to ascertain low resilience in healthcare workers through the measure of self-care (Decompression) and self-awareness (Activation). This instrument is applicable across clinical and non-clinical roles within the industry. Further validation of the SRS through longitudinal studies and against established resilience scales are needed. The goal of the SRS is to help facilitate development of tailored interventions that mitigate lapses in care and workforce attrition in the healthcare industry.

Declarations

Author contribution statement

Kristopher Morgan: Conceived and designed the experiments; Performed the experiments; Wrote the paper.

Nicholas Libby: Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Amy Weaver, Cindy Cai: Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare the following conflict of interests: the authors of this paper work for Press Ganey Associates Inc., which is developing survey products related to research described in this publication.

Additional information

No additional information is available for this paper.