Assessing Leadership Behavior in Health Care: Introducing the Local Leadership Scale of the SCORE Survey

Abstract

Background:

Engaged and accessible leadership is a key component of care excellence. However, the field lacks brief, reliable, and actionable measures of feedback and coaching-related behaviors of local leaders (for example, provides frequent feedback). The current study introduces a five-item Local Leadership (LL) scale by examining its psychometric properties, providing benchmarking across demographic factors and work settings, assessing its association with psychological safety, and testing whether LL predicts reports of restricted activities and absenteeism.

Methods:

In this cross-sectional study, 23,853 questionnaires were distributed across 31 Midwestern US hospitals. The survey included the LL scale, as well as safety culture and well-being scales. Psychometric analyses (Cronbach’s α, confirmatory factor analysis [CFA] fit: root square mean error of the approximation [RMSEA], comparative fit index [CFI], Tucker-Lewis index [TLI]), Spearman correlations, t-tests, and analyses of variance (ANOVAs) were used to test the properties of the LL scale and differences by health care worker and work setting characteristics.

Results:

A total of 16,797 surveys were returned (70.4% response rate). The LL scale exhibited strong psychometric properties (Cronbach’s α = 0.94; RMSEA = 0.079; CFI = 0.99; TLI = 0.98). LL scores differed by role, shift, shift length, and years in specialty. Of all roles, leaders (for example, managers) rated leaders most favorably. Nonclinical (vs. clinical) and nonsurgical (vs. surgical) work settings reported higher LL. LL scores correlated positively with psychological safety, absenteeism, and activities restricted due to illness.

Conclusion:

The LL scale exhibits strong psychometric properties, convergent validity with psychological safety, and variation by work setting, work setting type, role, shift, shift length, and specialty. The study indicates that assessing leadership behaviors with the LL scale is useful and offers actionable behaviors for leaders to improve safety culture within teams.

Patient safety culture surveys were practically nonexistent before the 1999 Institute of Medicine report To Err Is Human: Building a Safer Health System¹ but have been widely used in hospitals since 2005. From the beginning, measuring perceptions of leadership within a work setting was considered important enough to merit its own domain on each of the most commonly used safety culture instruments. An array of leadership scales have been developed both inside and outside of health care to capture specific leadership behaviors as well as various leadership styles. A 2010 systematic review found that leadership styles focused on relationship building, rather than orientation to tasks, were associated with higher nurse job satisfaction and work effectiveness.² However, relationship building can be particularly difficult in the hierarchical setting of health care.^{3 , 4} In a study on how leaders can promote learning on traditionally hierarchical teams, leader expectation setting, accessibility, and provision of feedback all served as coaching behaviors to reduce hierarchy and build psychological safety so that health care workers (HCWs) feel safe to freely speak up about concerns with little interpersonal risk.⁵

The Local Leadership (LL) scale of the Safety, Communication, Operational Reliability, and Engagement (SCORE)⁶ survey (mean SCORE α = 0.88) is a brief metric that is highly reliable (α = 0.94),⁷ designed to be actionable, simply phrased, and reflective of specific leadership behaviors related to feedback, availability, and expectations setting. The LL scale is one of 13 domains within the SCORE survey. LL can be used on its own or within SCORE. The full SCORE survey consists of 80 short questions and takes an estimated 8 to 10 minutes to complete. LL has already been linked to feeling supported,⁷ readiness to engage in quality improvement,⁸ work-life integration,⁹ and psychological safety.¹⁰

The LL scale focuses on three characteristics of leadership behaviors when providing feedback to team members (positive, frequent, useful), and two leadership behaviors (setting expectations and being available). These components serve as leadership coaching behaviors to build psychological safety and were included in Amy Edmondson’s seminal work on team psychological safety as scale items under the domains of “team leader coaching” and “clear direction.” These domains highlight the importance of having a clear expectation for the team and having an accessible leader who regularly discusses progress with team.^{11 , 12} However, these key antecedents for psychological safety are not typically assessed on traditional psychological safety scales.¹²

In addition to breaking down hierarchies and promoting psychological safety, these components relate to provision of feedback by leaders to promote quality improvement as well as well-being in the employees they supervise. Despite substantial literature on the elements of feedback necessary to promote behavior change, there is limited research specifically exploring feedback-related behaviors by leadership that can promote feedback acceptability and incorporation.^13–15 There is also limited research on the role of feedback provision and well-being, with some suggestion that feedback from supervisors can promote well-being of those they supervise if it is done well.^16–18 For instance, leaders who provide task performance feedback have lower rates of absenteeism due to illness.¹⁸ However, most studies have compared absence and presence of feedback or examined delivery mechanisms rather than exploring specifics of timing or the balance between positive and constructive feedback. In addition, setting clear goals and expectations for performance is an important component of feedback effectiveness.¹⁴ Similarly, feedback source trustworthiness is an important component of feedback effectiveness, which may be promoted by leader accessibility.¹⁵ To this end, the components of the LL scale measure coaching behaviors to promote psychological safety specifically surrounding the feedback process.

Given the role of coaching behaviors in promoting psychological safety and feedback behaviors and promoting well-being, the current study aims to examine the five-item LL scale by doing the following:

• Evaluating its psychometric properties

• Examining variation in LL scores—at the work setting and hospital levels, by various demographic factors (role, shift length, shift type, years in specialty), and, across different categories of work settings (nonclinical [vs. clinical] and nonsurgical [vs. surgical])

• Testing whether LL scores are associated with reports of absenteeism and activities restricted due to illness

• Testing whether LL scores are associated with psychological safety (convergent validity)

Our hypotheses regarding LL were that higher ratings would be associated with positive culture and organizational outcomes.

METHODS

Design and Study Population

This cross-sectional study was conducted during a two-month period in 2015 using the SCORE survey and demographic items.⁶ The survey was distributed via e-mail listservs through the Michigan Health & Hospital Association (MHA) Keystone Center within their routine patient safety and quality measurement. The MHA Keystone Center includes all 175 hospitals from 20 health systems within the state of Michigan. All employees working 0.5 full-time equivalent (FTE) or higher for four consecutive weeks prior to survey administration were asked to complete the survey. Confidentiality was guaranteed to respondents, participation was voluntary both at the individual and organizational level, and no incentives were offered. Participating organizations were offered feedback on their scores as well as safety improvement information. The Keystone Center administered the survey, with each organization allowed to encourage participation locally to maximize response rates. The study was approved by the Institutional Review Board at Duke University Medical Center (Pro00033155).

Measures

The SCORE survey reflects contemporary workplace issues and work setting norms, including workforce wellbeing.^{6–9 , 19 , 20} The LL domain of SCORE consists of five items based on work by Frankel and Leonard on leadership and safety culture.²¹ Items were selected based on the concepts that most often offered leaders useful insight and specific strategies to deploy as a result of safety culture debriefings of leadership within work settings.

The LL scale begins with the prompt “In this work setting, local management …” (or “In this work setting, local physician leaders …” for physicians). Then individual items are as follows:

• Is available at predictable times.

• Regularly makes time to provide positive feedback to me about how I am doing.

• Provides frequent feedback about my performance.

• Provides useful feedback about my performance.

• Communicates their expectations to me about my performance.

Responses were scored on a five-point Likert scale, “disagree strongly” to “agree strongly, and offered a “not applicable” option, which responses were not included in the current analyses. At the individual respondent level, LL was calculated by transposing the mean score of the five subdomain items onto a 0 to 100 scale.^{6–9 , 19 , 20}

Work Setting Categories.

To compare LL by work setting type, work settings were classified first as nonmedical or medical, and medical work settings were further classified as surgical, inpatient, or outpatient.

Absenteeism and Illness.

Single items assessed absenteeism and illness. “In the past month I have missed work (for any reason)” and “In the past month my activities have been restricted due to illness.” Response options were 1 to 5 (strongly disagree to strongly agree).

LL Associations Related to Psychological Safety.

To examine convergent validity, a set of six SCORE items were selected based on the definition of psychological safety as “the belief that it is safe to freely speak up about concerns.” These items form a psychological safety scale that we have used informally at the request of patient safety officers who ask for diagnostics of psychological safety across work settings. This psychological safety scale (not tied directly to leadership behaviors) was recently developed and validated¹⁰ and was not part of standard feedback to SCORE users at the time these data were debriefed. The psychological safety scale was composed of items drawn from the Teamwork Climate and Safety Climate domains of the SCORE survey.^{6 , 22} An aggregated score of the overall psychological safety scale, as well as its items, were computed using the same method as the LL scale. The specific items were as follows:

• It is easy for personnel here to ask questions when there is something that they do not understand.

• The culture in this work setting makes it easy to learn from the errors of others.

• My suggestions about quality would be acted upon if I expressed them to management.

• In this work setting, it is difficult to speak up if I perceive a problem with patient care (reversed).

• In this work setting, it is difficult to discuss errors (reversed).

• Disagreements in this work setting are appropriately resolved (that is, not who is right but what is best for the patient).

Demographic Items.

The survey also collected items asking about role, shift length, shift type, and length in specialty.

Statistical Analysis

For analyses that assess LL at the work setting level (Aims 2–4), an established technique aggregated percent positive scores for LL and all other SCORE domains.^{6–9 , 19 , 20 , 22} Aggregated scores were computed by taking the percentage of workers in a given setting who, on average, agreed slightly or strongly to the items. Because percent positive scores convey local climates, we refer to “LL climate” when it is aggregated at the work setting level.^{6 , 8 , 9 , 19} We excluded surveys that did not include LL responses and considered other missing data as informative missing.

The psychometric properties of the hypothesized one-factor model of the LL scale were evaluated with Cronbach’s α and a multigroup confirmatory factor analysis (CFA), clustering at the work-setting level. A CFA is a multivariate analysis to test how well a set of measured variables represents a number of constructs (factors). We used the following CFA fit indices: RMSEA (root mean square error of approximation; acceptable values are ≤ 0.08), TLI (Tucker-Lewis index; acceptable values are > 0.95), CFI (comparative fit index; acceptable values are > 0.95), and SRMR (standardized root mean squared residual; values < 0.08 are considered a good fit).^{23 , 24} Intraclass correlations (ICCs), a measure of reliability of the group means, assessed within vs. between work setting variability in LL scores. Univariate analysis of variance (ANOVA) with Scheffé post hoc tests examined variation in LL scores by hospital, work setting, respondents’ role, shift type, and years in specialty. Independent t-tests examined differences by work setting types. For t-tests that failed Levene’s test for equality of variances, we report values that did not assume equal variances. Spearman correlations assessed the relationship between LL, psychological safety, activities restricted due to illness, and absenteeism. Analyses were computed using IBM SPSS Statistics, version 24²⁵ and Mplus.²⁶

RESULTS

Respondent Demographics

Of 23,853 distributed questionnaires,16,797 were returned (70.4% response rate). Table 1 shows the demographic data for the entire sample. The most frequent job positions included nurses (27.2%), administrative support (11.7%) and technologists (6.8%). Approximately half of the study population was working an 8-hour shift (50.8%), and most (70.0%) reported working during the daytime. Almost a quarter (23.8%) of the respondents had been in their specialty for 11–20 years. Of 16,797 participants, 125 (0.7%) did not submit any LL responses, and 66 (0.4%) submitted incomplete LL responses; these were excluded. Of the 16,606 respondents completing the full scale, 8,985 (54.1%) reported positive LL. Of the 1,140 work settings (that is, discrete units or teams within participating health systems) that were invited to participate in the survey, 818 had five or more respondents, the threshold used to aggregate a work setting into percent positive/climate scores. Work settings were classified as indirect patient care (nonclinical; for example, administrative or billing; n = 423), or direct patient care (clinical; n = 325; which included the subgroups of surgical [n = 70], inpatient [n = 185], and outpatient [n = 36] work settings).

Table 1.

Respondent Demographics and Local Leadership (LL) Cronbach’s α

		N	LL Cronbach’s α	% of Total
Role (N = 16,672)	Administrator	807	0.920	4.8
	Administrative Support	1,961	0.937	11.8
	Clinical Social Worker	129	0.916	0.8
	Clinical Support	635	0.942	3.8
	Dietitian	93	0.938	0.6
	Environmental Support	350	0.927	2.1
	Nurse	4,555	0.944	27.3
	Nurse’s Aide	672	0.933	4.0
	Other	2,877	0.939	16.9
	Other Manager	482	0.938	2.9
	Pharmacist	240	0.951	1.4
	Physician Assistant	110	0.941	0.7
	Attending/Staff Physician	401	0.930	2.4
	Resident Physician	42	0.961	0.3
	Technician	939	0.947	5.6
	Technologist	1,134	0.935	6.8
	Therapist	742	0.937	4.5
	Role Missing	562	0.944	3.4
Shift Length	8 hours	8,462	0.939	50.8
	10 hours	1,351	0.947	8.1
	12 hours	4,320	0.941	25.9
	Flexible	968	0.942	5.8
	Other	1,342	0.940	8.0
	Missing	229	0.960	1.4
Shift	Days	11,678	0.942	70.0
	Nights	2,447	0.936	14.7
	Swing	877	0.937	5.3
	Other	1,351	0.941	8.1
	Missing	319	0.952	1.9
Years in Specialty	Less than 6 months	661	0.913	4.0
	6–11 months	872	0.924	5.2
	1–2 years	1,777	0.928	10.7
	3–4 years	2,059	0.934	12.4
	5–10 years	3,627	0.943	21.8
	11–20 years	3,963	0.946	23.8
	21 or more years	3,447	0.946	20.7
	Missing	266	0.944	1.6

Aim 1: The Psychometric Properties of the LL Scale

The LL scale demonstrated excellent internal consistency (Cronbach’s α = 0.941). Spearman correlations among the five items ranged from r = 0.609 to r = 0.897, p < 0.001 (see Table 2). The initial one-factor CFA model’s RMSEA revealed good fit to the data (RMSEA = 0.079, 90% confidence interval [CI] 0.073–0.085, CFI 0.99, TLI 0.98, SRMR 0.015). The work setting level ICC was 0.17, indicating that people within work settings generally share similar views about their local leadership, relative to assessments of local leadership between work settings. Items’ ICCs ranged from 0.13 to 0.15. Clustering was reflected in these results, as between-work setting differences contributed to 17% of total variability in LL scores and 13% to 15% of the total variability in the LL items.

Table 2.

Spearman Correlation Matrix for the Local Leadership (LL) Domain and LL Items (Intraclass Correlations on the Diagonal)

Variable	1	2	3	4	5
1. Local Leadership Domain	(.17)
2. Predictable Availability item	.765*	(.15)
3. Positive Feedback item	.922*	.664*	(.14)
4. Frequent Feedback item	.935*	.614*	.863*	(.14)
5. Useful Feedback item	.933*	.623*	.852*	.897*	(.14)
6. Communicates Expectations item	.876*	.609*	.774*	.803*	.830*	(.13)

^*p < 0.01 level (2-tailed).

Aim 2: Examine LL Scale Variation

Considerable variation in LL scores (from 0% positive to 100% positive) was observed across the 818 work settings (mean 59 ± 22, median 75, and an interquartile range of 40–72) (see Figure 1). Individual LL scores also differed among respondents from different hospitals (F(1,28) = 16.2, p < 0.001) and work settings (F(1,1138) = 3.86, p < 0.001) A small but significant negative association was found between the number of people in a work setting and LL scores (r = −0.159, p < 0.001), with larger units more likely to report lower LL scores.

Figure 1:

This bar graph shows responses to illness, missing work, and psychological safety items by local leadership (LL) climate quartile.

Managers and those working in administration reported the highest LL scores, whereas physician assistants, technologists, and environmental support reported the lowest LL scores. LL scores did not differ between nurses (mean 66.2, standard deviation [SD] 30.2) and physicians (attending, hospitalists, and residents); mean 68.0, SD 27.7; t (461.5) = 1.17, p = 0.24.

Significant differences in LL scores were observed by shift length (F(1,7) = 15.6, p < 0.001) and shift type (F(1,4) = 31.6, p < 0.001). Specifically, both flex and 8-hour shift workers rated LL significantly better than 12-hour shift workers did. Day shift workers reported significantly higher LL compared with all other shift types. Swing, night, and “other” shift workers’ LL scores were not significantly different from each other.

Nonclinical (mean 59.7, SD 22.3) work settings reported higher LL climate than clinical work settings (mean 54.6, SD 22.7, t (746) = 1.17, p = 0.002). Nonsurgical (mean 55.8, SD 22.3) work settings reported higher LL climate than surgical work settings (mean 47.3, SD 22.6, t (393) = −2.44, p = 0.015). Inpatient (mean 50.5, SD 22.4) and outpatient (mean 55.5, SD 25.0) work settings did not differ by LL climate (t (219) = −1.19, p = 0.235)

Aim 3: Test Whether LL Predicts Reports of Absenteeism and Activities Restricted Due to Illness

Lower LL climate significantly predicted activities restricted due to illness and missing work for any reason (r = −.202, p < 0.001; r = −.132, p < 0.001, respectively). Comparisons of the top and bottom LL climate quartiles revealed significant differences for activities restricted due to illness and for missing work due to any reason (see Figure 2). Respondents in the lowest LL quartile (relative to the highest) were, on average, 36.6% more likely to report activities restricted due to illness and 16.1% more likely to miss work.

Figure 2:

This graph shows reports of good local leadership climate across 818 work settings.

Aim 4: Test Whether LL Scores Predict Psychological Safety

The internal reliability for the psychological safety scale was good (Cronbach’s α = 0.80).¹² Higher LL scores were associated with higher psychological safety (r = 0.676, p < 0.001). Higher LL scores were also associated with each of the six psychological safety items; all correlations were moderate (Spearman’s rho ranging from 0.44 to 0.69) and significant at p < 0.001 (see Table 2 for correlations and Figure 2 for differences by LL quartile). The strongest correlation was found with the item “My suggestions about quality would be acted upon if I expressed them to management” (r = 0.69, p < 0.001).

DISCUSSION

The current study found robust support for the reliability and validity of the LL scale in a large sample of HCWs. LL was associated with less absenteeism, fewer activities restricted due to illness, and greater psychological safety. Psychometrically, the LL scale exceeded established thresholds for internal consistency and model fit (assessed by Cronbach’s α and CFA, respectively). LL scores varied widely by hospital, work setting, role, shift, years in specialty, and type of unit (for example, surgical, nonclinical).

Particularly during times of upheaval, such as the COVID-19 pandemic,²⁷ health care leaders have a powerful opportunity to create environments of continuous learning, support, and psychological safety. These factors are known to improve patient safety and HCW engagement.²⁸ The current study examined a novel measure specifically designed to assess LL behaviors related to coaching and feedback (for example, being available at predictable times, providing useful feedback, communicating clearly about expectations). Examining specific behaviors allows the measure to be highly actionable so that leaders can directly target improvement by adjusting their behavior based on results from different items. The overall LL and item-level ICC results demonstrate that HCWs share similar perceptions of their local leader along these specific behaviors.

Aim 1: The Psychometric Properties of the LL Scale

The LL scale demonstrated strong psychometric properties, (α = 0.94, and a well-fitting CFA). In comparison, CFAs of other frequently used safety culture surveys do not always meet or exceed minimal psychometric thresholds. For instance, the Agency for Healthcare Research and Quality (AHRQ) Hospital Survey on Patient Safety Culture domain of “Supervisor/Manager Expectations and Actions Promoting Patient Safety” revealed a CFI of 0.82, relative to the LL CFI of 0.99 (> 0.90 is generally accepted as reflecting good model fit)²⁴ and a lower Cronbach’s α (0.79),²⁹ than the LL scale (α = 0.94).

Aim 2: Variation with the LL Scale

Flexible and 8-hour shift workers reported the best leadership ratings, while 12-hour shifts reported substantially lower LL. Prior research has found nurses working 12-hour shifts to have lower levels of job satisfaction and higher rates of absenteeism than those working 8-hour shifts.³⁰ Day shift workers reported higher LL than night shift workers. This may be due to reduced staffing levels during night shifts, including leaders. Individuals in their specialty for less than 6 months reported the highest LL, followed by those having worked between 6 and 11 months, compared to all other lengths in specialty. This pattern of lower scores associated with increasing experience is similar to previous reports on safety culture.⁹

Different types of work settings differed in their LL scores: Nonclinical work settings reported higher LL than clinical work settings. Among clinical settings, medical work settings reported higher LL than surgical work settings. Perhaps the higher levels of demands (for example, production pressure, consequences of outcomes) of clinical, including surgical, work settings set a higher bar for effective LL norms. Conversely, it may be easier to rate a leader highly in a nonclinical work setting such as finance or procurement, where tasks and expectations may be more clearly defined. Of all roles, leaders (for example, managers) rated leaders most favorably. This result is similar to the finding that leaders are more likely to report higher improvement readiness norms in their work settings compared to all other roles.⁸

Aims 3 and 4: Associations with Absenteeism/Activities Restricted and Psychological Safety

Higher LL scores predicted lower rates of missing work for any reason. These findings are consistent with prior evidence relating transformational leadership styles to lower rates of sick absences and absenteeism in general^{31 , 32} Transformational leadership is characterized by motivating people to participate in the process of change and fosters autonomy, responsibility, and accountability, which leads to stronger feelings of self-worth and self-efficacy among employees.³¹ A study of nurses found that having a leader with the transformational style predicted their organizational commitment, productivity, and intention to stay in the organization.³³ An important component of transformational leadership is individualized consideration, or leadership behaviors supporting the individual needs of followers to promote their self-actualization.³¹ The LL behaviors of feedback, accessibility, and expectations theoretically map on well here, addressing specific modifiable behaviors.

LL scores were also positively correlated with all six psychological safety items (ask questions, learn from errors of others, speak up, discuss mistakes, feel that suggestions will be acted upon, and have disagreements appropriately resolved). These strong associations provide convergent validity between the novel LL scale (with the underlying behaviors of feedback provision, accessibility, and expectation-setting) and long-standing markers of psychological safety, teamwork, and patient safety norms.

Improving Psychological Safety Through Leadership Behaviors

The LL scale focuses on specific behaviors to measure leadership, rather than assessments of satisfaction with leadership or transformational leadership attributes, making the scale less subjective to personal sympathies or beliefs and tying leadership to the direct effects on HCWs. Moreover, leaders can directly adapt their behaviors based on ratings of specific items (for example, giving feedback about what is going well or deliberately making themselves more accessible at predictable times). This scale can be used in conjunction with assessment of psychological safety climate to offer specific coaching behaviors to promote around feedback practices. For those interested in learning more about giving and receiving feedback as well as setting expectations, several resources and courses are available.^34–37

Limitations

This study should be to be viewed in light of its design. It is limited in its use of self-report data which are at risk for response, selection, and social desirability biases. The cross-sectional design of the study precludes any causal explanations. Our response rate compares favorably with other studies,³⁸ and response rates at this level have been found to result in stable estimates. Due to data security and anonymity measures, we were unable to identify the leaders being reported on in the current dataset, precluding comparisons between self-rated and HCW–rated leadership. We did not have clinical outcomes in the current dataset, and therefore were unable to test associations between clinical outcomes and LL. More research will be required to study the relationships between LL and clinical and operational outcomes, particularly given the potential clinical implications of both providing effective feedback and promoting a psychologically safe work environment. Finally, although the current data were collected within the state of Michigan, the Michigan Health & Hospital Association Keystone Center includes all 175 hospitals from 20 health systems in the state of Michigan. This includes a variety of settings (for example, rural and urban) and populations of workers and patients. Therefore, our results are likely to generalize to a wide variety of community and academic hospitals across the United States.

CONCLUSION

The LL scale exhibits strong psychometric properties, internal reliability, and convergent validity and varies by work setting and by work setting type (for example, clinical vs. nonclinical), as well as by role, shift, shift length, and specialty. Higher LL ratings predict lower rates of absenteeism, having activities restricted due to illness, and higher rates of psychological safety. Not only is assessing specific leadership behaviors with the LL scale psychometrically valid and reliable, the face validity of the verbatim items provides leaders with insights into actionable behaviors they can engage in to improve their leadership.