Building a safety culture in global health: lessons from Guatemala

Abstract

Programmes to modify the safety culture have led to lasting improvements in patient safety and quality of care in high-income settings around the world, although their use in low-income and middle-income countries (LMICs) has been limited. This analysis explores (1) how to measure the safety culture using a health culture survey in an LMIC and (2) how to use survey data to develop targeted safety initiatives using a paediatric nephrology unit in Guatemala as a field test case. We used the Safety, Communication, Operational Reliability, and Engagement survey to assess staff views towards 13 health climate and engagement domains. Domains with low scores included personal burnout, local leadership, teamwork and work–life balance. We held a series of debriefings to implement interventions targeted towards areas of need as defined by the survey. Programmes included the use of morning briefings, expansion of staff break resources and use of teamwork tools. Implementation challenges included the need for education of leadership, limited resources and hierarchical work relationships. This report can serve as an operational guide for providers in LMICs for use of a health culture survey to promote a strong safety culture and to guide their quality improvement and safety programmes.

Key questions.

What is already known about this topic?

• The need to improve patient safety and healthcare quality is an increasing recognised priority within the global health agenda. The implementation of tools to improve patient safety and the quality of care remains challenging in low-income and middle-income countries (LMICs).

• Modification of the safety culture has been shown to lead to lasting improvements in patient outcomes, healthcare quality, as well as organisational performance in a variety of high-income countries, although tools to assess and promote a safety culture remain underused in LMICs.

• Culture surveys are commonly used in high-income settings to measure and promote a strong safety culture. These surveys offer valuable resources to guide the implementation of safety and quality improvement (QI) interventions.

What are the new findings?

• We confirmed at a paediatric nephrology unit in Guatemala that the Safety, Communication, Operational Reliability, and Engagement survey could be deployed within an LMIC and yield high-quality data about staff views towards several health climate and staff engagement domains.

• Staff debriefings are critical when interpreting health culture survey data in order to develop targeted options for QI interventions.

• Using health culture survey results, clinicians can deploy a number of low-cost tools to improve the safety culture in low-resource settings, such as morning briefings, leadership walk-rounds and interventions to reduce staff burnout.

Key questions.

Recommendations for policy

• Use of a health culture survey can define opportunities to promote a safety culture in LMICs. Future research should focus on validation efforts across different healthcare systems to define the role of health culture surveys to improve patient safety and healthcare quality.

• The use of healthcare surveys requires attention to cultural adaptation at all steps of the process to ensure that safety and QI efforts are contextualised to local culture and needs.

• Training of local safety leaders in safety and QI operations is required to ensure long-term adoption of these programmes in low-resource settings.

Introduction

Medical care is fraught with hazards around the world, particularly in low-income and middle-income countries (LMICs).^{1 2} Approximately two-thirds of all adverse events and associated disability-adjusted life-years occur in LMICs.^{2 3} Concern about patient safety and healthcare quality is increasingly recognised within the global health agenda as articulated by the Sustainable Development Goals Target 3.8,⁴ the World Health Assembly Resolution 55.18 on ‘Quality of Care’,⁵ the WHO Patient Safety Programme, as well as several collaborations in surgical and childbirth checklist use.^5–7 However, the impact of many individual safety programmes on patient outcomes is not clear.⁸ As well, the implementation of programmes to improve patient safety remains challenging in LMICs, with common barriers including costs, human resources and complex cultural, social and political constraints.^{1 8–13}

A safety culture refers to an environment that encourages collaboration, quality and safety.^{14 15} In To Err Is Human, the U.S. National Academy of Medicine (formerly the Institute of Medicine) highlights the importance of building a safety culture as a prerequisite to improving healthcare safety.¹⁶ Assessment of the safety culture is commonly performed using health culture surveys across high-income settings and is required as part of hospital accreditation by the Joint Commission in the United States.¹⁷ The use of health culture surveys can facilitate programmes to promote a safety culture, and these programmes have been associated with improved clinical outcomes (such as decreased infection rates) as well as enhanced organisational performance (such as decreased staff turnover) in many high-income settings.^18–23 However, the use of programmes to modify the safety culture has been limited in LMICs.

Previous work has confirmed concerns about safety and healthcare quality among patients and staff in Guatemala,^24–29 suggesting that fundamental improvement in the safety culture may be required to generate lasting improvements in healthcare quality. In this analysis, we explored how to assess the safety culture using a health culture survey and use these data to develop targeted programmes to improve the safety culture, using a paediatric nephrology unit in Guatemala as a test field case.

Safety culture assessment

We deployed the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey at the paediatric nephrology unit at the Roosevelt Hospital, which is the largest public hospital in Guatemala (online supplementary file 1). We selected this unit based on adequate size for assessment of the safety culture (generally as 15–20 people) and enthusiastic engagement of unit leadership. This unit is the largest centre in the country for the care of children with renal disease, and operates inpatient care, outpatient clinics, as well as six haemodialysis beds. Services offered include renal replacement therapies, support for renal transplantation and care for children with renal disease. This analysis summarises our progress over a 2-year period from 2016 to 2017.

As safety culture surveys are designed to measure culture at a hospital unit level, it is essential to engage as many unit staff as feasible in the process. Prior to survey deployment, staff should be organisationally mapped into their profession role (minimum of five staff/role to protect anonymity of survey data) to allow for comparison between roles. For example, we mapped staff (n=29) into four categories, including physicians, nurses, clinical support (nutritionist, psychologist, social worker, dialysis technician and pharmacist) and non-clinical staff (administration, secretary, maintenance).

The SCORE survey has been validated in a number of high-income settings and includes questions from the Safety Attitudes Questionnaire (SAQ) and the Agency for Healthcare Research and Quality (AHRQ) Hospital Survey on Patient Safety, with additional items on burnout, depression and work–life balance.^{19 30–35} SCORE measure staff views across 13 domains, including healthcare climate domains (learning, local leadership, burnout climate, personal burnout, teamwork, safety climate, work–life balance) and engagement domains (growth opportunities, workload strain, job certainty, intention to leave, advancement and decision-making) (tables 1 and 2).³⁵ All domains include 3–7 questions, with responses expressed using a five-point Likert scale and scored in line with standard methods. Full survey details are included as online supplementary file 2, and readers are referred to Sexton for operational details, including use of psychometric measure and sample size requirements.¹⁵

Table 1.

Results of safety culture domains for Safety, Communication, Operational Reliability, and Engagement survey at Pediatric Nephrology Unit/Roosevelt Hospital in Guatemala

Domain	What does it measure?	Pediatric Nephrology Unit scores	Congruence across roles	Potential Interventions
Learning environment	Whether staff feel that they are learning and that their input is considered; openness that exist in the work setting Strong learning environments foster open discussion about errors	Overall scores were favourable Most staff claim they are learning and input is valued Environment is conducive to discussion but necessarily about errors	Incongruent among roles Administrative staff showing less favourable views than other staff	Use staff input Encourage error reporting with just culture Review errors from an improvement perspective, not punitive
Local leadership	Staff views on leadership support, feedback and guidance Integral to promoting ‘psychological safety’, showing respect for staff	Mixed responses Need for regular feedback Lack of positive feedback	Incongruent among roles Physicians having more favourable views towards feedback and expectations than others	Patient safety walkrounds Increase local management access Enhance feedback loops
Burnout climate	How staff feel about burnout and conflict across unit Staff in settings with burnout are unable to welcome changes	Overall scores were unfavourable Most staff perceive others around them as burned out, leading to conflict and poor performance	Incongruent among roles Administrative and clinical support staff having unfavourable views	Team activities to enhance resilience (ie, ‘three good things’) Designated staff break area Optimise efficiency of workflows
Personal burnout	How staff feel about their own burnout and resilience Burnout associated with higher rate of medical errors	Overall scores unfavourable Most staff have high level of burnout and challenges with job demands, fatigue In combination with other domains, suggests burnout may be mitigated by better leadership support	Incongruent among roles Nursing expressed lower levels of personal burnout, although their views towards burnout climate was favourable	Debriefings Assuring staff breaks
Teamwork	Staff views of team performance and ability to voice concerns and communicate Predicts operational outcomes, including staff turnover, quality, costs, etc.	Overall favourable findings, including staff willingness to speak up and to ask questions Conflict resolution scored favourably Unfavourable scores related to dealing with difficult colleagues and interdepartmental communication Given associated burnout, teamwork is challenged by fatigue and stress	Moderately incongruent among roles Physicians having more favourable views towards interdepartmental communication Some roles showed concerns related to breakdowns in communication	Comprehensive team briefings Structured communication tools Identify and eliminate sources of conflict
Safety climate	Staff view of capacity to provide high-quality care; degree of transparency to discuss errors Predicts clinical outcomes	Most scores were favourable Discussion of errors rated unfavourably Responses are very favourable about quality of care	Incongruent among roles Nursing, administrative and clinical support staff having unfavourable views towards feedback Nursing staff perceive difficulty in discussing errors	Integrate non-punitive error discussion into daily briefings Standardise incident reporting methods ‘Learning from Defects’ tool
Work–life balance	Quantifies activities which measure work–life balance Associated with long-term resilience	Overall scores were unfavourable In concert with other domains, suggests staff are not strong in emotional and wellness domains and may not have resources for self-care	Congruent among roles	Support self-awareness efforts Organisational efforts to address burnout may offset unfavourable results

For each domain, definition of what the domain measures, survey results for Pediatric Nephrology Unit, congruence across professional roles in the unit and sample of tools to improve performance in that domain. Readers are referred to text for a full description of all information and appropriate references.

Table 2.

Results of staff engagement domains for Safety, Communication, Operational Reliability, and Engagement survey at Pediatric Nephrology Unit/Roosevelt Hospital in Guatemala

Domain	What does it measure?	Unit scores	Congruence across roles
Growth opportunities	How staff view opportunity to enhance their career	Scores were mixed Some favourable responses with career growth Some unfavourable responses towards ability to influence planning of career activities All staff feel they can contribute to something important	Incongruent among roles Nurses and administrative support staff had less favourable views towards growth opportunities
Job certainty	How staff view job security within institution as well as outside their institution	Overall scores favourable Most staff plan to be working at the same unit next year	Congruent among roles
Intentions to leave	Staff intention to stay in current job or plan to move to another job within short time frame	Mixed responses Staff may like to find another job Some staff question whether there are other jobs available May reflect lack of job opportunities and need to stay at current job regardless of stressors	Incongruent among roles Nurses having more favourable views towards staying at their current job
Decision-making	How staff view their ability to influence organisational decision-making	Mixed responses Some insufficient involvement in planning and organisational decisions Other staff have favourable views about ability to discuss problems	Incongruent among roles Physicians and clinical support staff having more favourable views than others
Advancement	How staff view advancement potential within the organisation	Mixed responses Staff may not feel like there is advancement potential, but they are not intending to leave	Incongruent among roles Clinical-related staff (physicians, nurses, and clinical support) having more favourable views towards educational, promotional and financial advancement opportunities
Workload strain	How staff view the balance of work demand with available resources	Mixed responses Most staff expressing high levels of workload strain in regards to the volume and/or complexity of their work demands	Incongruent among roles Physician, administrative and clinical support staff having unfavourable views

For each domain, definition of what the domain measures, survey results for Pediatric Nephrology Unit and congruence across professional roles in the unit. Readers are referred to text for a full description of all information and appropriate references.

Before deploying the SCORE survey in any non-English-speaking setting, it is important to perform translation using the WHO-recommended process.^36–38 Although SCORE has not been tested in other languages to date, its core elements (the SAQ and AHRQ surveys) have been validated in several non-English languages.^{22 39}

Interpretation of safety climate domains

For optimal health culture survey use, researchers should understand the role of individual domains as well as how data relate across domains. Three domains of SCORE (safety climate, teamwork and burnout climate) have been shown in high-income settings as the primary areas that describe the overall safety culture. A strong safety culture across these domains is essential to maintain psychological safety, which is the shared belief that the unit is a safe environment for individuals to speak up and ask questions that can contribute to learning and improve clinical outcomes.³¹ Further validation will be required to define the key domains of a safety culture in LMICs.

Of all health culture domains, burnout is increasingly recognised as a challenge to staff performance and safety in many high-income settings, although drivers of burnout are similar across healthcare contexts.^{30 40 41} In the presence of high levels of burnout, there is limited capacity of staff to implement any new healthcare programme. Some drivers of burnout in resource-constrained settings such as low salaries may be difficult to change within a single organisation. However, there are several low-cost approaches to reduce burnout that can be easily implemented in low-resource settings, such as provision of resources for staff breaks, confirmation of set break times and increased education around self-care such as nutrition and exercise. These practices can ensure that staff are caring for themselves, and in doing so, they will be more resilient and better able to care for their patients.

Proper interpretation of a health culture survey includes analysis of individual domains, with patterns across domains used to derive overall conclusion (tables 1 and 2). To facilitate understanding of survey data, we find it helpful to summarise findings using radar charts, in which each spoke represents the per cent of favourable responses to individual questions (see online supplementary file 3). As well, radar charts can summarise overall domain scores (figures 1 and 2). Colour coding is helpful for radar charts, with yellow and green demonstrating ≥60% favourable responses and red demonstrating <60% favourable responses. The 60% threshold goal is based on experiences with health surveys in high-income settings, in which units with <60% of favourable responses gain the most from quality improvement (QI) efforts.^{20 21 32} Finally, although overall domain scores provide an overview of staff views, they tend to score lower than scores from individual questions. This phenomenon is termed cultural instability and suggests that individual item scores help you discern the specific issues that underlie the domain performance.¹⁵

Figure 1.

Overall results of Safety, Communication, Operational Reliability, and Engagement survey for health climate domains (learning environment, local leadership, burnout climate, personal burnout, teamwork, safety climate, work–life balance). Scores summarise responses to all questions in that domain, with responses to each question defined as favourable for an answer of agree/strongly agree for positively worded questions or disagree/strongly disagree for negative (reverse-worded) questions. A positive individual-level score was assigned when ≥50% of staff responses in the domain were favourable. The overall score was calculated as the number of individuals with positive responses divided by the total number of responses. Data summarised using radar charts, with each spoke representing the overall score for each domain.

Figure 2.

Overall results of Safety, Communication, Operational Reliability, and Engagement survey for staff engagement domains (growth opportunities, workload strain, job certainty, intentions to leave, advancement and decision-making). Overall scores summarise responses to all questions in that domain, with responses to each question defined as favourable for an answer of agree/strongly agree for positively worded questions or disagree/strongly disagree for negative (reverse-worded) questions. A positive individual-level score was assigned when ≥50% of staff responses in the domain were favourable. The overall score was calculated as the number of individuals with positive responses divided by the total number of responses. Data summarised using radar charts, with each spoke representing the overall score for each domain.

Survey debriefing

A critical step for use of health culture data is unit staff debriefing. During staff debriefing, we summarise all survey data by overall domains as well as by individual questions. For each domain, we discuss how to interpret the data, potential reasons for survey findings and options to improve unit performance (in the setting of poor scores) or to sustain performance (in setting of high scores). We ask staff to identify issues that might contribute to their perspectives, as well as which types of changes could be implemented successfully in their current work environment. We emphasise that all survey data should be interpreted within local organisational and cultural contexts.

Debriefings should be led in a transparent and non-judgemental fashion. For example, we use open-ended questions, such as asking staff why a particular domain may be important to them and whether they can identify specific examples that illustrate how the survey data reflect their personal experiences. To maximise staff comfort discussing their views in a public setting, we find it helpful to debrief staff as a single group as well as by small groups of individual roles (ie, nursing, non-clinical staff). Furthermore, given the difference of views by staff roles, small group debriefing allows consideration of group-specific interventions.

As examples of how to discuss survey data, within the Local Leadership domain, we discuss how the data may show a need to improve the availability of front-line leadership to staff and the need for leadership to set clear expectations for staff performance and provide scheduled feedback. Within the personal burnout and work/life balance domains, we discuss how staff may have difficulties keeping up with the demands of a stressful work environment, as well as how they feel about their compensation and job security. We emphasise the need for any unit with high levels of burnout to focus first on recovery from burnout prior to tackling any major safety interventions as high levels of burnout in themselves limit the capacity of staff to implement large organisation changes. Readers are referred to several reviews of the survey debriefing process.^{15 22 35}

Use of survey to direct safety initiatives

Following debriefings, we used survey results to implement a targeted series of safety culture and QI interventions. To select specific initiatives, we used an iterative process led by a local unit leader working under the mentorship of an expert in healthcare safety. All planning was conducted either via teleconference or email every 3–4 weeks, with in-person visits to review progress every six months. All initiatives were developed using a Plan–Do–Study–Act (PDSA) framework.⁴² In doing such, we used the SCORE survey data and debriefings for the first portion of this process, namely identifying a driver of concern and developing options to improve processes within that domain of the safety culture.

We first identified the most pressing needs as defined by the survey, which in this unit were gaps in communication, teamwork and burnout. Additional key messages include the importance of identifying small and attainable goals as well as long-term goals to improve the safety culture, particularly in low-performing domains. In doing such, staff gain trust in their capacity to implement healthcare change by seeing results from attainable interventions. As well, both unit staff and the organisation over time can build the skills and capacity to undertake major systematic changes.⁴²

For example, one of the first initiatives we implemented was intended to address staff concerns surrounding intra-unit communication. The local safety leader led the implementation of a regularly scheduled morning briefing to begin each workday. During this 10–15 min session, all staff members collectively discuss unit needs of the day, identify any pressing concerns and gain perspectives of each team member. Additional low-cost and simple programmes included engagement of the unit dietician to help staff with personal eating plans as well as construction of a defined area for staff breaks, and institution of scheduled staff feedback sessions with unit leaders.

One additional tool we are implementing to improve communication and feedback is the use of regularly scheduled Leadership Walk Rounds, which is a structured tool to engage leaders with front-line staff and improve communication. Leadership Walk Rounds have been shown to improve the safety culture, clinical outcomes and organisation performance in several high-income settings.^{35 43} For discussion of other low-cost tools to promote a safety culture based on a health culture survey (table 1), readers are referred to several reviews.21 35 44–47

Impact assessment

Any comprehensive impact assessment on clinical outcomes or overall safety culture is beyond the capacity of this pilot analysis as these changes generally take 18–24 months to see significant changes. However, we did perform a rapid impact assessment 4 months after survey deployment to test the impact of early safety culture interventions on select areas of concern though use of a ‘Pulse Survey’.⁴⁸ This short focused survey used 18 questions from the larger SCORE survey to take the ‘pulse’ of the unit in specific domains and to assess traction of early interventions to modify the safety culture. We found that some domains showed substantial improvement, including teamwork and work–life balance. Minimal improvement was noted in interdepartmental communication and discussion of errors. Domains that scored less favourably included job certainty. Short and rapid surveys such as these offer quick feedback on success of interventions to modify the safety climate and provide guidance to target interventions towards areas of continued need.

Implementation challenges

Similar to other new healthcare programmes in resource-constrained settings, assessment and modification of the safety culture in LMICs faces several implementation challenges. Appropriate cultural adaptation guided by a theoretical framework is essential to guide the implementation process, including modification of safety culture surveys programmes in response to local needs, culture, language and context.^{37 38} For this project, we used the ecological validity model to guide cultural adaptation, which specifies areas of attention when merging interventions across settings, including language, persons, metaphors, content, concept and context.⁴⁹ We adhered to this framework to guide discussion of conflicts in language, content or when developing and deploying the SCORE survey, as well as to frame staff debriefings.

For example, we found early in our staff debriefings there was a culture gap in regards to views towards the value of reporting medical errors. As part of our ecological validity model, we recognised that the value of collecting medical errors was not commonly discussed by Guatemala hospital leadership or staff. To help frame discussion of this difference in views towards reporting medical errors between cultures, we identified events from other resource-constrained settings to demonstrate that errors in themselves can be valuable events to understand faults in clinical systems and improve patient safety. We emphasised through this discussion the value of medical errors as learning tools rather than as a guide for punitive action. Working within this cultural adaptation model, our discussion led to a common understanding of the role of collecting medical errors and transparency in medical error reporting.

Similar to other low-resource settings, Guatemala has complex cultural, economic and social barriers to implementing new healthcare programmes.^{28 29} Guatemala is generally considered a collectivist, bureaucratic and hierarchical culture.⁵⁰ As physicians are often in positions of power in these healthcare settings, we chose to develop physicians as the first safety and QI leaders to encourage adoption of new safety initiatives. Early in this process of building a safety culture, hospital leadership recognises the need to encourage the growth of safety leaders from other staff roles (eg, nursing, pharmacy). Finally, it is important to recognise that not all survey findings (such as frustration with low pay) can be addressed in any single organisation, particularly in settings of low health system resources.

The use of health culture surveys is a learnt skill that requires training and experience. Through this experience, we recognised that formal training of local safety officers in patient safety and QI was essential, such that these leaders have the requisite skills to direct local safety initiatives. Although a full description of our training programmes is beyond the capacity of this report, our curriculum emphasises patient safety and QI theory, experiential learning with experts in health culture survey use and QI operations, as well as focused training on tools to improve the safety culture. Following training, these leaders were matched with experienced mentors in QI and healthcare safety, such that over time they can develop capacity to lead local safety processes.

We examined the implementation process itself by recording staff comments from debriefings and classifying these comments into different themes. Our analysis showed several barriers to promotion of a safety culture, including a hierarchical work structure that limits the ability of nursing staff to report medical errors as well as organisational resistance to healthcare change. Importantly, we found that initial acceptance from leadership early in the process was critical. Many individuals involved, including hospital leaders, did not have prior knowledge of safety culture theory and operations, and therefore education of leadership with these processes was critical. Finally, the implementation process itself required iterative improvement as leaders and staff became comfortable with the programme. However, over time, most staff appeared to be quite enthusiastic to implement changes as driven by the SCORE survey.

Conclusion

Our analysis is intended to serve as a ‘how to’ guide to help providers in LMICs use a health culture survey to promote a safety culture in their organisation. We confirmed that the safety culture could be measured within a healthcare system that is plagued by chronic shortages of resources and bureaucratic constraints. Despite these challenges, we have shown that a heath culture survey can provide high-fidelity data in these settings, and that these data can guide targeted safety culture initiatives.

Essential elements for the use of health culture surveys include collecting and sharing survey data in a transparent and non-judgemental fashion, focusing on positive aspects of the health culture (‘celebrations of success’) as well as areas that may require improvement. In low-resource settings, emphasis should be made of identifying easily attainable goals (‘low-hanging fruit’) to demonstrate to staff the value of investment in their work culture, the importance of measuring clinical and organisational performance outcomes, and tailoring programmes to local resources and context.

Although beyond the capacity of this report, validation of a health culture survey in Guatemala will require assessment of clinical outcomes as well as organisational performance. Formal implementation analysis can help guide these processes, and iterative change based on an understanding of local barriers to programme implementation can facilitate long-term adoption of safety practices. As the global health agenda expands beyond simply increasing access to care towards ensuring the provision of safe and high-quality care, programmes to build a strong safety culture have enormous potential to reduce the global burden of unsafe medical care. Tools to assess and promote a safety culture may provide key support for healthcare system strengthening across many healthcare settings around the world.