Quality Improvement Initiative to Reduce Serious Safety Events and Improve Patient Safety Culture

Abstract

BACKGROUND AND OBJECTIVE:

Many thousands of patients die every year in the United States as a result of serious and largely preventable safety events or medical errors. Safety events are common in hospitalized children. We conducted a quality improvement initiative to implement cultural and system changes with the goal of reducing serious safety events (SSEs) by 80% within 4 years at our large, urban pediatric hospital.

METHODS:

A multidisciplinary SSE reduction team reviewed the safety literature, examined recent SSEs, interviewed internal leaders, and visited other leading organizations. Senior hospital leaders provided oversight, monitored progress, and helped to overcome barriers. Interventions focused on: (1) error prevention; (2) restructuring patient safety governance; (3) a new root cause analysis process and a common cause database; (4) a highly visible lessons learned program; and (5) specific tactical interventions for high-risk areas. Our outcome measures were the rate of SSEs and the change in patient safety culture.

RESULTS:

SSEs per 10 000 adjusted patient-days decreased from a mean of 0.9 at baseline to 0.3 (P < .0001). The days between SSEs increased from a mean of 19.4 at baseline to 55.2 (P < .0001). After a worsening of patient safety culture outcomes in the first year of intervention, significant improvements were observed between 2007 and 2009.

CONCLUSIONS:

Our multifaceted approach was associated with a significant and sustained reduction of SSEs and improvements in patient safety culture. Multisite studies are needed to better understand contextual factors and the significance of specific interventions.

A decade ago, 2 groundbreaking reports from the Institute of Medicine revealed that defective systems, rather than calloused or careless providers, were the cause of patient safety errors.^1,2 Since then, many hospitals have implemented or revised systems to improve patient safety.^3–5 In addition, many specialty societies and nongovernmental organizations have focused on improving patient safety.^3,6–12 Although there have been some impressive improvements,⁴ patients continue to suffer from serious and preventable safety events.^3,13 Recent multicenter data illustrate that patient harm remains common, and little, if at all, changed over a period from 2002 to 2007.¹⁴ Safety events in hospitalized children are common, and they are associated with increased length of stay, in-hospital mortality, and total charges.^15–19 Up to 60% of such events may be preventable.¹⁷

Cincinnati Children’s Hospital Medical Center (CCHMC) has maintained a significant focus on improving patient safety for many years.²⁰ Three strategic plans since 2001 confirmed quality as our core business strategy and committed us to transforming health care delivery and established organizational readiness via leadership focus, transparency, data infrastructure to support improvement, and continuously increasing improvement capability. Specific improvement initiatives led to measurable reductions in adverse events.^22–23 However, during this same time, the number of serious safety events (SSEs) at CCHMC, defined as deviation from generally accepted performance standards resulting in severe or permanent harm,²⁴ remained stable. Therefore, in 2006, leadership committed to an improvement effort focused on cultural and system changes, with the aim of reducing SSEs to 0.2 per 10 000 adjusted patient-days by June 30, 2010. We report here on the details and results of that initiative.

Methods

Setting

CCHMC is a large, urban pediatric academic medical center. In fiscal year 2010, CCHMC had >32 000 inpatient admissions and 125 000 emergency department (ED) visits and performed 31 000 surgical procedures. The hospital serves a diverse local, national, and international population; ∼43% of patients have Medicaid insurance.

Improvement Team

In 2006, a SSE reduction team was formed, consisting of quality and safety leaders, risk management representatives, project managers, a senior decision analyst, and a consultant human factors expert. The faculty leader devoted 50% of his time. The other team members amounted to 4.75 full-time–equivalent employees. Senior hospital leaders provided oversight, monitored progress toward goals, and helped to overcome barriers.

Diagnostic Phase

The team reviewed the safety literature (with particular attention focused on reliability in health care),^4,25,26 examined the 35 most-recent SSEs at CCHMC (applying methods from Healthcare Performance Improvement²⁴ to analyze causes of the events), and created a common cause database. The team then reviewed CCHMC results of the Hospital Survey on Patient Safety Culture (HSPSC) developed by the Agency for Healthcare Research and Quality (AHRQ).^27,28 Finally, the team hired a consultant from Healthcare Performance Improvement who conducted interviews with >100 CCHMC leaders, physicians, and clinical staff. Team members also participated with other health care organizations in AHRQ’s High Reliability Organization Learning Network,⁷ which included visits to organizations focused on reducing SSEs.^29,30 Evidence from the available literature and expert opinion were then used to develop a key driver diagram (Fig 1).

FIGURE 1.

Key driver diagram for reduction in SSEs. A key driver diagram describes the learning structure for a quality improvement project and includes the aim statement, key drivers, and change strategies to be tested or implemented during the project. The aim statement is developed by using SMART (specific, measurable, actionable, relevant, and time-bound) goals and states the primary objective of the project. The key drivers are the elements believed to be crucial to achieving the goal. OR, operating room.

Interventions

Error Prevention

• A set of expected safety behaviors for all clinical employees were selected by a multidisciplinary panel of local clinicians on the basis of evidence obtained from a literature review and consultant expertise (Table 1).

• An error prevention training program was designed for all direct patient caregivers, staff regularly assigned to a microsystem³¹ or clinical unit, and leaders with the opportunity to affect patient safety. Training objectives included recognizing the significance of safety and factors contributing to harm for patients and families, reviewing CCHMC patient safety history, identifying safety behaviors expected of all providers, practicing error prevention techniques, and discussing CCHMC accountability plans. To encourage teamwork, clinical microsystems trained together in interdisciplinary groups that included physicians. Didactic training involved dynamic lectures, videos, and interactive small-group discussions. Classes were taught by trained staff members. Continuing education credits were available to attendees. To date, >8100 employees have received training.

• Volunteer safety coaches reinforced the reliable use⁴ of expected safety behaviors with their colleagues, encouraged open communication about safety, identified and shared safety improvement opportunities, and built accountability throughout the organization. More than 300 employees, from many disciplines and every inpatient unit, volunteered to be safety coaches. Safety coach groups were organized according to microsystem or department. They conducted behavioral observations and coached staff on performance of safety behaviors. Each safety coach completed at least 4 observations per month that included specific and immediate positive and negative feedback. Observation details were submitted online, creating a database used to plan additional interventions. Safety coaches on some units regularly led rounds to encourage open discussion of patient safety successes and failures. Safety coach leaders from each microsystem met monthly to share findings, provide support, and honor a coach for outstanding contributions.

• Simulation training was used to improve communication and allow teams to practice expected safety behaviors.^32–34 During simulations, clinicians learned to find and eliminate latent safety threats, develop communication and teamwork skills, make adjustments to training and practice, and respond to emergencies. Significant capital investment was dedicated to upgrade the simulation center. The initial focus was on high-risk areas and procedures, such as perioperative practices, extracorporeal membrane oxygenation, bone marrow transplantation, and the ED. Additional in situ or on-site simulations occurred regularly across many microsystems.

• Safety training sessions were conducted for >600 leaders. They were held accountable for patient safety through rewards and recognition programs, yearly reviews, and performance-based privileging. For example, a portion of compensation for senior executives at the director level and above was tied to goal achievement.

• Processes were developed across ICUs to solicit safety concerns from family members throughout the day and to “stop the line”³⁵ until concerns were resolved. When a concern was identified, 2 staff members met with the family and developed a plan to address the concern before continuing care.

• One unit implemented, and later spread, a successful pediatric early warning score to detect clinical deterioration and prevent cardiopulmonary arrests.³⁶ Another team developed and tested processes across the entire delivery system for real-time situation awareness.⁷ The implementation of a medical emergency response team was associated with a reduction in the risk of respiratory and cardiopulmonary arrest outside of critical care areas.³⁷

TABLE 1.

Expected Safety Behaviors for All Clinical Employees

• Asking and encouraging clarifying questions

• Practicing and supporting consultations and coaching with frontline team members (eg, nurses, physicians, support staff)

• Committing to 200% accountability (responsibility for both personal and co-worker behaviors)

• Communicating clearly

• Paying attention to detail

• Having a questioning attitude

• Conducting effective patient handoffs

Restructuring Patient Safety Governance

• A patient safety oversight group, consisting of 5 members of the senior leadership team, the patient safety officer (a physician), and the director of patient safety, focused on accountability, balancing quick fixes and long-term solutions in response to safety events, allocating organizational resources and quality improvement infrastructure to strategic priorities,²⁰ and transparency to the organization and the public. They worked to remove barriers and maintain the emphasis on patient safety as the primary concern of the organization. They strategically used meetings with institutional leaders to keep patient safety on the agenda and as a means to drive accountability and execution on safety action plans developed from the cause analysis process (see following text). The patient safety officer called the chief executive officer and the board chairman if an SSE was identified. SSE details and progress on patient safety goals were the first agenda items at monthly meetings of senior leaders and board of trustee meetings.

Cause Analysis Program

• A new root cause analysis process for SSEs was implemented based on a 3-meeting model developed by Healthcare Performance Improvement.²⁴ After an extensive investigation, meeting 1 was devoted to establishing facts about the event; meeting 2 was used to determine root causes; and meeting 3 was used to make recommendations for changes and develop action plans. The focus was on root cause(s), systems thinking, effective investigation techniques, credible findings, and actionable improvement plans.

• A common cause database was developed to support analysis of inappropriate actions, system and individual failure modes, and error types and to examine trends and inform improvement opportunities.

• For each identified SSE, a senior executive was immediately assigned to oversee an analysis and completion of action plans to prevent reoccurrence of similar events.

Lessons Learned Program

• Staff were given access to information, creating a highly visible, transparent feedback mechanism, emphasizing key concepts to promote and advance a culture of safety. The CCHMC intranet site, available to all employees, provided monthly updates on all patient safety goals, a patient safety tracker that displayed the number of days since the last SSE, safety stories, and a weekly report from the safety officer designed to reinforce expected safety behaviors, increase staff mindfulness and awareness, celebrate successful interventions, and share details of failures.

Tactical Interventions for High-Risk Areas

• Interventions to reduce perioperative SSEs included a list of expected safety behaviors specific to operating room safety developed by surgeons, embedding “time outs” and “debriefs” into standard surgical practice, providing visual cues in the operating room, standardized checklists, smoothing patient flow into the ICU,³⁸ parent review, real-time feedback, educational videos, and executive rounds.

Outcome Measures

The primary outcome measure was the rate of SSEs per 10 000 adjusted patient-days from January 2003 through June 2011. The secondary outcome was patient safety culture over 5 years (2005–2009).

Data Collection

Serious or permanent harm was defined by using a framework developed by Healthcare Performance Improvement²⁴ (Table 2). Harm could result from errors of commission or omission. A review panel consisting of the chief of staff, patient safety officer, and 2 attorneys from risk management reviewed all potential SSEs identified by an anonymous, web-based patient safety reporting system, supplemented by a trigger tool system,¹⁹ telephone call alerts, and direct communications to the safety officer. The panel met biweekly and used a standardized evaluation to determine performance deviation and assessment of harm. Emphasis was on consistency and precedent in determination of whether an SSE had occurred. Clinicians with subspecialty knowledge expertise were consulted in most cases.

TABLE 2.

SSE Level of Harm

• Severe temporary harm: detectable harm, lasting for a limited time only, resulting in no permanent injury, yet causing great discomfort, injury, distress, and/or additional procedure, surgery, or resuscitation

• Moderate permanent harm: detectable harm, not expecting change in clinical status, and is greater than minimal harm but less than severe harm (eg, permanent, significant organ dysfunction [loss of neurologic function])

• Severe permanent harm: detectable harm, not expecting change in clinical status, and causing great discomfort, injury, and/or distress (eg, permanent loss of organ function [renal failure])

• Death: death attributed to deviation in care

Framework developed by Healthcare Performance Improvement.²⁴

Each year since 2005, all patient care providers at CCHMC have been asked to anonymously complete the web-based AHRQ HSPSC.^27,28 The survey asks respondents to share their experiences and perceptions over the previous 12 months regarding 7 unit-level and 3 hospital-level dimensions of safety culture and 4 outcome variables. Demographic data were not collected to protect respondent anonymity.

Human Subject Protection

The institutional review board determined that this was a local systems improvement project and did not require their review and approval.

Analytical Methods

Statistical process control charts were used to examine the impact of interventions on the number of SSEs.^39,40 The number of events per 10 000 adjusted patient-days (including inpatient admissions, ED visits, and short stays) was tracked on a U (unit) chart based on count data with a Poisson distribution. The days between SSEs were tracked on a T (time between) chart, used to track rare events and with the expectation that the number of days between events would grow after intervention.⁴¹ The upper and lower control limits on the charts establish the margins within which the measurement will be found ∼99% of the time. Wilcoxon rank-sum test analyses of the change in the rate of events were also conducted.

For survey data, as recommended by AHRQ,²⁸ the percentage of positive responses was calculated (strongly agree/agree or always/most of the time) across items in each HSPSC dimension. Responses to negatively worded questions were reversed before calculating the percentage of positive responses. For patient safety grade, a positive response was denoted by a response of excellent or very good. For the number of events reported, a positive response was assigned if a respondent submitted >1 report in the previous year. Finally, the mean percentages for each dimension and overall respondents were calculated. Scores for each dimension could range from 0% to 100%, with higher scores indicating a more positive response. The mean percentage over each year of the survey was adjusted to account for individual differences in how long the respondent worked at CCHMC, in the current hospital area or unit, and in the current specialty or profession; the number of weeks per year usually worked; and whether the respondent had direct or indirect patient contact.

All data analyses were performed by using SAS version 9.2 (SAS Institute, Inc, Cary, NC). Adjusted mean percentages and 95% confidence intervals were calculated by using the LSMEANS option in PROC GLM. Statistical differences across years were obtained by using pair-wise multiple comparison tests with Bonferroni corrections.

Results

SSEs

After intervention implementation, the number of SSEs per 10 000 adjusted patient-days (Fig 2) significantly decreased from a mean of 0.9 to 0.3 (P < .0001). Days between SSEs (Fig 3) increased from a mean of 19.4 to 55.2 (P < .0001). During this same time, patient volume increased; monthly average adjusted patient-days were 13 686 during the baseline period and 17 521 during the study period. The reduction in SSEs occurred gradually but then seemed to stabilize at 0.3 SSE per 10 000 adjusted patient-days. We believe this reflects a combined effect from system improvements and cultural change.

FIGURE 2.

Number of SSEs per 10 000 adjusted patient days. Patient-days were adjusted to include inpatient admissions, ED visits, and short stays. LCL, lower control limits; UCL, upper control limits.

FIGURE 3.

Days between SSEs. LCL, lower control limits; UCL, upper control limits.

This combination of interventions, by nature, required time and relentless clarity of vision by the organization, with dedicated resources and focus on key clinical areas. As more employees were trained, we discussed methods to observe and measure performance of expected safety behaviors. We decided to focus, instead, on using the perceptions and qualitative feedback of safety coaches to reinforce behaviors and estimate the penetration of behaviors at the microsystem level. Senior leaders made transparent sharing of information regarding SSEs the expected norm, modeled the new culture through their actions, and directed attention to key clinical areas and processes by continuously focusing on results. The overall measure reflects performance across the organization, but the interventions required changes in dozens of clinical microsystems. Individual clinical areas adopted changes at varying paces, so persistence over time was vital to achieving the overall organizational goal. Specific tactical interventions were developed and implemented where they were needed, and results were monitored continuously at the microsystem level.³¹ Together, these interventions helped to focus leaders, physicians, and staff attention on patient safety and potential risks each day.

Patient Safety Culture

Response rates generally increased over time, especially for nurses, who received encouragement and incentives (eg, pizza lunches) to complete the survey. Between 2005 and 2007, there was a general decreasing trend in the mean percentage of respondents giving a positive response across several patient safety culture outcomes and dimensions (Table 3). However, significant improvements in positive responses in several outcomes and dimensions were seen between 2005 and 2009 and between 2007 and 2009.

TABLE 3.

Trends in Average Percent Positive Rating for Outcome Measures and Safety Culture Dimensions by Year

Variable	2005	2006	2007	2008	2009
Respondents, n	1541	1672	2741	2661	3752
Total employees, n	8676	9391	10 297	11 262	11 995
Response rate, %	17.8	17.8	26.6	23.6	31.3
Profession, %
Physician	12.2	10.4	6.9	9.7	4.9
Nurse	31.5	44.7	43.8	42.4	46.0
Other	56.3	44.9	49.3	47.9	49.1
Total	100.0	100.0	100.0	100.0	100.0
Outcome measures, % (95% CI)
Frequency of events reporteda	55.2 (52.6–57.8)	55.2 (52.7–57.8)	52.7 (50.6–54.8)	54.7 (52.6–56.9)	61.5* † (59.5–63.5)
Overall perceptions of safetyb	72.5 (70.5–74.5)	70.2 (68.2–72.1)	66.5‡ (64.8–68.1)	67.9 (66.2–69.6)	70.4* (68.9–72.0)
Patient safety gradec	82.6 (80.1–85.2)	79.9 (77.4–82.3)	76.6‡ (74.5–78.7)	79.9 (77.8–82.0)	84.0* (82.1–85.9)
No. of events reportedc	30.9 (26.6–35.2)	40.7 (36.5–44.9)	38.8 (35.2–42.3)	32.9 (29.3–36.5)	44.6* † (41.3–47.9)
Safety culture dimension (unit level)
Supervisor/manager expectations and actions promoting safetyb	75.1 (73.3–76.9)	73.3 (71.5–75.1)	72.0§ (70.6–73.5)	72.0 (70.5–73.5)	74.1 (72.8–75.5)
Organizational; learning, continuous improvementa	76.5 (74.7–78.3)	78.2 (76.4–80.0)	76.7 (75.3–78.2)	76.4 (74.9–77.9)	81.9* † (80.5–83.3)
Teamwork within hospital unitsb	83.6 (81.9–85.2)	83.8 (82.2–85.4)	81.8 (80.3–83.1)	81.5 (80.1–82.8)	82.7 (81.5–84.0)
Communication opennessa	66.1 (63.8–68.3)	62.6 (60.4–64.8)	60.9‡ (59.1–62.7)	61.4 (59.6–63.3)	65.2* (63.5–66.9)
Feedback and communication about errora	61.5 (59.1–63.9)	60.4 (58.1–62.8)	58.2 (56.3–60.2)	61.0 (59.0–63.0)	63.8* (62.0–62.6)
Nonpunitive response to errora	43.1 (40.7–45.5)	44.2 (41.8–46.5)	43.1 (41.1–45.1)	41.6 (39.6–43.6)	43.0 (41.1–44.8)
Staffingb	59.0 (57.1–61.0)	59.9 (58.0–61.8)	56.2§ (54.6–57.8)	57.8 (56.2–59.4)	59.5* (58.1–61.0)
Safety culture dimension (hospital level)
Hospital management support for patient safetya	69.3 (67.3–71.3)	69.3 (67.4–71.3)	66.1§ (64.4–67.7)	70.2 (68.5–71.9)	72.6* ‖ (71.0–74.1)
Teamwork across hospital unitsb	54.0 (51.7–56.3)	54.9 (52.7–57.1)	53.0 (51.1–54.9)	55.2 (53.3–57.1)	58.1* ‖ (56.4–59.8)
Hospital handoffs and transitionsb	34.1 (31.9–36.4)	40.2 (38.0–42.4)	38.3§ (36.4–40.2)	38.2 (36.3–40.1)	40.9† (39.2–42.7)

Response rates based on total medical center employees. Survey percentages are adjusted for years worked at CCHMC, years worked in current work area/unit, years worked in current specialty or profession, hours a week worked in hospital, and whether there was direct contact with patients. CI, confidence interval.

^aComposite of 3 items.

^bComposite of 4 items.

^cOne item.

*Significant change (P < .001) between 2007 and 2009;

^†Significant change (P < .001) between 2005 and 2009;

^‡Significant change (P < .001) between 2005 and 2007;

^§Significant change (P < .05) between 2005 and 2007;

^‖Significant change (P < .05) between 2005 and 2009.

Discussion

After implementation of cultural and system changes to improve patient safety, we observed significant reductions in the number of SSEs and increases in the days between SSEs. Although we did not reach our goal of 0.2 events, both statistical process control and Wilcoxon rank-sum analyses confirm these improvements to be statistically significant. More importantly, they are unquestionably clinically significant, equating to 62 fewer SSEs over the last 5 years. During the initial phase of the interventions, results from the safety culture survey worsened. However, as the initiative progressed, there was improvement.

Our experience of significant reduction in SSEs is the first that we are aware of in the published literature. Although we do not have data on which drivers or interventions most affected this change, our bundle of an error prevention system, restructured safety governance, a cause analysis structure, lessons learned program, and specific tactical interventions was strongly associated with fewer SSEs. We believe this set of interventions, designed to reach every employee, is likely to achieve similar patient safety advances in other health systems if adjusted to the local context.

A more positive safety culture has been associated with fewer adverse events in hospitals.⁴² Many hospitals have measured and reported their patient safety culture; however, there are few reports of multiple surveys to determine the change in patient safety culture over time. Our leadership was acutely aware of the initial decrease in some patient safety culture outcomes in the second and third surveys. Such results have been reported previously and may be due to the increased focus on patient safety and error prevention and the perception that change is not happening fast enough.^43–46 In addition, change takes time. Our leadership was confident that our interventions would eventually result in an improved safety culture.

Our overall response rates were calculated by using total employees as the denominator, rather than number of clinicians. Nonetheless, the improvement in the overall patient safety grade from 2007 (1 year after the interventions began) to 2009 was statistically significant. Although our study design did not allow us to attribute causation, our experience with this and other quality improvement initiatives leads us to believe that our interventions drove the culture change that was a prerequisite to safety outcomes improvement. In addition, our experience seems to affirm that process improvement initiatives alone are not sufficient to drive necessary culture change.

The major theoretical model for our approach was based on 5 key concepts guiding high reliability organizations: sensitivity to operations, reluctance to simplify, preoccupation with failure, deference to expertise, and resilience.^4,5 These organizations strive to create a culture and processes that drastically reduce system failures and effectively respond when failures do occur.⁷ Our systematic, whole-system approach to reducing SSEs included a transparent, consistent method for identifying safety events; a standardized taxonomy for classifying failures; and a strict method to determine root causes, drive development of specific interventions, and focus organizational attention.

Some limitations of our study are that the interventions occurred at a single site, we do not have data from a control hospital, and multiple interventions were made simultaneously, resulting in what, in describing the etiology of disease, has been called a web of causation.^31,47 We believe the simultaneous, multifaceted approach was crucial in changing behavior and culture and that no single change was responsible for the results obtained. In addition, this work was conducted over several years at a large pediatric medical center with a long history of improving care²⁰ and a robust infrastructure for tracking outcomes and harm. However, we believe these methods can be generalized to other health care organizations if adapted to address contextual factors.^48,49 Multisite studies and a detailed analysis of the percent penetration of the interventions will allow further improvement. For example, we are currently in a collaborative with all 8 children’s hospitals in Ohio to build improvement capability, reduce SSEs, and improve patient safety culture.

Improving patient safety is an ongoing challenge. We continue to focus on changing behavior and understand that this will require years of persistence. We believe additional factors and experience from outside health care will be important to further improvement. We have begun a new, organization-wide focus on situation awareness (ie, knowing what is going on around you and what is likely to happen next) to eliminate unrecognized clinical deterioration and delayed diagnosis as a cause of serious harm.⁷ We are also increasing emphasis on human factors engineering; that is, using knowledge about human behavior in system design and redesign.^7,50

Conclusions

Our multifaceted approach, focusing on culture and system changes, was associated with a significant and sustained reduction of SSEs. This approach may be applicable to other sites. Multisite studies are warranted to better understand contextual factors and the significance of individual interventions.

Glossary

AHRQ

Agency for Healthcare Research and Quality

CCHMC

Cincinnati Children’s Hospital Medical Center

HSPSC

Hospital Survey on Patient Safety Culture

SSE

serious safety event