Highly Reliable Procedural Teams: The Journey to Spread the Universal Protocol in Diagnostic Imaging

The Joint Commission’s Universal Protocol has been widely implemented in surgical settings since publication in 2003, and the elements are applied to procedures occurring in other health care arenas, in particular, diagnostic imaging. The teams underwent human factors training and then adapted key interventions used in surgical suites to their workflows. Perception of the safety climate improved 25% in interventional radiology and 4.5% in mammography. Perception of the teamwork climate decreased 5.4% in interventional radiology and 16.6% in mammography. The study reveals unexpected challenges and requires long-term effort and focus.

Abstract

Context:

The Joint Commission’s Universal Protocol has been widely implemented in surgical settings since publication in 2003. The elements improved patient safety in operating rooms, and the same rigor is being applied to procedures occurring in other health care arenas, in particular, diagnostic imaging.

Objective:

In 2011, Kaiser Permanente West Los Angeles’s Diagnostic Imaging Department desired to adapt previous work on Universal Protocol implementation to improve patient safety in interventional radiology and mammography procedures.

Design:

The teams underwent human factors training and then adapted key interventions used in surgical suites to their workflows. Time-out posters, use of whiteboards, “glitch books,” and regular audits provided structure to overcome the risks that human factors present.

Main Outcome Measures:

Staff and physician perceptions of the teamwork and safety climates in their modalities were measured using the Safety Attitudes Questionnaire at baseline and at 18 months after training. Unusual Occurrence Reports were also reviewed to identify events and near misses that could be prevented. Implementation of key process changes were identified as process measures.

Results:

Perception of the safety climate improved 25% in interventional radiology and 4.5% in mammography. Perception of the teamwork climate decreased 5.4% in interventional radiology and 16.6% in mammography. Unusual occurrences were underreported at baseline, and there is ongoing reluctance to document near misses.

Conclusion:

This work provides important considerations of the impact of departmental cultures for the implementation of the Universal Protocol in procedural areas. It also reveals unexpected challenges, and requires long-term effort and focus.

Introduction

The Joint Commission’s Universal Protocol¹ has been widely implemented in surgical settings across the country since its publication in 2003. Following implementation of the 3 key elements of the protocol—1) pre-procedure verification, 2) site marking, and 3) a time-out—rates of wrong site, wrong procedure, and wrong person surgery were shown to significantly decrease.² For example, Haynes et al³ measured the impact of the implementation at 8 hospitals in 8 cities as part of the World Health Organization’s Safe Surgery Saves Lives program in 2007 and 2008. The total inhospital rate of death dropped from 1.5% to 0.8%, and the rate of complications fell from 11% to 7%. They also reported that “the overall rates of surgical-site infection and unplanned reoperation ... declined significantly.”³

Institutions promoting patient safety are advocating for implementation beyond the surgical suite. According to a recent article in the Pennsylvania Patient Safety Advisory, “Despite quality improvement efforts, the prevalence of these errors in other disciplines, namely, radiology services, may be more common than generally expected and reported in the literature.”⁴ The work described in this article is the first reported work of its kind at Kaiser Permanente (KP), where the Universal Protocol has been applied to procedural areas in a Diagnostic Imaging Department to decrease errors and improve patient safety.

A three-pronged approach to improve safety in these settings was proposed: 1) use human factors concepts and training to create highly reliable procedural teams and to reduce procedural errors, 2) identify a core bundle of metrics that measure performance improvement and rigorously support improvement, and 3) implement simulation-based education.⁵ The literature in this arena further shows that teams that “exhibited less teamwork behaviors were at a higher risk for death or complications [and therefore] supports arguments in favor of human factors training for surgical teams.”⁶

In KP West Los Angeles (WLA), the goal of the Diagnostic Imaging Department’s leadership was to build on the recommendations in the arena of patient safety in the surgical suite and to apply them to diagnostic imaging procedures. This article will provide lessons learned and tools that other Diagnostic Imaging Departments can use, leading to improved reliability of patient safety in their interventions over time.

Methods

Literature Search

Literature searches were conducted in 2010 and 2012 for state-of-the-art patient safety procedures in interventional radiology and mammography. There are emerging publications addressing the transfer of learnings from the surgical arena to interventional radiology. In 2006, Children’s Hospital of Boston published its work about developing a checklist for interventional radiology, taking into account the unique challenges of the diagnostic imaging environment relative to the rapidly changing scope of interventional radiology.⁷

... behaviors that have been shown to create risks to patient safety and clinical outcomes. This included a discussion of avoidable distractions and limits to human memory, as well as unresolved conflict and lack of respect among team members.

This early work has been followed by more general recommendations. In 2008, Angle et al² proposed guidelines for implementation of the Universal Protocol in interventional radiology, and recommended that “each institution/hospital ... define thresholds as needed for the process steps ... to meet its quality improvement program needs.” In 2011, the Cardiovascular and Interventional Society of Europe created a task force to create a checklist for interventional radiology based on the World Health Organization surgical safety checklist. The intention was to create a template for interventional radiology care across Europe in recognition of the effectiveness of checklists in surgical suites and to document complication rates in interventional radiology.⁸ For example, Lewis et al⁹ reported a 0.2% complication rate in interventional radiology when vascular surgical intervention was needed.

Literature searches revealed a dearth of information on Universal Protocol implementation in mammography. In September 2012, after this work began at KP WLA, the Institute for Clinical Systems Improvement published a Non-Operating Room Procedural Safety protocol. The Institute provided tools and workflows for pre-procedural verification and time-outs, listing breast biopsies as one of the “invasive, high-risk or surgical procedures” to which their work applies.¹⁰ The American College of Radiology references the Universal Protocol and time-outs in their practice guidelines for breast procedures.¹¹

Participants

Leadership at KP WLA initiated this project and titled it “Highly Reliable Procedural Teams” (HRPT), following the model of KP’s previous implementation of “Highly Reliable Surgical Teams” (HRST), which expands on use of the Universal Protocol in KP surgical suites. Interventional radiology and mammography were identified as the targets for the work. Interventional radiology was identified by KP WLA leadership as an area of vulnerability, and the modalities had been identified as such by the American College of Radiology, as cited in Miller et al,¹² and by the previously mentioned Pennsylvania Patient Safety Advisory.⁴ Mammography was included in response to staff vocalizing concern with the safety climate and their inability to effect change.

A Steering Committee was established with key leaders from across the Medical Center, as well as staff from the department. Invited members included the senior leaders from the service line; Diagnostic Imaging departmental administrative and physician leadership; staff and physician leaders from both modalities; the patient safety officer; the quality physician lead; the Director of Risk Management; the Director of Quality and Accreditation, Regulation, and Licensing; and a project manager.

Measures

Safety Attitudes Questionnaires were completed by staff and physicians in the mammography and interventional radiology modalities before kickoff and 18 months later. This questionnaire has proved to be a psychometrically sound and highly reliable instrument on the basis of analysis of responses from 10,843 health care workers in 203 clinical settings and 3 countries.¹³ The Teamwork and Safety Climate Survey version of the Safety Attitudes Questionnaire was chosen for this project. It is made up of 27 questions organized into 2 sections, the results of which give the team a sense of overall teamwork and safety climates.

The second piece of data came from Unusual Occurrence Reports submitted electronically by staff over the two years before the project and throughout the project implementation. The KP WLA Risk Management Department maintains these data. Reports of events, including near misses, were sorted according to modality and type of event and were reported to the Steering Committee. Annual updates are conducted to measure impact.

Several process measures were identified as well. First, data on the percentage of staff and physicians that attended human factors training were collected. Participation rates were used to show engagement by leadership and staff in both modalities, and to establish a strong foundation on which to build. Additional process measures identified were the completion of selected deliverables identified by the Steering Committee.

Procedure

The Steering Committee met monthly to direct the structure of the project and to support work teams in each modality. A charter was established, and KP’s Rapid Improvement Model served as a guide for goal development and change management.¹⁴ The next step was administering baseline Safety Attitudes Questionnaires in early 2011. This was overseen by a project manager who was independent of the department to eliminate potential response bias. The KP National Environmental Health and Safety team provided data analysis support.

The HRPT project was formally kicked off with human factors training for all staff and radiologists involved in procedures in both modalities. The modalities were closed down for a half day, and attendance was required. The service line administrative and physician leads attended the session to establish the case for the work and to show leadership support. KP leaders in patient safety adapted the surgical training to the specifics of imaging modalities for our project.

Human factors training covered three topics. First was to build the case for HRPT using examples of errors and the resulting costs, including patient suffering, staff morale, and financial consequences. Second was to review the research on behaviors that have been shown to create risks to patient safety and clinical outcomes. This included a discussion of avoidable distractions and limits to human memory, as well as unresolved conflict and lack of respect among team members. The third topic was to define a highly reliable team and highly reliable processes. The overarching message was that “the goal is to take a team of experts and create an expert team.”⁵

Following the training, the mammography and interventional radiology modalities established separate work teams to engage in targeted projects identified as best practices during previously implemented surgical safety work. The teams were directed to meet twice a month to build momentum and facilitate regular discussion of progress. The deliverables accomplished were as follows: solidifying all pieces of the time-out process, creating a poster of the checklist to reference in examination rooms, and implementing the new process for every procedure in both modalities. Ongoing projects included using “glitch books” (books used to capture glitches then document and track follow-up) for timely communication and follow-up on issues, use of whiteboards for patient information and flow, development and implementation of a monitoring tool, and an effective hand-off communication process. Work teams reported monthly to the Steering Committee.

Interventional radiology’s time-out poster (Figure 1) was posted and referenced in each procedure room. The poster clearly demarcated who was responsible for checking each element before the procedure and incorporated the patient in the time-out whenever possible. A debriefing process was also detailed. Use of this workflow was reinforced through monitoring and regular feedback. A similar poster was created for mammography procedures.

Figure 1.

Highly Reliable Procedural Teams’ time-out poster for interventional radiology.

DNR = do not resuscitate; KP = Kaiser Permanente; lab = laboratory; Med = medication; MR# = medical record number; prep = preparation; RN = registered nurse; Tech = technician.

Results

Safety Attitudes Questionnaire results from baseline in 2011 and follow-up in 2012 can be seen in Figure 2. A greater than 90% response rate for staff and physicians was achieved for both modalities surveyed. At baseline, 92.9% of respondents in interventional radiology rated the teamwork climate “good,” and 50% rated the safety climate “good.” A score above 80% indicated a good culture.^15,16 At the 2012 follow-up, the interventional radiology modality’s teamwork climate rating decreased 5.4% and the safety climate rating increased 25%. In contrast, at baseline just 42.9% of mammography respondents rated the teamwork climate as good, and 42.9% rated the safety climate as good. At follow-up, the mammography modality’s teamwork climate rating decreased 16.6%, and the safety climate rating increased 4.5%. As is seen in other settings, there was a discrepancy between physician and staff perception of teamwork in both modalities, with physician ratings more than 15% higher in both areas.^17,18 Notably, the drop in teamwork climate in mammography was entirely driven by a decrease in staff perception, whereas physician perception remained the same.

Figure 2.

Kaiser Permanente West Los Angeles Diagnostic Imaging Department’s teamwork and safety climate scores over time.

Unusual Occurrence Report data from 2009, 2010, and 2011 showed 4, 10, and 11 items reported per respective year for the interventional radiology modality and 5, 0, and 5 items reported per year for mammography. Since kickoff, no sentinel events or near misses were reported in either modality. The goal continues to be accurate reporting of sentinel events and near misses, as well as to have zero sentinel events and minimal near misses in 2011 and beyond.

The percentage of staff and physicians that attended human factors training was tracked. For both modalities, 94% of staff and 100% of radiologists attended the training. Critical event team training was more recently completed for the interventional radiology team. This was done using simulation, a recommended alternate way to improve teamwork and safety outcomes.

Discussion

The HRPT implementation journey at KP WLA led us down 2 disparate paths. At the outset, it was clear that the cultures in the mammography and interventional radiology modalities were very different, as the baseline Safety Attitudes Questionnaire data reflected. In particular, the teamwork climate varied greatly between the 2 groups (92.9% for interventional radiology and 42.9% for mammography).

The interventional radiology team is historically a small and stable group, with strong internal leadership. The team’s culture has promoted effective communication among all team members, evidenced by the high baseline and follow-up results for teamwork climate. This culture, which has historically been a strength, revealed itself as a challenge during implementation. As Jim Collins¹⁹ writes in Good to Great, “To go from good to great requires transcending the curse of competence.... Just because you’ve been doing it for years, or perhaps even decades does not necessarily mean you can be the best in the world at it.” The interventional radiology team’s strong self-perception of doing high-quality work became a hurdle for implementing the changes necessary to hardwire the highest level of safety. This is evidenced by inconsistently used checklists, debriefings, and glitch books as well as underreporting of near misses that continues after implementation. Safety climate scores increased to 75% because of HRPT efforts, and with continued focus should continue to move above the 80% benchmark for a good score.

Before HRPT, the mammography staff voiced concern over their discomfort in speaking up around issues of patient safety. Human factors training provided the needed entrée to open lines of communication in this modality. This increased awareness of the shared responsibility to improve coordination and communication was likely the cause of the decreased teamwork scores, because inconsistent physician engagement continues to be a barrier in achieving true collaboration and has increased staff’s frustration. The safety climate scores remained stable, and as new members join the department, we are encouraged by their commitment to this work and their impact on future results of the Safety Attitudes Questionnaire.

Although the issues in either modality are unique, a higher-level reason these presented as problems is that the program was initiated by senior leadership, rather than driven by departmental leadership. A complicating factor was that since kickoff of the project, the Diagnostic Imaging Department and both modalities have hired new managers. This turnover made it difficult to remain consistent in use of the improved processes. Most recently, the Steering Committee has reengaged around consistent audits, structured time-outs, and use of whiteboards. We look forward to the impact of our refined focus as the new managers and physicians engage around tightened requirements. In addition, as part of orientation, staff is oriented to the HRPT standards to ensure they are clear on expectations for procedures from Day 1.

The Steering Committee believed that the baseline data for Unusual Occurrence Reports reflected an overall low level of reporting, a problem regularly discussed in the patient safety literature.^20,21 This is evidenced by near misses that have been discussed during Steering Committee meetings but continue to not be entered into our Unusual Occurrence Report system. Continued underreporting of unusual occurrences leaves us unaware of the true levels of errors and near misses. It has been communicated to staff that increased reporting of errors or near misses is evidence of a more open and just environment, and that the group’s intention is to focus on learning from mistakes in the system.

Conclusion

Implementation of the Universal Protocol in procedural areas in diagnostic imaging is increasingly important as the scope and volume expands. This work has shown that cultural change is paramount in achieving patient safety goals. Cultural change here includes physician and staff engagement in changing how work is done. Health care professionals must gain comfort with and a willingness to report, discuss, and learn from errors. Finally, health care teams should not allow a culture of good to hinder them from moving to a culture of great.

To Be Better

Freedom is nothing but the chance to be better.

—Albert Camus, 1913–1960, French Nobel Prize-winning author, journalist, and philosopher