Implementation of a Surgical Safety Checklist: Impact on Surgical Team Perspectives

Abstract

Background

The World Health Organization (WHO) surgical safety checklist has been shown to decrease mortality and complications and has been adopted worldwide. However, system flaws and human errors persist. Identifying provider perspectives of patient safety initiatives may identify strategies for improvement. The purpose of this study was to determine provider perspectives of surgical safety checklist implementation in an effort to improve initiatives that enhance surgical patients' safety.

Methods

In September 2010, a WHO-adapted surgical safety checklist was implemented at our institution. Surgical teams were invited to complete a checklist-focused questionnaire 1 month before and 1 year after implementation. Baseline and follow-up results were compared.

Results

A total of 437 surgical care providers responded to the survey: 45% of providers responded at baseline and 64% of providers responded at follow-up. Of the total respondents, 153 (35%) were nurses, 104 (24%) were anesthesia providers, and 180 (41%) were surgeons. Overall, we found an improvement in the awareness of patient safety and quality of care, with significant improvements in the perception of the value of and participation in the time-out process, in surgical team communication, and in the establishment and clarity of patient care needs. Some discordance was noted between surgeons and other surgical team members, indicating that barriers in communication still exist. Overall, approximately 65% of respondents perceived that the checklist improved patient safety and patient care; however, we found a strong negative perception of operating room efficiency.

Conclusion

Implementation of a surgical safety checklist improves perceptions of surgical safety. Barriers to implementation exist, but staff feedback may be used to enhance the sustainability and success of patient safety initiatives.

INTRODUCTION

Since the Institute of Medicine published To Err Is Human, a significant focus in surgery has been to identify strategies to improve patient safety and prevent postoperative complications and adverse events.¹ The World Health Organization (WHO) surgical safety checklist developed from the WHO Global Safety Challenge “Safe Surgery Saves Lives” campaign² and has decreased mortality and complication rates in the perioperative period.^3-5 Humans are fallible, and this checklist enhances consistency in surgical team performance at critical times, fostering good communication, teamwork, and a culture of patient safety.⁶ The simplicity of the checklist has been cited as a benefit that allows for rapid (within 1 month) and effective implementation without significant cost.³ The checklist has garnered significant worldwide enthusiasm, with programs implemented in 26 countries and more than 3,000 hospitals worldwide within 3 years of its introduction.²

Despite the high level of enthusiasm and global use of the checklist, barriers to its effective implementation and adoption exist.^7-10 Common barrier themes include redundancy, poor communication between surgical team members, negative perceptions of efficiency, and a lack of understanding about and commitment to the process. Although effective implementation strategies have been proposed,⁷ the high degree of variability in operational and cultural factors among hospitals, surgical services, and surgical team members requires flexibility and even modification of these strategies. Quantitative and qualitative assessment questionnaires have been used to assess surgical team member perspectives and attitudes toward quality improvement and patient safety initiatives, as well as to identify communication gaps between surgical team providers.^6,8,11-13 Using surveys as tools for pre- and postimplementation evaluation can provide valuable information for identifying and implementing meaningful and effective change. The purpose of this study was to evaluate provider perspectives of team communication, patient safety, patient care, and operative efficiency before and after implementation of a WHO-adapted surgical safety checklist in an effort to enhance surgical safety at our institution.

METHODS

Study Design

A pre/post study design evaluated surgical team provider perspectives using multimethod (quantitative/qualitative) analysis before and after implementation of a WHO-adapted surgical safety checklist. The study was primarily quantitative, with complementary qualitative strands. The study was approved by the institutional review board.

Setting

The study was conducted at Scott & White Memorial Hospital, a >500-bed tertiary care hospital affiliated with the Texas A&M University Health Science Center College of Medicine in Temple, Texas. Our institution has a strong commitment to patient quality and safety, with leadership positions at the system, institutional, and department levels. Checklist implementation was a team effort led by individual members/champions from administration, perioperative services, and surgical-related departments. This effort was well supported and started 1 year before the checklist start date of September 1, 2010 (Figure 1). The preimplementation period involved a multidisciplinary checklist design team instructed to use the WHO surgical safety checklist as a framework and encouraged to incorporate institutional needs, including quality performance goals. Three surgeon champions conducted a focused trial.

Figure 1.

Design and implementation timeline of the Scott & White Memorial Hospital Surgical Safety Checklist (S&W SSCL). Perceptions surveys were distributed 1 month before (PRE) and 1 year after (POST) checklist implementation.

After adjustments were made, hospital leadership (the chief medical officer, marketing managers, champions in the Departments of Surgery and Anesthesiology, and operating room [OR] nursing leadership) and the design team created an education campaign focused on the checklist and implementation date (September 1, 2010). The education campaign consisted of a mandatory, online education module with a posttest for continuing medical education credit, an educational video showing the checklist process and elements involved, campaign visuals located in the OR and perioperative service area, and weekly frequently asked question communications to surgical team members regarding the benefits of the checklist and the checklist process.

During the first week after implementation of the checklist, program champions were available in the preoperative holding area and OR to monitor completion, reinforce checklist use, and provide coaching on how to go through the checklist. For the first year, random spot-auditing, monitoring, and reinforcement helped to ensure surgical team participation and completion. A survey regarding elements of the checklist and provider perceptions of team communication, patient safety, patient care, and OR efficiency was distributed to all surgical team members 1 month before (PRE) and 1 year after (POST) implementation.

Participants

All surgical team members—including surgeons, anesthesia providers (physicians and nurse anesthetists), and nursing staff—were invited to complete the questionnaire. A total of 824 providers were included in the electronic distribution list: 469 in the PRE group and 355 in the POST group. The POST group had a lower distribution number because of natural attrition; and only those who were in the PRE group were included in the POST group. Demographic information about provider subgroups was collected; however, the questionnaire did not ask for specific participants' demographics to ensure anonymity and encourage completion of the survey.

WHO-Adapted Surgical Safety Checklist

The intent of the checklist is to enhance safety, quality, and surgical outcomes by improving teamwork, communication, and consistency. The Scott & White WHO-adapted checklist (Figure 2) includes 4 specific phases of patient care: check in (before entering the OR), sign in (before anesthesia), time out (before skin incision), and sign out (before exiting the OR). The members of the surgical team responsible for the checklist are the surgeon, anesthesia provider, and nurses.

Figure 2.

Scott & White Memorial Hospital's World Health Organization–adapted Surgical Safety Checklist.

Questionnaire Development

After reviewing the literature, we designed a series of questions to determine the providers' perspectives of our checklist and its implementation. Survey questions focused on topics and themes related to surgical team communication, patient safety, patient care and quality measures, and teamwork. The final survey consisted of 28 questions: a Likert-type scale and yes/no and multiple choice questions. Exclusive to the POST survey were 3 additional Likert-type questions asking about the impact of our checklist on patient care, patient safety, and OR efficiency. The POST survey also included open-ended questions about quality improvement, such as “What recommendation(s) do you have to improve the Scott & White surgical safety checklist?”

Questionnaire Administration

The Scott & White WHO-adapted checklist was implemented on September 1, 2010. Baseline evaluation of providers' perspectives was collected 1 month prior to the launch date. During the month following the 1-year anniversary of implementation, providers' perspectives were collected again. The questionnaire was confidential, anonymous, and administered via SurveyMonkey (Palo Alto, CA). The request to complete the questionnaire was sent to all surgical team providers via email, along with a link that directed providers to the web-based questionnaire. Participants were given 1 month to complete the survey. Three email reminders were sent before the survey response period closed. No incentives were provided for completing the questionnaire.

Data Analysis

Summary statistics, such as frequencies and percentages, were presented for all questionnaire items. The primary outcome measured was a change in perceptions 1 year following implementation of the surgical safety checklist. Baseline and follow-up outcomes were compared using chi-square tests for all provider subgroups and separately within each stratum of respondents (nurses, anesthesia providers, and surgeons). Statistical significance was defined as P≤0.05. Analysis was completed using SAS software, version 9.2 (SAS Institute Inc., Cary, NC).

Open-ended questions were analyzed using an iterative thematic qualitative approach. Study investigators completed independent content analyses of the data to generate broad themes. Themes were refined through an iterative process involving face-to-face meetings, reexamination of data, coding, and further refinement until a consensus was reached.¹⁴ Exemplars for each theme were identified and coded by hand. The frequency and percentage of codes were analyzed using Microsoft Office Excel 2007 (Microsoft Corp., Redmond, WA).

RESULTS

Survey participants and response rates are shown in Table 1. A total of 824 surgical team members were invited to participate in the survey study, and the overall response rate was 53% (n=437). The response rate was significantly higher in the POST group compared to the PRE group (64% vs 45%; P<0.01). Surgeons represented the majority of respondents (41%), followed by nurses (35%), and anesthesia providers (24%). However, the proportion of surgical team members responding in the PRE group was significantly different from the proportion of respondents in the POST group, with a lower percentage for nursing and higher percentages for the anesthesia and surgeon subgroups (P<0.05).

Table 1.

Participants and Response Rates of Surgical Team Perspectives Surveys

Questions in our survey focused on surgical team perceptions of patient safety, patient care, surgical team communication, and teamwork. The time-out process is a critical component of patient safety and our checklist; therefore, we evaluated surgical team perceptions of necessity, information verification, and surgical team member participation in the time-out process (Table 2) before and after implementation of our checklist.

Table 2.

Surgical Team Perceptions Related to the Time-Out Process

First, we determined whether surgical team members felt the time out was really necessary in all cases. We found a significant overall improvement in the total percentage of Yes responses from 81.9% in the PRE group to 94.3% in the POST group (P<0.01). The greatest improvement was seen in the surgeon (67.1% vs 90.1%; P<0.01) and anesthesia (77.3% vs 95.0%; P<0.01) subgroups.

Furthermore, a significantly higher proportion of respondents in the POST group reported 100% compliance in confirming patient identity, site of procedure (if indicated), and operative consent form once the patient entered the operating room (79.5% vs 92.1%; P<0.01). Although improvement was seen in all surgical team subgroups, the most significant improvement was seen in the anesthesia (59.1% vs 86.7%; P<0.01) and surgeon (87.3% vs 98.0%; P<0.05) subgroups.

An effective time-out process requires active participation of all surgical team members; therefore, we chose to determine the perception of effective and satisfactory participation in the time-out process according to each surgical team subgroup. We found no significant change in perception of effective participation in the nursing and surgeon subgroups. The anesthesia providers were the only surgical team subgroup that showed significant improvement following checklist implementation (64.8% vs 77.5%; P<0.01). Further evaluation of the data indicates that although only 71.8% of the POST respondents felt surgeons effectively participated in the time-out process, there was a large discordance in the surgical team perceptions of the surgeons' effective participation: 47% and 58.3% of nursing and anesthesia providers, respectively, responded Yes, while 96% of surgeons reported Yes. Collectively, these data indicate that implementation of a checklist improves perceptions of value in the time-out process and improves perceptions of participation in the components of the time out. However, perceptions of effective participation of surgical team members vary, with dramatic discordance regarding the surgeon group.

One of the proposed benefits of the checklist is to improve surgical team communication. Figure 3A-D shows perceptions of surgical team communication related to patient care and operative procedure. We found overall significant improvement in the perceptions of effective communication regarding equipment needs and availability (14.8% vs 23.8%, P<0.05 [Figure 3A]), critical events or anticipated difficulties during the operation (35.2% vs 46.3%, P<0.05 [Figure 3B]), and surgical team debriefing for patient recovery and postoperative management (51.4% vs 65.6%, P<0.01 [Figure 3D]). We found no significant difference in perceived awareness of potential blood loss >500 mL in the PRE and POST groups (Figure 3C).

Figure 3.

Surgical team perceptions of key communication elements in the operating room. Survey questions corresponding to the graphs are as follows: (A) Do the circulator and scrub nurse always receive enough information about the case to ensure equipment and instruments are reserved and ready before the patient enters the operating room (OR)? (B) Does the surgeon always inform you of critical elements or anticipated difficulties before they occur? (C) Before the patient enters the OR for an operative case, are you always aware of a potential blood loss of >500 mL if it exists? (D) In your opinion, are the key concerns for patient recovery and management reviewed by the surgeon, nursing, and anesthesia? The scoring system was binomial Yes or No, except question 3A, which also included Do Not Know. The graphic representations of the results are reported as the percentage of Yes responses. *P<0.05 and ^†P<0.01 compared to the PRE values.

Furthermore, we found an overall improvement in the surgical team's perception that the surgeon is fully aware of the specific patient or procedure to be performed prior to entering the operating room (P<0.001) (Table 3). These data reveal significant improvement in 4 of 5 of the communication events. However, the responses of the surgeon subgroup and the nursing/anesthesia subgroups were sharply discordant. This finding suggests that the surgeons perceive improvement in patient care–related communication that the other members of the surgical team do not reciprocate or acknowledge.

Table 3.

Surgical Team Perceptions of Surgeon Awareness of the Operative Case

Next, we aimed to determine the effect of the checklist on surgical team perceptions of patient safety, patient care, and OR efficiency (Figure 4A-C). Figures 4A and 4B show that approximately 65% of all respondents agreed and only 10% disagreed that patient safety and patient care were improved by the checklist. We found no difference in the proportion of response categories (Agree, Neutral, and Disagree) among the surgical team subgroups. However, perceptions of OR efficiency (Figure 4C) were more negative, with nearly 35% of total respondents indicating that they disagree that the checklist improved OR efficiency. Although responses among the surgical team subgroups were not statistically significant (P=0.06), the greatest proportion of negative responses came from the anesthesia (44% disagree) and surgeon (35% disagree) subgroups. Collectively, these data indicate that implementation of a checklist is associated with an improved perception of patient safety and patient care by surgical team providers. However, this perception is accompanied by a concern about the negative impacts of a checklist on OR efficiency.

Figure 4.

Survey participants were asked whether the Scott & White Surgical Safety Checklist improved patient safety (4A), patient care (4B), and operating room efficiency (4C). The scoring system was based on a Likert scale ranging from 1 to 5, Strongly Disagree to Strongly Agree, respectively. Strongly Disagree and Disagree were grouped as Disagree, and Strongly Agree and Agree were grouped as Agree. The graphic representations of the results are reported as the percentages responding Disagree, Neutral, and Agree. We found no significant differences in the responses between provider subgroups.

The open-ended question asked in the POST survey—What recommendation(s) do you have to improve the Scott & White surgical safety checklist?—was answered by 86 (38%, n=227) respondents. The major themes identified were modification of the checklist (56%, n=48) and improve buy-in of the process (29%, n=25). Modification was further divided into subcategories, with the most common themes in decreasing frequency related to improvements in the checklist execution process, efficiency of its use, wording, and content. Although 2 (2%) respondents recommended elimination of the checklist, more than 17% recommended no change, and most respondents indicated that the checklist was complete and comprehensive.

DISCUSSION

The implementation of our WHO-adapted surgical safety checklist was successful, with overall improved awareness of patient safety, communication, and quality of care. Improvements were most noted in the perception of value and participation in the time-out process, surgical team communication, and the establishment and clarity of patient care needs. However, we found some significant disparities between the perceptions of surgeons and other members of the surgical team, indicating that barriers to effective communication exist. Although the general perception was that the checklist improved patient safety and patient care, respondents had significant concern about its perceived effect on OR efficiency. Use of this information will help us determine effective modifications and iterative changes to the checklist and process of execution as an ongoing quality improvement project at our institution.

It has been suggested that the simplicity of the checklist facilitates a rapid and effective implementation process³; however, questions have been raised about the ease of introduction.^15,16 Furthermore, introduction without the adequate training and education of team members may result in a check-box exercise of no relevance or perceived benefit. Our implementation process and timeline preceded the report by Conley and colleagues⁷ in which they describe effective surgical safety checklist implementation. Comparing our experience with their observations of an institution with highly effective implementation is a valuable exercise. They highlight 2 important themes in effective implementation: explaining why and showing how. Furthermore, institutions with highly effective implementation processes achieved these goals through active leadership, deliberate enrollment, extensive discussion and training, piloting, multidisciplinary communication, real-time coaching, and ongoing feedback.⁷ Figure 1 shows our implementation process through which we believe we achieved those goals. A surgeon champion was identified and worked with a multidisciplinary team for content development, a focused pilot trial, and implementation strategy for our checklist. The education campaign included a comprehensive mandatory online learning module (explaining why) with video examples of how to effectively perform the elements of the checklist (showing how). Each surgical team provider was required to successfully complete a posttest for certification. Our strategy for sustainability was to provide onsite coaching that progressively evolved to observation and monitoring. Our preimplementation time period lasted approximately 1 year, which may be perceived as protracted and not ideal for effective implementation.⁷ Currently, we are in the outcome and review phase and will be looking at checklist completion, compliance and accuracy data, and patient outcomes to determine the effectiveness of our surgical safety program. These data will provide direct feedback to strengthen teamwork, communication, and the commitment to a culture of safety within our OR.

The surgical safety checklist has been shown to improve postoperative morbidity and mortality.^3-5 These observations are associated with enhanced perceptions of teamwork and communication.^3,6,11 We found an overall improvement in perceptions of communication and teamwork after implementation of the checklist; however, there was obvious evidence of disparity between surgeons and nurses or anesthesia providers. The greatest discrepancy was in the perception of surgeon participation in the time-out process (Table 2) and communication of critical patient care information (Figure 3A-D). Our surgeons rated their participation or perception of patient care communication higher than the nursing and anesthesia provider subgroups. Our results are similar to those of other studies in which surgeons report a more favorable perception of teamwork and communication than other surgical team members.^11,17 These findings likely represent communication gaps that must be addressed because failures of communication in the OR are a common root cause of medical failures.^1,18 Given these findings, we anticipate the need to develop team-building programs and assess safety climate scores using the previously validated modified OR Safety Attitudes Questionnaire.^6,19 It would be beneficial to use this tool before and after any modification of our checklist or patient safety processes to determine the effectiveness of programmatic change on teamwork and communication.

Barriers to the implementation of quality initiatives exist on both cultural and structural levels. In our study, the disparity of perception between surgeon and nurse/anesthesia subgroups likely represents communication gaps. Poor communication between surgeons and surgical team members has been previously reported, and team training may be required for effective checklist use.^8,11,17 Other major barriers for the checklist include the duplication of items in existing checklists, time spent completing the checklist for no perceived benefit, lack of procedural understanding, and ambiguity.⁸ We found similar results in the answers to our open-ended question on recommendations for the improvement of our checklist. The most frequent theme was modification of the checklist, including the execution process, wording and content, and efficiency of use. The second most common theme was the improvement of buy-in from all surgical care providers, indicating that we may have incomplete fidelity of our checklist implementation. Overall, it was encouraging to see that the majority of respondents (∼65%) agreed that our checklist improved patient safety and patient care. However, we were surprised that the perception of OR efficiency was so negative (Figure 4C) because the checklist only takes about 1-2 minutes to complete. We anticipated that direct communication of provider needs would improve the perception of OR and surgical team efficiency. Although practical time and cost implications of checklist implementation have been made,^10,15,16 Hurlbert and Garrett reported²⁰ an improvement in OR efficiencies and outcomes following team training and implementation of preoperative briefings. They showed a consistent reduction in team issues (46% vs 21%), equipment issues (46% vs 21%), procedural events (46% vs 16%), and average number of circulator trips out of the OR per case (2.1 vs 1.6). Although these authors did not evaluate operative time or OR time, we anticipate these findings would reduce OR resource utilization and decrease average cost per case (direct and indirect). Given the strong negative feelings of our surgical care providers regarding the survey's impact on OR efficiency, it will be important for us to provide feedback on the effect of our checklist on operative time, OR time, cost, and other OR efficiency metrics.

Several limitations of our study are important to mention. First, we did not report surgical safety checklist completion, compliance, or accuracy rates; these data would have provided evidence about whether the checklist was being used as intended. We are in the process of obtaining these data because they are needed to compare perceptions with actual utilization and performance data; incomplete checklist compliance may signify a poor implementation process or sustainability strategy.^8,16,21 Second, our study did not report patient outcomes. These data are beyond the scope of this study but important in validating surgical safety checklist use. Investigators have used established databases such as the American College of Surgeons' National Surgical Quality Improvement Project to measure the effects of the checklist on patient outcomes⁵; its validation will provide means for rapid identification of objective evidence on outcome, establish visible institutional value, and strengthen effective participation. Third, some of the improvements seen in our provider perceptions survey may have been caused by respondents' desire to provide responses that demonstrated success of our checklist implementation. To minimize this effect, the checklist distribution and reporting were anonymous and voluntary, demographics were focused on surgical team provider subgroups that could not be traced back to individuals, and no compensation was provided for participation. Although we cannot determine if bias influenced providers' responses, we anticipate that process stakeholders with strong feelings (positive or negative) would respond honestly and in an unfiltered way if anonymity were protected. Finally, our data represent responses from individuals within teams rather than entire team perspectives. Therefore, our study provides a cultural or institutional perception of our checklist. Linking responses to direct team members would be valuable in determining high- and low-functioning surgical teams, which would allow for specific intervention and team-training exercises to improve effectiveness.

CONCLUSIONS

The implementation of our checklist improved perceptions of surgical safety and patient care themes across surgical care provider subgroups. We intend to use these quantitative and qualitative data for iterative changes and modifications to our checklist and surgical safety improvement programs. Institutions should consider staff feedback as a means to enhance the sustainability and success of implementation of a surgical safety checklist. These survey results can be used to identify barriers to implementation, determine needed areas of educational development, and distinguish system gaps that have the potential to improve patient safety.

This article meets the Accreditation Council for Graduate Medical Education and the American Board of Medical Specialties Maintenance of Certification competencies for Patient Care, Professionalism, Systems-Based Practice, and Practice-Based Learning and Improvement.