The World Health Organization Surgical Safety Checklist: An audit of quality of implementation at a tertiary care high volume cancer institution

Abstract

Background and Aims:

In 2007, the World Health Organization (WHO) implemented the Surgical Safety Checklist (SSC), which has enhanced the communication between the surgical team members, improved outcomes, decreased complications, and improved patient safety. However, for the checklist to be effective, proper implementation and compliance with the checklist are imperative. The aim of this study was to evaluate the quality of implementation of the WHO SSC during elective surgery at a tertiary referral cancer hospital in India.

Material and Methods:

In this prospective observational study, a trained research nurse passively observed the implementation of selected items from the modified version of the WHO SSC during elective surgeries and evaluated the compliance with the checklist, percentage of items for which the use of the SSC prompted an action, and level of interaction between the key team players during the conduct of the checklist.

Results:

We studied 200 surgeries for each part of the SSC. Compliance was 200 (100%), 156 (78%), and 153 (76.5%) for the first, second, and third part of the SSC, respectively. All the three parts were mostly initiated by surgeons [197 (98.5%), 92 (59%), and 136 (88.9%), respectively]. Overall, 131/2200 (5.95%) items in the checklist were carried out only after being prompted during the conduct of the checklist. The interaction between all three representatives was found in only 265/509 (52%) cases.

Conclusion:

The quality of implementation of the SSC was found to be suboptimal, with a definite scope for improvement. Compliance with all items on the checklist and active participation by all team members are crucial for successful implementation of the checklist.

Introduction

The delivery of healthcare is complex and riddled with the potential for errors due to human factors, system failure, or a combination of both. Surgery forms an important treatment modality with millions of surgical procedures performed world over. Complications are not uncommon and occur in 3–16% of all surgical procedures, with permanent disability or mortality rates ranging between 0.4 and 0.8% in all surgical procedures.[1,2] These figures are from the Western world, and it is likely that the incidence of these complications is higher in developing countries such as India. Many of these complications may be due to preventable or modifiable causes.[3]

Checklists or protocols are a common tool for preventing human errors in complex and high intensity areas of work. They have been shown to be valuable in various professions such as aviation and the armed services. In 2007, the World Health Organization (WHO) launched the “Safe Surgery Saves Lives” global campaign during which it identified key processes in the operative period that could potentially affect patient outcomes. These included inadequate anesthetic safety practices, avoidable surgical infection, and poor communication among team members. Based on these processes, the WHO implemented a Surgical Safety Checklist (SSC) [Appendix 1] for briefings in the operating room (OR).[4]

In 2009, a modified version of the WHO checklist [Appendix 2] was introduced in the major operation theatres (OT) at our hospital, which is a tertiary level cancer center in India. Previous research has indicated that the implementation of the SSC leads to a decrease in perioperative complications and the number of communication failures in the OT.[5,6] It has been observed that the use of the SSC is associated with the development of a better safety attitude among the operating personnel.[5] It has also been shown that there is a direct relationship between improved clinical outcomes associated not just with the introduction of checklist but with compliance to the checklist.[6] Therefore, all the benefits of the checklist are attainable only if the compliance and implementation are proper.

We decided to conduct an audit that could generate data on the quality of implementation of the checklist, 5 years after its introduction at our hospital.

Material and Methods

We conducted a prospective observational study from March 2015 to June 2015. The Institutional Ethics Committee of our hospital approved the study and granted waiver of patient consent. The trial was registered with the Clinical Trial Registry of India (CTRI/2015/03/005646). There are 13 major ORs in our hospital carrying out an average of two elective surgeries per OR per day. For the study, two ORs were randomly selected by a study investigator using randomly numbered chits daily.

In the selected OR, implementation of a modified version of the WHO SSC was observed passively by a trained research nurse. The nurse observed the first two parts of the checklist in the same patient; the third part was observed in the same or other patients depending on the duration of the surgery. On an average, four observations were made per day. OR personnel were completely unaware of the audit. The research nurse documented the following:

1. Whether the checklist was performed

2. Whether the checklist prompted a corrective action because an item was not performed (e.g., confirming availability of blood, procuring equipment, administering antibiotic)

3. Whether members of all three teams needed for the checklist were present and participating actively in the implementation of the checklist

4. Which team member initiated each part of the checklist.

Among the various items in the three parts of the checklist, we studied six elements in the first part (Sign-in), four elements in the second part (Time-out), and one element in the third part (Sign-out) of the checklist. We chose these particular elements because they prompted changes in behavior which could be easily recognized by a research nurse.

The primary outcomes of the study were to analyze the following:

1. The effectiveness of implementation of the SSC by looking at

   • (a)
     The overall compliance with the checklist (measured as the percentage of cases in which the items of SSC were implemented) and
   • (b)
     The number of items important in the perioperative period, which were picked up by the OT team members only after they were brought up during the conduct of the checklist.

2. The quality of implementation of the checklist which was studied by the level of interaction between the three team members (surgeon, anesthetist, and/or nurse) involved in the implementation of the checklist. This was measured as the percentage of cases in which members of all three teams needed for the checklist were present and participated actively, separately for the Sign-in, Time-out, and Sign-out parts of the checklist.

The secondary outcome was to identify the member of the operating team (surgeon, anesthetist, or nurse) who initiated each part of the checklist.

We selected a convenience sample of 200 cases for each part of the checklist. Data were entered and analyzed using SPSS software version 21.0 (SPSS Inc., CA, USA). Summary data are expressed as percentages.

Results

During a period of 6 months, we audited the implementation of checklist in 352 patients in whom 600 observations were analyzed (200 Sign-in, 200 Sign-out, and 200 Time-out).

The SSC was implemented in 509, out of the total 600 observations. Implementation of the first part of the checklist was seen to be done in all cases (200 observations – 100%). The second part was implemented in 156 cases (78%) and the third part in 153 cases (76.5%) [Figure 1].

Figure 1.

Compliance with SSC

Surgeons initiated the checklist in most of the cases in all the three parts of the checklist (98.5% – first part, 59% – second part, and 88.9% – third part). Anesthetists initiated the second part of the checklist in 41% of cases (1.5% – first part and 9.8% – third part). Nurses initiated the third part of the checklist in 1.3% of the cases (None – first part and second part) [Figure 2].

Figure 2.

Initiation of the SSC

The interaction among all three team members (surgeons, anesthetists, and nurses) was best during implementation of the second part of the checklist (78%). It was 24.5% for the first part and 62% for the third part of the checklist. Overall, there was interaction among all three team members in only 52% of cases [Figure 3].

Figure 3.

Interaction between team members

In the first part of the checklist, 71/1200 items (5.9%) were followed up only after being prompted during the conduct of the checklist. Twenty-four (3%) items in the second part and 36 (18%) items in the third part were followed only after prompting during the checklist. The most frequent item being prompted was during the third part of the checklist (instrument, sponge, and needle count) – 36 times (18%) [Table 1].

Table 1.

Items prompted by the SSC

Discussion

The WHO checklist [Appendix 1] consists of three main parts, which are implemented at specific time-points during the surgery: The first part (Sign-in) is done before administering anesthesia to the patient; the second part (Time-out) is done before taking the surgical incision; and the third part (Sign-out) is done before shifting the patient to recovery room. At each of these time-points, important information can be checked, communicated, and shared between all team members participating in the surgery. The WHO encourages modification of the checklist to suit local practices. In 2009, a modified version of the WHO checklist [Appendix 2] was introduced in the major OT in our hospital, a tertiary level cancer center in India. Six elements/items were added to the original checklist according to the requirements in our hospital. Because there is no circulating nurse in our OTs, it was decided that the checklist would be implemented by one of the three OT team members (surgeons, anesthetist, or nurse). In addition, it was decided that the third part (Sign-out) would be done before the closure of the surgical incision.

Surgery is an integral part of healthcare system all over the world. Literature has shown that at least half of the surgical complications are avoidable.[2] Various studies conducted in the Western world have shown that introduction and implementation of the SSC has significantly reduced surgical complications and improved patient outcomes.[7,8,9,10] The use of the SSC not only improves patient safety but also introduces a safety culture among the OR team members.[5,6,11] For a checklist to be effective, compliance is of vital importance.

In a systematic review in 2012, Borchard et al. observed that the overall compliance to the checklist ranged between 12 and 100% (mean-75%) with maximum compliance of 90% during the Time-out part.[11] In our study, the overall compliance to the checklist was 84.8%, but the maximum compliance was found during the Sign-in part, which was 100% in our study. The compliance rates for second (Time-out) and third part (Sign-out) were 78 and 76.5%, respectively. This reduced compliance for the third part of the checklist is consistent with the results from studies done in the Western world where the completeness of the checklist was lowest for the third part.[12] Our study results can be explained by the absence of a dedicated circulating nurse for implementation of the SSC. For the first part of the checklist, all three members of the operating team (surgeon, nurse, and anesthetist) are relatively unoccupied and free to perform the checklist, whereas during the remaining two parts, they are involved with clinical work.

In a retrospective study by Fourcade, the frequency of SSC items raising concern ranged from 1.5 to 1.9%, with common ones including forgotten administration of antibiotics, unexpectedly high risk of bleeding, incomplete preparation, and incomplete orders.[13] In our study, 6% (131) of important items were identified only during the conduct of the checklist. In terms of the individual items, the item that prompted an action maximum times was the “instrument count” done during the third part (23.5%) followed by specific instrument required for surgery, which is done in the first part (16%) and administration of antibiotic during the second part (12.2%). This shows the importance of conducting the checklist, as these important items would have been forgotten if not for the checklist and that would have put the patient at risk. It is also worrying that the item prompting a change in behavior maximum times was in the third part of the checklist which had the lowest compliance.

In a retrospective study by Paugham-Burtz, adequate communication between the team members was found in only 4% of cases with no communication at all in 27%.[14] In our study, all three staff members of the OT team (surgeons, anesthetists, and nurses) interacted with each other while implementing the checklist in 52% of the cases. In the rest of the cases, one or more of the team members did not participate in the checklist implementation. The interaction between all three members was highest (78%) during the second part of the checklist and lowest (24.5%) during the first part. The second part of the checklist is conducted just before the surgery starts. Hence all the team members have their undivided and complete attention toward its implementation unlike in the last part of the checklist where they are busy in the clinical work. This could explain the highest interaction between all the team members for the second part.

De Vries et al. observed differences in compliance with completion of the checklist between surgeons and anesthetists, in which 77% of surgeons completed the checklist, whereas only 35% of anesthesiologists completed the checklist.[15]

Our results showed that the surgeons initiated the checklist in 83.5% of all the cases, whereas the anesthetists initiated it in 16.1% cases followed by nurses who initiated it in only 0.4% cases. This is probably due to the OR culture in India, where nurses do not take a leading role. Equal participation of all the team members in the checklist is essential for the successful implementation of the checklist. Reduced initiation by anesthetists and nurses in our study strongly indicates a requirement of change in OR culture.

Studies have shown that teamwork in surgery improves outcomes, with high-functioning teams achieving significantly reduced rates of adverse events.[16]

Lingard et al. studied the effect of interprofessional checklist briefings in OR between the surgical team members and observed that these briefings reduced the number of communication failures in the OR and promoted team communication that was proactive and collaborative which helped in preventing errors.[17]

Conley et al. undertook semi-structured interviews with implementation leaders and surgeons using the checklist. The results showed that for a highly effective implementation it is important that it is clearly communicated “why” and “how” the checklist should be used. “How” refers not only to actual checklist execution but also to checklist introduction and support. Key points to explaining “why” were, for example, providing a rationale for checklist implementation, highlighting values that aligned the institution with the checklist and surgical staff recognizing their own role in patient safety. The success of the implementation of the checklist was much higher when it was led by a multidisciplinary team, which met regularly and spontaneously, than when the implementation was led and mandated by a single surgical staff member.[18]

Our study results indicate that appropriate measures are required to improve teamwork and communication between all the team members. Explaining the importance of the checklist, educating, motivating, and training all the staff members included in conduct of checklist is the key to make it more effective.

Our study has given us important inputs with respect to the compliance to the checklist and interaction between different team members after 5 years of the checklist implementation. Though we have 100% compliance during the first part of the checklist, the compliance during the second and third part needs improvement. The level of interaction between all three members is poor during the first and third part of the checklist. The initiation of the checklist is done by surgeons in most of the cases. In spite of the varied compliance, the percentage of change prompted by the items in checklist is 6%, which is significant.

From the available literature in the past, it is proven that compliance to the checklist is paramount in increasing the effectiveness of the checklist and in bringing a safety culture to the OR. Our study has provided us with an opportunity to take measures to further increase the compliance to our checklist, to encourage the interaction between the team members, and to be actively involved with greater participation and ownership of the process.

Several large studies have looked at the impact of the implementation of the SSC on patient outcomes such as infection rates, morbidity, and mortality.[19,20] One of the chief limitations of our study is that we did not have the data to study the effect of the SSC on these outcomes. Also, as the observations were carried out by a research nurse, we had to limit the observations to those items which could be easily picked up by the nurse. We did not look at hard outcomes such as infection, morbidity, and mortality. Also, we did not look at emergency cases. The compliance for the checklist is likely to be lower for emergency cases but more crucial at the same time, as more items are likely to be missed in an emergency situation. The most important strength of our study is that all the operation theatre personnel were blinded to the audit. This is the first such study from the Indian. Because most hospitals in India do not have a circulating nurse, it is likely that our results will be representative of OTs across India.

To improve the implementation of the checklist at our hospital, we have renewed our efforts by introducing mandatory department-wide training and reinforcement about the checklist and its benefits, involvement of multidisciplinary team to identify barriers in implementation, interactive sessions, immediate real-time feedback regarding the implementation of the checklist, and opinion from end-users. We have also suggested that compliance with the checklist should be used as a performance indicator with different surgical specialties. In conclusion, though the SSC is an important tool for improving patient safety, the quality of its implementation was found to be suboptimal, with scope for improvement. If the compliance to SSC is not good, efforts such as multidisciplinary teaching, training, and strict formulation of hospital or institutional policy should be done for improvement in implication.

Conclusion

The quality of implementation of the SSC was found to be suboptimal, with a definite scope for improvement. Compliance with all items on the checklist and active participation by all team members are crucial for successful implementation of the checklist.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Appendix 1: World Health Organization SSC

Appendix 2: Modified SSC