Abstract
Background
Surgical checklists (SCs) have been developed to enhance teamwork and facilitate handovers, thereby improving the safety of surgical patients in health care organisations. The aim of this study was to determine whether the implementation of a 39-item SC reduced mortality and surgical adverse events (AEs) in patients undergoing inpatient surgery.
Methods
A retrospective pre- and post-intervention study of two cohorts of surgical patients was conducted (n = 1602) in a tertiary teaching hospital. The patients' homogeneity was confirmed by studying 40 comorbidities, 13 analytical determinations and 14 patient- and intervention-related variables. A 39-item SC adapted from one by the World Health Organization was used. The primary endpoint was the occurrence of any AE, including death, within 30 days of the operation. Twenty-three types of AEs were analysed.
Results
Following implementation of the checklist, the rate of AEs per 100 patients decreased from 31.5% to 26.5% (P = 0.39), the rate of infectious AEs decreased from 13.9 to 9.6 (P = 0.037) and non-infectious AEs decreased from 17.5 to 16.8 (P = 0.82). For non-elective patients, total AEs decreased from 60.4 to 37.0 (P = 0.017). The proportion of patients with one or more AE decreased from 18.1% to 16.2% (P = 0.35), and the death rate at 30 days decreased from 1.5% to 0.9% (P = 0.35).
Conclusion
The overall AE rate did not decrease significantly between the two periods. However, the rate of infectious AEs and overall AEs in patients with non-elective admissions had statistically significant reductions. Further research is needed to determine how and in which patients SC introduction can work successfully.
Editorial comment: what this article tells us
This article describes the work needed to implement the Safe Surgery Checklist in a big hospital. It confirms how difficult it is to document a convincing effect on the rate of adverse events and mortality after implementing the checklist in a modern hospital where the quality level is already high and where there are many confounding factors.
Surgical interventions have been used in health care for over a hundred years.1 An estimated 234 million operations are performed every year.2,3 However, surgical interventions are not completely risk-free. The mortality rates for surgical patients in developed countries range from 0.4% to 0.8%, with an adverse event (AE) rate of between 3% and 17%.3,4
In Spain, the incidence of patients with AEs directly related to hospital care is 8.4%.5 Surgical specialities had the highest number of AEs, particularly vascular surgery (16.1%), whereas medical specialities presented with the lowest rates (3.6%).5 Additionally, several studies3,4 have suggested that at least half of surgical AEs may be preventable.
In 2008, the World Health Organization (WHO) published a set of recommendations for improving the safety of surgical patients. These recommendations included a surgical checklist (SC) containing 19 items to be checked in the operating room (OR) at three different times.6
Since the publication of the WHO checklist, more than 1790 health-care centres worldwide have implemented this SC.*
As is the case for any health-care-related technology, and despite being based on a valid theoretical model, the actual utility of the SC must be proved on the basis of prevented morbidity and mortality, with consideration of the characteristics of the organisations in which it is implemented. Despite the increasing use of SCs and hospitals' efforts to implement them,7,8 very few studies to date have analysed their effectiveness in terms of AEs and mortality reductions in patients undergoing surgery.1,9–14
The main aim of this study was to determine whether the use of SCs reduced AEs and 30-day mortality rates in patients undergoing surgical intervention at a tertiary teaching hospital.
Methods
Study design
This study was conducted at the Hospital de Navarra, a tertiary teaching hospital in northern Spain with 500 beds and 10 ORs (9 for planned operations and 1 dedicated to emergency and urgent surgeries) that performs approximately 5300 surgical interventions a year.
The Navarra Ethics Research Committee (Pyto 55/2014) approved this study (Comité Ético De Investigación Clínica, Pabellón de Docencia. Irunlarrea, 3. 31008 Pamplona. Spain, Protocol number Pyto 55/2014, 30 April 2014).
A pre- and post-intervention study with two retrospective cohorts of surgical patients was conducted. The cohorts corresponded to two samples of patients who underwent surgery in 2008, before the introduction of the checklist, and in 2010, after the checklist was implemented throughout the hospital. A multi-stage method was used for sample extraction. An initial stratification was performed to ensure that the samples maintained the proportion of patients treated by each surgical department. Once the number of patients corresponding to each department was determined, the sample was extracted by simple randomisation.
The reference population comprised 10,121 patients who underwent surgery in 2008 and 2010. The sample included 1602 patients (801 for each study period). Given the variability of the AE rates before and after the introduction of the checklist among the sites in the seminal article published by Haynes et al.,1 the sample size was calculated using the average AE rate. The parameters used to calculate the sample size were a pre-intervention AE rate of 11%, a post-intervention rate of 7% and respective alpha and beta values of 0.05 and 0.8.
Inclusion criteria
All adults with a minimum hospital stay of 24 h who underwent surgery were included in the study.
Data collection
The information required for the patients who met the inclusion criteria was obtained from the clinical and administrative database (CADB), the haemotherapy database and medical records. Qualified personnel who were trained to perform this task extracted the information according to a standardised procedure. To avoid bias during data collection, the data managers were unaware of the working hypothesis.
Table 1 displays the variables used to describe the general characteristics of the patients. Overall, 40 comorbidities, 13 pre-operative analytical determinations, 23 surgical AEs in the 30 days following the intervention and intra-operative mortality and mortality at 30 days post-intervention were analysed following the definitions established by the American College of Surgeons National Quality Improvement Program (ACS NSQIP).15
Table 1.
General characteristics of the patients studied.
| Pre-implementation | Post-implementation | |
|---|---|---|
| General characteristics | (n = 801) | (n = 801) |
| Female, no. (%) | 312 (38.9) | 348 (43.4) |
| Age, year | 61.2 ± 18,0 | 60.4 ± 18.4 |
| Non-elective hospital admission, no. (%) | 220 (27.5) | 216 (27.0) |
| Non-elective intervention, no. (%) | 138 (17.2) | 147 (18.4) |
| Length of preop stay | 1.65 (3.3) | 1.65 (3.5) |
| Diagnoses (CADB) | 4.3 (3.6) | 3.9 (3.1) |
| Procedures (CADB) | 2.3 (2.0) | 2.2 (2.1) |
| Comorbidities, risk factor or analytical alterations | 2.2 (1.76) | 2.2 (1.60) |
| Diagnoses related group weight | 2.7 (4.2) | 2.7 (3.6) |
| Duration of the surgical intervention (min) | 81.8 (71.3) | 90.1 (77.8) |
| Anaesthesia type, no. (%)† | ||
| General | 430 (53.7) | 520 (64.9) |
| Spinal | 279 (34.8) | 199 (24.8) |
| Epidural | 3 (0.4) | 5 (0.6) |
| Others | 89 (11.1) | 77 (9.6) |
| Surgical wound classification, no. (%) | ||
| Clean | 367 (45.8) | 341 (42.6) |
| Clean-contaminated | 325 (40.6) | 359 (44.8) |
| Contaminated | 35 (4.4) | 29 (3.6) |
| Dirty | 74 (9.2) | 72 (9.0) |
| ASA classification, no. (%) | ||
| 1 | 198 (26.0) | 195 (25.4) |
| 2 | 293 (38.4) | 292 (38.1) |
| 3 | 203 (26.6) | 222 (28.9) |
| 4 | 69 (9.0) | 58 (7.6) |
| Surgical specialty, no. (%) | ||
| Ophthalmology | 4 (0.5) | 2 (0.2) |
| Otorhinolaryngology | 20 (2.5) | 19 (2.4) |
| General surgery | 259 (32.3) | 249 (31.1) |
| Traumatology | 217 (27.1) | 208 (26.0) |
| Urology | 89 (11.1) | 98 (12.2) |
| Neurosurgery | 55 (6.9) | 56 (7.0) |
| Cardiac surgery | 65 (8.1) | 70 (8.7) |
| Thoracic surgery | 31 (3.9) | 32 (4.0) |
| Vascular surgery | 61 (7.6) | 67 (8.4) |
P < 0.05 in the comparison of pre-implementation vs. post-implementation phase. ASA, American Society of Anesthesiologists Physical Status; CADB, clinical and administrative database.
Intervention
In 2009, before the checklist was introduced, a pilot study was conducted in the Orthopaedics-Traumatology and Vascular Surgery Departments to evaluate the SC completion rate in 237 surgical interventions. The analysis of this pilot study led to some corrective actions, including a change in where pre-operative prophylactic antibiotherapy was administered (from the ward to the OR). Furthermore, 22 safe-surgery training sessions were held to raise awareness of the importance of completing the SC. A multidisciplinary team that included the medical heads of the Surgical and Anaesthesiology Departments, the head of the Surgical Nursing Department and staff members from the Preventive Medicine and Quality Control Department participated as lecturers in these sessions. The training sessions were evaluated using an anonymous questionnaire (n = 191, response rate 65%). Respondents gave the sessions a mean score of 7.2 (scale: 0–10) for both their overall satisfaction with them and the applicability of the material to their daily work.
In January 2010, the use of the SC was extended to all patients undergoing surgery at our hospital. The 39-item SC used (Table 2) is an adaptation of the SC issued by the WHO and was prepared by the Safe Surgery Committee at our hospital. The different elements were checked by the appropriate health-care professionals, who signed the corresponding section after completing the verification process. A paper format was used for the SC, and the Preventive Medicine and Quality Control Department assessed its completion.
Table 2.
Checklist items.
| Before the patient goes to the OR | |
|---|---|
| Nurse reviews | The patient is informed of the question reiteration |
| Identity | |
| Antibiotic prophylaxis | |
| Medical record including drug delivery sheet | |
| Cross-matching blood tests | |
| In the OR | |
|---|---|
| Anaesthesia professional reviews | Anaesthesia consent |
| Pre-operative medication | |
| Airway assessment | |
| Aspiration assessment | |
| Equipment checked | |
| ECG, pulse oximeter and non-invasive blood pressure | |
| Surgeon and anaesthesia professional review | Identity |
| Site and marked site | |
| Procedure | |
| Known allergy | |
| Risk for > 500 ml blood loss | |
| Cross-matching blood tests | |
| Surgeon reviews | Antibiotic prophylaxis within last 60 min |
| Blood transfusion consent | |
| Surgery consent | |
| Surgeon, anaesthesia professional and nursing team review | Critical or unexpected steps |
| Sterility, equipment issues or any other concerns | |
| Nurse before patient leaves OR | Sponge counts |
| Instrument and needle counts | |
| Specimen labelled | |
| OR book fulfilled | |
| Identity wristband placed | |
| Cross-matching wristband placed | |
ECG, electrocardiography; OR, operating room.
Outcomes
The primary end point was the occurrence of any AE, including death, within 30 days after the operation, even if the patient was no longer in the hospital at the time (Table 3). The definitions of AEs established by the ACS NSQIP were used.15
Table 3.
Adverse events, number and percentage of adverse events per 100 patients studied before and after introduction of checklist.
| Adverse events | Before |
After |
P value | ||
|---|---|---|---|---|---|
| (n = 801) |
(n = 801) |
||||
| No. | % (CI) | No. | % (CI) | ||
| Wound | |||||
| Wound infection | 57 | 7.1 (5.3 to 8.9) | 48 | 6.0 (4.3 to 7.6) | 0.419 |
| Wound disruption | 38 | 4.7 (3.3 to 6.2) | 52 | 6.5 (4.8 to 8.2) | 0.158 |
| Lung | |||||
| Pneumonia | 22 | 2.8 (1.6 to 3.9) | 11 | 1.4 (0.6 to 2.2) | 0.077 |
| Unplanned intubation | 8 | 1.0 (0.3 to 1.7) | 5 | 0.6 (0.1 to 1.2) | 0.579 |
| Pulmonary embolism | 1 | 0.1 (−0.1 to 0.4) | 0 | 1 | |
| Ventilator > 48 h | 18 | 2.2 (1.2 to 3.3) | 10 | 1.2 (0.5 to 2.0) | 0.181 |
| Kidney | |||||
| Renal insufficiency | 4 | 0.05 (0.0 to 0.1) | 1 | 0.01 (−0.1 to 0.4) | 0.374 |
| Urinary tract infection | 17 | 2.1 (1.1 to 3.1) | 14 | 1.7 (0.8 to 2.7) | 0.717 |
| Nervous system | |||||
| Stroke/CVA | 3 | 0.04 (0.0 to 0.8) | 3 | 0.04 (0.0 to 0.8) | 1 |
| Coma > 24 h | 7 | 0.09 (0.2 to 0.15) | 3 | 0.4 (0.0 to 0.8) | 0.342 |
| Peripheral nerve injury | 0 | 3 | 0.4 (0.0 to 0.8) | 0.250 | |
| Heart and vessels | |||||
| Cardiac arrest requiring CPR | 1 | 0.1 (− 0.1 to 0.4) | 5 | 0.6 (0.1 to 1.2) | 0.218 |
| Myocardial infarction | 0 | 1 | 0.1 (−0.1 to 0.4) | 0.317 | |
| Bleeding transfusions > 4 units | 12 | 1.5 (0.6 to 2.3) | 14 | 1.7 (0.8 to 2.7) | 0.844 |
| Thrombophlebitis | 4 | 0.5 (0.0 to 1.0) | 3 | 0.4 (0.0 to 0.8) | 1 |
| Sepsis | |||||
| Sepsis | 16 | 2.0 (1.0 to 3.0) | 4 | 0.5 (0.0 to 1.0) | 0.011 |
| Others | |||||
| Reintervention | 44 | 5.5 (3.9 to 7.1) | 34 | 4.4 (0.2 to 1.5) | 0.356 |
CI, confidence interval; CPR, cardiopulmonary resuscitation; CVA, cerebrovascular accident.
Information regarding retained foreign objects after surgery and wrong-site procedures was also collected. The AEs were classified into two groups: infectious and non-infectious. Surgical site infections, pneumonia, urinary tract infections and sepsis were considered infectious AEs, whereas the remainders were considered non-infectious AEs.
The proportion of patients with one or more complications and the number of AEs per 100 patients were reported.
For the 2010 cohort, the SC completion rate was evaluated in a consecutive sample of 452 interventions. Because written proof of how the different sections of the checklist evaluated was available, it was possible to evaluate both the percentage of surgical interventions with a checklist and the degree of completion of the checklist.
Confounders
The association between the checklist's implementation and AEs at 30 days, including mortality, may have been influenced by temporal changes in the baseline characteristics of the patient population. We considered gender, age, emergency status of surgery, type of anaesthesia, American Society of Anesthesiologists physical status and a set of comorbidities used by the ACS NSQIP15 as potential confounding factors in the analysis.
The values established by the Medical Lab Observatory were considered to be critical.16
Statistical analysis
Statistical analyses were performed using SPSS release 20 for Windows (IBM SPSS, Chicago, IL, USA). Pearson's χ2 tests or Student's t-tests were used to compare the baseline characteristics of patients before and after the checklist was implemented. Levene's test was used to check the homogeneity of the variances. A multivariate logistic regression analysis was used to adjust the association between SC implementation and outcomes for confounding factors. A P value of less than 0.05 was considered to indicate statistical significance.
Results
Proportion of patients with one or more AE
The proportion of patients with one or more AE was 18.1% during the baseline period and 16.2% after implementation of the checklist (P > 0.35) (Table 4). The odds of AEs were 14% higher for the patients in the baseline period than the period after the checklist was introduced, and the risk fraction due to the checklist was 12.3%. None of these values were statistically significant.
Table 4.
Outcomes: number and percentage of patients with one or more adverse events or exitus before and after introduction of checklist.
| Before |
After |
P value | Percentage difference after/before | OR | Etiologic fraction in exposed (EFe/OR) | |||
|---|---|---|---|---|---|---|---|---|
| No. | % (CI) | No. | % (CI) | |||||
| Patients with adverse events | ||||||||
| Total | 145 | 18.1 (15.5–20.8) | 130 | 16.2 (13.7–18.8) | 0.35 | −10.0.5 | 1.14 (0.9–1.8) | 12.3 (−13.7 to 35.4) |
| Infectious events | 88 | 11.0 (8.8–13.2) | 71 | 8.9 (6.9–10.8) | 0.18 | −19.4 | 1.26 (0.9–1.8) | 21.2 (−9.5 to 43.3) |
| Non-infectious events | 93 | 11.6 (9.4–13.8) | 93 | 11.6 (9.4–13.8) | 1 | 0 | ||
| Non-elective admission | 71 | 31.8 (25.6–37.9) | 44 | 20.4 (14.9–25.8) | 0.006 | −35.8 | 1.86 (1.2–2.9) | 46.3 (17.0 to 65.3) |
| Elective admission | 75 | 12.9 (10.2–15.6) | 86 | 14.7 (11.8–17.6) | 0.42 | 13.9 | 0.86 (0.6–1.2) | 14.0 (20.0 to 38.4) |
| Exitus | ||||||||
| Exitus day of surgery | 0 | 0 | ||||||
| Exitus within 30 days of surgery | 12 | 1.5 (0.7–2.3) | 7 | 0.9 (0.2–1.5) | 0.356 | −40.0 | 1.72 (0.7–4.4) | 42.3 (−47.9 to 77.3) |
CI, confidence interval; OR, operating room.
Although the proportion of patients with infectious AEs decreased from 11.0% in the baseline period to 8.9% after the checklist was implemented (P = 0.18), the proportion of patients with a non-infectious AE (11.6%) remained stable.
A total of 52 patients (6.5%) experienced more than one AE at baseline, whereas 48 (5.9%) did so after the checklist was implemented (P = 0.756).
The biggest decrease in the proportion of patients with AEs was observed for patients who were admitted on a non-elective basis (P = 0.006) (Table 4).
Number of AEs per 100 patients
The number of AEs per 100 patients was 31.5 at baseline and 26.5 after implementation of the checklist (P = 0.39) (Table 5).
Table 5.
Outcomes: number and percentage of adverse events per 100 patients studied before and after introduction of checklist.
| Before |
After |
Percentage difference before/after | P value | Difference |
||||
|---|---|---|---|---|---|---|---|---|
| No. | % (CI) | No. | % (CI) | No. | % (CI) | |||
| Total | 252 | 31.5 (25.3–37.6) | 212 | 26.5 (21.5–31.5) | −15.9 | 0.39 | 40 | 4.9 (−2.9 to 12.8) |
| Infectious | 112 | 13.9 (10.9–17.0) | 77 | 9.6 (7.4–11.9) | −30.9 | 0.037 | 35 | 4.3 (0.6 to 8.1) |
| Non-infectious | 140 | 17.5 (13.4–21.5) | 135 | 16.8 (13.2–20.5) | −4.0 | 0.82 | 5 | 0.6 (−4.8 to 6.1) |
| Non-elective admission | 133 | 60.4 (45.2–75.6) | 80 | 37.0 (25.2–48.8) | −38.7 | 0.017 | 53 | 23.4 (4.2 to 42.6) |
| Elective admission | 119 | 20.5 (14.5–26.4) | 132 | 22.6 (17.3–27.8) | 10.2 | 0.608 | 13 | − 2.0 (−10.0 to 5.9) |
CI, confidence interval.
Although the rate of infectious AEs dropped from 13.9 at baseline to 9.6 in the post-implementation period (P = 0.037), there was no significant change in the rate for non-infectious AEs.
A more detailed analysis of the reported AEs showed that only the rate of sepsis decreased significantly between the two periods (P = 0.011) (Table 3).
For non-elective patients, the AE rate was 60.4 at baseline and 37.0 after checklist implementation (P = 0.017). There were no significant changes in AE rates between the two periods of time for electively admitted patients.
Mortality
No patient died on the day of the intervention in either of the periods studied. Mortality after 30 days decreased from 1.5% in the baseline period to 0.9% after implementation of the checklist (P = 0.356) (Table 4).
Patient characteristics
The general patient characteristics did not differ between the pre- and post-implementation periods (Table 1) except for the type of anaesthesia used.
Only alcohol consumption (12.8% vs. 8.74%) and cerebrovascular disease with no neurological involvement (4.0% vs. 2.12%) decreased significantly after the SC was. Despite these differences, neither of these variables was related to a higher risk of suffering an AE in the multivariate logistic regression analysis.
Checklist verification
The percentage of patients who underwent a surgical intervention and had at least one SC item completed was 87% in 2010. For these patients 88% of the items were verified.
Discussion
Outcomes
AE
After the introduction of the SC, the percentage of patients with one or more AEs dropped from 18.1% to 16.2%, the post-operative AE rate per 100 patients dropped from 31.5 to 26.5, and the death rate went from 1.5% to 0.9%. These reductions did not reach statistical significance.
The results obtained differ from some articles published in the literature to date, which have demonstrated the effectiveness of the SC in preventing AEs and/or mortality in patients in a hospital setting.11
While the AE rate fell by 16% in our study, the AE rates fell by 36–65%1,12,14,17 in other hospitals, which are all statistically significant changes.
In contrast, mandatory adoption of SCs at hospitals in Ontario, Canada, did not produce significantly better outcomes.18
However, our study showed that after checklist implementation, the percentage of patients with non-elective admissions who experienced AEs fell by 39% and the AE rate fell by 36% (P < 0.05). The decrease in AEs in this specific patient population was also observed by other authors.13 These data support the importance of the SC in non-elective admissions, suggesting that the checklist effect might not be the same across patients or procedures with different risks.
Mortality
The mortality rates reported in previous studies showed statistically significant reductions after checklists were implemented.1,12,13,19 The mortality decrease in our study was very similar to the reported studies (40%), although the difference was not statistically significant, which is most likely due to a sample size effect. Bliss et al.17 found no statistically significant differences after implementation of an SC.
Baseline AE rates
The AE rates observed in our study are higher than those obtained in other studies,1,12–14,17 most likely due to the mix of patient cases (the hospital has Cardiothoracic, Vascular, Hepatobiliary, and Neurosurgery Departments but no Gynaecology/Obstetrics or Paediatric ones). Patients at tertiary teaching hospitals generally have a larger number of coexisting conditions and undergo more extensive procedures, increasing the likelihood of complications. The mix of cases could have an effect on the moderate reduction in AEs after the SC was introduced in our hospital.
Similarly, we analysed the 30-day AE and mortality rates, even for patients who had been discharged. The values reported would have been markedly lower if the AE and mortality rates had been studied only during hospital stays. In our study, 42% of the AEs occurred after patient discharge.
Methodology
The AE and comorbidity classification systems used in our study are the same as in the initial studies designed to measure SC effectiveness.1,13,17
Type of checklist and verification system
Our SC implementation strategies and the SC itself differ from those published previously, which could also help explain the different results obtained. For example, our SC has 39 items vs. the 19 on WHO's SC. Although some items on our SC can only be completed with the participation of various members of the surgical team, each professional involved in the surgical process was responsible for each item's verification. This requirement is a marked difference with respect to the SC proposed by the WHO, which recommends verification as a team. One of the advantages of our SC is that verification begins before the patient goes into the OR, thereby creating extra safety filters. Moreover, the SC used was the only one for which consensus was built among all of the professionals involved.
Implementation of the SC was highly recommended in our hospital, but not mandatory. There has recently been some controversy over the advantages of regulations to ensure compliance.18,20,21
Source of information
The fact that the information was collected retrospectively and at different periods of time could affect also the quality of the information collected. The source of information could also influence the results. We used clinical records, but other studies18 have used the CADB.
This study was conducted using data from 2008 and 2010. Data from 2009 were excluded because a pilot test was being performed in two surgical departments to study how to best introduce the SC throughout the hospital. During the pilot test, it was deemed impossible to comply with the SC without changing certain processes at the hospital. For example, in 2010, antibiotic prophylaxis was administered in the OR to ensure timely administration of it. These changes were considered to be part of the SC's implementation. Indeed, some authors12 have shown that the use of an SC optimises proper timing before an incision is made. However, initiatives involving more than one component make it more difficult to evaluate the SC's effectiveness. The 22 training sessions could have also influenced attitudes toward patient safety even before the intervention period began.
Study limitations
Design
A pre- and post-intervention study has some obvious limitations. However, the design of another type of study in which cases (patients with SCs) and controls (patients with no SCs) are assessed simultaneously would also present potential biases. For example, the introduction of an SC in some departments but not others would make a comparison of the results more difficult. The fact that some personnel (for example, anaesthesiologists) would treat cases and controls at the same time means that they could apply the SC to controls unconsciously.
During the period analysed, other quality-related factors might have been introduced, so it is not possible to conclude that the moderate improvement obtained was due exclusively to the SC. Furthermore, despite using a 39-item SC, there is no direct causal relationship between the SC and each of the AEs studied.
Confounders
Although all possible confounding factors have been considered by studying factors such as comorbidities and analytical alterations, there may nevertheless be some residual confounding factors that could explain the results obtained.
Sample size
The sample size was calculated based on the study performed by Haynes et al.1 However, the starting AE rate, and especially the lower than expected decrease after the SC was implemented, could have been related to the results not reaching statistical significance. The study may be underpowered for detecting clinically significant changes in overall AEs.
SC completion
The completion rate for the SC is another relevant factor. Indeed, to be effective, the SC completion rate must clearly exceed 80%.12 Other authors19 have demonstrated the effectiveness of an SC in reducing mortality only in patients for whom all of the SC items are verified, with no statistically significant differences observed for patients with partially completed SCs.
SC completion was measured in 2010 by analysing an SC sample. In contrast to other studies, there were no direct observers.1 The completion rates observed in our study (87% of interventions with checklists and 88% of items verified) did not differ markedly from those reported by authors such as de Vries et al.12 (80%), although the values reported in other studies ranged from 97.26%17 to 39%19 for patients with a completed SC.
The fact that the completion rate in our study was less than 100% could lead to an underestimation of the SC effect.
The implementation of an SC in any hospital is a major challenge. Indeed, numerous barriers to the effective implementation of a checklist have been identified.22 Emphasis has also been placed on the keys to success, such as the importance of leadership, adaptation of the SC to the local setting and the participation of professionals.23 In our hospital, health-care professionals and the management team led the project. Moreover, the professionals' perceptions regarding the utility of the SC and the main barriers to the SC implementation process have been evaluated. The health-care professionals considered the SC to be moderately useful (average 6.6; 1–10 scale), and 11.6% noted that actual errors had been avoided due to the SC.24
The impact of the SC can also be measured in terms of the health-care professionals' patient safety cultural change.25–27 Our study has concentrated exclusively on measuring final patient health outcomes. However, divulging the results is part of the strategy to improve the safety culture of our organisation. As a matter of fact, Bosk et al.28 stated ‘what a simple checklist can achieve is; on its own not much’. Safer care is achieved when organisations ‘summarise and simplify what to do; measure and provide feedback on outcomes; and improve culture by building expectations of performance standards into work processes’.28
In conclusion, our study showed that after the introduction of an SC, the proportion of patients with one or more AE, the number of AEs per 100 patients and the death rate decreased (11%, 16% and 40%, respectively), but these reductions did not reach statistical significance. Nonetheless, the rate of infectious AEs and the overall AEs in patients with non-elective procedures had statistically significant reductions. Further research is needed to determine the most effective way of introducing the SC and in which patients the SC can produce the highest reductions in post-operative AEs.
Acknowledgments
The authors thank Sergio Santana and Cristina Eslava for their valuable input during the study period.
Notes
World Health Organization. 2012 Surgical Safety Web Map. Accessed 18 January 2012 http://maps.cga.harvard.edu:8080/Hospital/
References
- 1.Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat A-HS, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MCM, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AA Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–499. doi: 10.1056/NEJMsa0810119. for the. [DOI] [PubMed] [Google Scholar]
- 2.Weiser T, Regenbogen S, Thompson K, Haynes A, Lipsitz S, Berry W, Gawande A. An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet. 2008;372:139–144. doi: 10.1016/S0140-6736(08)60878-8. [DOI] [PubMed] [Google Scholar]
- 3.Gawande A, Thomas E, Zinner M, Brennan T. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery. 1999;126:66–75. doi: 10.1067/msy.1999.98664. [DOI] [PubMed] [Google Scholar]
- 4.Kable A, Gibberd R, Spigelman A. Adverse events in surgical patients in Australia. Int J Qual Health Care. 2002;14:269–276. doi: 10.1093/intqhc/14.4.269. [DOI] [PubMed] [Google Scholar]
- 5.Aranaz J, Aibar C, Vitaller J, Ruiz P. Estudio Nacional sobre los Efectos Adversos ligados a la Hospitalización. Madrid: ENEAS, Ministerio de Sanidad y Consumo; 2005. [Google Scholar]
- 6.World Health Organization. Word alliance for patient safety. implementation manual, surgical safety checklist (first edition). Available at: http://www.who.int/patientsafety/safesurgery/tools_resources/SSSL_Manual_finalJun08.pdf?ua=1 (accessed 20 January 2009)
- 7.Clark S, Hamilton L. WHO surgical checklists. Needs to be customised by specialty. BMJ. 2010;340:c589. doi: 10.1136/bmj.c589. [DOI] [PubMed] [Google Scholar]
- 8.Mahaffey PJ. Seductions of the WHO safe surgery checklist. BMJ. 2010;340:c915. doi: 10.1136/bmj.c915. [DOI] [PubMed] [Google Scholar]
- 9.Borchard A, Schwappach D, Barbir A, Bezzola P. A systematic review of the effectiveness, compliance, and critical factors for implementation on safety checklist in surgery. Ann Surg. 2012;256:925–933. doi: 10.1097/SLA.0b013e3182682f27. [DOI] [PubMed] [Google Scholar]
- 10.Sewell M, Adebibe M, Jayakumar P, Jowett C, Kong K, Vemulapalli K, Levack B. Use of the WHO surgical safety checklist in trauma and orthopaedic patients. Int Orthop. 2011;35:897–901. doi: 10.1007/s00264-010-1112-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Thomassen Ø, Storesund A, Søfteland E, Brattebø G. The effects of safety checklists in medicine: a systematic review. Acta Anaesthesiol Scand. 2014;58:5–18. doi: 10.1111/aas.12207. [DOI] [PubMed] [Google Scholar]
- 12.de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, van Helden SH, Schlack WS, van Putten MA, Gouma DJ, Dijkgraaf MG, Smorenburg SM, Boermeester MA, Group SC. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010;363:1928–1937. doi: 10.1056/NEJMsa0911535. [DOI] [PubMed] [Google Scholar]
- 13.Weiser TG, Haynes AB, Dziekan G, Berry WR, Lipsitz SR, Gawande AA Safe Surgery Saves Lives Investigators and Study Group. Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg. 2010;251:976–980. doi: 10.1097/SLA.0b013e3181d970e3. [DOI] [PubMed] [Google Scholar]
- 14.Askarian M, Kouchak F, Palenik CJ. Effect of surgical safety checklists on postoperative morbidity and mortality rates, Shiraz, FaghihyHospital, a 1-year study. Qual Manag Health Care. 2011;20:293–297. doi: 10.1097/QMH.0b013e318231357c. [DOI] [PubMed] [Google Scholar]
- 15.American College of Surgeons. User guide for the 2010 participant use data file. National Surgical Quality Improvement Program. Available at: http://site.acsnsqip.org/wp-content/uploads/2012/03/2010-User-Guide_FINAL.pdf (accessed 18 January 2012)
- 16.Medical Laboratory Observer. Clinical laboratory reference. Available at: http://www.clr-online.com/CLR201213-Table-of-Critical-Limits.pdf (accessed 13 January 2013)
- 17.Bliss LA, Ross-Richardson CB, Sanzari LJ, Shapiro DS, Lukianoff AE, Bernstein BA, Ellner SJ. Thirty-day outcomes support implementation of a surgical safety checklist. J Am Coll Surg. 2012;215:766–776. doi: 10.1016/j.jamcollsurg.2012.07.015. [DOI] [PubMed] [Google Scholar]
- 18.Urbach DR, Govindarajan A, Saskin R, Wilton AS, Baxter NN. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370:1029–1038. doi: 10.1056/NEJMsa1308261. [DOI] [PubMed] [Google Scholar]
- 19.van Klei WA, Hoff RG, van Aarnhem EE, Simmermacher RK, Regli LP, Kappen TH, van Wolfswinkel L, Kalkman CJ, Buhre WF, Peelen LM. Effects of the introduction of the WHO ‘Surgical Safety Checklist’ on in-hospital mortality: a cohort study. Ann Surg. 2012;255:44–49. doi: 10.1097/SLA.0b013e31823779ae. [DOI] [PubMed] [Google Scholar]
- 20.Leape LL. The checklist conundrum. N Engl J Med. 2014;370:1063–1064. doi: 10.1056/NEJMe1315851. [DOI] [PubMed] [Google Scholar]
- 21.Robblee JA. Surgical safety checklists in Ontario, Canada. N Engl J Med. 370:2349. doi: 10.1056/NEJMc1404583. [DOI] [PubMed] [Google Scholar]
- 22.Fourcade A, Blache JL, Grenier C, Bourgain JL, Minvielle E. Barriers to staff adoption of a surgical safety checklist. BMJ Qual Saf. 2014;21:191–197. doi: 10.1136/bmjqs-2011-000094. 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Conley DM, Singer SJ, Edmondson L, Berry WR, Gawande AA. Effective surgical safety checklist implementation. J Am Coll Surg. 2011;212:873–879. doi: 10.1016/j.jamcollsurg.2011.01.052. [DOI] [PubMed] [Google Scholar]
- 24.Rodrigo-Rincón MI, Tirapu-León B, Zabalza-López P, Martín-Vizcaino MP, de la Fuente-Calixto A, Villalgordo-Ortín P, Domench-Mañero L, Gost-Garde J. [The healthcare professional's perceptions on the implementation and usefulness of the surgical safety checklist] Rev Calid Asist. 2011;26:380–385. doi: 10.1016/j.cali.2011.03.005. [DOI] [PubMed] [Google Scholar]
- 25.Böhmer AB, Kindermann P, Schwanke U, Bellendir M, Tinschmann T, Schmidt C, Bouillon B, Wappler F, Gerbershagen MU. Long-term effects of a perioperative safety checklist from the viewpoint of personnel. Acta Anaesthesiol Scand. 2013;57:150–157. doi: 10.1111/aas.12020. [DOI] [PubMed] [Google Scholar]
- 26.Haugen AS, Søfteland E, Eide GE, Sevdalis N, Vincent CA, Nortvedt MW, Harthug S. Impact of the World Health Organization's Surgical Safety Checklist on safety culture in the operating theatre: a controlled intervention study. Br J Anaesth. 2013;110:807–815. doi: 10.1093/bja/aet005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Morgan PJ, Cunningham L, Mitra S, Wong N, Wu W, Noguera V, Li M, Semple J. Surgical safety checklist: implementation in an ambulatory surgical facility. Can J Anaesth. 2013;60:528–538. doi: 10.1007/s12630-013-9916-8. [DOI] [PubMed] [Google Scholar]
- 28.Bosk CL, Dixon-Woods M, Goeschel CA, Pronovost PJ. Reality check for checklists. Lancet. 2009;374:444–445. doi: 10.1016/s0140-6736(09)61440-9. [DOI] [PubMed] [Google Scholar]