Skip to main content
PMC home page
The BMJ logoLink to The BMJ
. 2020 Aug 25;370:m2917. doi: 10.1136/bmj.m2917

Prospective validation of classification of intraoperative adverse events (ClassIntra): international, multicentre cohort study

Salome Dell-Kuster 1 ,2 ,3,✉, Nuno V Gomes 1 ,3, Larsa Gawria 2 ,4, Soheila Aghlmandi 2, Maame Aduse-Poku 5, Ian Bissett 6, Catherine Blanc 7, Christian Brandt 8, Richard B ten Broek 4, Heinz R Bruppacher 9, Cillian Clancy 10, Paolo Delrio 11, Eloy Espin 12, Konstantinos Galanos-Demiris 13, I Ethem Gecim 14, Shahbaz Ghaffari 15, Olivier Gié 16, Barbara Goebel 17, Dieter Hahnloser 16, Friedrich Herbst 15, Ioannidis Orestis 13, Sonja Joller 18, Soojin Kang 5, Rocio Martín 12, Johannes Mayr 17, Sonja Meier 5, Jothi Murugesan 19, Deirdre Nally 10, Menekse Ozcelik 20, Ugo Pace 11, Michael Passeri 21, Simone Rabanser 22, Barbara Ranter 23, Daniela Rega 11, Paul F Ridgway 10, Camiel Rosman 4, Roger Schmid 24, Philippe Schumacher 8, Alejandro Solis-Pena 12, Laura Villarino 25, Dionisios Vrochides 21, Alexander Engel 19, Greg O’Grady 6, Benjamin Loveday 6, Luzius A Steiner 1 ,3, Harry Van Goor 4, Heiner C Bucher 2 ,3, Pierre-Alain Clavien 26, Philipp Kirchhoff 27, Rachel Rosenthal 28
PMCID: PMC7500355  PMID: 32843333

Abstract

Objective

To prospectively assess the construct and criterion validity of ClassIntra version 1.0, a newly developed classification for assessing intraoperative adverse events.

Design

International, multicentre cohort study.

Setting

18 secondary and tertiary centres from 12 countries in Europe, Oceania, and North America.

Participants

The cohort study included a representative sample of 2520 patients in hospital having any type of surgery, followed up until discharge. A follow-up to assess mortality at 30 days was performed in 2372 patients (94%). A survey was sent to a representative sample of 163 surgeons and anaesthetists from participating centres.

Main outcome measures

Intraoperative complications were assessed according to ClassIntra. Postoperative complications were assessed daily until discharge from hospital with the Clavien-Dindo classification. The primary endpoint was construct validity by investigating the risk adjusted association between the most severe intraoperative and postoperative complications, measured in a multivariable hierarchical proportional odds model. For criterion validity, inter-rater reliability was evaluated in a survey of 10 fictitious case scenarios describing intraoperative complications.

Results

Of 2520 patients enrolled, 610 (24%) experienced at least one intraoperative adverse event and 838 (33%) at least one postoperative complication. Multivariable analysis showed a gradual increase in risk for a more severe postoperative complication with increasing grade of ClassIntra: ClassIntra grade I versus grade 0, odds ratio 0.99 (95% confidence interval 0.69 to 1.42); grade II versus grade 0, 1.39 (0.97 to 2.00); grade III versus grade 0, 2.62 (1.31 to 5.26); and grade IV versus grade 0, 3.81 (1.19 to 12.2). ClassIntra showed high criterion validity with an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91) in the survey (response rate 83%).

Conclusions

ClassIntra is the first prospectively validated classification for assessing intraoperative adverse events in a standardised way, linking them to postoperative complications with the well established Clavien-Dindo classification. ClassIntra can be incorporated into routine practice in perioperative surgical safety checklists, or used as a monitoring and outcome reporting tool for different surgical disciplines. Future studies should investigate whether the tool is useful to stratify patients to the appropriate postoperative care, to enhance the quality of surgical interventions, and to improve long term outcomes of surgical patients.

Trial registration

ClinicalTrials.gov NCT03009929.

Introduction

Over 310 million surgeries are conducted worldwide each year.1 One in six patients having elective surgery experiences at least one postoperative adverse event.2 The growing annual surgical volume1 3 and the increasing complexity of comorbid conditions in patients having surgery4 5 has caused a rise in adverse events related to surgery. Intraoperative adverse events are associated with adverse outcomes,6 7 8 9 10 but in contrast with postoperative complications,11 12 13 a prospectively validated instrument for standardised and transparent reporting of intraoperative adverse events is lacking. Uniform definitions of perioperative outcomes, however, are required for programmes to improve the safety of surgery, for benchmarking, and to adequately compare results from clinical research.14

In 2015, our group developed a CLASSification of Intraoperative Complications (ClassIntra version 1.0, formerly known as CLASSIC, supplementary table S1) in a Delphi process involving international, interdisciplinary experts, which was retrospectively validated.15 This tool defines intraoperative adverse events as any deviation from the ideal intraoperative course between skin incision and skin closure, and includes events related to surgery and anaesthesia. Similar to the widely used Clavien-Dindo classification for reporting postoperative complications (supplementary table S2),11 we sought to develop a generic classification, broadly applicable to all surgical disciplines, and including anaesthesia. To align with the validated Clavien-Dindo classification,16 17 CLASSIC was updated by including five severity grades, maintaining the decision criterion on the need for treatment and severity of symptoms.

The aim of this international, multicentre, prospective cohort study was to investigate the validity and practicability of the updated classification in many different surgical disciplines.

Methods

Study oversight

This investigator initiated validation study of ClassIntra version 1.0 consisted of two parts. An international, multicentre, prospective cohort study was conducted in 18 centres in 12 countries in patients having any type of surgery requiring hospital admission for at least one night. The objective was to assess construct validity. Because no gold standard exists for the severity of intraoperative adverse events, we assessed the relation between the ClassIntra grades and other indirect measures that we expected to be related.18 In particular, we investigated the stepwise increasing effect of the severity grades of ClassIntra and relevant postoperative outcomes, such as postoperative complications, length of hospital stay, and length of surgery. The second part involved a survey of 10 fictitious case scenarios describing intraoperative adverse events. The survey was sent at the beginning of the prospective cohort study to physicians from the same international centres to grade severity. The aim was to assess criterion validity and reliability by comparing severity ratings by physicians with benchmark ratings set by the core team.

Across all centres, collaborators were trained in standardised data collection. Local study investigators confirmed the validity of the data from their centres. The results are reported according to STROBE (strengthening the reporting of observational studies in epidemiology).19

Patients and study procedures (prospective cohort study)

The centres prospectively included consecutive patients or a representative group of patients having any type of surgery from any surgical discipline. Excluded were day case surgeries, procedures not taking place in the operating room, procedures with no anaesthesia, and follow-up procedures.

The surgical and anaesthesia team prospectively recorded the severity grade and details of intraoperative adverse events that occurred between skin incision and skin closure, according to ClassIntra version 1.0 (table 1), during team sign-out20 at the end of surgery. Disagreement was resolved by consensus. All patients were followed up until discharged from hospital and were assessed daily by the postoperative care team to identify and grade postoperative complications according to the Clavien-Dindo classification.11 Because this was a pragmatic cohort study, performed during daily clinical practice, the postoperative team could not be blinded to the intraoperative adverse events, but was not aware of the ClassIntra grade. Postoperative complications after discharge, including mortality within 30 days, were assessed during structured follow-up visits or telephone calls to patients or their relatives.

Table 1.

ClassIntra version 1.0 classification of intraoperative adverse events. The classification defines intraoperative adverse events as any deviation from the ideal intraoperative course occurring between skin incision and skin closure. Any event related to surgery and anaesthesia during the index surgery must be considered and should be rated directly after surgery.* A requirement is that the indication for surgery and the interventions conform to current guidelines

Grade Definition Examples
Grade 0 No deviation from the ideal intraoperative course —
Grade I Any deviation from the ideal intraoperative course:
• Without the need for any additional treatment or intervention
• Patient with no or mild symptoms		 • Bleeding: bleeding above average from small calibre vessel, self-limiting or definitively manageable without additional treatment than routine coagulation
• Injury: minimal serosal intestinal lesion, not requiring any additional treatment
• Cautery: small burn of the skin, no treatment necessary
• Arrhythmia: arrhythmia (eg, extrasystoles) without relevance
Grade II Any deviation from the ideal intraoperative course:
• With the need for any additional minor treatment or intervention
• Patient with moderate symptoms, not life threatening, and not leading to permanent disability		 • Bleeding: bleeding from medium calibre artery or vein, ligation; use of tranexamic acid
• Injury: non-transmural intestinal lesion requiring suture(s)
• Cautery: moderate burn requiring non-invasive wound care
• Arrhythmia: arrhythmia requiring administration of antiarrhythmic drug, no haemodynamic effect
Grade III Any deviation from the ideal intraoperative course:
• With the need for any additional moderate treatment or intervention
• Patient with severe symptoms, potentially life threatening or potentially leading to permanent disability		 • Bleeding: bleeding from large calibre artery or vein with transient haemodynamic instability, ligation or suture; blood transfusion
• Injury: transmural intestinal lesion requiring segmental resection
• Cautery: severe burn requiring surgical debridement
• Arrhythmia: arrhythmia requiring administration of antiarrhythmic drug, transient haemodynamic effect
Grade IV Any deviation from the ideal intraoperative course:
• With the need for any additional major and urgent treatment or intervention
• Patient with life threatening symptoms or leading to permanent disability		 • Bleeding: life threatening bleeding with splenectomy; massive blood transfusion; stay at intensive care unit
• Injury: injury of central artery or vein requiring extended intestinal resection
• Cautery: life threatening burn injury by cautery leading to fire requiring intensive care treatment
• Arrhythmia: arrhythmia requiring electroconversion, defibrillation, or admission to intensive care
Grade V Any deviation from the ideal intraoperative course with intraoperative death of the patient —
Open in a new tab
*

These events were not defined as intraoperative adverse events: sequelae, failures of cure, events related to the underlying disease, incorrect site or incorrect patient surgery, or errors in indication.

Data describing the patients enrolled in the study were anonymised before entry into a secure web based electronic database. The database had automated checks for plausibility, consistency, and completeness of the data entries. Further consistency checks were performed after recruitment was finished.

ClassIntra

ClassIntra version 1.0 was derived from CLASSIC,15 which had four severity grades, but the same definition depending on the intervention required to treat the adverse event and the severity of symptoms (supplementary table S1). The only difference in the updated ClassIntra is that grade II has been divided into two grades. The additional grade (new grade III) describes a moderate deviation from the ideal course, which requires moderate treatment and results in severe symptoms. The rationale for updating the classification was to align with the number of grades in the Clavien-Dindo classification to avoid misclassification in grading intraoperative or postoperative death (supplementary table S2).11 A retrospective (unpublished) pilot study showed high reliability for ClassIntra.

We secured the rights to the ClassIntra classification by filing a trademark with the Swiss Federal Institute of Intellectual Property. The trademark was extended to relevant countries for our activities by international registration at the World Intellectual Property Organisation.

Outcomes

The primary outcome was the association of the most severe intraoperative adverse events, according to ClassIntra, with the most severe postoperative complications, according to the Clavien-Dindo classification.11 Secondary outcomes were the association of the most severe intraoperative adverse events with the sum of all postoperative complications, calculated by the comprehensive complication index,12 postoperative length of hospital stay, length of surgery, and 30 day mortality. Thirty day mortality was defined as death within 30 days of surgery, either in hospital or after discharge.

All models for the primary and secondary outcomes were adjusted for confounders: American Society of Anesthesiologists physical status,21 as a summary measure of perioperatively relevant comorbidities; age; length of surgery; complexity of surgery, graded as one of five categories according to the British United Provident Association (BUPA)22 23; urgency of the procedure (planned or emergency); experience of the surgical and anaesthesia team (for a definition, see the statistical analysis plan in the supplementary file); and wound class.24 All confounders were defined according to biological plausibility and scientific rationale. Type of operation and complexity were recorded according to the BUPA classification.22 23 BUPA categorises all commonly performed surgical procedures into five complexity grades (minor, intermediate, major, major plus, and complex major; supplementary table S15) and has been used previously for scientific purposes.25 26

Inter-rater reliability and practicability of ClassIntra

An online anonymous survey (SurveyMonkey, San Mateo, CA, USA) of 10 fictitious case scenarios (supplementary table S3) describing intraoperative adverse events was sent to a representative sample of physicians from the participating centres. The scenarios were designed to cover a range of severity grades and medical specialties. The benchmark rating for each case scenario was determined by consensus of the core team. The physicians also rated the practicability of the instrument on a nine point scale, with end anchors of “very practical” and “not practical at all.”

Statistical analysis

A multivariable hierarchical proportional odds model was used to investigate the association between the most severe intraoperative and postoperative adverse event considering clustering within study centres. The proportional odds assumption was mainly assessed with the Brant test. Because of the sparseness of the data for the higher severity grades for intraoperative and postoperative adverse events, and breach of the proportionality assumption for the five categories of the Clavien-Dindo system, a three category classification (Clavien-Dindo grade 0, grades I-II, and grades III-V) was used. The rationale for these cut-offs was to distinguish an uneventful course from a minor or major deviation, and was in line with cut-offs used in most publications combining categories.16 The consistency of the coefficients was checked by performing an internal validation with bootstrapping methods, with 100 replications of samples from 2520 patients. A model with a binary endpoint (any v no postoperative complication) was used as a sensitivity analysis.

The association of the most severe intraoperative adverse event and the weighted sum of all postoperative complications (comprehensive complication index)12 was explored with a multivariable two level mixed effect regression model accounting for correlation between patients from the same study centre. Associations between the most severe intraoperative adverse event and the secondary endpoints of postoperative length of hospital stay and length of surgery were investigated in a multivariable two level mixed effect log linear regression model. The coefficients of the log linear regression models were back transformed to improve the interpretability of the results showing percentage changes27 in hospital days or duration of surgery for one unit increase in the corresponding covariate.

In the survey, the average agreement between the ratings of the physicians and the core team was calculated. Inter-rater reliability was assessed with the intraclass correlation coefficient based on the mean absolute agreement, estimated in a two way random effects model.28

The participating hospitals were a voluntary convenience sample. The planned sample size of 2500 patients allowed for robust estimation in the multivariable models, assuming at least one postoperative complication in 10% of the patients (that is, at least 250 events; for details, see the statistical analysis plan in the supplementary file).

Analyses were performed with Stata 14 (StataCorp, College Station, TX). Further details on data management and statistical analysis are in the statistical analysis plan (supplementary file).

Missing data

When an undefined complexity grade in the BUPA classification system (in 4% of procedures) was found, a grade corresponding to a similar procedure was used. This analysis was then compared with a complete case analysis. If no information was available on the presence of the anaesthesia resident or nurse, or whether they were in training or had graduated, we assumed that these team members were not present or were in training. There were no other missing values for outcomes or covariates, except for data on 30 day mortality in patients lost to follow-up.

ClassIntra study group

The ClassIntra study group consisted of a multidisciplinary, international team of anaesthetists, surgeons with a range of surgical specialties, epidemiologists, and statisticians with extensive expertise in perioperative care.

Patient and public involvement

This research was a multicentre collaboration to evaluate a newly developed classification for intraoperative adverse events. The data and software were not publicly available for analysis by patients or members of the public. Our protocol did not include patient and public involvement in the planning or implementation of this research. No patients were asked to advise on interpretation or writing up of the results. However, future research will evaluate the effect of intraoperative adverse events measured with ClassIntra on patient reported outcomes.

Results

Patient and procedural characteristics

Of 2640 patients screened, 2520 were enrolled in 18 secondary and tertiary centres in North America, Europe, and Oceania, representing 12 countries. All patients were recruited between 14 February 2017 and 27 July 2018. The median number of patients enrolled per site was 130 (interquartile range 100-176). The main reason for exclusion was day surgery (fig 1). Patients had a range of surgical procedures from different surgical disciplines, except ophthalmic surgery (table 2), with 44% (n=1090) of surgeries corresponding to the two highest complexity levels. Patient and procedural characteristics are described in table 2 (examples of the most frequent types of surgical procedures are in supplementary table S15). Baseline characteristics were similar in patients with and without intraoperative adverse events. The American Society of Anesthesiologists’ physical status classes, and the complexity of the surgical procedure, however, were higher in patients with an intraoperative adverse event.

Fig 1.

Fig 1

Open in a new tab

Flow diagram of patients in the validation study of ClassIntra

Table 2.

Patient and procedural characteristics for the whole study population and for subgroups with and with no intraoperative adverse events

All patients (n=2520) Patients with no intraoperative adverse events (n=1910) Patients with intraoperative adverse events (n=610) Missing data
(No (%))
American Society of Anesthesiologists (ASA) physical status
 ASA I 503 (20) 431 (23) 72 (12) —
 ASA II 1118 (44) 852 (45) 266 (44) —
 ASA III 805 (32) 565 (30) 240 (39) —
 ASA IV            92 (3.7) 62 (3.3) 30 (4.9) —
 ASA V 2 (0.1) — 2 (0.3) —
Age (median (IQR; range))
 In adults (age ≥18; n=2340) 61 (46-72; 18-97) 60 (45-71; 18-97) 64 (49-74; 18-93) —
 In children (age <18; n=180) 9 (5-14; 0-17) 9 (5-14; 0-17) 8 (3-13; 0-16) —
Sex
 Male 1382 (55) 1038 (54) 344 (56) —
 Female 1138 (45) 872 (46) 266 (44) —
Body mass index in adults (median (IQR); n=2340) 26 (23-30) 26 (23-30) 26 (23-30) 14 (1)
Weight in children (kg; median (IQR, range); n=180) 32 (19-53; 4-137) 33 (19-54; 4-137) 26 (17-37; 7.9-85) —
Surgical discipline
 Gastrointestinal surgery 1437 (57) 1085 (57) 352 (58) —
 Orthopaedic surgery and traumatology 297 (12) 260 (14) 37 (6.1) —
 Vascular surgery 169 (6.7) 121 (6.3) 48 (7.9) —
 Urology 134 (5.3) 109 (5.7) 25 (4.1) —
 Ear, nose, throat, and maxillofacial surgery 122 (4.8) 99 (5.2) 23 (3.8) —
 Neurosurgery and spine surgery 96 (3.8) 53 (2.8) 43 (7.1) —
 Cardiac surgery 73 (2.9) 41 (2.2) 32 (5.3) —
 Paediatric surgery 54 (2.1) 48 (2.5) 6 (1.0) —
 Gynaecology 46 (1.8) 29 (1.5) 17 (2.8) —
 Obstetrics 44 (1.8) 31 (1.6) 13 (2.1) —
 Reconstructive and hand surgery 26 (1.0) 21 (1.1) 5 (0.8) —
 Thoracic surgery 22 (0.9) 13 (0.7) 9 (1.5) —
Urgency of the procedure
 Planned 2153 (85) 1627 (85) 526 (86) —
 Emergency 367 (15) 283 (15) 84 (14) —
Complexity of surgical procedure (original)
 Minor 105 (4.2) 94 (4.9) 11 (1.8) 98 (4)
 Intermediate 437 (17) 383 (20) 54 (8.9)
 Major 790 (31) 613 (32) 177 (29)
 Major plus 442 (18) 323 (17) 119 (20)
 Complex major operation 648 (26) 431 (23) 217 (36)
Additional procedure 406 (16) 266 (14) 140 (23)
Wound class
 Clean 1341 (53) 1033 (54) 308 (50) —
 Clean contaminated 1001 (40) 746 (39) 255 (42) —
 Contaminated 147 (5.8) 103 (5.4) 44 (7.2) —
 Dirty 31 (1.2) 28 (1.5) 3 (0.5) —
Operating surgeon
 Senior consultant 1662 (66) 1239 (65) 423 (69) —
 Junior consultant 544 (22) 427 (22) 117 (19) —
 Resident 314 (12) 244 (13) 70 (11) —
Assisting surgeon 2371 (94) 1781 (93) 590 (97) —
 Senior consultant 555/2371 (23) 385/1781 (22) 170/590 (29) —
 Junior consultant 578/2371 (24) 432/1781 (24) 146/590 (25) —
 Resident 1238/2371 (52) 964/1781 (54) 274/590 (46) —
Anaesthesia consultant present 2311 (92) 1746 (91) 565 (93) —
 Senior consultant 1481/2311 (64) 1112/1746 (64) 369/565 (65) —
 Junior consultant 830/2311 (36) 634/1746 (36) 196/565 (35) —
Anaesthesia resident present 1443 (57) 1017 (53) 426 (70) 1 (0.0)
Anaesthesia nurse present 1866 (74) 1405 (74) 461 (76) 2 (0.1)
 Proportion who graduated 1699 (91) 1268 (90) 431 (93) 3 (0.2)
Anaesthesia technique
 General anaesthesia 2035 (81) 1558 (82) 477 (78) —
 Regional anaesthesia 198 (7.9) 158 (8.3) 40 (6.6) —
 Combined techniques 265 (11) 176 (9.2) 89 (15) —
 Monitored anaesthesia care 22 (0.9) 18 (0.9) 4 (0.7) —
Open in a new tab

Data are number (%) unless indicated otherwise. IQR=interquartile range.

Intraoperative adverse events according to ClassIntra

Of 2520 patients enrolled, 610 (24%) experienced 778 intraoperative adverse events (table 3). The most severe intraoperative adverse event per patient was grade I in 161 patients (6.4%), grade II in 309 (12%), grade III in 122 (4.8%), and grade IV in 18 patients (0.7%). No intraoperative deaths occurred (table 3).

Table 3.

Outcomes for the whole study population and for subgroups with and with no intraoperative adverse events

All patients (n=2520) Patients with no intraoperative adverse events (n=1910) Patients with intraoperative adverse events (n=610) Missing data
(No (%))
All iAE (several events per patient possible)
 0 1910 (71) 1910 (100) — —
 I 198 (7.4) — 198 (25) —
 II 417 (16) — 417 (54) —
 III 142 (5.3) — 142 (18) —
 IV 21 (0.8) — 21 (3.0) —
 V — — — —
Total No of iAE per patient
 0 1910 (76) 1910 (100) — —
 1 485 (19) — 485 (80) —
 2 92 (3.7) — 92 (15) —
 3 27 (1.1) — 27 (4.4) —
 4 3 (0.1) — 3 (0.5) —
 5 2 (0.1) — 2 (0.3) —
 6 1 (0.0) — 1 (0.2) —
Transfer after operation 677 (27) 403 (21) 274 (45) —
 Intermediate care unit 347/677 (51) 244/403 (61) 103/274 (38) —
 Intensive care unit 330/677 (49) 159/403 (39) 171/274 (62) —
 Unplanned 45/677 (6.7) 6/403 (1.5) 39/274 (14) —
Length of surgery (minutes; median (IQR, range)) 100 (60-170, 4-760) 90 (55-147, 4-760) 151 (93-240, 12-673) —
Most severe postoperative complication per patient
 0 1682 (67) 1367 (72) 315 (52) —
 I 349 (14) 257 (13) 92 (15) —
 II 277 (11) 162 (8.5) 115 (19) —
 IIIa 72 (2.9) 45 (2.4) 27 (4.4) — 
 IIIb 55 (2.2) 40 (2.1) 15 (2.5) —
 IVa 53 (2.1) 23 (1.2) 30 (4.9) —
 IVb 7 (0.3) 3 (0.2) 4 (0.7) —
 V 25 (1.0) 13 (0.7) 12 (2.0) —
All postoperative complications (several events per patient possible)
 I 1106 (44) 747 (48) 359 (37) —
 II 983 (39) 557 (36) 426 (44) —
 IIIa 160 (6.4) 94 (6.1) 66 (6.8) —
 IIIb 105 (4.2) 72 (4.7) 33 (3.4) —
 IVa 110 (4.4) 51 (3.3) 59 (6.1) —
 IVb 20 (0.8) 9 (0.6) 11 (1.1) —
 V 25 (1.0) 13 (0.8) 12 (1.2) —
Total No of postoperative complications per patient
 None 1682 (67) 1367 (72) 315 (52) —
 1 412 (16) 288 (15) 124 (20) —
 2-5 329 (13) 198 (10) 131 (21) —
 >5 97 (3.9) 57 (3.0) 40 (6.6) —
Reoperation
 No 2421 (96) 1844 (97) 577 (95) —
 Any reoperation 99 (3.9) 66 (3.5) 33 (5.4) —
Total No of reoperations
 1 68 (69) 46 (70) 22 (67) —
 2 16 (16) 11 (17) 5 (15) —
 3 6 (6.1) 4 (6.1) 2 (6.1) —
 4 8 (8.1) 5 (7.6) 3 (9.1) —
 5 1 (1.0) ·· 1 (3.0) —
Postoperative length of hospital stay (days; median (IQR, range)) 3 (2-6, 0-191) 3 (1-5, 0-106) 6 (3-9, 1-191) —
Total length of hospital stay (days; median (IQR, range)) 4 (2-8, 1-193) 3 (2-7, 1-113) 7 (3-11, 1-193) —
Stay during postoperative course 68 (2.7) 40 (2.1) 28 (4.6) —
 Intermediate care unit 18 (26) 15 (38) 3 (11) —
 Intensive care unit 50 (74) 25 (63) 25 (89) —
30 day mortality 26 (1.1) 13 (0.7) 13 (2.1) 148 (5.9)
Open in a new tab

Data are number (%) unless stated otherwise. IQR=interquartile range; iAE= intraoperative adverse events.

Postoperative complications

During the postoperative course, 2509 complications were seen in 838 patients (33%). Table 3 gives details of the severity grades and number of postoperative complications.

Multivariable analysis showed a gradual increase in the odds of a more severe postoperative complication with an increasing grade of the most severe intraoperative adverse event (fig 2; supplementary table S5). Consistently, a stepwise increase in the sum of all postoperative complications (comprehensive complication index) with an increasing grade of ClassIntra was seen (supplementary table S6).

Fig 2.

Fig 2

Open in a new tab

Multivariable hierarchical proportional odds model for the most severe postoperative complications according to the Clavien-Dindo classification (three categories for outcome: no postoperative complications, grades I-II, and grades III-V), adjusted for the most relevant confounders. ASA=American Society of Anesthesiologists’ physical status level

More secondary outcomes

Median postoperative length of hospital stay was 3 days (interquartile range 2-6) in all patients, and was doubled in patients with at least one intraoperative adverse event (median 6 days; interquartile range 3-9 days) compared with patients with no intraoperative adverse events (3; 1-5 days) (table 3). In the multivariable analysis, median postoperative length of hospital stay increased with each ClassIntra grade by 6.1% (95% confidence interval 2.8 to 9.3%; supplementary table S7).

The median length of surgery was 100 minutes (interquartile range 60-170) with a maximum length of 760 minutes (table 3). The length of surgery was longer in patients experiencing an intraoperative adverse event, with an increase of 16% in median length of surgery with each grade increase in ClassIntra (95% confidence interval 13% to 18%; supplementary table S8). Thirty day follow-up was available for 2372 (94%) patients (fig 1). Mortality at 30 days was 1.1% (n=26), with a higher incidence in patients with an intraoperative event (table 3 and table S9).

Survey

The response rate to the survey was 83% (136/163 physicians). About a third of respondents were anaesthetists and two thirds were surgeons from various surgical disciplines with a range of years’ experience (supplementary table S4). Reliability among physicians showed an intraclass correlation coefficient of 0.76 (95% confidence interval 0.59 to 0.91). Overall agreement with the ratings of the core team was 61% (interquartile range 42-71%; supplementary figure S1), with deviation usually explained by physicians rating one grade lower than the core team. Physicians rated the practicability of ClassIntra as 6 out of 9 (interquartile range 5-7).

Discussion

Principal findings

This international multicentre study showed that ClassIntra is a valid and standardised tool for assessing intraoperative adverse events, with high construct validity, given the increasing effects of severity grades on important patient outcomes. ClassIntra also showed good criterion validity and practicability across different surgical disciplines.

The association between severe intraoperative and postoperative complications, well established in abdominal surgery,6 8 was replicated in a heterogeneous population of surgical patients from a range of surgical disciplines. The association of intraoperative adverse events and postoperative length of hospital stay might have been mediated by postoperative complications (that is, more postoperative complications resulted in a longer stay in hospital after operation). The association between intraoperative adverse events and length of surgery was most likely related (that is, more adverse events might have prolonged surgery, which in turn could have led to more intraoperative adverse events).

Strengths and limitations of the study

ClassIntra is a comprehensive classification, with a patient centred perspective, because it records any event related to anaesthesia or surgery, and grades the outcomes according to those relevant to the patient, the symptoms caused by the event, and the measures taken to treat the event. ClassIntra was derived from CLASSIC, which was developed by an interdisciplinary, international group of anaesthetists, surgeons, epidemiologists, and statisticians.15 ClassIntra aligns with the Clavien-Dindo classification of postoperative complications,11 allowing the reporting of intraoperative and postoperative adverse events.

Our robust study design, based on a large sample size involving a range of surgical disciplines, and on prospective multicentre data collection, implies good generalisability. Good generalisability always introduces some heterogeneity, which was accounted for by adjusting the analyses for the complexity of the surgical procedure and patient related factors. Whether the results are mainly driven by the largest subgroup of patients having an intra-abdominal procedure will be investigated in a secondary analysis comparing the results in several subgroups. The primary advantage of a broad classification is that it increases uniformity in reporting intraoperative adverse events and allows comparison of results between different centres and times. In our study, details of each intraoperative adverse event were collected. Our group is currently investigating how the additional information can be used to support quality improvement in the future. We therefore encourage adding details about the events and intervention specific intraoperative outcomes of interest to the ClassIntra grade, whenever possible within a framework of standardised core outcomes.29

To keep it practical and simple, the classification uses generic definitions of the five severity grades; individual judgment is necessary for interpretation. Similarly, defining an event as deviation from the ideal intraoperative course requires personal judgment. Inter-rater reliability in the survey was strong and disagreement was mostly by one grade. We found the inter-rater reliability results reassuring, given that the physicians had various backgrounds and were grading intraoperative adverse events in different surgical disciplines and in anaesthesia. In clinical practice and in our prospective cohort study, joint assessment by surgeons and anaesthetists during sign-out20 should lower this variability and the risk of under-reporting and downgrading of intraoperative adverse events. As clinicians become more familiar with ClassIntra, we believe the system will be easier to use, similar to the experience with the Clavien-Dindo classification after its introduction into routine clinical practice. Generally, different professional backgrounds, levels of experience, and cultural diversity among physicians might have explained the practicability ratings of less than five in 25% of the survey participants and ratings greater than seven in another 25%.

Because our project was conducted as part of daily clinical practice, having an independent person to record intraoperative and postoperative events would not have been feasible, and blinding of those assessing postoperative outcomes would have been unethical. The direction of the potential bias is unclear, however, as under-reporting and over-reporting could, theoretically, occur. Under-reporting and downgrading of complications is a problem in healthcare settings and should be handled with a positive, non-discriminative learning culture. A classification for complications should be sensitive enough to monitor and detect rating variability caused by downgrading.30 31

Our secondary endpoints included length of hospital stay. We acknowledge that this outcome might be affected by the variables included in the model (that is, the most severe intraoperative adverse event and the confounding variables) and also by cultural norms, logistical issues, and economic considerations. This analysis should be viewed as supporting evidence as it showed consistent stepwise increasing association in alignment with all other postoperative endpoints.

Findings in relation to other studies

Other classification systems for intraoperative adverse events have been published.32 Kaafarani and colleagues8 classified intraoperative adverse events into six grades. Grades I-IV depend on the intervention required to treat the intraoperative adverse events and the deviation from the planned procedure. Grade V refers to an intraoperative injury missed during surgery and requiring re-operation within seven days. Grade VI refers to intraoperative death. This classification was based on a retrospective analysis of routinely collected data in selected patients with accidental puncture or laceration. Although their classification approach differs considerably from ClassIntra, the authors confirmed an association between major intraoperative and postoperative adverse events.8 Our study was based on prospective data in unselected patients from a range of surgical interventions and considered any event related to surgery or anaesthesia. We did not include re-operation because of the difficulty in determining if an intraoperative injury or a postoperative complication was responsible for the re-operation.

A classification of surgical intraoperative adverse events for laparoscopic surgery was proposed, with five grades similar to ClassIntra, but excluded adverse events related to anaesthesia.33 Others used readily available information, such as quality indicators of the intraoperative period, to predict postoperative outcome. Coulson and colleagues estimated the acute risk change in the probability of death before and after operation based on two validated risk scores.34 35 The 10 point Apgar score for surgery was developed in patients having a colectomy and was evaluated in a broader surgical population.36 37 Prediction of postoperative complications was based on estimated blood loss, lowest mean arterial pressure, and heart rate. Although both scores showed good predictive ability, they provided only a surrogate marker for adverse intraoperative events.

Bothner and colleagues developed an incidents, events, and complications scale related to anaesthesia, based on a set of 63 events with five ascending severity grades depending on the effect on postoperative care and outcome.38 The scale was implemented for quality assurance and covered the whole period of anaesthesia, from induction to discharge to the ward or to the intensive care unit. Their reported rate of 22% adverse events was relatively higher than our findings where events related to anaesthesia and surgery were taken together, which could partly be explained by the restricted period covered in our study.

Implications for clinicians or policy makers

Patient safety in the perioperative period has gained considerable interest and successful interventions, such as the reduction in morbidity and mortality after introduction of a surgical safety checklist, have been seen.20 ClassIntra covers a major unmet need, as the first prospectively validated classification system that is simple and generally applicable to all surgical disciplines, to our knowledge. It takes a holistic view by integrating any adverse event, regardless of origin. Also, because innovative surgical techniques are increasingly complex and implemented quickly, preventing morbidity during the implementation period is important.39 ClassIntra should be included in routine practice to alert the postoperative care team to patients with intraoperative adverse events and at risk of further postoperative complications.

Future research

We acknowledge that our definition of intraoperative adverse events included only the time between skin incision and closure. Induction of and emergence from anaesthesia were not covered, which can have risks for more adverse events. Assessing the construct validity of a new classification is an ongoing process, which requires confirmation by more convergent and discriminative validity assessments to increase credibility. In future research, we will evaluate the properties of ClassIntra extending the assessment period to include the periods related to anaesthesia (that is, induction and emergence), focusing on patient reported outcomes, thus expanding the hypothetical construct for assessing validity.

ClassIntra can be used in clinical research as a defined core outcomes measure as it provides standard definitions, allowing accurate communication of research results and comparisons across research studies.11 40 ClassIntra can help evaluate the origin of the complications during the surgical procedure, providing more granularity to the complication profile for each procedure or surgical unit. Also, ClassIntra can be used in critical incidence reporting systems,41 for measuring individual performance in educational settings and system performance for benchmarking by institutions, and for outcome research.Future implementation studies should evaluate this tool in daily clinical practice for risk stratification of postoperative care or for planning resources and hospital capacities, in view of the increase in postoperative length of stay with the increasing severity of the ClassIntra grade. For day surgeries, the risk of perioperative complications is minimal. Future research should investigate whether assessing intraoperative complications is associated with early visits to the general practitioner or readmission.

Conclusion

ClassIntra version 1.0 is a validated and standardised tool to quantify and qualify intraoperative adverse events in clinical practice and research. It showed good reliability and practicability for all surgical disciplines and for anaesthesia. We reproduced the well established association between the severity grade for intraoperative adverse events and important postoperative outcomes. Hence ClassIntra is a useful tool for recording intraoperative adverse events and should be incorporated into the perioperative surgical safety checklist to highlight recurrent and potentially avoidable complications. ClassIntra could improve the technical aspects of intraoperative care and provide clarity in audits of practice.

What is already known on this topic

  • Intraoperative adverse events are a concern for patients, anaesthetists, and surgeons because of their association with adverse postoperative outcomes

  • In contrast with postoperative complications, prospectively validated systems for reporting intraoperative adverse events are lacking

  • Uniform definitions and transparent reporting of intraoperative adverse events are required for programmes to improve the safety of surgery and to compare results from clinical research

What this study adds

  • ClassIntra is the first prospectively validated classification system with good reliability and practicability that includes intraoperative adverse events related to surgery and anaesthesia

  • ClassIntra covers a major unmet need as it provides a standardised instrument to quantify and qualify intraoperative adverse events in clinical practice and clinical research for a range of surgical disciplines and for anaesthesia

Acknowledgments

We thank Ramon Saccilotto who programmed and maintained the online database; Allison Dwileski for editorial assistance on the manuscript; Christian Schindler for statistical advice; and all of the centres in this investigator initiated study for their support. In particular, we thank Esther Seeberger, Claudia Blaesi, Franziska Merz, Samuel Heim, and all participating surgical disciplines from University Hospital Basel, Basel, Switzerland; Martin Taube, Kata Papp, and Blanca Schuster, University Children’s Hospital Basel, Basel, Switzerland; Michael Winiker, University Hospital Lausanne, Lausanne, Switzerland; Victoria Koenig-Oberhuber and the preoperative ambulatory unit team, Cantonal Hospital Graubünden, Chur, Switzerland; David Iannitti and Hiral Amin, Carolinas Medical Centre, Charlotte, NC, USA; Deniz Yalman, Ankara University Medical School, Ankara, Turkey; Christos Chatzakis, Savvas Symeonidis, Natalia Antigoni Savvala, Nikolaos Antoniou, Dimitrios Konstantaras, Lydia Loutzidou, Loukiani Kitsikosta, Grigorios Gkasdaris, Stefanos Bitsianis, Ioannis Mantzoros, Manousos George Pramateftakis, Efstathios Kotidis, and Konstantinos Tsalis, medical school at Aristotle University of Thessaloniki, Greece; the surgical trials unit team at University of Auckland, Auckland, New Zealand; Olivia Kirkland, Royal North Shore Hospital, Sydney, Australia; Astrid Chiari and René Schmutz, Department of Anaesthesiology, and Bernhard Dauser and Tudor Birsan, Department of Surgery, Hospital of St John of God Vienna, Austria; and Kariem El-Boghdadly and Gill Arbane, Guy’s and St Thomas’ Hospital, London, UK.

Part of the preliminary findings of this research were presented at the following meetings: (11 November 2017) poster presentation at the Congress of the Swiss Society for Anaesthesiology and Resuscitation (Interlaken, Switzerland); (2 November 2018) poster presentation at the Congress of the German Society of Gynaecology and Obstetrics (Berlin, Germany); (6 February 2019) QuickShot presentation at the 14th Annual Academic Surgical Congress (Houston, TX, USA); (9 May 2019) short paper at the International Surgical Congress of the Associations of Surgeons of Great Britain and Ireland (Telford, UK); (9 May 2019) oral presentation in the general surgery section of the Royal Australasian College of Surgeons’ annual scientific congress (Bangkok, Thailand); (16 May 2019) oral presentation at the 106th Annual Meeting of the Swiss Society of Surgery (Bern, Switzerland), and abstract published as society abstract supplement (Br J Surg 2019;106(S4):9); (15 May 2019) oral presentation at the XXIII National Meeting of the Spanish Association of Coloproctology (Valladolid, Spain); (3 June 2019) oral presentation at the European Anaesthesiology Congress Euroanaesthesia of the European Society of Anaesthesiology (Vienna, Austria), and abstract published as society abstract supplement (Eur J Anaesthesiol 2019;36(S57):313); (26 June 2019) oral presentation at the Swiss Congress of Gynaecology (St Gallen, Switzerland); (15 August 2019) oral presentation at the World Congress of Surgery (Krakow, Poland); (26 September 2019) free paper presentation at the 14th scientific and annual meeting of the European Society of Coloproctology (Vienna, Austria); (7 November 2019) oral presentation at the Congress of the Swiss Society for Anaesthesiology and Resuscitation (Interlaken, Switzerland; prize for second best oral presentation); and (8 November 2019) oral presentation at the National Surgery Congress of Spain (Santander, Spain). The authors were awarded the first prize for Innovation Qualité by the Swiss Academy of Quality in Medicine (SAQM) of the Swiss Medical Association.

Web extra.

Extra material supplied by authors

Web appendix: Supplemental file

dels054044.ww.pdf (519.7KB, pdf)

Contributors: SD-K and NVG contributed equally as first authors. PK and RR contributed equally as last authors. SD-K, NVG, and RR did the literature search. SD-K, NVG, PFR, HCB, P-AC, LAS, PK, and RR designed the study. S D-K, NVG, LG, RM, PFR, AE, GO, BL, HV, and PK collected the data. SD-K, LG, and SA analysed the data. SD-K, LG, SA, AE, GO, BL, HV, HCB, and RR interpreted the data. SD-K, NVG, RM, HCB, and RR wrote the manuscript. LG, SA, PFR, AE, GO, BL, HV, CR, P-AC, LAS, and PK provided critical feedback on the manuscript. MA-P, IB, C Bl, CBr, RTB, HRB, CC, PD, EE, KG-D, IEG, SG, OG, BG, DH, FH, OI, SJ, SK, JMa, SM, JMu, DN, MO, UP, MP, SR, BR, DR, CR, RS, PS, AS-P, LV, and DV collected the data and provided critical feedback on the manuscript.

All authors revised the manuscript for important intellectual content, approved the final version of the manuscript, and agreed to be accountable for all aspects of the work. SD-K is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: The research was supported by grants from the Goldschmidt-Jacobson Foundation, University of Basel, Foundation for Research and Education in Anaesthesiology, Switzerland, and VSB fonds (Verenigde Spaar Bankenfonds), Netherlands. The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all data.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Goldschmidt-Jacobson Foundation, University of Basel, Foundation for Research and Education in Anaesthesiology, Switzerland, and VSB fonds (Verenigde Spaar Bankenfonds) for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work. RR is an employee of F Hoffmann-La Roche (includes Stock-Settled Appreciation Rights). The study has no connection to her employment by the company. RR continues to be affiliated to the University of Basel.

Ethical approval: The study was approved by the regional ethics committee of Ethikkommission Nordwest- und Zentralschweiz (reference No Req-2016-00469) as Swiss lead ethics committee, and by the local ethics committees of all international centres. Participants gave oral or written informed consent compliant with the relevant legal requirements of each centre.

Data sharing: Anonymised patient level data are available for investigators whose proposed use of the data has been approved by a review committee identified for this purpose.

The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Dissemination to participants and related patient and public communities: To reach wide dissemination, the results of the study were presented at many international meetings. After publication, the results will be disseminated to relevant medical societies and patient organisations. The use of an (adapted) version of our web based database could easily be broadly implemented in other hospitals other than the study sites, which are interested to use ClassIntra version 1.0 as a tool for internal quality control in daily clinical practice and in research.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

  • 1. Weiser TG, Haynes AB, Molina G, et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet 2015;385(Suppl 2):S11. 10.1016/S0140-6736(15)60806-6  [DOI] [PubMed] [Google Scholar]
  • 2. International Surgical Outcomes Study group Global patient outcomes after elective surgery: prospective cohort study in 27 low-, middle- and high-income countries. Br J Anaesth 2016;117:601-9. 10.1093/bja/aew316  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Weiser TG, Regenbogen SE, Thompson KD, et al. An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet 2008;372:139-44. 10.1016/S0140-6736(08)60878-8  [DOI] [PubMed] [Google Scholar]
  • 4. Bainbridge D, Martin J, Arango M, Cheng D, Evidence-based Peri-operative Clinical Outcomes Research (EPiCOR) Group Perioperative and anaesthetic-related mortality in developed and developing countries: a systematic review and meta-analysis. Lancet 2012;380:1075-81. 10.1016/S0140-6736(12)60990-8  [DOI] [PubMed] [Google Scholar]
  • 5. The Lancet Making more of multimorbidity: an emerging priority. Lancet 2018;391:1637. 10.1016/S0140-6736(18)30941-3  [DOI] [PubMed] [Google Scholar]
  • 6. Kinaci E, Sevinc MM, Bayrak S, Erdoğan E, Ozakay A, Sari S. Is the classification of intraoperative complications (CLASSIC) related to postoperative course? Int J Surg 2016;29:171-5. 10.1016/j.ijsu.2016.03.068  [DOI] [PubMed] [Google Scholar]
  • 7. Bohnen JD, Mavros MN, Ramly EP, et al. Intraoperative adverse events in abdominal surgery: what happens in the operating room does not stay in the operating room. Ann Surg 2017;265:1119-25. 10.1097/SLA.0000000000001906  [DOI] [PubMed] [Google Scholar]
  • 8. Kaafarani HM, Mavros MN, Hwabejire J, et al. Derivation and validation of a novel severity classification for intraoperative adverse events. J Am Coll Surg 2014;218:1120-8. 10.1016/j.jamcollsurg.2013.12.060  [DOI] [PubMed] [Google Scholar]
  • 9. Chen Q, Rosen AK, Amirfarzan H, Rochman A, Itani KMF. Improving detection of intraoperative medical errors (iMEs) and intraoperative adverse events (iAEs) and their contribution to postoperative outcomes. Am J Surg 2018;216:846-50. 10.1016/j.amjsurg.2018.03.005  [DOI] [PubMed] [Google Scholar]
  • 10. Ramly EP, Larentzakis A, Bohnen JD, et al. The financial impact of intraoperative adverse events in abdominal surgery. Surgery 2015;158:1382-8. 10.1016/j.surg.2015.04.023  [DOI] [PubMed] [Google Scholar]
  • 11. Dindo D, Demartines N, Clavien PA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 2004;240:205-13. 10.1097/01.sla.0000133083.54934.ae  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Slankamenac K, Graf R, Barkun J, Puhan MA, Clavien PA. The comprehensive complication index: a novel continuous scale to measure surgical morbidity. Ann Surg 2013;258:1-7. 10.1097/SLA.0b013e318296c732  [DOI] [PubMed] [Google Scholar]
  • 13. Mazeh H, Cohen O, Mizrahi I, et al. Prospective validation of a surgical complications grading system in a cohort of 2114 patients. J Surg Res 2014;188:30-6. 10.1016/j.jss.2013.12.004  [DOI] [PubMed] [Google Scholar]
  • 14. Boney O, Moonesinghe SR, Myles PS, Grocott MP. Standardizing endpoints in perioperative research. Can J Anaesth 2016;63:159-68. 10.1007/s12630-015-0565-y  [DOI] [PubMed] [Google Scholar]
  • 15. Rosenthal R, Hoffmann H, Clavien PA, Bucher HC, Dell-Kuster S. Definition and classification of intraoperative complications (CLASSIC): Delphi study and pilot evaluation. World J Surg 2015;39:1663-71. 10.1007/s00268-015-3003-y  [DOI] [PubMed] [Google Scholar]
  • 16. Clavien PA, Barkun J, de Oliveira ML, et al. The Clavien-Dindo classification of surgical complications: five-year experience. Ann Surg 2009;250:187-96. 10.1097/SLA.0b013e3181b13ca2  [DOI] [PubMed] [Google Scholar]
  • 17. Müller M, Gloor B, Candinas D, Malinka T. The 100 most-cited articles in visceral surgery: a systematic review. Dig Surg 2016;33:509-19. 10.1159/000446930  [DOI] [PubMed] [Google Scholar]
  • 18. Bland JM, Altman DG. Statistics notes: validating scales and indexes. BMJ 2002;324:606-7. 10.1136/bmj.324.7337.606  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Vandenbroucke JP, von Elm E, Altman DG, et al. STROBE Initiative Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology 2007;18:805-35. 10.1097/EDE.0b013e3181577511  [DOI] [PubMed] [Google Scholar]
  • 20. Haynes AB, Weiser TG, Berry WR, et al. Safe Surgery Saves Lives Study Group A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491-9. 10.1056/NEJMsa0810119  [DOI] [PubMed] [Google Scholar]
  • 21. ASA Physical Status Classification System October 2014. http://www.asahq.org/resources/clinical-information/asa-physical-status-classification-system.
  • 22.BUPA (British United Provident Association). Schedule of procedures. https://bupa.secure.force.com/#/procedures.
  • 23. BUPA (British United Provident Association) http://www.cambridgevascularaccess.com/userfiles/BUPAScheduleofProcedures1.pdf.
  • 24. Mangram AJ, Horan TC, Pearson ML, Silver LC, Jarvis WR, Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee Guideline for prevention of surgical site infection, 1999. Am J Infect Control 1999;27:97-132, quiz 133-4, discussion 96. 10.1016/S0196-6553(99)70088-X  [DOI] [PubMed] [Google Scholar]
  • 25. Sutton R, Bann S, Brooks M, Sarin S. The Surgical Risk Scale as an improved tool for risk-adjusted analysis in comparative surgical audit. Br J Surg 2002;89:763-8. 10.1046/j.1365-2168.2002.02080.x  [DOI] [PubMed] [Google Scholar]
  • 26. Lee MJ, Daniels SL, Wild JRL, Wilson TR, SYSuRG RAGeS Group Readmissions after general surgery: a prospective multicenter audit. J Surg Res 2017;209:53-9. 10.1016/j.jss.2016.09.020  [DOI] [PubMed] [Google Scholar]
  • 27. Cole TJ, Altman DG. Statistics notes: percentage differences, symmetry, and natural logarithms. BMJ 2017;358:j3683. 10.1136/bmj.j3683  [DOI] [PubMed] [Google Scholar]
  • 28. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996;1:30-46 10.1037/1082-989X.1.1.30. [DOI] [Google Scholar]
  • 29. Gargon E, Williamson PR, Altman DG, Blazeby JM, Tunis S, Clarke M. The COMET Initiative database: progress and activities update (2015). Trials 2017;18:54. 10.1186/s13063-017-1788-8  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Hutter MM, Rowell KS, Devaney LA, et al. Identification of surgical complications and deaths: an assessment of the traditional surgical morbidity and mortality conference compared with the American College of Surgeons-National Surgical Quality Improvement Program. J Am Coll Surg 2006;203:618-24. 10.1016/j.jamcollsurg.2006.07.010  [DOI] [PubMed] [Google Scholar]
  • 31. Peponis T, Baekgaard JS, Bohnen JD, et al. Are surgeons reluctant to accurately report intraoperative adverse events? A prospective study of 1,989 patients. Surgery 2018;164:525-9. 10.1016/j.surg.2018.04.035  [DOI] [PubMed] [Google Scholar]
  • 32. Dell-Kuster S, Rentsch CA, Steiner LA, Rosenthal R. The importance of standardised recording of intraoperative adverse events: Key features of an ideal classification system. Eur Urol 2020 doi:101016/jeururo202001024. [DOI] [PubMed] [Google Scholar]
  • 33. Francis NK, Curtis NJ, Conti JA, Foster JD, Bonjer HJ, Hanna GB, EAES committees EAES classification of intraoperative adverse events in laparoscopic surgery. Surg Endosc 2018;32:3822-9. 10.1007/s00464-018-6108-1  [DOI] [PubMed] [Google Scholar]
  • 34. Coulson TG, Bailey M, Reid CM, et al. Acute Risk Change for Cardiothoracic Admissions to Intensive Care (ARCTIC index): a new measure of quality in cardiac surgery. J Thorac Cardiovasc Surg 2014;148:3076-81.e1. 10.1016/j.jtcvs.2014.06.069  [DOI] [PubMed] [Google Scholar]
  • 35. Coulson TG, Gregson B, Sandys S, et al. Acute Risk Change: an innovative measure of operative adverse events and perioperative team performance. J Cardiothorac Vasc Anesth 2018;32:2160-6. 10.1053/j.jvca.2018.01.014  [DOI] [PubMed] [Google Scholar]
  • 36. Haynes AB, Regenbogen SE, Weiser TG, et al. Surgical outcome measurement for a global patient population: validation of the Surgical Apgar Score in 8 countries. Surgery 2011;149:519-24. 10.1016/j.surg.2010.10.019  [DOI] [PubMed] [Google Scholar]
  • 37. Gawande AA, Kwaan MR, Regenbogen SE, Lipsitz SA, Zinner MJ. An Apgar score for surgery. J Am Coll Surg 2007;204:201-8. 10.1016/j.jamcollsurg.2006.11.011  [DOI] [PubMed] [Google Scholar]
  • 38. Bothner U, Georgieff M, Schwilk B. Building a large-scale perioperative anaesthesia outcome-tracking database: methodology, implementation, and experiences from one provider within the German quality project. Br J Anaesth 2000;85:271-80. 10.1093/bja/85.2.271  [DOI] [PubMed] [Google Scholar]
  • 39. Claassen L, van Workum F, Rosman C. Learning curve and postoperative outcomes of minimally invasive esophagectomy. J Thorac Dis 2019;11(Suppl 5):S777-85. 10.21037/jtd.2018.12.54  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Ioannidis JP, Evans SJ, Gøtzsche PC, et al. CONSORT Group Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004;141:781-8. 10.7326/0003-4819-141-10-200411160-00009  [DOI] [PubMed] [Google Scholar]
  • 41. Neuhaus C, Huck M, Hofmann G, St Pierre M, Weigand MA, Lichtenstern C. Applying the human factors analysis and classification system to critical incident reports in anaesthesiology. Acta Anaesthesiol Scand 2018;62:1403-11. 10.1111/aas.13213  [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Web appendix: Supplemental file

dels054044.ww.pdf (519.7KB, pdf)

Articles from The BMJ are provided here courtesy of BMJ Publishing Group

RESOURCES