Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Aug 13.
Published in final edited form as: Anaesthesia. 2019 Jun 20;74(9):1165–1174. doi: 10.1111/anae.14742

The development and validation of the Score for the Prediction of Postoperative Respiratory Complications (SPORC-2) to predict the requirement for early postoperative tracheal re-intubation: a hospital registry study

C Lukannek 1,2, S Shaefi 3, K Platzbecker 4, D Raub 1,2, P Santer 4, S Nabel 5, HS Lecamwasam 6,7, TT Houle 8, M Eikermann 9,10
PMCID: PMC7424553  NIHMSID: NIHMS1614119  PMID: 31222727

Abstract

Postoperative pulmonary complications are associated with an increase in mortality, morbidity and healthcare utilisation. The Agency for Healthcare Research and Quality recommends risk assessment for postoperative respiratory complications in patients undergoing surgery. In this hospital registry study of adult patients undergoing non-cardiac surgery between 2005 and 2017 at two independent healthcare networks, a prediction instrument for early postoperative tracheal re-intubation was developed and externally validated. This was based on the development of the Score for Prediction Of Postoperative Respiratory Complications. For predictor selection, stepwise backward logistic regression and bootstrap resampling were applied. Development and validation cohorts were represented by 90,893 patients at Partners Healthcare and 67,046 patients at Beth Israel Deaconess Medical Center, of whom 699 (0.8%) and 587 (0.9%) patients, respectively, had their tracheas reintubated. In addition to five pre-operative predictors identified in the Score for Prediction Of Postoperative Respiratory Complications, the final model included seven additional intra-operative predictors: early post-tracheal intubation desaturation; prolonged duration of surgery; high fraction of inspired oxygen; high vasopressor dose; blood transfusion; the absence of volatile anaesthetic use; and the absence of lung-protective ventilation. The area under the receiver operating characteristic curve for the new score was significantly greater than that of the original Score for Prediction Of Postoperative Respiratory Complications (0.84 [95%CI 0.82–0.85] vs. 0.76 [95%CI 0.75–0.78], respectively; p < 0.001). This may allow clinicians to develop and implement strategies to decrease the risk of early postoperative tracheal re-intubation.

Keywords: clinical epidemiology, critical care, postoperative ventilation, quality measures, risk assessment

Introduction

Each year, 3–8% of patients develop postoperative pulmonary complications (PPC) amounting to approximately 2.5 million patients in the USA alone [1, 2]. Postoperative pulmonary complications are associated with an increased risk of mortality, morbidity and postoperative discharge to a nursing home [3–7]. Commonly considered respiratory complications after surgery include pneumonia, pulmonary oedema, atelectasis and acute respiratory failure, potentially resulting in tracheal re-intubation [8]. Unplanned tracheal re-intubation after surgery was included as a quality benchmark measure in the Quality and Resource Use reports by the Centers for Medicare and Medicaid Services [9].

To enhance the quality and safety of peri-operative patient care, the Agency for Healthcare Research and Quality recommends risk assessment for postoperative respiratory failure [10]. We developed previously the Score for Prediction of Postoperative Respiratory Complications (SPORC); this is a widely accepted, simple prediction model for development of PPCs with the primary endpoint of early (within 72 h of surgery) tracheal re-intubation [11]. The SPORC has been advocated by the Agency for Healthcare Research and Quality and has been implemented in recommendations reflecting state of the art knowledge for pre-operative patient screening [10, 12].

Our primary aim was to develop and externally validate an improved prediction score (SPORC-2) that considered both pre- and intra-operative predictors, and which could be used by clinicians to assess patients for the risk of early postoperative tracheal re-intubation. Our secondary aim was to compare this new tool for stratified risk assessment to existing models that focus only on pre-operative predictors. We hypothesised that the addition of intra-operative variables to pre-operative risk assessment would significantly improve the predictive value of the model.

Methods

In this hospital registry study, patient data from two independent healthcare networks (Partners Healthcare and Beth Israel Deaconess Medical Center (BIDMC), Boston, MA, USA) were analysed. Both the Partners Institutional Review Board and the Committee on Clinical Investigations at Beth Israel Deaconess Medical Center approved this study and waived requirement for patient consent. This manuscript adheres to the applicable TRIPOD guidelines [13].

For score development, all patients undergoing non-cardiac surgery between 1 January 2007 and 31 December 2015 at Massachusetts General Hospital, Boston, and two community hospitals were screened for eligibility. For external validation, we considered all non-cardiac surgical patients at BIDMC between 1 October 2005 and 30 September 2017. All patients included in the study underwent general anaesthesia with tracheal intubation and post-procedural extubation in the operating theatre. For both cohorts, patients with any of the following criteria were not studied: ambulatory surgery; age < 18 years; ASA physical status 6; surgery within 10 days before the index procedure; and missing data-points of predictor variables. We accounted for missing data by applying multiple imputation by chained equations in both cohorts (see also Supporting Information, Section 7).

The primary outcome was early tracheal re-intubation after post-procedural extubation. This was defined as requirement for tracheal re-intubation within 72 h of surgery with subsequent mechanical ventilation in the operating theatre, recovery area or intensive care unit (ICU). Tracheal intubations within 72 h of the index procedure for an additional surgical procedure were not included in the primary outcome. In the development cohort, tracheal re-intubation was captured based on current procedural terminology codes for tracheal intubation and mechanical ventilation management. The primary outcome variable has been validated previously in the development cohort (see also Supporting Information, Section 1.1) [11]. In the validation cohort, tracheal re-intubation was captured based on time stamps for post-procedural tracheal extubation and mechanical ventilation after surgery available from the respiratory therapists’ database.

We considered both pre-operative (comorbidities and pre-operatively available measures of procedural risk) and intra-operative variables for the development of SPORC-2. A detailed definition of all predictors can be found in the Supporting Information (Section 3.1, Table S1). All five predictors of the previously reported SPORC were considered: ASA physical status ≥ 3; history of chronic pulmonary disease; history of heart failure; emergency surgery; and high-risk surgical services (vascular surgery; transplant surgery; neurosurgery; thoracic surgery; general surgery; and burn surgery) [11]. Furthermore, we considered surgical complexity as quantified by the procedural severity score [14]. Intra-operative candidate variables were selected a priori based on recent literature, biological plausibility and clinical reasoning and included: desaturation (SpO2 ≤ 90%) 5 min after tracheal intubation [7]; duration of surgery; median fraction of inspired oxygen (FIO2) [15]; noradrenaline equivalent dose of vasopressors; dose of neuromuscular blocking agents (NMBA) (expressed as multiples of NMBA dose needed to reduce twitch height by 95%) [16, 17]; fluid volume [18]; oral morphine equivalent dose [19, 20]; total administered fentanyl dose; intra-operative transfusion of packed red blood cells; use of volatile anaesthetic agents [21]; and the absence of lung-protective ventilation (defined as driving pressure [plateau pressure − positive end-expiratory pressure (PEEP)] > 15 mmHg) [22, 23].

The initial full multivariable logistic regression model contained all a priori defined pre- and intra-operative candidate predictors. To derive the final model, we used stepwise backward regression with p values < 0.05 to retain predictors from the set of candidate variables. This was followed by bootstrap resampling (n = 500 repetitions) to confirm predictor robustness and avoid overfitting. All predictors retained in the final model were assigned a weighting value by dividing the respective beta coefficient by the smallest beta coefficient among all final predictors. Weighting values were rounded to the nearest integer and multiplied by the respective predictor, thus estimating the final prediction score. All analyses were performed using Stata version 13 (Stata Corp LLC, College Station, TX, USA) or RStudio v1.0136 (RStudio. Inc, Boston, MA, USA).

Model discrimination was assessed by performing C-statistics. The Brier Score was calculated to examine model accuracy. We further assessed model calibration by creating a calibration plot. The Hosmer-Lemeshow test was performed with a p value > 0.05 indicating good model fit [24]. Analyses on model performance refer to the regression model with the prediction score as a single independent variable. The score variable represents the sum of all final predictors multiplied by their respective weighting values. In the independent validation cohort, we applied SPORC-2 to all patients and assessed model performance utilising C-statistics, Brier Score, and calibration plot. We compared the newly derived SPORC-2 with the original SPORC. Comparison of the predictive value of the two models was performed based on the respective C-statistics results. Furthermore, we performed net reclassification improvement analysis to assess whether the addition of unique procedure-related and intra-operative predictors to the original SPORC led to enhanced risk assessment.

Postoperative pulmonary complications may lead to increases in overall organ dysfunction, mortality and utilisation of healthcare resources [25]. As part of an exploratory analysis, we examined the association between SPORC-2 and further outcomes including: postoperative hospital duration of stay (defined as the number of hospitalised days after surgery); 30-day all-cause mortality; and 30-day re-admission rate. Analyses on the consequences of a high SPORC-2 were conducted in a dataset combining the development and validation cohorts. We compared the discriminatory ability of SPORC-2 with another existing model, the Assess Respiratory Risk in Surgical Patients in Catalonia (ARISCAT), applying the same comparison analyses as described above [26]. Results are described in the Supporting Information (Section 6.1.1). The pre-operative SPORC-2 consists of all pre-operatively available predictors (see also Supporting Information, Table S3). We compared the predictive value of the pre-operative SPORC-2 to the full SPORC-2. Results are described in the Supporting Information (Section 6.1.2).

Results

The final development cohort consisted of 90,893 patients, of whom 699 patients (0.8%) had their trachea re-intubated within 72 h of surgery (Table 1). At BIDMC, a total of 67,046 patients were studied; in the final validation cohort, 587 patients (0.9%) required tracheal re-intubation within 72 h of surgery (Fig. 1 and Supporting Information, Section 2).

Table 1.

Clinical characteristics of the development and validation cohort stratified by the presence or absence of early postoperative tracheal re-intubation. Values are number (proportion), mean (SD) or median (IQR [range]).

Development cohort (n = 90,893)
 Validation cohort (n = 67,046)
No tracheal re-intubation (n = 90,194) Tracheal re-intubation (n = 699) No tracheal re-intubation (n = 66,459) Tracheal re-intubation (n = 587)
Sex; male 40,523 (45%) 372 (53%) 28,050 (42%) 298 (51%)
Age; years 56 (17) 63 (16) 56 (17) 63 (15)
Body mass index; kg.m−2 28.6 (7.1) 28.5 (7.8) 28.8 (7.3) 28.9 (7.9)
ASA physical status 2 (2–3 [1–5]) 3 (2–3 [1–5]) 2 (2–3 [1–5]) 3 (2–3 [1–5])
Chronic pulmonary disease 16,943 (19%) 228 (33%) 12,353 (19%) 188 (32%)
Heart failure 7395 (8%) 240 (34%) 4219 (6%) 101 (17%)
Emergency surgery 4,711 (5%) 104 (15%) 4,965 (8%) 88 (15%)
High-risk surgical servicesa 40,348 (45%) 413 (59%) 30,267 (46%) 301 (51%)
Duration of procedure; min 177 (122–253 [9–1153]) 232 (146–343 [31–885]) 156 (112–222 [0–1147]) 183 (124–267 [15–874]
Duration of intra-operative MAP< 55 mmHg; min 0 (0–2 [0–614]) 2 (0–6 [0–258]) 1 (0–3 [0–243]) 1 (0–4 [0–59])
Procedural severity score for morbidity 41 (18) 54 (17) 39 (16) 49 (17)
Packed red blood cells; 0 units 86,299 (96%) 572 (82%) 64,250 (97%) 496 (85%)
Packed red blood cells; 1–2 units 3189 (4%) 93 (13%) 1981 (3%) 76 (13%)
Packed red blood cells; ≥ 3 units 706 (1%) 34 (5%) 228 (0%) 15 (3%)
Fluids; lb 1.5 (1.0–2.5 [0.0–30.8]) 2.0 (1.0–3.5 [0.0–20.2]) 3.5 (2.5–5.0 [0.0–34.0]) 4.0 (2.5–6.5 [0.0–24.9])
Oral morphine equivalent of total long-acting intra-operative opioid dose; mg 15 (2–27 [0–545] 7 (0–24 [0–408]) 14 (0–24 [0–558]) 14 (0–24 [0–102])
95% effective dose of total NMBA; mg 2.7 (1.7–4.1 [0.0–46.7]) 3.3 (2.1–5.1 [0.0–19.0]) 2.2 (1.2–3.2 [0.0–40.7]) 2.6 (1.5–4.0 [0.0–17.8])
Neostigmine dose;mg 2 (0–4 [0–23]) 3 (0–4 [0–10] 3 (0–4 [0–20]) 3 (0–4[0–8])
Dose equivalent of inhalational anaesthetic; MAC 0.6 (0.4–0.8 [0.0–2.5]) 0.5 (0.3–0.8 [0.0–1.6]) 0.9 (0.7–1.1 [0.0–3.0]) 0.9 (0.7–1.1 [0.0–2.4])
Noradrenaline equivalent of vasopressor dose; mg 0.1 (0.0–0.3 [0.0–97.5] 0.3 (0.1–0.8 [0.0–91.0] 0.0 (0.0–0.1 [0.0–72.5]) 0.1 (0.0–0.5 [0.0–45.7])
SpO2 5 min post-tracheal intubation; % 100 (99–100 [45–100]) 100 (99–100 [80–100]) 99 (99–100 [60–100]) 100 (98–100 [82–100]
FIO2; % 52 (38–58 [21–100]) 57 (46–77 [21–100] 54 (48–60 [21–100]) 58 (52–78 [21–100])
Lung-protective ventilationc 55,417 (61%) 354 (51%) 19,163 (29%) 148 (25%)
Open in a new tab

ASA, American Society of Anesthesiologists; MAP, mean arterial pressure; NMBA, neuromuscular blocking agent; MAC, minimum alveolar concentration; FIO2, fraction of inspired oxygen; SpO2, peripheral capillary oxygen saturation.

a

Includes vascular surgery, transplant surgery, neurosurgery, thoracic surgery, general surgery and burns.

b

Crystalloid:colloid ratio 1:1.5.

c

Lung-protective ventilation was defined as driving pressure ≤ 15 mmHg(median plateau pressure–median positive end-expiratory pressure).

Figure 1.

Figure 1

Open in a new tab

Study flow chart. Multiple exclusion criteria may apply.

A total of 12 predictors were retained in the final model for SPORC-2: five pre-operative variables (ASA physical status ≥ 3; heart failure; chronic pulmonary disease; emergency surgery; and high procedural severity score (32–48 and ≥ 49)) [14]; and seven intra-operative factors (desaturation (SpO2 ≤ 90%) 5 min after tracheal intubation; duration of surgery > 140 min and > 225 min; high median FIO2 (> 0.61); high total noradrenaline equivalent dose (> 0.18 mg); intra-operative transfusion of packed red blood cells; the absence of volatile anaesthetic agents; and lack of lung-protective ventilation patterns). The final prediction score reached a maximum score point value of 46 points (Fig. 2). Predicted risks of postoperative tracheal re-intubation for increasing score point values derived from the development cohort are shown in the Supporting Information (Table S4).

Figure 2.

Figure 2

Open in a new tab

Factors used in the score for the prediction of early tracheal re-intubation after surgery (Score for Prediction of Postoperative Respiratory Complications-2). Score point values of applicable predictors are summed up to create total score. PSS, procedural severity score.

In the development cohort, the final model had an area under the receiver operating characteristic curve (AUC) of 0.84 (95%CI 0.82–0.85) showing excellent discriminative ability (Fig. 3) [24]. The uncertainty of 0.01 has to be considered in relation to the relatively low incidence of the outcome. Predictions ranged from 0.2% to 60.3% and were well calibrated (reliability 0.002). The Hosmer–Lemeshow test was not significant (p = 0.06), indicating good model fit. In the validation cohort, C-statistics proved good discriminative ability (AUC 0.75 (95%CI 0.73–0.77)) (Fig. 3). The Brier Score of 0.012 (uncertainty 0.010, reliability 0.002) confirmed the results of the development cohort. Figure 4 shows the model calibration in the development and validation cohorts.

Figure 3.

Figure 3

Open in a new tab

Model discrimination (C-statistics) for the score for the prediction of early tracheal re-intubation after surgery (Score for Prediction of Postoperative Respiratory Complications-2) in development (Fig. 3a) and external validation (Fig. 3b) cohorts.

Figure 4.

Figure 4

Open in a new tab

Calibration plots for the score for the prediction of early tracheal re-intubation after surgery (Score for Prediction of Postoperative Respiratory Complications-2) in development (Fig. 4a) and external validation (Fig. 4b) cohorts.

When comparing the newly derived SPORC-2 to SPORC, the AUCs showed significantly improved model discrimination in the development cohort (0.84 (95%CI 0.82–0.85) vs. 0.75 (95%CI 0.73–0.77), respectively; p < 0.001) (Fig. 5a). In addition, risk categorisation was significantly improved in net reclassification improvement (NRI) analyses (additive NRI 0.27; p < 0.001). Overall, 232 (33.1%) patients were adequately reclassified into a higher, and 16,209 (18.0%) patients into a lower risk category when considering the additional intra-operative predictors of SPORC-2 (see also Supporting Information, Section 5 and Table S2a). In the external validation cohort, SPORC-2 performed significantly better than SPORC (AUC 0.75 (95% CI 0.73–0.77 vs. 0.69 (95%CI 0.67–0.70), respectively; p < 0.001) (Fig. 5b). Net reclassification improvement results derived from the external validation cohort are described in the Supporting Information (Section 5, Table S2b).

Figure 5.

Figure 5

Open in a new tab

Comparison of model discrimination (C-statistics) for Score for Prediction of Postoperative Respiratory Complications (SPORC-2) (dashed lines) and SPORC (solid lines) to predict early tracheal re-intubation after surgery in development (Fig. 5a) and validation (Fig. 5b) cohorts.

In the exploratory analyses, when combining the development and validation cohorts, 1286 (0.8%) of 157,939 patients required early tracheal re-intubation after surgery, compared with 156,653 patients (99.2%) who did not. The requirement for early tracheal re-intubation was associated with increased 30-day mortality (11.0% vs. 0.7%, absolute risk difference 10.4% (95%CI 8.7–12.1%)), 30-day re-admission rates (15.8% vs. 8.3%, absolute risk difference 7.5% (95%CI 5.5–9.5%)) and postoperative hospital duration of stay (median (IQR [range]), 10 (6–16 [1–326]) days vs. 4 (2–6 [1–288]) days).

Discussion

We developed and externally validated SPORC-2, a prediction instrument for early tracheal re-intubation after surgery. The incorporation of intra-operative predictors resulted in a significantly improved and clinically more relevant prediction model, with enhanced discriminative ability and risk categorisation in both the development and validation cohorts.

The primary end-point of the present study was tracheal re-intubation within 72 h of surgery. Early postoperative tracheal re-intubation is one of the most rigorous markers for persistent respiratory failure [3, 27]. In our study, early tracheal re-intubation was associated with increased all-cause mortality within 30 days of surgery, 30-day re-admission rate and hospital duration of stay. The Agency for Healthcare Research and Quality’s recommendation to assess the risk of early tracheal re-intubation underlines the need for evidence-based prediction instruments for PPCs. The inclusion of tracheal re-intubation as a quality benchmark in the Quality and Resource Use reports by the Centers for Medicare and Medicaid Services confirms its importance as an adverse patient-centred outcome [9].

Several predictors in the presented score relate to procedural risk independent of the individual intra-operative course, such as emergency versus elective surgery, procedural severity score and duration of surgery. In many academic centres, predicted duration of surgery data are available pre-operatively, and administrators use these data for optimal operating room scheduling [28]. Our prediction instrument includes several potentially modifiable intra-operative predictors, such as high median FIO2, absence of volatile anaesthetic agents, high vasopressor dose, absence of lung-protective ventilation and intra-operative transfusion of packed red blood cells.

We found a strong association between high median FIO2 (> 0.61) and early tracheal re-intubation after surgery, which is in line with previous findings [15]. A high FIO2 in general may be a marker for pulmonary dysfunction that would not be accounted for by other predictor variables. Adverse effects of higher inspiratory oxygen concentrations may be explained by increased absorption atelectasis [29], as well as increased production of reactive oxygen species with resultant oxidative stress injury to the lungs and other organ systems [30]. Atelectasis may contribute to additional pulmonary complications, thereby increasing the likelihood of requiring postoperative tracheal re-intubation [31]. The negative effects of high inspired oxygen concentrations need to be weighed against the clinical requirement of avoiding and safeguarding against potentially injurious hypoxaemia.

Our finding of the absence of volatile anaesthetic agents as a significant predictor for early tracheal re-intubation confirms previous results. Volatile anaesthetic agents are bronchodilators and immunomodulators [32], and their intra-operative use is dose-dependently associated with lower odds of PPCs [21].

We observed intra-operative haemodynamic instability, expressed as the requirement of high vasopressor doses and transfusion of packed red blood cells, to be associated with an increased risk of early tracheal re-intubation. The predictive value of high vasopressor dose as a risk factor for the development of PPCs was shown previously in the trial by Neto et al. [33]. Both high vasopressor dose and intra-operative transfusion of packed red blood cells are indicators of haemodynamic instability due to ongoing intra-operative blood loss and/or a higher degree of distributive shock, conditions that are associated with an increased risk of PPCs.

Increasing and individualising PEEP levels, as well as decreasing plateau airway pressure, may have lung-protective effects [22, 34]. Accordingly, the absence of lung-protective ventilation patterns was associated with an increased risk of early tracheal re-intubation, which is in line with previous work [22, 35]. It is possible to modify intra-operative driving pressure by optimising flow characteristics during intra-operative mechanical ventilation [35].

Comparison of SPORC-2 with other published tools for pre-operative risk assessment (e.g. SPORC, ARISCAT) showed enhanced performance of SPORC-2 [11, 26]. One-third of patients in the score development cohort were adequately reclassified into higher risk categories when applying SPORC-2 instead of SPORC. To be optimally effective, the SPORC-2 tool should be used for stratified risk prediction. For example, SPORC-2 may be able to assign a patient score using data from pre-operative anaesthetic assessment that includes medical history (chronic pulmonary disease, heart failure), ASA physical status, emergency status and an automatically calculated procedural severity score. This information will be important for intra-operative resource allocation and postoperative critical care requirements. Intra-operatively, the clinician will integrate information provided by device data and documentation of events (e.g. lack of lung-protective ventilation, use of blood products, etc.) to generate a full SPORC-2 in real time. This information may change postoperative management; for example, ICU bed allocation depends on intra-operative risk assessment [36]. Given the expansion of electronic health records and anaesthesia information management systems, one would presume that generating SPORC-2 in real time using patient baseline characteristics, biomedical device and medical record data would be straightforward. It is encouraging to see a new breed of technologies with advanced medical device connectivity, powerful processing capability and real-time decision support enter the anaesthesia software market [37–39].

The utilisation of intra-operative predictors is an important distinctive feature of our prediction model. The majority of previous studies on the prediction of PPCs focused on predictors only available pre-operatively [26, 40, 41]. The trial by Neto et al, which derived a moderately performing prediction model comprising six pre-operative, five intra-operative and two procedure-related risk factors, was limited by its relatively small sample size and lack of external validation [33]. Further studies investigating intra-operative predictors of PPCs were similarly limited by small sample size and lack of external validation [2, 42]. In the present study, we externally validated our score in an independent cohort with a different case mix and outcome incidence. The utilisation of two different cohorts with high-quality, validated data allowed us to prove the generalisability of the newly developed model.

This study has several limitations. The observational nature of the study and use of administrative data and billing codes confer risks of unmeasured confounding factors and bias. Second, despite the granularity of the databases, the independence of the cohorts used for SPORC-2 development and validation, and the robustness of our findings, there are some limitations to the scope of the subjects captured. Regarding PPCs, risk assessment of subjects whose tracheas remained intubated at the end of the procedure would be of comparative interest. It remains unclear whether there are differences in characteristics and outcomes between patients whose tracheas remain intubated at the end of surgery compared with those whose tracheas are primarily extubated and subsequently re-intubated later.

In summary, we have derived and validated SPORC-2, an instrument for stratified assessment of a patient’s risk of early tracheal re-intubation after surgery. This tool may improve peri-operative resource allocation for surgical patients and will help clinicians fulfil impending reporting requirements. It may also prove beneficial in supporting clinicians in their efforts to advance patient safety and improve overall quality of care.

Supplementary Material

Supplemental Material

Data S1 Score for Stratified Prediction of Early Postoperative Re-intubation (SPORC-2): a hospital registry study.

Table S1. Definition of candidate predictors.

Table S2a. SPORC-2 vs. SPORC: net reclassification improvement analysis in the development cohort. Risk categorisation for early tracheal re-intubation after surgery within 72 h. The accuracy of the two models SPORC-2 vs. SPORC was compared using net reclassification improvement analysis: grey cells indicate no reclassification; green cells indicate improved classification; and yellow cells indicate worsened classification. Two hundred and thirty-two patients (33.1%) were adequately reclassified in higher risk categories, n = 16,209 (18.0%) were adequately reclassified in lower-risk categories when adding the additional SPORC-2 variables to SPORC.

Table S2b. SPORC-2 vs. SPORC: net reclassification improvement analysis in the validation cohort. Risk categorisation for early tracheal re-intubation after surgery within 72 h. The risk classification of the two models SPORC-2 vs. SPORC was compared using net reclassification improvement: grey cells indicate no reclassification; green cells indicate improved classification; and yellow cells indicate worse classification. One hundred and seventy-two patients (29.3%) were adequately reclassified in higher risk categories, whereas 12,470 (18.8%) patients were adequately reclassified in lower-risk categories when adding the additional SPORC-2 predictors to SPORC.

Table S3. “Pre-operative” SPORC-2. This model utilises all predictors of SPORC-2 that are available pre-operatively.

Table S4. Predicted risk of unplanned tracheal reintubation for increasing score values. Table shows the predicted risk for early tracheal re-intubation within 72 h after surgery for increasing score values of SPORC-2 estimated in the score development cohort.

Table S5. Comparison of results from logistic regression and penalised maximum likelihood estimation in the development cohort.

Figure S1. Model discrimination of pre-operative SPORC-2 in development cohort. The pre-operative SPORC-2 yielded an AUC of 0.79 [95%CI 0.78–0.81] in the development cohort.

Figure S2a. SPORC-2 vs. pre-operative SPORC-2: ROC comparison in development cohort. In the development cohort, compare SPORC-2 AUC of 0.84 [95% CI 0.83–0.85] to pre-operative SPORC-2 AUC of 0.79 [95% CI 0.78–0.81] (p < 0.001).

Figure S2b. SPORC-2 vs. pre-operative SPORC-2: ROC comparison in validation cohort. In the validation cohort, compare SPORC-2 AUC of 0.75 [95% CI 0.73–0.77] to preoperative SPORC-2 AUC of 0.72 [95% CI 0.70–0.74] (p < 0.001).

Figure S3. SPORC-2 vs. ARISCAT: ROC comparison in development cohort. In a subcohort of 30,578 patients in the development cohort, compare SPORC-2 AUC of 0.80 [95% CI 0.78–0.83] to ARISCAT AUC of 0.63 [95% CI 0.59–0.66](p < 0.001).

Acknowledgements

This work was funded by philanthropic donors Dr Jeffrey and Judy Buzen. The donors were not involved in the study design, data collection, data analysis, data interpretation or writing of the manuscript and supporting materials. HL is an employee at Talis Clinical LLC, a company that offers secure web-based online medical records. TH reports grants from NINDS and NIGMS, and personal fees from Headache, Anesthesiology and Cephalalgia, outside the submitted work. ME receives funding from philanthropic donors Dr Jeffrey and Judy Buzen. No other external funding or competing interests declared.

References

  • 1.Johnson RG, Arozullah AM, Neumayer L, Henderson WG, Hosokawa P, Khuri SF. Multivariable predictors of postoperative respiratory failure after general and vascular surgery: results from the patient safety in surgery study. Journal of the American College of Surgeons 2007; 204: 1188–98. [DOI] [PubMed] [Google Scholar]
  • 2.McAlister FA, Bertsch K, Man J, Bradley J, Jacka M. Incidence of and risk factors for pulmonary complications after nonthoracic surgery. American Journal of Respiratory and Critical Care Medicine 2005; 171:514–17. [DOI] [PubMed] [Google Scholar]
  • 3.Nafiu OO, Ramachandran SK, Ackwerh R, Tremper KK, Campbell DA Jr, Stanley JC. Factors associated with and consequences of unplanned post-operative intubation in elderly vascular and general surgery patients. European Journal of Anaesthesiology 2011; 28: 220–4. [DOI] [PubMed] [Google Scholar]
  • 4.Bailey JG, Davis PJ, Levy AR, Molinari M, Johnson PM. The impact of adverse events on health care costs for older adults undergoing nonelective abdominal surgery. Canadian Journal of Surgery 2016; 59: 172–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Dimick JB, Chen SL, Taheri PA, Henderson WG, Khuri SF, Campbell DA Jr. Hospital costs associated with surgical complications: a report from the private-sector National Surgical Quality Improvement Program. Journal of the American College of Surgeons 2004; 199: 531–7. [DOI] [PubMed] [Google Scholar]
  • 6.Fernandez-Bustamante A, Frendl G, Sprung J, et al. Postoperative pulmonary complications, early mortality, and hospital stay following noncardiothoracic surgery: a multicenter study by the perioperative research network investigators. Journal of the American Medical Association Surgery 2017; 152:157–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Rostin P, Teja BJ, Friedrich S, et al. The association of early postoperative desaturation in the operating theatre with hospital discharge to a skilled nursing or long-term care facility. Anaesthesia 2019; 74:457–67. [DOI] [PubMed] [Google Scholar]
  • 8.Miskovic A, Lumb AB. Postoperative pulmonary complications. British Journal of Anaesthesia 2017; 118: 317–34. [DOI] [PubMed] [Google Scholar]
  • 9.U.S. Centers for Medicare & Medicaid Services. Benchmarks for measures included in the performance year 2016 quality and resource use reports. 2017. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/PhysicianFeedbackProgram/Downloads/PY2016-Prior-Year-Benchmarks.pdf (accessed 10/25/2018).
  • 10.Agency for Healthcare Research and Quality. Toolkit for using the AHRQ quality indicators. Selected best practices and suggestions for improvement - PSI 11: postoperative respiratory failure. 2016. https://www.ahrq.gov/sites/default/files/wysiwyg/professionals/systems/hospital/qitoolkit/combined/d4h_combo_psi11-postoprespfailure-bestpractices.pdf (accessed 09/20/2018).
  • 11.Brueckmann B, Villa-Uribe JL, Bateman BT, et al. Development and validation of a score for prediction of postoperative respiratory complications. Anesthesiology 2013; 118: 1276–85. [DOI] [PubMed] [Google Scholar]
  • 12.Joint recommendation of the German Society of Anaesthesiology and Intensive Care Medicine, the German Society of Surgery, and the German Society of Internal Medicine. Preoperative evaluation of adult patients before elective, non-cardiothoracic surgery. Anästhesiologie, Intensivmedizin, Notfallmedizin und Schmerztherapie: AINS. 2017; 52:446–62. [Google Scholar]
  • 13.Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Annals of Internal Medicine 2015; 162: 55–63. [DOI] [PubMed] [Google Scholar]
  • 14.Dalton JE, Kurz A, Turan A, Mascha EJ, Sessler DI, Saager L. Development and validation of a risk quantification index for 30-day postoperative mortality and morbidity in noncardiac surgical patients. Anesthesiology 2011; 114: 1336–44. [DOI] [PubMed] [Google Scholar]
  • 15.Staehr-Rye AK, Meyhoff CS, Scheffenbichler FT, et al. High intraoperative inspiratory oxygen fraction and risk of major respiratory complications. British Journal of Anaesthesia 2017; 119:140–9. [DOI] [PubMed] [Google Scholar]
  • 16.McLean DJ, Diaz-Gil D, Farhan HN, Ladha KS, Kurth T, Eikermann M. Dose-dependent association between intermediate-acting neuromuscular-blocking agents and postoperative respiratory complications. Anesthesiology 2015; 122:1201–13. [DOI] [PubMed] [Google Scholar]
  • 17.deBacker J, Hart N, Fan E. Neuromuscular blockade in the 21st century management of the critically Ill patient. Chest 2017; 151:697–706. [DOI] [PubMed] [Google Scholar]
  • 18.Shin CH, Grabitz SD, Timm FP, et al. Development and validation of a Score for Preoperative Prediction of Obstructive Sleep Apnea (SPOSA) and its perioperative outcomes. BMC Anesthesiology 2017; 17:71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Friedrich S, Raub D, Teja BJ, et al. Effects of low-dose intraoperative fentanyl on postoperative respiratory complication rate: a pre-specified, retrospective analysis. British Journal of Anaesthesia 2019; 122:e180–8. [DOI] [PubMed] [Google Scholar]
  • 20.Long DR, Lihn AL, Friedrich S, et al. Association between intraoperative opioid administration and 30-day readmission: a pre-specified analysis of registry data from a healthcare network in New England. British Journal of Anaesthesia 2018; 120:1090–102. [DOI] [PubMed] [Google Scholar]
  • 21.Grabitz SD, Farhan HN, Ruscic KJ, et al. Dose-dependent protective effect of inhalational anesthetics against postoperative respiratory complications: a prospective analysis of data on file from three hospitals in New England. Critical Care Medicine 2017; 45: e30–9. [DOI] [PubMed] [Google Scholar]
  • 22.Ladha K, Vidal Melo MF, McLean DJ, et al. Intraoperative protective mechanical ventilation and risk of postoperative respiratory complications: hospital based registry study. British Medical Journal 2015; 351: h3646. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Bagchi A, Rudolph MI, Ng PY, et al. The association of postoperative pulmonary complications in 109,360 patients with pressure-controlled or volume-controlled ventilation. Anaesthesia 2017; 72: 1334–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Hosmer DW, Lemeshow S. Applied logistic regression In: Cressie NAC, Fisher NI, Johnstone IM, et al. , eds. Wiley series in probability and statistics. New York, NY: John Wiley & Sons, Inc, 2000. [Google Scholar]
  • 25.Kim M, Brady JE, Li G. Interaction effects of acute kidney injury, acute respiratory failure, and sepsis on 30-day postoperative mortality in patients undergoing high-risk intraabdominal general surgical procedures. Anesthesia and Analgesia 2015; 121:1536–46. [DOI] [PubMed] [Google Scholar]
  • 26.Canet J, Gallart L, Gomar C, et al. Prediction of postoperative pulmonary complications in a population-based surgical cohort. Anesthesiology 2010; 113:1338–50. [DOI] [PubMed] [Google Scholar]
  • 27.Benedetto WJ, Hess DR, Gettings E, et al. Urgent tracheal intubation in general hospital units: an observational study. Journal of Clinical Anesthesia 2007; 19: 20–4. [DOI] [PubMed] [Google Scholar]
  • 28.Zenteno AC, Carnes T, Levi R, Daily BJ, Dunn PF. Systematic OR block allocation at a large academic medical center: comprehensive review on a data-driven surgical scheduling strategy. Annals of Surgery 2016; 264: 973–81. [DOI] [PubMed] [Google Scholar]
  • 29.Edmark L, Kostova-Aherdan K, Enlund M, Hedenstierna G. Optimal oxygen concentration during induction of general anesthesia. Anesthesiology 2003; 98: 28–33. [DOI] [PubMed] [Google Scholar]
  • 30.Martin DS, Grocott MP. Oxygen therapy and anaesthesia: too much of a good thing? Anaesthesia 2015; 70: 522–7. [DOI] [PubMed] [Google Scholar]
  • 31.Hedenstierna G, Edmark L. Mechanisms of atelectasis in the perioperative period. Best Practice and Research: Clinical Anaesthesiology 2010; 24: 157–69. [DOI] [PubMed] [Google Scholar]
  • 32.Stollings LM, Jia LJ, Tang P, Dou H, Lu B, Xu Y. Immune modulation by volatile anesthetics. Anesthesiology 2016; 125: 399–411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Neto AS, da Costa LGV, Hemmes SNT, et al. The LAS VEGAS risk score for prediction of postoperative pulmonary complications: an observational study. European Journal of Anaesthesiology 2018; 35: 691–701. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Hodgson LE, Murphy PB, Hart N. Respiratory management of the obese patient undergoing surgery. Journal of Thoracic Disease 2015; 7:943–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.O’Gara B, Talmor D. Perioperative lung protective ventilation. British Medical Journal 2018; 362: k3030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Thevathasan T, Copeland CC, Long DR, et al. The impact of postoperative intensive care unit admission on postoperative hospital length of stay and costs: a prespecified propensity-matched cohort study. Anesthesia and Analgesia 2018. [DOI] [PubMed] [Google Scholar]
  • 37.Nair BG, Horibe M, Newman SF, Wu WY, Peterson GN, Schwid HA. Anesthesia information management system-based near real-time decision support to manage intraoperative hypotension and hypertension. Anesthesia and Analgesia 2014; 118:206–14. [DOI] [PubMed] [Google Scholar]
  • 38.Epstein RH, Dexter F, Patel N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesthesia and Analgesia 2015; 121: 678–92. [DOI] [PubMed] [Google Scholar]
  • 39.Simpao AF, Rehman MA. Anesthesia information management systems. Anesthesia and Analgesia 2018; 127: 90–4. [DOI] [PubMed] [Google Scholar]
  • 40.Canet J, Sabate S, Mazo V, et al. Development and validation of a score to predict postoperative respiratory failure in a multicentre European cohort: a prospective, observational study. European Journal of Anaesthesiology 2015; 32: 458–70. [DOI] [PubMed] [Google Scholar]
  • 41.Gupta H, Gupta PK, Fang X, et al. Development and validation of a risk calculator predicting postoperative respiratory failure. Chest 2011; 140:1207–15. [DOI] [PubMed] [Google Scholar]
  • 42.Jeong BH, Shin B, Eom JS, et al. Development of a prediction rule for estimating postoperative pulmonary complications. PLoS ONE 2014; 9: e113656. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material

Data S1 Score for Stratified Prediction of Early Postoperative Re-intubation (SPORC-2): a hospital registry study.

Table S1. Definition of candidate predictors.

Table S2a. SPORC-2 vs. SPORC: net reclassification improvement analysis in the development cohort. Risk categorisation for early tracheal re-intubation after surgery within 72 h. The accuracy of the two models SPORC-2 vs. SPORC was compared using net reclassification improvement analysis: grey cells indicate no reclassification; green cells indicate improved classification; and yellow cells indicate worsened classification. Two hundred and thirty-two patients (33.1%) were adequately reclassified in higher risk categories, n = 16,209 (18.0%) were adequately reclassified in lower-risk categories when adding the additional SPORC-2 variables to SPORC.

Table S2b. SPORC-2 vs. SPORC: net reclassification improvement analysis in the validation cohort. Risk categorisation for early tracheal re-intubation after surgery within 72 h. The risk classification of the two models SPORC-2 vs. SPORC was compared using net reclassification improvement: grey cells indicate no reclassification; green cells indicate improved classification; and yellow cells indicate worse classification. One hundred and seventy-two patients (29.3%) were adequately reclassified in higher risk categories, whereas 12,470 (18.8%) patients were adequately reclassified in lower-risk categories when adding the additional SPORC-2 predictors to SPORC.

Table S3. “Pre-operative” SPORC-2. This model utilises all predictors of SPORC-2 that are available pre-operatively.

Table S4. Predicted risk of unplanned tracheal reintubation for increasing score values. Table shows the predicted risk for early tracheal re-intubation within 72 h after surgery for increasing score values of SPORC-2 estimated in the score development cohort.

Table S5. Comparison of results from logistic regression and penalised maximum likelihood estimation in the development cohort.

Figure S1. Model discrimination of pre-operative SPORC-2 in development cohort. The pre-operative SPORC-2 yielded an AUC of 0.79 [95%CI 0.78–0.81] in the development cohort.

Figure S2a. SPORC-2 vs. pre-operative SPORC-2: ROC comparison in development cohort. In the development cohort, compare SPORC-2 AUC of 0.84 [95% CI 0.83–0.85] to pre-operative SPORC-2 AUC of 0.79 [95% CI 0.78–0.81] (p < 0.001).

Figure S2b. SPORC-2 vs. pre-operative SPORC-2: ROC comparison in validation cohort. In the validation cohort, compare SPORC-2 AUC of 0.75 [95% CI 0.73–0.77] to preoperative SPORC-2 AUC of 0.72 [95% CI 0.70–0.74] (p < 0.001).

Figure S3. SPORC-2 vs. ARISCAT: ROC comparison in development cohort. In a subcohort of 30,578 patients in the development cohort, compare SPORC-2 AUC of 0.80 [95% CI 0.78–0.83] to ARISCAT AUC of 0.63 [95% CI 0.59–0.66](p < 0.001).

NIHMS1614119-supplement-Supplemental_Material.docx (153KB, docx)

RESOURCES