Abstract
Background
Surgical risk prediction tools can facilitate shared decision-making and efficient allocation of perioperative resources. Such tools should be externally validated in target populations before implementation.
Methods
Predicted risk of 30-day mortality was retrospectively derived for surgical patients at Royal Perth Hospital from 2014 to 2021 using the Surgical Outcome Risk Tool (SORT) and the related NZRISK (n=44 031, 53 395 operations). In a sub-population (n=31 153), the Physiology and Operative Severity Score for the enumeration of Mortality (POSSUM) and the Portsmouth variant of this (P-POSSUM) were matched from the Copeland Risk Adjusted Barometer (C2-Ai, Cambridge, UK). The primary outcome was risk score discrimination of 30-day mortality as evaluated by area-under-receiver operator characteristic curve (AUROC) statistics. Calibration plots and outcomes according to risk decile and time were also explored.
Results
All four risk scores showed high discrimination (AUROC) for 30-day mortality (SORT=0.922, NZRISK=0.909, P-POSSUM=0.893; POSSUM=0.881) but consistently over-predicted risk. SORT exhibited the best discrimination and calibration. Thresholds to denote the highest and second-highest deciles of SORT risk (>3.92% and 1.52–3.92%) captured the majority of deaths (76% and 13%, respectively) and hospital-acquired complications. Year-on-year SORT calibration performance drifted towards over-prediction, reflecting a decrease in 30-day mortality over time despite an increase in the surgical population risk.
Conclusions
SORT was the best performing risk score in predicting 30-day mortality after surgery. Categorising patients based on SORT into low, medium (80–90th percentile), and high risk (90–100th percentile) might guide future allocation of perioperative resources. No tools were sufficiently calibrated to support shared decision-making based on absolute predictions of risk.
Keywords: external validation, resource allocation, risk prediction, shared decision making, surgical mortality
Surgical risk prediction tools can facilitate shared decision-making and efficient allocation of perioperative resources.1 Many such tools have demonstrated excellent predictive performance in the populations in which they are developed (internal validation) but perform less well in external validation studies without recalibration.2 3 For shared decision-making, where a patient may decline an operation or treatment based on the quoted risk, accurate predictive performance across the entire risk range (calibration) is important.4
The advent of locally developed or commercially available tools to calculate risk scores and record surgical outcomes in large data sets allows performance evaluation (external validation) at a hospital level before implementation. This evaluation can guide which, if any, available risk tools can be successfully applied to the local population, or whether bespoke risk tools should be developed. Furthermore, local data can define appropriate risk thresholds, above which patients can systematically be allocated extra resources with the aim of improving outcomes.
At Royal Perth Hospital, a data and digital innovation unit was recently established, linking a range of perioperative information in a single data warehouse. In parallel, the Copeland Risk Adjusted Barometer (C2-Ai, Cambridge, UK) system was introduced, primarily to evaluate risk-adjusted surgical outcomes and benchmark them against other hospitals in the system database. We set out to use these information systems to externally validate the performance of four common surgical risk tools, when applied in our hospital over a 7-yr period.
Methods
The study protocol, incorporating a waiver of consent, was approved by the Royal Perth Hospital Human Research Ethics Committee on August 12, 2021 (RGS0000004853). This retrospective observational study is reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.5
Risk tools assessed
Four commonly used risk scores were evaluated for the prediction of 30-day mortality. First, version one of the Surgical Outcome Risk Tool (SORT)6 and the related NZRISK tool2 were evaluated, as they were recently identified as suitable candidates for adaptation in Australian hospitals,7 and their component variables were available within the data and digital innovation warehouse. Second, the Physiology and Operative Severity Score for the enumeration of Mortality (POSSUM)8 and the Portsmouth variant of this (P-POSSUM)9 were evaluated as they were available within the Copeland Risk Adjusted Barometer system. This system applies proprietary algorithms to derive risk scores retrospectively from coding-based information after hospital discharge.
Eligibility, data extraction, and selection
Surgical episodes taking place between July 1, 2014 and June 30, 2021 were examined if the following eligibility criteria were met: age ≥18 yr; procedure undertaken in the main theatre complex by a surgeon; planned postoperative stay ≥1 night; non-indigenous ethnicity (to allow a fair comparison across risk scores with varying approaches to ethnicity).
The perioperative data required to apply the SORT and NZRISK tools were extracted from the data and digital innovation warehouse. Sources of data in this extract included the Theatre Management System and WebPas software used across public hospitals in Perth to schedule and track operations and outcomes. These were supplemented by the International Classification of Diseases, 10th Revision, Australian Modification (ICD-10-AM) codes10 and the Australian Classification of Health Interventions (ACHI) procedure codes,11 entered by the coding department after the end of an inpatient episode of care.
The data extract was further refined by taking the following sequential steps: Step 1, removal of all operative data other than the first surgery within an episode of care; Step 2, removal of procedures performed under local anaesthesia or sedation without an upper or lower limb block; Step 3, removal of surgical specialties that were not included in the original risk score development studies or were not fully represented during the study period; Step 4, removal of procedures with missing SORT or NZRISK variables; Step 5, matching of procedures captured by the Copeland Risk Adjusted Barometer; Step 6, restriction to the first episode of surgical care within the 7-yr study period.
Calculating 30-day mortality risk
The SORT and NZRISK predicted risks of 30-day mortality were calculated by mapping the component variables for each eligible surgical episode from the data warehouse extract (Supplementary Table S1). Age, sex, and surgical urgency were obtained directly from fields in the Theatre Management System. Other variables required processing of coded data. Specifically, ACHI procedure codes were matched to a Johns Hopkins Pasternak Operative Severity Score (1–5)12 applying the classification table prepared and used by the NZRISK group.2 If multiple procedures were coded for the same operative episode, the procedure with the highest severity score was selected to represent the overall complexity. The ACHI procedure code block denoting the body or organ system targeted by that procedure was used to assign the surgical speciality type, again following the NZRISK methodology. The ACHI anaesthetic code provided the ASA physical status score, but if this was missing, the value entered in the Theatre Management System during the operation was used. Finally, ICD-10-AM codes recorded during care at Royal Perth Hospital in the 5 yr preceding an eligible surgical episode were examined, and if codes were present in the C01–C96 or D00–D09 range, then the patient was determined to have a positive cancer status.
After variable mapping, the open-access regression equations outlined below were used to summate the risk score and calculate the 30-day mortality prediction for each patient.
SORT predicted risk of 30-day mortality (R): ln(R∕(1−R))=–7.336+risk score.
NZRISK predicted risk of 30-day mortality (R): ln(R∕(1−R))=–10.625+risk score.
Outcome data
Routinely collected outcome data were extracted from the data and digital innovation warehouse. This included 30-day mortality, hospital-acquired complications (HACs), ICU bed hours, and length of stay. As per Australian government policy (www.safetyandquality.gov.au/our-work/indicators/hospital-acquired-complications), HACs are recorded if the required ICD-10-AM code(s) and the condition onset are linked to an episode of care. As the criteria to define a HAC evolves over time, the most recent iteration at the time of the analysis (Excel Groupers – Version 3.0) was applied across the whole data set. Days alive and out of hospital in the first 30 days (DAH-30), estimated at Royal Perth Hospital from public healthcare facility admission data in the metropolitan area, were also extracted.
Statistical analysis
Predictive performance was evaluated with receiver operator characteristic (ROC) curves and calibration plots, restricted to the first episode of care in the study period if multiple episodes took place (avoiding violation of the independent and identically distributed data assumption). The primary outcome was risk score discrimination of 30-day mortality as evaluated by area under ROC curve (AUROC) statistics. Calibration plots were constructed to examine predicted risks against observed risks. Non-parametric smoothed best-fit curves were added to calibration plots to aid visual evaluation. Calibration plots were re-scaled to reflect the risk range where predictions were relatively precise. To assess predictive performance year-on-year, select risk score(s) were further evaluated with calibration plots using annualised data.
Surgical population risk and 30-day mortality trends over time were assessed with a mixed-effects regression model, applied to all episodes of care within the 7-yr study period. Surgical outcomes in the highest and next-highest deciles of risk were also compared with lower risk patients, applying mixed-effects models with either logistic or negative binomial regression as appropriate.
Analyses were completed in Stata (StataCorp, 2019; Stata Statistical Software: Release 16; StataCorp LLC, College Station, TX, USA).
Results
The number of surgical episodes meeting eligibility criteria for the data extract was 70 846 (Fig 1). After the various exclusion criteria were applied, SORT and NZRISK predicted 30-day mortality were calculated for 44 031 patients, undergoing 53 395 distinct surgical episodes. Mortality according to patient and operative characteristics in the total population is summarised in Table 1.
Figure 1.
Flow diagram for data extraction and selection. GA, general anaesthesia; ACHI, Australian Classification of Health Interventions; SORT, Surgical Outcome Risk Tool.
Table 1.
Patient and operative characteristics in the total study population.
| Surgical episodes, n=53 395 | Deaths, n (%) | |
|---|---|---|
| Age (yr) | ||
| <65 | 36 210 | 97 (0.27) |
| 65–79 | 11 745 | 153 (1.30) |
| ≥80 | 5440 | 308 (5.66) |
| Sex | ||
| Male | 31 664 | 332 (1.05) |
| Female | 21 731 | 226 (1.04) |
| ASA physical status | ||
| 1 | 11 020 | 3 (0.03) |
| 2 | 24 196 | 25 (0.10) |
| 3 | 15 719 | 239 (1.52) |
| 4 | 2413 | 270 (11.19) |
| 5 | 47 | 21 (44.68) |
| Surgical severity | ||
| Minor/intermediate/major | 46 917 | 397 (0.85) |
| Xmajor/complex | 6478 | 161 (2.49) |
| Surgical urgency | ||
| Elective | 19 846 | 68 (0.34) |
| Expedited | 23 820 | 300 (1.26) |
| Urgent | 8993 | 156 (1.73) |
| Immediate | 736 | 34 (4.62) |
| Surgical type | ||
| Other | 12 396 | 33 (0.27) |
| Musculoskeletal | 23 905 | 317 (1.33) |
| Urological | 5086 | 32 (0.63) |
| Digestive | 10 166 | 137 (1.35) |
| Vascular | 1842 | 39 (2.12) |
| Cancer in past 5 yr | ||
| No | 44 986 | 451 (1.00) |
| Yes | 8409 | 107 (1.27) |
Matching to the Copeland Risk Adjusted Barometer resulted in 31 152 patients with all four risk scores available. In this population, the risk scores showed high discrimination (AUROC) for 30-day mortality (SORT=0.922, NZRISK=0.909, P-POSSUM=0.893; POSSUM=0.881) but consistently over-predicted risk (Fig 2). On visual inspection of calibration plots in the 0–10% range, SORT was marginally better calibrated than NZRISK and substantially better calibrated than P-POSSUM OR POSSUM. The proportion of patients with predicted 30-day mortality >10% was low for all the risk scores (SORT=2.87%, NZRISK=3.70%, P-POSSUM=4.98%, and POSSUM=5.98% respectively), resulting in widened confidence intervals (CIs) in the 10–100% range (Supplementary Fig. S1).
Figure 2.
Calibration plots for predictions of 30-day mortality between 0% and 10% using SORT (a), NZRISK (b), P-POSSUM (c), and POSSUM (d). Dashed line represents perfect calibration, blue line is non-parametric smoothed best-fit curve, green bars are 95% confidence intervals. SORT, Surgical Outcome Risk Tool; POSSUM, Physiology and Operative Severity Score for the enumeration of Mortality; P-POSSUM, Portsmouth variant of POSSUM. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
As SORT exhibited the best discrimination and overall calibration, and was also the most parsimonious risk score, further in-depth analyses focused on this measure only. Annualised evaluation of SORT revealed good visual calibration in the 0–10% risk range in the first 3 yr of the study period, followed by a steady calibration drift towards over-predicted risk (Fig 3). This drift reflected a significant decrease in annualised 30-day mortality over time (odds ratio [OR]=0.865 per elapsed year; 95% CI, 0.828–0.903; P<0.001) despite an increase in the predicted 30-day mortality (OR=0.073% increase per elapsed year; 95% CI, 0.057–0.089%; P<0.001) (see Fig 4).
Figure 3.
Annualised calibration plots for SORT predictions of 30-day mortality according to financial year end (July 1 to June 30). Dashed line represents perfect calibration, blue line is non-parametric smoothed best-fit curve, green bars are 95% confidence intervals. SORT, Surgical Outcome Risk Tool. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
Figure 4.
Annualised median (IQR) SORT risk and 30-day mortality. SORT, Surgical Outcome Risk Tool; IQR, inter-quartile range.
Thresholds to denote the highest and next-highest deciles of risk (SORT 30-day mortality predicted >3.92% and 1.52–3.92%) across the 7-yr period captured the majority of deaths (76% and 13%, respectively). Applying these thresholds to classify low-, medium-, and high-risk surgical patients resulted in a significant increase in the odds of 30-day mortality and major HACs in the medium and high risk categories, manifesting in prolonged length of hospitalisation and fewer days alive and out of hospital (Table 2). The likelihood of HACs indicative of a greater nursing burden also increased across the risk groups (Supplementary Table S2).
Table 2.
Mortality and major morbidity in low, medium and high SORT risk groups. All odds ratios and 95% confidence intervals (CIs) reflect a pairwise comparison with the low risk group and have a significance level of P<0.001. SORT, Surgical Outcome Risk Tool; IQR, inter-quartile range.
| Low risk SORT <1.52% n=42 705 | Medium risk SORT 1.52–3.92% n=5516 | High risk SORT >3.92% n=5174 | |
|---|---|---|---|
| Mean predicted 30-day mortality (%) | 0.394 | 2.46 | 9.84 |
| Median predicted 30-day mortality (%) | 0.266 | 2.38 | 6.63 |
| Observed 30-day mortality, n (%) | 59 (0.14) | 74 (1.34) | 425 (8.21) |
| Odds ratio (95% CI) | 11.1 (7.45–16.6) | 98.2 (54.7–176) | |
| Any hospital-acquired complication, n (%) | 1203 (2.82) | 694 (12.6) | 1110 (21.5) |
| Odds ratio (95% CI) | 5.20 (4.67–5.79) | 10.3 (9.23–11.5) | |
| Healthcare associated infection, n (%) | 635 (1.49) | 384 (6.96) | 519 (10.0) |
| Odds ratio (95% CI) | 5.20 (4.52–5.99) | 7.98 (6.96–9.16) | |
| Delirium, n (%) | 200 (0.47) | 156 (2.83) | 330 (6.38) |
| Odds ratio (95% CI) | 6.31 (5.07–7.85) | 15.3 (12.5–18.7) | |
| Cardiac complications, n (%) | 141 (0.33) | 120 (2.18) | 218 (4.21) |
| Odds ratio (95% CI) | 6.71 (5.25–8.58) | 13.3 (10.7–16.4) | |
| Respiratory complications, n (%) | 144 (0.34) | 112 (2.03) | 202 (3.90) |
| Odds ratio (95% CI) | 6.27 (4.84–8.12) | 12.6 (9.84–16.1) | |
| Venous thromboembolism, n (%) | 88 (0.21) | 36 (0.65) | 45 (0.87) |
| Odds ratio (95% CI) | 3.18 (2.16–4.69) | 4.25 (2.96–6.09) | |
| Gastrointestinal bleeding, n (%) | 27 (0.06) | 21 (0.38) | 45 (0.87) |
| Odds ratio (95% CI) | 6.04 (3.41–10.7) | 13.9 (8.59–22.4) | |
| Surgical complications requiring unplanned return to theatre, n (%) | 76 (0.18) | 36 (0.65) | 44 (0.85) |
| Odds ratio (95% CI) | 3.74 (2.48–5.64) | 4.98 (3.37–7.35) | |
| Renal failure | 4 (0.01) | 8 (0.15) | 18 (0.35) |
| 15.5 (4.67–51.4) | 37.2 (12.6–110) | ||
| ICU admission, n (%) | 1339 (3.14) | 751 (13.6) | 1023 (19.8) |
| Odds ratio (95% CI) | 5.46 (4.86–6.13) | 9.17 (8.11–10.4) | |
| Length of hospital stay in days, median (IQR) [range] | 2.0 (1.2–4.2) [0.1–283.6] | 5.1 (2.3–9.2) [0.2–300.0] | 6.6 (3.9–12.1) [0.1–211.1] |
| Coefficient, (95% CI) | 0.723 (0.693–0.754) | 1.01 (0.986–1.04) | |
| DAH-30, median (IQR) [range] | 29 (26–29) [0–30] | 25 (18–28) [0–30] | 20 (6–25) [0–30] |
| Coefficient, (95% CI) | –5.00 (–5.25 to –4.75) | –10.0 (–10.3 to –9.75) |
Discussion
In this large single-centre study, SORT, NZRISK, P-POSSUM, and POSSUM risk scores exhibited high levels of discrimination for 30-day mortality for patients undergoing surgery, but calibration displayed varying degrees of over-prediction. The SORT mortality risk demonstrated the best external validity in our population and proved an effective basis for a broad categorisation of patients into low, medium, and high risk, based on predicted risk deciles. These categories were associated with HACs, length of stay, and derived DAH-30, in addition to 30-day mortality.
Although it has long been recognised that the majority of postoperative complications and deaths occur in a minority of high-risk patients,13 there is a paucity of data in the literature on how thresholds for a specific risk score should be set to identify such patients. Indeed, as hospitals will vary in the number of high-risk surgical patients they encounter annually and the resources available to them, thresholds to designate high-risk can only be usefully set at an institution level in order to identify a manageable patient volume that can reliably receive enhanced care. For our hospital, this manifested in the use of the top two deciles of SORT 30-day mortality predictions to define medium and high-risk patients, and is an example of how the advent of large institution-level risk and outcome data facilitates this approach.
The calibration findings in our study are consistent with recent prospective studies that report suboptimal risk score calibration in surgical populations beyond the original development populations.2,3 It is only when risk equations are recalibrated or refined within a target population that subsequent risk scores meet the exacting calibration standards required to support shared decision-making on the basis of absolute risk. Despite the suboptimal calibration reported in this and other external validation studies, it can be argued that the levels of predictive performance observed are sufficient to support shared decision-making on the basis of relative risk, either by classifying patients into broad risk groups, or by interpreting absolute risk predictions alongside contemporaneous calibration plots. In this regard, given the low number of patients with predicted 30-day mortality above 10%, calibration plots to assist shared decision-making should incorporate confidence intervals at each level of predicted risk or be restricted to the 0–10% risk range.
In contrast to the calibration observed using the entire data set, an annual analysis of SORT demonstrated excellent calibration for the first 3 yr of the study period (July 1, 2014 to June 30, 2017). It is worth noting that the original version of SORT was derived from 2010 data in the UK and first published in 2014. After 2017, a steady calibration drift towards over-predicted risk was observed. This drift reflected a significant decrease in annualised 30-day mortality despite an increase in the surgical population risk. Similar trends should be evident in most high-performing hospitals, where the patients treated become older and more co-morbid with time, but perioperative processes improve. For example, at Royal Perth Hospital a number of initiatives were implemented and completed during the study period, including a ‘Safety Afterhours For Everyone’ (SAFE) team to enhance management of patients at risk of clinical deterioration. The observed calibration drift highlights the importance of incorporating periodic recalibration into systems aiming to maximally support shared decision-making.
The acquisition of institution-level risk and event rate data as in our study can guide strategic decisions with respect to the allocation of perioperative resources. For example, one of the most common clinical applications of surgical risk prediction is to decide who is admitted to an ICU bed on the day of surgery. This type of intervention is expensive (approximately A$5000 per bed-day14) and often captures only a small portion of the at-risk period before ward discharge.15 Even optimistically assuming a 25% relative risk reduction in patients experiencing at least one HAC by allocation of a postoperative ICU bed, implementing this intervention in all high-risk patients at Royal Perth Hospital where the current default is a standard postoperative ward bed, requires $3.7 million or $99 000 per complication prevented. Although formal evaluations would also include the cost savings from preventing HACs, it is clear that more cost-effective interventions that span the entire at-risk period and accommodate increased nursing requirements are needed. One such intervention introduced in 2021 in our hospital is the availability of enhanced postoperative ward beds that incorporate remote monitoring of vital signs and automated clinical deterioration alerts. Future studies are planned to evaluate the impact of this Healthcare in a Virtual Environment (HIVE) approach.
A limitation of this study is the retrospective calculation of the risk scores, derived in large part from hospital administrative data after discharge from the index hospital episode. In particular, we were only able to obtain a cancer diagnosis in the past 5 yr if the patient had been treated as an inpatient at Royal Perth Hospital and if this diagnosis had been coded. A more reliable indicator of this cancer field might in future be obtained via the statewide data linkage system. It is therefore possible that the predictive performance limitations we observed, in particular the calibration findings, reflect inaccuracies in the retrospective methodology and that prospectively acquired risk scores would perform better. However, in their recent prospective study, Wong and colleagues3 reported remarkably similar findings to the current work, showing that SORT outperformed other more complex risk scores for 30-day mortality including P-POSSUM, with all risk scores over-predicting risk. These similarities indicate that surgical risk depends consistently on a small number of objective variables that do not change perioperatively,16 and coding approaches to risk estimation that can detect such objective variables are likely valid. As most hospitals either currently collect such data, or will do so with increasing healthcare digitisation, we consider integrated national and institution-specific risk prediction systems highly feasible.
There are other limitations to our study. By opting to exclude patients of indigenous ethnicity (linked to high levels of social deprivation and chronic disease), we may have underestimated the true 30-day mortality in our hospital. Furthermore, there was limited distinction between the SORT and NZRISK scores and this was reflected in their very similar performance characteristics. Some of the retrospective outcome data collected in our study also has limitations. The nationally determined methodology to record HACs will underestimate morbidity relative to prospective audit.17 This is especially true where clinical note keeping is lacking or where complications that occur frequently after hospital discharge are assessed, such as surgical site infection.18 Finally, our measure of DAH-30 does not account for admissions in the private, non-metropolitan sectors, and likely over-estimates performance. Nevertheless, these methodological limitations are consistent for all our patients and thus provide useful insights into the outcome differences across surgical risk groups.
In conclusion, SORT, NZRISK, P-POSSUM, and POSSUM risk scores exhibited high levels of discrimination but suboptimal calibration for 30-day mortality at Royal Perth Hospital over a 7-yr period. SORT was the best performing surgical risk tool and effectively categorised patients into low (0–80th percentile), medium (80–90th percentile), and high (90–100th percentile) risk. Defining these risk category thresholds for efficient and reliable allocation of perioperative resources is the key advantage of locally developed surgical risk and outcome databases. Risk tools sufficiently calibrated for shared decision-making based on absolute risk may also be feasible but will likely require the development of region or institution specific risk models that incorporate periodic recalibration.
Funding
This work was supported by Royal Perth Hospital and the Department of Anaesthesia and Pain Medicine.
Authors' contributions
Study conception and design: AT.
Data extraction: FT, AT.
Data processing: FT, CYY, AT.
Statistical analysis: CYY, MP.
Data interpretation: JR, DW, TC, KH.
First draft of paper: AT.
Revision of paper: FT, CYY, JR, DW, TC, KH.
Declarations of interest
The authors declare that they have no conflict of interest.
Acknowledgements
Access to NZRISK surgical severity classification table: D. Campbell (Auckland City Hospital, Auckland); L. Boyle, University of Auckland, Auckland. Extracts from data and digital innovation warehouse: J Mamas (Royal Perth Hospital, Perth); G. North (Royal Perth Hospital, Perth). Development of DAH-30 at Royal Perth Hospital: D. Brooke (Royal Perth Hospital, Perth).
Handling editor: Phil Hopkins
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.bjao.2022.100018.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
Fig. s1.
Calibration plots for predictions of 30-day mortality between 0–100% using SORT (A), NZRISK (B), P-POSSUM (C) and POSSUM (D). Dashed line represents perfect calibration, blue line is non-parametric smoothed best-fit curve, green bars are 95% confidence intervals.
References
- 1.Moonesinghe S.R., Mythen M.G., Das P., Rowan K.M., Grocott M.P. Risk stratification tools for predicting morbidity and mortality in adult patients undergoing major surgery: qualitative systematic review. Anesthesiology. 2013;119:959–981. doi: 10.1097/ALN.0b013e3182a4e94d. [DOI] [PubMed] [Google Scholar]
- 2.Campbell D., Boyle L., Soakell-Ho M., et al. National risk prediction model for perioperative mortality in non-cardiac surgery. Br J Surg. 2019;106:1549–1557. doi: 10.1002/bjs.11232. [DOI] [PubMed] [Google Scholar]
- 3.Wong D.J.N., Harris S., Sahni A., et al. Developing and validating subjective and objective risk-assessment measures for predicting mortality after major surgery: an international prospective cohort study. PLoS Med. 2020;17 doi: 10.1371/journal.pmed.1003253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Balachandran V.P., Gonen M., Smith J.J., DeMatteo R.P. Nomograms in oncology: more than meets the eye. Lancet Oncol. 2015;16:e173–e180. doi: 10.1016/S1470-2045(14)71116-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Collins G.S., Reitsma J.B., Altman D.G., Moons K.G. Transparent reporting of a multivariable prediction model for individual Prognosis or diagnosis (TRIPOD) Ann Intern Med. 2015;162:735–736. doi: 10.7326/L15-5093-2. [DOI] [PubMed] [Google Scholar]
- 6.Protopapa K.L., Simpson J.C., Smith N.C., Moonesinghe S.R. Development and validation of the surgical outcome risk tool (SORT) Br J Surg. 2014;101:1774–1783. doi: 10.1002/bjs.9638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Reilly J.R., Gabbe B.J., Brown W.A., Hodgson C.L., Myles P.S. Systematic review of perioperative mortality risk prediction models for adults undergoing inpatient non-cardiac surgery. ANZ J Surg. 2021;91:860–870. doi: 10.1111/ans.16255. [DOI] [PubMed] [Google Scholar]
- 8.Copeland G.P., Jones D., Walters M. POSSUM: a scoring system for surgical audit. Br J Surg. 1991;78:355–360. doi: 10.1002/bjs.1800780327. [DOI] [PubMed] [Google Scholar]
- 9.Prytherch D.R., Whiteley M.S., Higgins B., Weaver P.C., Prout W.G., Powell S.J. POSSUM and Portsmouth POSSUM for predicting mortality. Physiological and operative severity score for the enUmeration of mortality and morbidity. Br J Surg. 1998;85:1217–1220. doi: 10.1046/j.1365-2168.1998.00840.x. [DOI] [PubMed] [Google Scholar]
- 10.Roberts R.F., Innes K.C., Walker S.M. Introducing ICD-10-AM in Australian hospitals. Med J Aust. 1998;169:S32–S35. doi: 10.5694/j.1326-5377.1998.tb123473.x. [DOI] [PubMed] [Google Scholar]
- 11.Best L. Australian classification of Health interventions — adapted for international use (ACHI-I) Health Inf Manag. 2003;31:14–17. doi: 10.1177/183335830303100106. [DOI] [PubMed] [Google Scholar]
- 12.Paternak L.R. Preanesthesia evaluation of the surgical patient. ASA Refresh Courses Anesthesiol. 1996;24:205–219. [Google Scholar]
- 13.Pearse R.M., Harrison D.A., James P., et al. Identification and characterisation of the high-risk surgical population in the United Kingdom. Crit Care. 2006;10:R81. doi: 10.1186/cc4928. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hicks P., Huckson S., Fenney E., Leggett I., Pilcher D., Litton E. The financial cost of intensive care in Australia: a multicentre registry study. Med J Aust. 2019;211:324–325. doi: 10.5694/mja2.50309. [DOI] [PubMed] [Google Scholar]
- 15.Sessler D.I., Meyhoff C.S., Zimmerman N.M., et al. Period-dependent associations between hypotension during and for four days after noncardiac surgery and a composite of myocardial infarction and death: a substudy of the POISE-2 trial. Anesthesiology. 2018;128:317–327. doi: 10.1097/ALN.0000000000001985. [DOI] [PubMed] [Google Scholar]
- 16.Gutierrez C.S., Passos S.C., Castro S.M.J., et al. Few and feasible preoperative variables can identify high-risk surgical patients: derivation and validation of the Ex-Care risk model. Br J Anaesth. 2021;126:525–532. doi: 10.1016/j.bja.2020.09.036. [DOI] [PubMed] [Google Scholar]
- 17.Koch C.G., Li L., Hixson E., Tang A., Phillips S., Henderson J.M. What are the real rates of postoperative complications: elucidating inconsistencies between administrative and clinical data sources. J Am Coll Surg. 2012;214:798–805. doi: 10.1016/j.jamcollsurg.2011.12.037. [DOI] [PubMed] [Google Scholar]
- 18.Corcoran T.B., Myles P.S., Forbes A.B., et al. Dexamethasone and surgical-site infection. N Engl J Med. 2021;384:1731–1741. doi: 10.1056/NEJMoa2028982. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.