Comparative Evaluation of Mortality Risk Prediction Tools After Emergency Laparotomy

ABSTRACT

Background

Accurate pre‐operative prediction of mortality after emergency laparotomy (EL) is essential as it may facilitate shared decision‐making. The aim of our study was to compare the prognostic accuracy of three risk prediction tools following EL.

Methods

This retrospective cohort study included all consecutive adult patients (≥ 18 years) who underwent EL for any non‐traumatic indication between December 2020 and October 2024 at Wellington Hospital (Aotearoa New Zealand). Demographic, physiological, comorbidity, and imaging data were extracted to enable risk calculation. Risk of 30‐day mortality was determined using the National Emergency Laparotomy Audit prognostic model (P‐NELA), American College of Surgeons National Surgical Quality Improvement Programme (ACS‐NSQIP), and the New Zealand Surgical Risk (NZRISK) tool. Calibration was assessed using McFadden's pseudo‐R ² (R ² _McF ≥ 0.40, excellent; ≥ 0.30, good; ≥ 0.20, moderate; < 0.20, poor) and accuracy was evaluated with the c‐statistic (< 1.0, excellent; < 0.90, good; < 0.80, fair; < 0.70, poor).

Results

A total of 312 EL procedures were performed, with observed 30‐day mortality of 7.7%. P‐NELA was most accurate in predicting 30‐day mortality after EL (c = 0.91), followed closely by the ACS‐NSQIP tool (c = 0.88; p = 0.43). NZRISK was the least accurate (c = 0.75) relative to both P‐NELA (p = 0.0005) and ACS‐NSQIP (p = 0.0008). Compared with ACS‐NSQIP (R ² _McF = 0.25) and NZRISK (R ² _McF = 0.062), P‐NELA was the only well‐calibrated risk prediction tool (R ² _McF = 0.38).

Conclusions

P‐NELA was most accurate in predicting 30‐day mortality after EL, while NZRISK performed least reliably in our cohort. These findings highlight the importance of providing external validation for different risk prediction tools within diverse populations to improve peri‐operative planning.

1. Introduction

Emergency laparotomy (EL) encompasses a group of high‐risk surgical procedures, with 30‐day mortality in excess of 20% in some earlier studies [1]. These surgeries are often performed for a range of indications and on critically unwell patients with complex conditions, which leads to challenges in counselling patients and planning peri‐operative care [2]. Accurate pre‐operative prediction of prognosis after EL could help facilitate shared decision‐making with patients and their families, as well as optimise allocation of resources in the post‐operative setting [2].

The National Emergency Laparotomy Audit (NELA) in the United Kingdom (UK) is the most comprehensive audit and quality improvement initiative for EL [3]. A risk prediction tool was subsequently developed from the NELA database, and has been shown to most consistently predict mortality within 30 days after EL, both in the Aotearoa New Zealand (AoNZ) and Australian context [3, 4]. More recently, the NELA calculator has also been externally validated for mortality prediction up to after 5 years after EL [5]. However, the NELA tool relied upon a significant number of variables that were inaccurate or difficult to obtain. Other predictors required the estimation of anticipated intra‐operative findings, such as the degree of peritoneal soiling, which is not available at the time of pre‐operative decision‐making [4, 6]. For this reason, the NELA tool has recently been revised to include eight fewer redundant variables (13 pre‐operative variables from a previous total of 21 pre and intra‐operative factors) and has also demonstrated improved accuracy [7].

This revised risk calculator, termed the NELA prognostic model (P‐NELA), is yet to be validated in geographically and ethnically diverse patient populations such as in AoNZ [7]. In addition, the American College of Surgeons National Surgical Quality Improvement Programme (ACS‐NSQIP) is still widely used despite previous studies highlighting its comparatively poor calibration for 30‐day mortality [4]. Furthermore, the NZRISK score was developed specifically for use in an AoNZ context for predicting mortality after elective surgery, but its utility has not been validated for EL [8].

The aim of this study was therefore to provide external validation for the P‐NELA risk score within our AoNZ cohort, as well as determine its comparative prognostic utility for 30‐day mortality prediction after EL against the ACS‐NSQIP and NZRISK tools.

2. Material and Methods

Institutional approval for the conduct of this study was granted by the University of Otago (Wellington, AoNZ) ethics committee (H24/0325). Study methodology and reporting complied with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines (Appendix S1) [9].

2.1. Study Setting

This research was conducted at Wellington Regional Hospital (Te Whatu Ora—Capital, Coast and Hutt Valley, Wellington, AoNZ), which is a large tertiary healthcare and teaching institution serving a catchment population in excess of 300 000 patients.

2.2. Study Design and Population

This retrospective cohort study included all consecutive adult patients (≥ 18 years) who underwent EL between December 2020 and October 2024. Patients who underwent emergency gastrointestinal surgery during an acute admission were included, mirroring the eligibility criteria of the UK NELA and the Australian and New Zealand Emergency Laparotomy Audit—Quality Improvement (ANZELA‐QI) [6, 10]. In addition, patients undergoing staged re‐operation following their initial EL, or unplanned re‐operation due to complications from elective surgery were also considered for inclusion. Patients undergoing appendicectomy, cholecystectomy, obstetric or gynaecological surgery, and major vascular surgeries were excluded, as were damage‐control laparotomy for trauma. Selection criteria were applied through verification of operative notes for each identified patient by a senior author.

2.3. Data Extraction

Electronic hospital databases were interrogated for the purposes of data extraction, which was undertaken using a Microsoft Excel (Microsoft Corporation, Redmond, Washington, USA) spreadsheet, with cross‐checking and validation of 10% of entries by a senior author. Variables collected included baseline patient demographics, comorbidities, physiological status, urgency of surgical intervention, indication, specific EL procedure performed, anticipated degree of peritoneal soiling, type of surgery, as well as blood tests and imaging results (refer to Appendix S2 for a comprehensive list of extracted variables). Although not included within the P‐NELA risk calculation, demographic variables such as body mass index (BMI) were ascertained from peri‐operative anaesthetic records at the time of EL for the purposes of risk calculation using the ACS‐NSQIP tool. Data were inputted into online risk prediction calculators for P‐NELA (https://data.nela.org.uk/riskcalculator/), ACS‐NSQIP (https://riskcalculator.facs.org/RiskCalculator), and NZRISK (https://nzrisk.com/#calculate), respectively.

2.4. Outcome Measure

The primary outcome was all‐cause mortality measured at 30‐days after EL. The AoNZ Mortality cOllection RegisTer (MORT) were used to accurately verify the date of death [11]. These mortality data are accessed from hospital clinical databases, and are linked to a patient's unique National Health Index (NHI) identification number [11].

2.5. Statistical Analysis

All statistical analyses were conducted using RStudio (version 4.5.2; Posit PBC, Boston, MA). Continuous variables were summarised with median and interquartile range (IQR) values, while categorical variables were summarised as frequencies (n) and percentage (%). Accuracy of each risk prediction tool were evaluated with the c‐statistic, which represents the area under the receiver‐operator characteristic (AU‐ROC) curve [12, 13]. Interpretation of the c‐statistic were as follows: 1.0 as a perfect test, 0.90–0.99 as an excellent test, 0.80–0.89 as a good test, 0.70–0.79 as a fair test, and 0.51–0.69 as a poor test [13, 14]. The performance of each tool were compared using the methods of Delong et al., with probability (p) values of < 0.05 considered to indicate a statistically significant difference [15]. Calibration is a measure of the degree of agreement between predicted and observed mortality, and was assessed using McFadden's pseudo‐R ² (R ² _McF) [16]. Excellent calibration indicates accurate alignment, whereas poor calibration may underestimate mortality risk. The following thresholds were used: R ² _McF ≥ 0.40, excellent; ≥ 0.30, good; ≥ 0.20, moderate and < 0.20, poor calibration.

3. Results

3.1. Patient Selection

A total of 312 EL procedures were performed on 291 patients. Sixteen patients (5.5%) underwent more than one acute operation for separate indications during the 4 year study interval. EL were most frequently indicated for small bowel obstruction (n = 121, 38.8%), perforation (n = 53, 17.0%), and large bowel obstruction (n = 31, 9.9%). Small bowel resection or repair with primary anastomosis (n = 97, 31.8%) was the most common operation, followed by large bowel resection (n = 80, 25.6%) and adhesiolysis (n = 61, 19.6%). Two patients underwent staged re‐look laparotomy following their initial EL, which included large bowel resection for ischaemia (n = 1) and abdominal washout, temporary ileostomy formation and closure for large bowel perforation (n = 1). One additional patient underwent re‐operation for bleeding after an elective infra‐renal endovascular abdominal aortic aneurysm repair. Indications for EL are outlined in Appendix S3, and the different procedures performed are denoted in Figure 1.

FIGURE 1.

Different emergency laparotomy operations performed for the cohort of included patients (*Includes drainage of intra‐abdominal abscess (n = 2), splenectomy (n = 2), debridement and lavage (n = 2), gastric surgery (n = 1), bladder repair (n = 1), and haemostasis (n = 1)).

3.2. Demographic Characteristics

Baseline demographic characteristics with respect to the included cohort of patients are described in Table 1. The median age of our EL cohort was 66 years (IQR 54–77), with an equal sex distribution (n = 157 female patients, 50.3%). Most patients identified as NZ European (72.1%).

TABLE 1.

Baseline demographic characteristics for the included cohort of patients.

Characteristic	Value
Age at time of EL in years, median (IQR)	66 (54–77)
Sex, n (%)
Female	157 (50.3)
Male	155 (49.7)
Ethnicity, n (%)
NZ European	225 (72.1)
Māori	44 (14.1)
Pacific Islander	16 (5.1)
Asian	24 (7.7)
Other	3 (1.0)
ASA classification, median (IQR)	3 (2–3)
1 (No systemic disease), n (%)	15 (4.8)
2 (Mild systemic disease), n (%)	97 (31.1)
3 (Non‐life‐threatening systemic disease), n (%)	147 (47.1)
4 (Life‐threatening systemic disease), n (%)	51 (16.3)
5 (Moribund patient), n (%)	2 (0.6)
Admission type, n (%)
Acute	304 (97.4)
Elective (arranged admission)	8 (2.6)
Admitting service, n (%)
General surgery	245 (78.5)
Other	67 (21.5)
Operative time in minutes, median (range)	111 (18–497)
Post‐operative LOS in days, median (range)	10 (1–242)
30‐day mortality, n (%)	24 (7.7)

Abbreviations: ASA, American Society of Anaesthesiologists; EL, emergency laparotomy; IQR, interquartile range; LOS, length of stay; n, number; NZ, New Zealand.

3.3. Post‐Operative Outcomes

Median post‐operative length of hospital stay was 10 days (range 1–242), and the observed 30‐day mortality after EL was 7.7% (n = 24). The median P‐NELA score was 4.2% (IQR 1.6%–10.6%; Table 2).

TABLE 2.

Median risk scores for 30‐day mortality prediction.

Risk prediction tool	Median (IQR), %
P‐NELA	4.2 (1.6–10.6)
ACS‐NSQIP	2.3 (0.6–8.1)
NZRISK	7.5 (3.3–14.3)

Abbreviations: ACS‐NSQIP, American College of Surgeons National Surgical Quality Improvement Programme; P‐NELA, National Emergency Laparotomy Audit prognostic model.

3.3.1. Diagnostic Accuracy

Diagnostic accuracy for 30‐day mortality prediction after EL with respect to each of the three risk prediction tools are pictorially depicted via ROC curves (Figure 2). P‐NELA demonstrated excellent accuracy in predicting 30‐day mortality after EL, with a c‐statistic of 0.91. This was followed closely by the ACS‐NSQIP tool (c = 0.88), although there was no significant difference in terms of accuracy of the two risk scores (p = 0.43). NZRISK was the least accurate tool in the context of EL (c = 0.75) relative to both P‐NELA (p = 0.0005) and ACS‐NSQIP (p = 0.0008).

FIGURE 2.

Receiver‐operator characteristic (ROC) curve demonstrating the diagnostic accuracy among the three examined risk prediction tools for 30‐day post‐operative mortality after emergency laparotomy.

3.3.2. Calibration

P‐NELA was the only well‐calibrated risk prediction tool (R ² _McF = 0.38; Table 3), which meant that the predicted mortality risk closely matched the actual observed mortality in our cohort of patients. Comparatively, ACS‐NSQIP showed moderate calibration (R ² _McF = 0.25), whereas the NZRISK was poorly calibrated (R ² _McF = 0.062).

TABLE 3.

Calibration of each risk prediction tool for mortality after emergency laparotomy.

Risk prediction tool	McFadden's pseudo‐R 2 (R 2 MF) a
P‐NELA	0.38
ACS‐NSQIP	0.25
NZRISK	0.062

Abbreviations: ACS‐NSQIP, American College of Surgeons National Surgical Quality Improvement Programme; P‐NELA, National Emergency Laparotomy Audit prognostic model.

^a The following thresholds were used: R ² _McF ≥ 0.40, excellent; ≥ 0.30, good; ≥ 0.20, moderate and < 0.20, poor calibration.

4. Discussion

This study provided external validation for the new NELA prognostic model (P‐NELA), whilst performing direct comparisons with two other risk prediction tools in the context of adult patients undergoing EL. P‐NELA most accurately predicted 30‐day mortality after EL, and was the only risk prediction tool to have good calibration. While the accuracy of ACS‐NSQIP did not differ significantly from that of P‐NELA, it was only moderately well‐calibrated for short‐term (30‐day) mortality prediction. In comparison, NZRISK was the least reliable in our cohort and demonstrated poor calibration.

The revised NELA prognostic model (P‐NELA) demonstrated strong predictive accuracy for 30‐day mortality in our external cohort of patients from AoNZ undergoing EL. This is likely a reflection of its purpose‐built design, wherein model development and internal validation occurred specifically in patients undergoing EL, and was constructed solely for risk estimation and to guide escalation of care for acutely unwell patients in the pre‐operative setting [7]. In contrast, the ACS‐NSQIP was designed to risk‐adjust hospital outcomes for benchmarking purposes and thus contains a cumbersome number of variables which may not be practical to measure in the pre‐operative context. Additionally, both the ACS‐NSQIP and NZRISK were developed through the inclusion of an elective case‐mix, which is illustrated by their comparatively poor performance in the EL context [8, 17]. Inclusion of patients undergoing planned procedures during model development also does not acknowledge the high‐risk profile of EL, and limits the applicability of the tools in aiding with pre‐operative counselling of patients. Our findings provide the only external validation of the revised P‐NELA outside of Denmark populations, and parallel their results, placing further emphasis on the need to evaluate the utility of different risk prediction tools in geographically diverse populations and settings [18, 19]. Knowledge of the most accurate risk estimation tool would assist clinicians with appropriately allocating post‐operative resources, while also facilitating shared decision‐making with patients and their families, particularly at a vulnerable time such as prior to EL, when providing accurate estimation of risk is paramount.

The original NELA model included 21 pre‐ and intra‐operative variables for predicting mortality risk after EL [6]. This has made it cumbersome to use in the pre‐operative setting, whilst also requiring a clinicians crude estimate for intra‐operative data such as the anticipated degree of peritoneal soiling [20]. We have demonstrated that the revised P‐NELA model leads to an improved accuracy, whilst maintaining good calibration for short‐term (30‐day) mortality prediction after EL despite comprising of only 13 variables [7]. In addition, the initial performance of the P‐NELA model in the UK were developed and internally validated across a range of indications for EL including obstruction, sepsis, ischaemia, and haemorrhage, among others [7]. This recognises the dynamic nature of EL and the range of contexts in which it may be required. Resultantly, these factors make P‐NELA an attractive tool for clinicians due to its comparatively easier bedside application, and generalisability in a diverse range of acute situations.

Interestingly, we found the NZRISK tool to have the worst performance for short‐term mortality after EL in our AoNZ cohort. This is despite its development within a large population of patients (> 360 000) nationally, and after accounting for the diverse demographic and ethnicity distribution within the AoNZ population [8]. Moreover, NZRISK were constructed using a national administrative database of patients undergoing both elective and emergency surgery, and thus was not designed specifically for populations undergoing EL. Additionally, such databases are too granular with limited collected administrative data, which meant that physiological variables or blood test results could not be incorporated within the risk prediction model [8]. This highlights the methodological flaws in devising a risk prediction model in this way, as these factors are fundamental to capturing the highly dynamic nature of EL.

There is potential to further refine and improve current risk prediction models for EL through the inclusion of frailty and nutritional status, as well as recalibration to predict longer‐term mortality and patient‐reported outcome measures (PROMs) such as quality‐of‐life [4, 20, 21, 22]. Increased short‐term morbidity and mortality have been demonstrated among frail patients undergoing EL, independent of age, suggesting that frailty is a progressive illness which ultimately results in an inability to cope with the physiological insult of an EL [21, 22, 23]. However, the development of a novel risk prediction tool which includes these factors is precluded by the current lack of standardized measurement tools for frailty and nutrition [4, 22, 24, 25, 26].

This study presents some limitations. The single‐centre study design limits the generalizability of our results. However, our patient selection criteria largely replicated that of the NELA in the UK [6], as well as providing validation for currently utilised risk prediction tools in the EL setting within an external cohort of patients. Furthermore, Wellington Regional Hospital is a large tertiary healthcare and academic teaching institution which services an ethnically and geographically diverse population of ~300 000 patients. We did not provide comparative analyses between the myriad of other risk prediction models which have been described in the EL setting [24, 25]. Previous efforts have already demonstrated superiority of the original NELA risk prediction score [4, 5], and the strength of our present study methodology stems from utilization of the recently developed NZRISK and revised P‐NELA scores [6, 7]. The accuracy of our risk estimation may have been potentially limited by missing data resulting from the retrospective study design. However, patient demographic and comorbidity variables were ascertained from peri‐operative anaesthetic records at time of EL and thus were subject to limitations in the accuracy of documentation, with missing variables assumed to indicate their absence for the purposes of risk calculation. Lastly, whilst our sample size was substantially smaller relative to the populations among which the NELA and revised P‐NELA risk prediction tools were developed, it is comparable to other recent external validation studies [3, 18, 19], supporting the robustness of our study findings.

5. Conclusion

The NELA prognostic model (P‐NELA) most accurately predicted 30‐day mortality after EL compared with ACS‐NSQIP and NZRISK in our cohort of patients, and was the only tool to demonstrate good calibration. These findings highlight the importance of providing external validation for different risk prediction tools within geographically diverse populations to facilitate improved planning for peri‐operative care. Further research into the development of a built‐for‐purpose risk tool utilizing only pre‐operative variables, which may predict longer‐term mortality as well as functional outcomes, is required.

Author Contributions

Sameer Bhat: conceptualization, investigation, writing – original draft, methodology, software, formal analysis, visualization. Campbell Hodges: conceptualization, investigation, writing – review and editing, methodology, validation, data curation. Shauna O'Brien: investigation, writing – review and editing, data curation. Wiremu MacFater: conceptualization, investigation, methodology, writing – review and editing, project administration. Anthony Y. Lin: project administration, writing – review and editing, methodology, investigation, conceptualization, supervision. Ahmed W. H. Barazanchi: supervision, project administration, formal analysis, data curation, visualization, methodology, conceptualization, investigation, funding acquisition, writing – review and editing.

Funding

This work was supported by the Surgical Research Trust (Dunedin School of Medicine, University of Otago, Dunedin, Aotearoa New Zealand).

Consent

Informed consent was not required due to the retrospective design of our study.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Appendix S1: Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Checklist.

Appendix S2: Comprehensive list of extracted variables.

Appendix S3: Indications for emergency laparotomy.

Bhat S., Hodges C., O'Brien S., MacFater W., Lin A. Y., and Barazanchi A. W. H., “Comparative Evaluation of Mortality Risk Prediction Tools After Emergency Laparotomy,” ANZ Journal of Surgery 96, no. 6 (2026): 1768–1774, 10.1111/ans.70730.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.