The National Clinical Database Risk Calculator and the 5‐Item Modified Frailty Index Predict the Development of Postoperative Delirium After Surgery for Hepatocellular Carcinoma

ABSTRACT

Aim

Postoperative delirium in elderly patients is a critical clinical issue because of its adverse effect on recovery and prognosis. The National Clinical Database (NCD) Risk Calculator, a clinical tool developed in Japan, is widely used to predict various postoperative complications, including delirium. In this study, we aimed to develop a more accurate delirium prediction model by integrating the NCD Risk Calculator with additional clinical indicators.

Methods

This study included 106 patients who underwent segmentectomy or more extensive procedures for hepatocellular carcinoma between 2013 and 2023. The training cohort comprised patients treated between 2013 and 2017, and those treated afterward formed the validation cohort. In the training cohort, we assessed the NCD Risk Calculator and additional indices for the favorable‐risk group, and validated the best‐performing index in combination with the NCD Risk Calculator.

Results

Using the median predicted risk (6.4%) as a threshold, the adverse‐risk group had a significantly higher incidence of delirium than the favorable‐risk group (p < 0.01). Among several indices, the 5‐Item Modified Frailty Index (mFI‐5) showed the strongest association with delirium (AUC 0.76, p < 0.01). Patients classified as high‐risk by both the NCD Risk Calculator and mFI‐5 had significantly higher delirium incidence in the training (p < 0.01) and validation (p = 0.05) cohorts. Multivariate analysis identified high‐risk status as an independent predictor of postoperative delirium (p < 0.01).

Conclusion

The combination of the NCD Risk Calculator and mFI‐5 enhances predictive accuracy and serves as a reliable tool for assessing delirium risk in older patients.

This study demonstrates that combining the NCD Risk Calculator with the mFI‐5 enables more precise stratification of postoperative delirium risk in patients undergoing surgery for HCC. Patients classified as high risk by the combined model showed a markedly higher incidence of delirium than those in the intermediate‐ and low‐risk groups. This integrated approach improves risk prediction and supports more tailored perioperative management.

Abbreviations

AUC

area under the curve

CI

confidence interval

HCC

hepatocellular carcinoma

IBPSs

inflammation‐based prognostic scores

LR⁺

likelihood ratio

mFI‐5

the 5‐Item Modified Frailty Index

NCD

National Clinical Database

OR

odds ratio

POD

postoperative delirium

ROC

receiver operating characteristic

1. Introduction

Delirium is an acute, reversible disturbance of cognition [1] and is a common complication following gastrointestinal surgery [2]. Emerging evidence indicates that postoperative delirium (POD) may negatively impact long‐term outcomes, highlighting the importance of accurate prediction and effective prevention in older patients [3, 4, 5]. Several studies have sought to identify preoperative risk factors and to develop predictive models for POD [6, 7, 8, 9].

In Japan, the National Clinical Database (NCD) has been implemented since 2011 and now captures data from over 95% of all surgical cases. Owing to this comprehensive coverage, NCD data have been employed in various recent studies [10, 11]. Based on these data, the NCD Risk Calculator was developed to estimate surgical risks for specific procedures and is now widely used in clinical practice in Japan. Recently, this calculator was further expanded to include a feature for predicting the onset of POD, providing a broader range of clinical applications.

In this study, we aimed to identify additional indices that could complement the NCD Risk Calculator and enhance its predictive performance for POD in patients with hepatocellular carcinoma (HCC).

2. Methods

2.1. Study Design and Patient Population

This study included 106 patients who underwent Couinaud's mono‐segmentectomy or more extensive hepatic resection for HCC at the Division of Gastroenterological, Hepato‐Biliary‐Pancreatic, Transplantation and Pediatric Surgery, Department of Surgery, Shinshu University Hospital, between January 2013 and December 2023. Patients treated between 2013 and 2017 were allocated to the training cohort, while those treated thereafter were allocated to the validation cohort (Figure 1).

FIGURE 1.

Patient selection criteria flowchart.

Delirium was retrospectively diagnosed from medical records in accordance with the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition criteria, based on diagnoses made by the treating physicians, while formal CAM‐based screening was not consistently performed. Nutritional markers, inflammation‐based prognostic scores, and frailty indicators were assessed for associations with POD using receiver operating characteristic (ROC) curve analysis. In the training cohort, patients were stratified into the adverse‐ and the favorable‐risk groups based on the median predicted risk of POD, serving as an initial cut‐off for balanced stratification in this exploratory study, given that no clinically validated threshold is currently available. The most predictive index in the favorable‐risk group was identified, and a novel index combining the NCD Risk Calculator with this index was subsequently evaluated and validated. The risk factors for POD were also examined.

The Ethics Committee of Shinshu University School of Medicine approved this study (approval number: 5244). The study was conducted in line with the Declaration of Helsinki and the institutional protocol. Informed consent was obtained using an opt‐out approach, whereby individuals who refused participation were excluded.

2.2. Endpoint Assessment and Data Collection

Patient demographics and operative characteristics were compared between the adverse‐ and the favorable‐risk groups. The diagnostic performance of various indices for predicting POD in the favorable‐risk group was assessed using the area under the curve (AUC), to identify the index with the highest diagnostic accuracy. A new composite index was constructed by combining the NCD Risk Calculator with the most predictive index identified in the favorable‐risk group, and its clinical utility was evaluated in the training cohort and subsequently validated in the validation cohort. Univariate and multivariate analyses were performed to identify risk factors associated with POD.

The medical records of the enrolled patients were retrospectively reviewed to obtain background information, operative details, and surgical outcomes.

2.3. Perioperative Management and Follow‐Up

The indications for surgery and the extent of liver resection were determined based on Makuuchi's criteria [12], indocyanine green clearance of the remnant liver [13], and the remnant hepatocellular uptake index [14], all of which were derived from laboratory and imaging findings. Perioperative management and postoperative monitoring were performed as outlined in our earlier reports [15].

2.4. Statistical Analysis

The predictive performance of the various scores was assessed using ROC curve analysis. The AUC and estimates of sensitivity and specificity at different cut‐off values were derived from the ROC curves. The optimal threshold for each score was defined using the Youden index. Internal validation was conducted using bootstrap resampling (1000 iterations) in the training cohort to estimate optimism‐corrected model discrimination. Model calibration was assessed using calibration plots and further quantified by calculating the calibration intercept and slope. Quantitative variables were compared using the Wilcoxon rank‐sum test, and categorical variables were analyzed using the χ ² test or Fisher's exact test, as appropriate. Statistical significance was set at p < 0.05. Univariate and multivariate logistic regression analyses were performed to determine independent risk factors associated with the development of POD. For each variable, odds ratios (ORs) and their corresponding 95% confidence intervals (CIs) were calculated to quantify the strength and precision of the associations. All statistical analyses were performed using JMP software, version 13.2.1 (SAS Institute Inc., Cary, NC, USA) and R v. 4.5.2 (R Core Team, Vienna, Austria).

3. Results

3.1. Incidence of POD and Risk Stratification by NCD Risk Calculator in the Training Cohort

POD was observed in 20 out of 54 patients (37.0%) in the training cohort. The NCD Risk Calculator predicted a significantly higher incidence of POD in patients who developed POD (median: 9.2%) than in those who did not (median: 5.6%) (p < 0.01).

Then, the median predicted incidence of POD (6.4%) was used as a cut‐off to stratify patients into the adverse‐ and the favorable‐risk groups (Figure 1). The adverse‐risk group had a significantly higher incidence of POD than the favorable‐risk group (53.6% vs. 19.2%, p < 0.01). Table 1 summarizes the characteristics and surgical outcomes of patients in the adverse‐ and the favorable‐risk groups, revealing that patients in the adverse‐risk group were older and had a higher incidence of diabetes mellitus and POD. No other significant differences were observed between the two groups.

TABLE 1.

Patients' background data, operative characteristics, and surgical outcomes.

Variables	Adeverse‐risk, n = 28		Favorable‐risk, n = 26		p
Age*, year	77	(65–85)	70	(65–74)	< 0.001**
Sex
Male	18	(64.3)	16	(61.5)	0.834
Female	10	(35.7)	10	(38.5)
Body mass index*, kg/m2	22.3	(17.1–29.9)	23.0	(17.1–28.3)	0.580
Underlying liver disease
Hepatitis B virus	10	(35.7)	14	(53.9)	0.179
Hepatitis C virus	11	(39.3)	8	(30.8)	0.512
Alcohol	1	(3.6)	2	(7.7)	0.506
MASH/MASLD	0	(0)	2	(7.7)	0.227
Diabetes mellitus	14	(50.0)	6	(23.1)	0.039**
Total bilirubin*, mg/dL	0.72	(0.36–1.51)	0.83	(0.37–1.67)	0.411
Indocyanine green retention rate at 15 min*, %	9.2	(2.7–17.7)	9.7	(4.8–26.6)	0.354
Operative procedure
Hemihepatectomy	10	(35.7)	7	(26.9)	0.077
Sectionectomy	12	(42.9)	6	(23.1)
Segmentectomy	6	(21.4)	13	(50.0)
Operation time*, min	335.5	(129–570)	392.5	(121–689)	0.146
Blood loss*, mL	450	(0–1530)	390	(50–1300)	0.890
Blood transfusion	6	(21.4)	4	(15.4)	0.567
Mortality	0	(0)	0	(0)	1.00
Morbidity
All grade	21	(75.0)	17	(65.4)	0.439
Clavien‐Dindo classification grade ≥ IIIa	7	(25.0)	5	(19.2)	0.610
Delirium	15	(53.6)	5	(19.2)	0.008**
Postoperative hospital stay*, day	13	(7–49)	12	(9–39)	0.814

Note: Figures in parentheses are percentage unless otherwise specified.

Abbreviations: MASH, metabolic dysfunction‐associated steatohepatitis; MASLD, metabolic dysfunction‐associated steatotic liver disease.

^*Median (range).

^** p < 0.05.

3.2. ROC Curve Analysis in the Favorable‐Risk Group and Development of a Three‐Tier Risk Model

The predictive performance of various markers for POD was assessed using ROC curve analyses in the favorable‐risk group, and the results are presented in Figure 2 and Table 2. Among these indices, the 5‐Item Modified Frailty Index (mFI‐5) showed the strongest association with POD, with an AUC of 0.76 at a cut‐off value of 2 (p < 0.01). Based on the NCD Risk Calculator–predicted POD risk and the mFI‐5 cut‐off (> 2), patients were stratified into three groups: a double‐positive or high‐risk group meeting both criteria (NCD adverse‐risk and mFI‐5 > 2), a single‐positive or intermediate‐risk group meeting exactly one criterion (NCD adverse risk or mFI‐5 > 2, but not both), and a double‐negative or low‐risk group meeting neither criterion. The high‐risk group had the highest incidence of POD (83.3%, n = 5), followed by the intermediate‐risk group (47.8%, n = 11) and the low‐risk group (16.0%, n = 4). Similar findings were observed in the validation cohort, where the incidences were 50.0% (n = 2), 39.4% (n = 13), and 13.3% (n = 2), respectively (Figure 3).

FIGURE 2.

Receiver operating characteristic curve analysis of various markers for postoperative delirium after liver resection for hepatocellular carcinoma. The area under the curve (AUC) values of inflammation‐based markers in the favorable‐risk group. The 5‐Item Modified Frailty Index showed the strongest association with postoperative delirium onset, achieving an AUC of 0.76 at a cut‐off value of 2.

TABLE 2.

Receiver operating characteristic curve analysis for postoperative delirium after liver resection for hepatocellular carcinoma.

	AUC	95% CI	Odds ratio	95% CI	p
CRP‐to‐albumin ratio	0.71	0.56–0.86	398.29	5.04–249029.69	0.003*
Lymphocyte‐to‐monocyte ratio	0.50	0.33–0.67	1.97	0.06–61.07	0.679
Neutrophil‐to‐lymphocyte ratio	0.58	0.41–0.74	7.08	0.26–315.89	0.248
Platelet‐to‐lymphocyte ratio	0.55	0.37–0.72	4.11	0.37–52.05	0.247
Prognostic nutritional index	0.58	0.41–0.75	0.29	0.02–3.34	0.320
Glasgow prognostic score	0.59	0.48–0.71	10.25	0.85–181.14	0.068
Controlling nutrition status	0.50	0.34–0.66	1.42	0.07–8.05	0.862
mFI‐5	0.76	0.64–0.88	42.37	4.92–584.35	0.001

Abbreviations: AUC, area under curve; CI, confidence interval; CRP, C‐reactive protein; mFI‐5, the 5‐Item Modified Frailty Index.

^* p < 0.05.

FIGURE 3.

Incidence of postoperative delirium by risk group. In the training cohort, the high‐risk group showed the highest incidence (83.3%), followed by the intermediate‐risk group (47.8%) and the low‐risk group (16.0%) (A). In the validation cohort, the high‐risk group showed the highest incidence (50.0%), followed by the intermediate‐risk group (39.4%) and the low‐risk group (13.3%) (B).

3.3. Assessment of the Predicted Value of the New Risk Model

Subsequently, the predictive value of the new risk model for POD was evaluated by comparing the low‐risk group with the combined high‐risk categories (high‐ and intermediate‐risk groups). Patients in the high‐risk categories exhibited a significantly higher incidence of POD in the training (p < 0.01) and validation (p = 0.05) cohort. The new risk model showed a higher positive likelihood ratio (LR⁺) (3.09) compared with the NCD risk calculator alone (2.47) or the mFI‐5 alone (1.39), indicating superior ability to identify patients truly at risk of POD when the marker was positive in the training cohort. This advantage was reproduced in the validation cohort, where the corresponding LR⁺ values were 3.15, 1.26, and 1.34 for the new model, NCD risk calculator, and mFI‐5, respectively.

In addition, the apparent AUC of the combined model in the training cohort was 0.71. Bootstrap resampling showed negligible optimism, resulting in an optimism‐corrected AUC of 0.71, indicating minimal overfitting. In the validation cohort, the AUC was 0.63, demonstrating preserved but modest discriminatory performance compared with the training cohort. Moreover, the calibration intercept was close to 0 and the calibration slope was close to 1, suggesting good calibration of the model (Figure 4).

FIGURE 4.

Calibration plot of the combined model. The plot shows the relationship between predicted probabilities and observed event rates. The dashed diagonal line represents perfect calibration.

3.4. Analysis of Risk Factors for POD

Univariate and multivariate analyses were performed to identify risk factors associated with the development of POD. In the univariate analysis, the high‐ and intermediate‐risk groups (p < 0.01), operation time > 326 min (p < 0.01), blood loss > 530 mL (p = 0.01), and age > 77 years (p = 0.04) were significantly associated with the development of POD. Multivariate logistic regression analysis identified the high‐ and intermediate‐risk groups (OR, 5.86; 95% CI, 1.78–19.29; p < 0.01) and operation time > 326 min (OR, 5.21; 95% CI, 1.51–17.89; p < 0.01) as independent risk factors for POD (Table 3).

TABLE 3.

Uni‐ and multivariate analyses of risk factors for severe complications.

Variables	Univariate			Multivariate
	Odds ratio	95% CI	p	Odds ratio	95% CI	p
Age
≤ 77	Ref
> 77	2.81	1.19–6.61	0.018*	1.57	0.53‐4.62	0.413
Sex
Female	Ref
Male	1.37	0.54–3.54	0.506
Operation time
≤ 326 min	Ref			Ref
> 326 min	4.37	1.52–12.59	0.006*	5.21	1.42‐17.90	0.008*
Blood loss
≤ 530 mL	Ref			Ref
> 530 mL	3.06	1.29–7.26	0.011*	1.90	0.67‐5.44	0.230
Serum level of albumin
≤ 3.6 g/dL	Ref
> 3.6 g/dL	1.95	0.80–4.72	0.139
Risk calculated by the NCD calculator and mFI‐5
Low‐risk	Ref			Ref
High‐ and intermediate‐risk	5.01	1.86–13.55	0.002*	5.87	1.78‐19.29	0.004*

Abbreviations: CI, confidence interval; mFI‐5, the 5‐Item Modified Frailty Index.

^* p < 0.05.

4. Discussion

The present analysis demonstrated that combining the NCD Risk Calculator and the mFI‐5 provides a novel predictive marker for the development of POD in patients with HCC undergoing liver resection. Given the established association between POD and postoperative outcomes [3], precise prediction of its occurrence is important, and this marker has potential clinical value.

Reports on POD following liver resection remain limited [8, 9, 16, 17, 18, 19, 20, 21, 22]; however, consistent with our findings, age, prolonged operative time, and greater intraoperative blood loss have been identified as important risk factors [8, 16, 18, 22]. Furthermore, other studies have also suggested that sex, preoperative hemoglobin concentration, and preoperative serum albumin level may also contribute to POD risk, although these factors were not confirmed in our study [16, 17, 18]. Despite its relatively low positive likelihood ratio, the marker identified in this study offers practical value by allowing the prediction of POD solely from preoperative testing. Such early risk identification provides an opportunity to plan the surgical approach and postoperative management, including the use of minimally invasive procedures [21], prophylactic administration of ramelteon [20], and substitution of H₂ blockers with proton pump inhibitors [19], which are interventions reported to lower the incidence of POD, as well as individualized perioperative management for high‐risk patients, such as optimization of analgesia and fluid management, careful medication adjustment, and coordination of multidisciplinary care. Furthermore, because the proposed marker combines an established risk prediction tool with an assessment of frailty, it provides notable novelty and clinical utility.

The NCD is a nationwide surgical patient registry in Japan. Using these data, the NCD Risk Calculator was developed to estimate the risk of various postoperative complications, including POD. This web‐based tool is freely accessible to all surgeons in Japan and estimates the risk of postoperative complications based on multiple clinical factors. POD arises from a complex interaction of multiple factors rather than a single cause; therefore, prediction models that integrate several variables, including our marker and the multivariable models of Yan [8] and Zhu [9], are likely to yield more accurate risk estimates than approaches focusing on any single factor. Frailty, a clinical condition marked by diminished physiological reserves and heightened susceptibility to stress, has been associated with postoperative complications following liver resection [23, 24, 25, 26], and a relationship with POD has also been reported [27]. Since the activity and vulnerability reflected by frailty cannot be entirely quantified by conventional clinical parameters, frailty assessment may offer additional predictive value that complements the NCD Risk Calculator in estimating POD risk. This combined approach enables clinically meaningful refinement of risk stratification by identifying a subgroup with increased vulnerability to postoperative delirium among patients initially classified as the favorable‐risk by the NCD Risk Calculator using only routine preoperative information.

This study has some limitations that warrant emphasis. First, its retrospective and non‐randomized design inherently introduces the possibility of selection bias. Second, the analysis was conducted at a single institution with a relatively small patient cohort, which may limit the external generalizability of our findings, including laparoscopic or robotic hepatectomy. In addition, because the training and validation cohorts were separated by treatment period, temporal bias related to changes in surgical techniques and perioperative management cannot be excluded. Consequently, our findings may not fully represent real‐world clinical practice and should be clarified in future large‐scale multicenter studies. As previously mentioned, a further limitation is that the NCD Risk Calculator is presently limited to use in Japan; its broader international use is expected in the future.

In conclusion, combining the NCD Risk Calculator with mFI‐5 markedly improves predictive accuracy, providing a reliable approach for assessing POD risk.

Author Contributions

Kiyotaka Hosoda: conceptualization, writing – original draft, investigation. Akira Shimizu: conceptualization, writing – review and editing. Koji Kubota: conceptualization, data curation. Tsuyoshi Notake: conceptualization, data curation. Hiroki Sakai: data curation, formal analysis, investigation. Kentaro Yoshikawa: data curation, investigation, formal analysis. Tomohiko Ikehara: data curation, investigation, formal analysis. Hikaru Hayashi: data curation, formal analysis, investigation. Koya Yasukawa: data curation, formal analysis, investigation. Yuji Soejima: supervision, conceptualization.

Funding

The authors have nothing to report.

Ethics Statement

This protocol for this research project was approved by a suitably constituted Ethics Committee of Shinshu University School of Medicine (Approval No. 5244), and it conforms to the provisions of the Declaration of Helsinki. Informed consent was obtained from the subjects.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that supports the findings of this study are available from the corresponding author, upon reasonable request.