Mortality Predictions Including Pre‐Admission Functional Status in ICU Patients With Delirium—A Substudy of the AID‐ICU Trial

ABSTRACT

Purpose

Pre‐admission functional status affects patients' ability to overcome the deteriorating effects of acute critical illness. We aimed to develop a clinical prediction model for 90‐day and 1‐year mortality based on pre‐admission data, including functional status, in adult delirious ICU patients.

Methods

We included participants randomized to the three highest‐enrolling hospitals in the Agents Intervening against Delirium in the Intensive Care Unit (AID‐ICU) trial, with pre‐admission data on the Clinical Frailty Scale, Comorbidity‐Polypharmacy Score, and Barthel‐20 score. Ten candidate models were evaluated using multiple modeling approaches. Final models were chosen based on the hyperparameter setting maximizing the Cox‐Snell pseudo R ². All baseline variables and trial allocation were included. Final models were retrained on the full dataset, with internal validation performed using bootstrapping validation adjusting for optimism.

Results

Of 1000 participants in AID‐ICU, 632 were included: 630 provided data on 90‐day mortality, and 610 on 1‐year mortality. The elastic net regression models demonstrated stable, robust performance. The optimism‐adjusted areas under the receiver operating characteristic curves were 0.74 (95% confidence interval [CI]: 0.70–0.78) and 0.74 (95% CI: 0.70–0.77) for the 90‐day and 1‐year mortality models, respectively. Calibration was good across the risk spectrum. Frailty, age, the Simplified Mortality Score for the Intensive Care Unit (SMS‐ICU), advanced cancer, and surgical admission contributed most to the prediction models.

Conclusions

We developed models to predict 90‐day and 1‐year mortality at ICU admission in patients enrolled in the AID‐ICU trial, using baseline variables, including functional status measures. The models showed fair discrimination and good calibration, with frailty, age, SMS‐ICU, advanced cancer, and surgical admission as key predictors. Future studies are needed to test whether the model is valid in other ICU settings and whether its performance is sufficient to have clinical value.

Editorial Comment

This article presents mortality prediction models for ICU patients with delirium that incorporate pre‐admission functional status and apply several modern statistical learning approaches, providing an instructive and transparent example of contemporary prediction modeling. The resulting elastic net regression models showed fair discrimination and good calibration for predicting 90‐day and 1‐year mortality, with frailty, age, and illness severity emerging as the strongest predictors. However, such models should be interpreted cautiously at the individual patient level and may be most useful for identifying patients at increased risk who may benefit from careful clinical assessment, individualized treatment, and close follow‐up.

1. Introduction

Pre‐morbid functional status significantly impacts mortality and functional outcomes after critical illness [1, 2, 3]. Age and severity of diseases at admission to the Intensive Care Unit (ICU) cannot alone account for the long‐term outcome [4, 5, 6]. Evaluation of functional status before hospitalization involves assessing activities of daily living (ADL), comorbidities, and frailty [6, 7, 8, 9].

Prediction models can be used in clinical settings to quantify patients' risk of future adverse events and, ultimately, stratify patients at risk of poor outcomes [10]. In ICU populations, multiple prediction models and scoring systems have been developed; however, none have proven feasible for use at the individual level [11]. Incorporating premorbid functional status in prediction models may be a step toward a more individualized care based on the risk of adverse outcomes [2, 12].

In this study, we aimed to develop models predicting 90‐day and 1‐year mortality using baseline variables from a prospective cohort of patients with delirium enrolled in the Agents Intervening against Delirium in the Intensive Care Unit (AID‐ICU) trial, along with pre‐admission functional status, and to evaluate their performance.

2. Materials and Methods

2.1. Design

This is a pre‐planned exploratory study of the randomized clinical trial, the AID‐ICU trial (Appendix 1 in Supporting Information). The study was reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines (Appendix 2, in Supporting Information) [13]. The protocol is uploaded to OSF.io [14].

2.2. Data Collection

Data were collected as part of the AID‐ICU trial and during the 90‐day and 1‐year follow‐up between June 14, 2018, and April 20, 2023. Baseline variables included age, sex, hematological malignancy, metastatic cancer, current smoking, current alcohol abuse, admission type (surgical or medical), and the Simplified Mortality Score for the Intensive Care Unit (SMS‐ICU) [15], and the stratification variables site and delirium motor subtype (hyperactive or hypoactive) (Appendix 3, in Supporting Information). Vital status was obtained by the primary investigators 90 days and 1 year after randomization. Moreover, pre‐admission limitations on life‐sustaining treatment and, after completion of the primary trial, treatment allocation (haloperidol or placebo) were obtained. At the time of inclusion in the AID‐ICU trial and of predictor collection, research personnel were blinded to trial allocation and outcomes.

2.3. Participants

The selected cohort included all participants with pre‐admission functional status data who were enrolled at Rigshospitalet, Copenhagen, Aalborg University Hospital, Aalborg, and Zealand University Hospital, Køge. At the time of inclusion, participants were diagnosed with delirium. The intervention was haloperidol versus placebo (Appendix 1, in Supporting Information).

2.4. Approvals and Informed Consent

The AID‐ICU trial adhered to the principles outlined in the Declaration of Helsinki. It was approved by the Regional Health Research Ethics Committee (Title: Agents Intervening against Delirium in the Intensive Care Unit (AID‐ICU); approval date: April 16, 2018; ClinicalTrials.gov number: NCT03392376; EudraCT number: 2017‐003829‐15), the Danish Data Protection Agency, and the Danish Medical Agency [16]. Consent for participation was obtained in accordance with national law.

2.5. Outcomes

The pre‐specified outcomes were 90‐day and 1‐year mortality. The time point was chosen for pragmatic reasons, as the study was embedded in the pre‐planned 90‐day and 1‐year follow‐ups of participants in the AID‐ICU trial.

2.6. Predictors

All baseline and stratification variables, trial allocation, and pre‐admission limitation of life‐sustaining treatment described above were entered into the models as candidate predictors. Moreover, three measures of pre‐admission functional status were collected at inclusion in the AID‐ICU trial: the Clinical Frailty Scale (CFS), the Comorbidity‐Polypharmacy Score (CPS), and the Barthel‐20 ADL score.

CFS version 2.0 summarizes the overall level of frailty on a 9‐point scale, ranging from very fit (1) to terminally ill (8–9) [17]. Scores of 5–9 indicate frailty, and 4 indicates vulnerability. The scale was initially developed and validated in individuals aged > 65 years, but has also been applied to patients as young as 40 years [17, 18]. It has been validated for ICU populations, demonstrating acceptable inter‐rater reliability [10, 17, 18]. Research personnel received special training before conducting the pre‐admission evaluation, which was obtained either by proxy or reconstructed from the patient's file [10, 19].

The CPS summarizes overall chronic illness by adding the number of ICD‐10 diagnoses of chronic conditions to the number of medications (prescription and “over the counter”) listed in the patient's file [8]. Higher scores indicate greater comorbidity. The CPS has mainly been validated in elderly, trauma, and burn patients, but also in ICU populations [10, 20]. Research personnel calculated the score based on information from the patient's file.

The Barthel‐20 ADL score is a shortened version of the Barthel‐100 score, assessing 10 physical domains essential for independent living: feeding, moving from chair to bed, personal hygiene, toileting, bathing, walking on a level surface, ascending/descending stairs, dressing, and bowel and bladder control. Each domain is scored on a scale of 0 to 1–3, yielding a total score of 0–20, with higher scores indicating better function [21]. Although widely used by occupational therapists and geriatricians, the Barthel Index has not been explicitly validated for ICU patients [22, 23, 24]. Pre‐admission evaluation was obtained by proxy or reconstructed from the patient's file.

2.7. Statistical Analysis

Continuous variables were summarized as mean (SD) or median (IQR), and categorical variables as proportions. Distributions were visualized using histograms, density plots, and bar charts, stratified by survival status (survivors, non‐survivors) and overall. Correlations between variables were assessed using Spearman's rank correlation.

Statistical analyses and explanations of the components of the models and the performance metrics are described in detail in Appendix 4, in Supporting Information.

Data were inspected for unusual patterns, and the selection of variables adjusted accordingly. Before model training, missing predictor data were imputed using the K‐nearest neighbors' algorithm with k = 5. A filter was set up to remove the least number of variables to ensure all variable pairs had a Pearson's correlation coefficient ≤ 0.9, all categorical variables were converted into dummy features, and all numeric predictors were scaled between 0 and 1 using min‐max scaling.

The prediction model was developed using the following steps and applied to the entire dataset: a set of widely used statistical learning techniques was investigated as classifiers for patients' outcome status. We empirically investigated different statistical learning methods because we did not know a priori which would be superior. Hyperparameter tuning was performed, and five‐fold cross‐validation was used to evaluate metrics of discrimination: area under the receiver‐operating curve (AUROC), area under the precision‐recall curve (AUPRC), calibration (observed/expected ratio, calibration intercept, and calibration slope), and overall fit (Brier score and, for regression‐based methods, the Cox‐Snell pseudo R ²) during initial training.

A final learning method was selected based on the overall performance of the statistical learning methods across metrics and interpretability. The hyperparameter settings that maximized the Cox‐Snell pseudo R ² were selected to retrain the final model on the entire dataset [25]. Both the models' apparent and optimism‐adjusted performance were reported. The basic bootstrap confidence interval (CI) method was used to construct 95% CIs, and optimism adjustment of CIs was performed using the location‐shifted method [26, 27]. Furthermore, flexible calibration curves were constructed and visually inspected to assess whether the model was well calibrated across the full range of predicted risk.

SHapley Additive exPlanations (SHAP) values were reported for the final model as a model‐agnostic method for quantifying variable contributions to predictions.

2.8. Subgroup Analyses

We dichotomized the functional variables and the SMS‐ICU based on the distribution of mortality. The dichotomization resulted in the following groups: CFS 1–4 and 5–9; CPS 0–13 and 14+; Barthel‐20 ADL score 0–16 and 17–20; and SMS‐ICU ≤ 25 and ≥ 26. We analyzed the final models' performance on the subgroups, adjusting for the overall model optimism.

Analyses were performed in R (R Core Team, Foundation for Statistical Computing, Vienna, Austria), version 4.3.2. The script is published online (https://github.com/CentreForAnaesthesiologicalResearch/aid_icu_pred).

3. Results

3.1. Participants

A total of 632 out of 1000 participants in the AID‐ICU trial were included in the three ICUs participating in this study. Median age was 71 years (IQR 63–76), and the female proportion was 33.9% (Table 1). Due to consent withdrawals, two participants had missing outcomes at 90 days, and 22 at 1 year, resulting in 630 participants contributing to the 90‐day mortality analyses and 610 to the 1‐year analyses (Figure 1).

TABLE 1.

Baseline variables of the model predicting 1‐year mortality.

	Total (n = 632)	Alive (n = 326)	Dead at 1 year (n = 284)	Missing outcome (n = 22)
Age, median (IQR) a	71 (63–76)	68 (60–74)	74 (68–78)	67.5 (59–72)
Sex, female n (%) a	214 (33.9)	107 (32.8)	101 (35.6)	6 (27.3)
Site
Rigshospitalet, n (%) a	228 (36.1)	119 (36.5)	100 (35.2)	9 (40.9)
AaUH, n (%) a	168 (26.6)	101 (31.0)	61 (21.5)	6 (27.3)
ZUH, n (%) a	236 (37.3)	106 (32.5)	123 (43.3)	7 (31.8)
Pre‐admission Barthel‐20, median (IQR), missing (n)	20 (19–20), (59)	20 (19–20), (17)	20 (18–20), (21)	20 (20–20), (21)
Pre‐admission CFS, median (IQR), missing (n)	4 (3–5), (23)	3 (3–4), (0)	4 (3–6), (2)	4 (4–4), (21)
Pre‐admission CPS, median (IQR), missing (n) b	12 (7–17), (81)	11 (6–15), (45)	13 (8–18), (15)	13 (13–13), (21)
Smoking, n (%), missing (n)	187 (29.6), (1)	104 (31.9), (1)	72 (25.4)	11 (50.0)
Alcohol abuse, n (%) a	90 (14.2)	50 (15.3)	37 (13.0)	3 (13.6)
Surgical admission, n (%) a , g	267 (42.3)	151 (46.3)	107 (37.7)	9 (40.9)
Advanced cancer, n (%) a , g	58 (9.2)	16 (4.9)	39 (13.7)	3 (13.6)
SMS‐ICU, median (IQR) a , b , f	20 (16–24)	18 (14–22)	22 (17–25)	18 (16–22)
Respiratory support, n (%) a , c	394 (62.3)	206 (63.2)	174 (61.3)	14 (63.6)
Vasopressor, n (%) a , d	351 (55.5)	158 (48.5)	184 (64.8)	9 (40.9)
Renal replacement therapy, n (%) a , e	98 (15.5)	54 (16.6)	41 (14.4)	3 (13.6)
No treatment limitation pre‐ICU, n (%) a , h	337 (53.3)	261 (80.1)	76 (26.8)	0
Hypoactive delirium at inclusion, n (%) a	334 (52.9)	170 (52.2)	155 (54.6)	9 (40.9)
Allocated to haloperidol, n (%) a	322 (51.0)	176 (54.0)	136 (47.9)	10 (45.5)

Note: Data on baseline characteristics were available for all 632 participants, except for one participant who was missing the smoking variable. For the pre‐admission functional status, 59, 23, and 81 participants were missing the Barthel‐20, CFS, and CPS variables, respectively.

Abbreviations: AaUH, aalborg university hospital; Barthel‐20, barthel‐20 activities of daily living score; CFS, clinical frailty scale; CPS, comorbidity‐polypharmacy score; SMS‐ICU, simplified mortality score for the intensive care unit; ZUH, zealand university hospital.

^a Missing values in variable = 0.

^b Information on coexisting conditions was collected from the SMS‐ICU and CPS.

^c The variable surgical admission was constructed by adding the baseline variables: elective surgical admission and emergency surgical admission. The variable advanced cancer was constructed by adding the baseline variables: hematologic cancer and metastatic cancer.

^d The predicted 90‐day mortality was calculated with the use of the SMS‐ICU, on which scores range from 0 to 42 (corresponding with a range of predicted 90‐day mortality of 3.3%–91.0%).

^e Respiratory support was defined as invasive or noninvasive mechanical ventilation, including use of a continuous positive airway pressure (CPAP) mask or CPAP by means of tracheostomy within 24 h before randomization. Intermittent CPAP was not defined as respiratory support.

^f Use of vasopressors or inotropes was defined as continuous infusion of norepinephrine, epinephrine, phenylephrine, vasopressin analogues, dopamine, dobutamine, milrinone, or levosimendan within the 24 h before randomization.

^g Renal replacement therapy was defined as short‐term or prolonged intermittent or continuous renal‐replacement therapy within the 24 h before randomization.

^h Treatment limitations pre‐ICU were defined as the decision to one. Do not resuscitate order, two. Not offer any of the advanced treatments, invasive ventilation, vasopressor therapy, and renal replacement therapy. The decision must have been made before ICU admission or on the day of admission.

FIGURE 1.

Consort diagram. A total of 368 participants were excluded because they were enrolled at sites that did not contribute to the estimation of pre‐admission functional status. Data on the primary outcomes were missing for 2 participants at 90 days and for 20 participants at 1 year. *Study sites: Rigshospitalet, Copenhagen, Aalborg University Hospital, Aalborg, Zealand University Hospital, Køge.

Mortality at 90 days and 1 year post‐randomization was 39.0% and 46.6%, respectively. Deceased participants were older, had a higher SMS‐ICU, and had more pre‐admission treatment limitations than the survivors (Tables 1 and E1). Missing values were observed almost exclusively in the three functional status variables (Tables 1 and E1). Ninety‐day and 1‐year mortality rates according to all baseline variables are shown in Figures E1–E22 and E40–E61.

3.2. Candidate Predictive Models

All baseline variables except smoking were included in the model training (Figures E15 and E54). Spearman's rank correlation between CFS, CPS, and Barthel‐20 ADL score yielded coefficients of 0.3 and 0.6, with the highest correlation observed between CFS and Barthel‐20 ADL score (Figures E23 and E62).

Upon inspecting the performance metrics of the 10 models, only minor differences were observed; however, the regression‐based models performed best and showed the most consistent results (Figures 2, E24–29, E63–68, and Table 2).

FIGURE 2.

Discriminative performance of the models. (A) 90‐day prediction models. (B) 1‐year prediction models. Illustrations of the area under the receiver‐operating curve (AUROC) for all 10 models trained on the dataset. LASSO, least absolute shrinkage and selection operator.

TABLE 2.

Performance of the apparent elastic net model and the model adjusted for optimism.

	AUPRC (95% CI)	AUROC (95% CI)	Brier score (95% CI)	Cox‐Snell R2 score (95% CI)	O/E ratio (95% CI)	Calibration slope (95% CI)	Calibration intercept (95% CI)
Prediction of 90‐day mortality
Apparent performance	0.64 (0.57–0.71)	0.75 (0.71–0.79)	0.20 (0.19–0.21)	0.16 (0.11–0.20)	1.00 (0.91–1.08)	1.20 (0.94–1.47)	0.00 (−0.16–0.16)
Optimism adjusted performance	0.63 (0.56–0.70)	0.74 (0.70–0.78)	0.20 (0.19–0.21)	0.15 (0.10–0.19)	1.01 (0.93–1.10)	1.11 (0.85–1.38)	0.03 (−0.14–0.18)
Prediction of 1‐year mortality
Apparent performance	0.71 (0.65–0.77)	0.75 (0.71–0.78)	0.20 (0.19–0.22)	0.17 (0.12–0.22)	1.00 (0.92–1.08)	1.12 (0.88–1.35)	0.00 (−0.17–0.18)
Optimism adjusted performance	0.74 (0.70–0.77)	0.74 (0.70–0.77)	0.21 (0.20–0.22)	0.16 (0.11–0.20)	1.01 (0.94–1.09)	1.02 (0.78–1.25)	0.02 (−0.15–0.20)

Note: The apparent performance was calculated by the elastic net model using all baseline and functional status variables (Table 1). The optimism‐adjusted performance was constructed using bootstrapped validation with 500 resamples on the original dataset.

Abbreviations: AUPRC, area under the precision‐recall curve; AUROC, area under the receiver operating curve; O/E ratio, observed event/estimated event ratio.

3.3. Final Models

Among the candidate models, the elastic net models were selected, as they consistently performed well across all hyperparameter settings and combine the advantage of automated feature selection with the ability to handle correlated variables. Hyperparameters that maximized the Cox‐Snell pseudo R ² during cross‐validation were used in the final model [28]. The apparent and optimism‐adjusted performance is summarized in Table 2, showing very similar performance for the 90‐day and 1‐year models, with fair discrimination (AUROC 0.74 [95% CI: 0.70–0.78] and 0.74 [95% CI: 0.70–0.77], respectively) for the 90‐days and 1‐year mortality model, respectively, and a tendency to underpredict risk, as reflected by calibration slopes of 1.11 (95% CI: 0.85–1.38) and 1.02 (95% CI: 0.78–1.25), respectively. Visual inspection of the calibration plots, however, indicated good calibration across the full range of predicted risks (Figures E37 and E76). Overall, the fit appeared acceptable; the optimism‐adjusted Cox‐Snell R ² for outcomes of 0.15 (95% CI: 0.10–0.19) and 0.16 (95% CI: 0.11–0.20) are out of a theoretical maximum of 0.74 for 90‐day mortality and 0.75 for 1‐year mortality, respectively. This showed an improved model likelihood of the fitted models relative to the null model (Table 2) [29].

SHAP analyses identified CFS, age, and SMS‐ICU as the strongest predictors of 90‐day and 1‐year mortality, while surgical admission and advanced cancer also emerged as important predictors of 90‐day mortality (Figure 3). This aligned with the regression coefficients for both models (Table E3). For each predictor, dependency plots were created by plotting SHAP values against observed values, with coloring based on the variable with the strongest potential interaction. These plots suggested no evidence of strong unmodeled interaction effects among the predictors (Figures E39 and E78).

FIGURE 3.

SHapley Additive exPlanations (SHAP) values for predictors of the model. (A) 90‐day prediction models. (B) 1‐year prediction models. Illustration of the predictor variables that contributed the most to the elastic net regression model. SHAP shows the contribution or importance of each variable to the model's prediction, but not the direction or clinical importance. The directionality of the variables “surgical admission” and “allocation to haloperidol” indicated that they reduced the risk of mortality.

3.4. Subgroup Analyses

Overall, model performance declined when predictors were dichotomized (Table E2). An exception was observed among participants with higher SMS‐ICU, suggesting that the models might perform best among the most acutely ill participants.

4. Discussion

In this study, we developed clinical prediction models for 90‐day and 1‐year mortality using data from delirious ICU patients included in the AID‐ICU trial. Multiple modeling approaches were evaluated, with no single model clearly outperforming the others across performance metrics. The elastic net models were ultimately chosen for their consistently strong performance across all hyperparameter settings and metrics. The models demonstrated fair discriminatory ability and acceptable calibration. The strongest predictors of models were CFS, age, SMS‐ICU, advanced cancer, and surgical admission. Generally, the association between input variables and predicted mortality aligns with clinical experience: higher age, CFS, and SMS‐ICU are associated with higher predicted mortality risk, and surgical admission and allocation to haloperidol treatment are associated with lower predicted mortality risk (Figures E38 and E77).

Prediction models based primarily on acute critical illness have achieved AUROCs of 0.72–0.88, with a primary focus on hospital mortality [11]. Their discriminatory ability decreased with older age, in patients with lower critical illness severity scores, and appeared sensitive to case‐mix and site [4, 5]. In elderly patients, this reduced performance may reflect the limited representation of older patients and those with chronic diseases in the historical cohorts used to develop critical illness severity scores. This limitation has prompted consideration of replacing chronological age with biological age to improve accuracy [30]. Incorporating independence in ADL into predictive models for elderly ICU patients has been shown to improve the prediction of 1‐year mortality, although not short‐term outcomes [3]. Pietiläinen et al. [31] developed a score to predict 1‐year mortality in a broad case mix of ICU patients, incorporating five domains of physical activity, and demonstrated fair discrimination (AUROC of 0.78) and good calibration in both the development and validation cohorts. However, physical performance, as measured by a single index such as ADL, has been shown to be insufficient to account for overall functional status [10]. Guidet et al. [10] investigated prediction models, including CFS, CPS, and Katz ADL score, in elderly ICU patients and found that CFS had the highest predictive ability for 30‐day survival. The CFS and Katz ADL were highly correlated, and the CPS did not appear to affect 30‐day mortality. Apart from age distribution, our younger study population was comparable to that of Guidet et al. [10]. We similarly found that CFS, age, and the SMS‐ICU were the strongest predictors. In contrast, the CPS and Barthel‐20 ADL scores were highly correlated with age and CFS, adding only limited predictive value (Figure 3). Two recent meta‐analyses confirm the substantial predictive value of frailty for mortality in ICU patients [2, 12]. Bruno et al. [12] further demonstrated that, when adjusting for acute critical illness using the Acute Physiology and Chronic Health Evaluation (APACHE) II, the predictive ability of CFS was attenuated.

In medical statistics, regression‐based models are the primary methods used for prediction, offering the advantage of being relatively easy for clinicians to understand and interpret [32]. In this study, no method was dominant across all performance metrics. While adjusting hyperparameter settings or incorporating interaction terms and higher‐order effects can influence results, our findings support the idea that regression‐based models can serve as a reasonable default choice. The elastic net model, an extension of traditional regression, combines both L1 and L2 regularization. The L1 term, also used in LASSO regression, enables feature selection by shrinking model coefficients to zero, thereby removing their influence on the model output. The L2 term offers some theoretical advantages, particularly for handling correlated predictors.

A typical challenge in statistical learning is dealing with datasets where the number of candidate predictors is significantly larger than the number of observations. Developing a model based on such datasets requires reducing the candidate predictors, either through data‐driven methods or expert opinion [28]. In contrast, our study relied on a limited set of baseline variables from the AID‐ICU trial, which limited the scope of this data‐driven approach. The traditionally high number of candidate variables applied in a machine learning model challenges the operationality. For daily practice, a model with as few variables as possible would be preferred. We do, however, not know how much predictive performance should be sacrificed in favor of simplicity. The Cox‐Snell pseudo‐R ²s in our study show that the fitted models had higher model likelihood than the null models. However, the theoretical upper bounds were not reached, suggesting that unmeasured variables could improve mortality risk models. However, the optimism‐adjusted Cox‐Snell pseudo R ² is just below a recently suggested threshold for a good fit [29]. Given the available variables, we believe our model captures the variance as effectively as possible. Simplifying it by reducing predictors would likely come at the cost of reduced performance.

We do not report positive or negative predictive values, sensitivity, or specificity, as these require predefined thresholds for classifying events. Dichotomizing predictions into “dead” or “alive” within a given time‐at‐risk may encourage a deterministic interpretation and, in turn, lead to a prescriptive use of the model. Instead, we view the model as a decision‐support tool for newly admitted ICU patients, such as for determining whether to escalate treatment, where expected survival is one of several factors to consider.

Dichotomizing the variables did not improve our models. All but one subgroup—those with an SMS‐ICU of 26 or higher—performed worse across all performance metrics than models that respected the ordinal and continuous nature of the variable (Table E2). This illustrates the disadvantage of losing power of the sample when dichotomizing.

The strength of this study lies in the high‐quality, prospectively collected dataset used to build the prediction model. This ensures a high degree of completeness and a low degree of bias. Moreover, missing values were imputed. Another strength of the study is that we performed a thorough model evaluation, including internal validation and performance metrics [25].

There are also limitations to our study. First, the study population was relatively small and consisted of Danish patients with delirium included in the AID‐ICU trial according to predefined criteria, which limits validity and generalizability. Nevertheless, despite being a selected group due to the randomized design and requirement of delirium, the population still represents a highly relevant intensive care cohort. Second, we have similar concerns about selection bias arising from the pre‐selected variables in our prediction models. In other machine learning models, candidate predictors were selected from a broader set of variables, whereas the selection criteria in the AID‐ICU trial restricted the range. However, the development of the present model focused exclusively on patients' pre‐admission status prior to the ICU admission. Therefore, variables related to organ support during the ICU stay and post‐discharge status were not included. Age, selected comorbidities, aspects of organ dysfunction, physiological status, and laboratory variables are already incorporated into the SMS‐ICU, while chronic disease is integrated into the CPS. Therefore, including these variables separately in the models could lead to overfitting. Although the model demonstrates reasonable performance as a binary classifier, our findings do not support the conclusion that it provides sufficient clinical value. Therefore, we do not suggest how the model could support clinical decision‐making. Without clearly defined decisions against how to evaluate the consequences of false positives, false negatives, true positives, and true negatives, the potential harms and benefits of using the model remain unknown. Finally, we did not perform external validation of the prediction model, which again reduces external validity.

Personalized treatment pathways could be tailored to the predicted risk of 90‐day and 1‐year mortality, with specific interventions adjusted accordingly. A recently published study of patients undergoing surgery for colorectal cancer successfully developed and implemented a prediction model for decision support [33]. This model, though, did not incorporate direct measurements of functional status, which offers a clear inspiration for future model development. Our models are based on a cohort of ICU patients with delirium enrolled in a randomized trial and thus require prospective external validation across a broader case mix of ICU patients. Additionally, to enable its integration into clinical practice, the model must be capable of identifying distinct risk groups that can be validated against clinical outcomes, preferably within a randomized clinical trial.

5. Conclusions

We developed elastic net models predicting 90‐day and 1‐year mortality in delirious patients at the time of admission to the ICU using AID‐ICU baseline variables and functional status measures. The models showed fair discrimination and good calibration, with frailty, age, SMS‐ICU, advanced cancer, and surgical admission as key predictors. Future studies are needed to test whether the model is valid in other ICU settings and whether its performance is sufficient to have clinical value.

Author Contributions

All author contributions meet the criteria for authorship as outlined by the ICMJE guidelines (https://www.ICMJE.org). L.M.P. formulated the research question, wrote the protocol and manuscript. L.M.P. and A.W.R. wrote the statistical analysis plan and performed the statistical analysis. L.M.P., C.B.M., N.C.A.‐R., and S.E. collected the data and constructed the data sheets for analysis. A.P., S.E., B.S.R., and O.M. supervised the process. All authors contributed to critical reading and editing the manuscript.

Funding

The Innovation Fund Denmark, The Zealand Region Research Fund, Foght Foundation, Hindsgavl Intensive Care Symposium, and the Regions of Denmark Medical Research Fund funded the AID‐ICU trial. The funders had no influence on the trial design, conduct, or report.

Conflicts of Interest

A.P. and S.E. are affiliated with the Department of Intensive Care at Rigshospitalet, which has received funding for other projects from the Novo Nordisk Foundation and Sygeforsikringen “danmark”. B.S.R. is affiliated with the Department of Anaesthesia and Intensive Care, Aalborg University Hospital, Aalborg, Denmark, which has received funding for another project from the Beckett Foundation and Kjærgaard Family Foundation. All other authors have no conflicts to disclose.

Supporting information

Data S1: aas70225‐sup‐0001‐Supinfo.docx.

Data Availability Statement

The script is published online (https://github.com/CentreForAnaesthesiologicalResearch/aid_icu_pred).