Development and validation of a prediction score for early identification of palliative care needs for patients in the intensive care unit: a multicentre retrospective cohort study

Summary

Background

Many critically ill patients and their families face serious physical, psychosocial, and spiritual burdens which can be addressed through specialist palliative care (PC). Identifying PC need and patients who would benefit from PC consultation remains challenging. We aimed to develop and validate a simple and accurate score for predicting PC involvement during intensive care unit (ICU) treatment with predictors routinely collected within 24 h of ICU admission to enable early integration.

Methods

This multicentre retrospective cohort study included adult patients admitted to an ICU between Jan 01, 2011 and Dec 31, 2022 in Boston, MA (development cohort), Omaha, NE, and Atlanta, GA (validation cohort [VC] I and II, respectively) in the United States of America. For score development, cases with missing data were excluded. PC involvement was defined as a specialist PC consult request or note in the patient's medical record. Candidate predictors were selected using adaptive-lasso-logistic regression-models with 10-fold cross-validation. We developed a comprehensive epidemiological score and subsequently a simplified version (PC-ICU) for clinical use. Score performance was quantified using the area-under the receiver-operating-characteristic (AU-ROC) curve and the PC-ICU score was externally validated in two independent cohorts. The PC-ICU score is available at: www.pc-icu.com.

Findings

99,582 patients (55.1% male, 44.9% female) were included. Score development was performed in 60,091 patients (19,705/79,796 excluded due to missing data), of which 5.5% (n = 3294) received PC (VC I: 1449 included patients [9.7%, n = 141 received PC], VC II: 38,042 included patients [13.2%, n = 5038 received PC]). The PC-ICU score consists of ten factors, including demographics, comorbidities, admission data, and the current patient status. AU-ROC curves in the development cohort were 0.85 (95% CI 0.84–0.86) and 0.81 (95% CI 0.81–0.82] for the comprehensive and PC-ICU score, respectively, indicating excellent discriminative ability. External validation of the PC-ICU score indicated good accuracy (AU-ROC of 0.78 [95% CI 0.74–0.82, VC I]; 0.67 [95% CI 0.66–0.67, VC II]).

Interpretation

The PC-ICU score predicts PC consultation during intensive care upon ICU admission. This externally validated instrument can identify patients with potential PC needs and may facilitate integration of multi-disciplinary care. Additional prospective evaluation of the PC-ICU score in different national and international locations, healthcare settings and patient populations is needed to further enhance generalisability.

Funding

The Deutsche Forschungsgemeinschaft (German Research Foundation).

Research in context.

Evidence before this study

A PubMed search conducted on Jun 04, 2025 using the terms "palliative care" and "intensive care" and "early" and "identification" revealed 39 results (seven paediatric studies, one in Chinese). No multivariable prediction tool has been published. This study addresses this gap by developing and validating the PC-ICU score, a clinically applicable instrument that allows to identify patients in need of specialist palliative care involvement early after admission to the intensive care unit.

Added value of this study

99,582 patients were included in this retrospective study conducted across three academic hospitals in the United States, one development and two validation centres. Specialist palliative care involvement ranged from 5.5% to 13.2%. The PC-ICU score includes ten parameters encompassing demographics, comorbidities, admission data, and current patient status, all available within 24 h following admission. The score predicts specialist palliative care consultation during intensive care early upon intensive care unit admission with good discriminative and predictive abilities and demonstrated superior performance compared to previously published trigger subsets.

Implications of all the available evidence

We developed and validated the first instrument to be used upon intensive care unit admission to predict the need for specialist palliative care involvement. Integration of the score into routine clinical practice may facilitate the early identification of patients with potential palliative care needs and support timely initiation of palliative care interventions, in accordance with current recommendations. Although this study used real-world data from three independent US-institutions, future prospective studies evaluating the PC-ICU score in different geographical regions, healthcare settings and populations are necessary.

Introduction

More than 20 million hospitalizations involved intensive care unit (ICU) admissions in the United States of America (U.S.) between 2008 and 2019.¹ This parallels an increasing trend of comprehensive treatments and ICU admissions at the end of life.1, 2, 3, 4, 5, 6 Critical illness is associated with physical, psychological, and spiritual distress with impacts lasting well beyond the time of admission.⁷ Specialist palliative care (PC) provided by a trained multi-professional team can relieve symptoms and improve the quality of life for patients and their families.8, 9, 10 Prior studies indicate that up to 20% of patients admitted to the ICU develop PC needs during their stay.¹¹

The integration of specialist PC in the ICU setting has been shown to improve goal-concordant care, reduce healthcare utilization, and increase the overall satisfaction with intensive care treatment without evidence of additional harm.^8,12, 13, 14, 15, 16, 17 However, early identification of those patients who need specialist PC at ICU admission remains challenging, and studies have shown a persistent underutilization of PC in the critically ill.^14,18,19 The European Society of Intensive Care Medicine (ESICM) has developed evidence-based recommendations and expert opinions for PC in the ICU, suggesting the use of standardised tools for symptom assessment in patients at high risk of dying.²⁰ While PC consult triggers such as hospital length of stay prior to ICU admission, end-stage malignancy, and status post cardiac arrest have been suggested,21, 22, 23 they have low performance in identifying PC needs (e.g., 45% sensitivity and 55% specificity), are often not accepted among physicians or nurses, and are not consistently implemented.^11,21,23, 24, 25 Hua et al. showed the insensitive ability of a trigger subset including seven factors (1. age >80 with two or more life-threatening comorbidities; 2. active stage IV malignancy; 3. status post cardiac arrest; 4. intracerebral haemorrhage requiring mechanical ventilation; 5. global cerebral ischemia; 6. multi-system organ failure and 7. advanced stage dementia) to predict 6-month mortality, while Secunda et al. further revealed that mortality-based triggers for specialist PC poorly reflect actual use of specialist PC in the ICU.^23,26Also, existing approaches do not incorporate individualised, weighted variables to reflect the complexity of clinical decision-making, and often reduce complex clinical situations to binary triggers.

To date, there is no tool available to predict individual PC involvement during the ICU stay. Identifying patients as early as possible, ideally during their admission to the ICU, is crucial to facilitate early initiation of multi-disciplinary care and promote resource-optimised involvement of specialist PC teams. In this study, we aimed to investigate what factors predict specialist PC involvement in the ICU based on previous integration and develop and subsequently validate an easily applicable score (Palliative Care in the Intensive Care Unit, PC-ICU score) assessed within 24 h following ICU admission to allow early and targeted identification of candidates for receipt of and benefits from specialist PC integration.

Methods

Study design and population

This multi-centre retrospective cohort study was approved by the institutional review board at Beth Israel Deaconess Medical Center in Boston, MA, U.S. (protocol number: 2023P001125) and the requirement for informed consent was waived, in accordance with the U.S. Department of Health and Human Services regulations (45 CFR 46.116), as the study posed minimal risk and used de-identified data from routine clinical care. Ethical approval was also obtained for the validation centres University of Nebraska Medical Center in Omaha, NE (protocol number: 0831-21-EP) and Emory University School of Medicine in Atlanta, GA (protocol number: 00007491; see Supplemental digital content 1.1–1.3). All data were collected in routine clinical care, extracted from electronic hospital databases and de-identified. A detailed description of data sources and the specific ICU environments and relationship with specialist PC services at each study site is provided in the Supplemental digital content 1.1–1.3. This manuscript adheres to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines,²⁷ and the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.²⁸

Adult patients (age ≥18 years) admitted to the ICU were eligible for inclusion. Cases with missing information on predictor variables were excluded in the development cohort. Thus, a complete-case approach including data from Jan 01, 2011 until Dec 31, 2022 was applied for score development.

Data analyses

Data analyses including score development, internal and external validation as well as sensitivity analyses were conducted following an a priori-defined statistical analysis plan. Exploratory analyses are marked accordingly.

Score development

The primary aim was to predict PC involvement, defined as a specialist PC encounter (request or note) in the patient's electronic medical record during the ICU stay. Patient characteristics between those receiving and those not receiving specialist PC were compared by standardised mean differences. To identify candidate predictors for PC, an extensive review of international literature with discussion of presumed clinically relevant factors among PC specialists was done (see Supplemental digital content 2.1). Importantly, all potential predictors were chosen from routinely collected data and had to be available within the first 24 h after ICU admission. Multiple continuous variables were categorised to facilitate clinical use of the score. As shown in the Supplemental digital content 2.2, a total of 62 candidate predictor variables were included: 47 referred to pre-ICU information and 15 to the patient's status in the ICU. Predictor selection was performed using adaptive-lasso-logistic regression-models with a 10-fold internal cross validation and subsequently the prediction model (comprehensive score) was created.

To address our study aim of developing a simple score for clinical practice, we then developed the PC-ICU score. To this end, we selected the most predictive factors based on ranked coefficient weights and a second adaptive-lasso-logistic regression was applied to confirm the same predictor set, supporting selection stability. For ease of calculation, the coefficients of each predictor were multiplied by two and rounded to the nearest integer. To ensure valid agreement between the comprehensive instrument and the PC-ICU score, both score values were compared by the mean (± standard deviation) difference. The predicted probability of PC involvement by each PC-ICU score values was visualised. We determined the cut-off point value of the PC-ICU score by applying the Youden index.²⁹ Additionally, the discriminative and predictive performances were assessed using C-statistics and Hosmer–Lemeshow goodness of fit tests.³⁰ We calculated the area-under the receiver-operating-characteristic (AU-ROC) curve, the calibration slope and intercept, and the Brier score.^31,32 The Brier score incorporates both discrimination and calibration, and ranges from 0 to 1, with smaller Brier scores indicating better prediction.³² Visualizations of AU-ROC curves, calibration plots, and decision curve analysis were performed.²⁹

External validation and score comparisons

Following internal validation, we validated the performance of the PC-ICU score in two independent, external cohorts using all available patient data: University of Nebraska Medical Center in Omaha, NE (validation cohort I) and Emory University School of Medicine in Atlanta, GA (validation cohort II). Validation metrics were assessed using AU-ROC, Brier score, and calibration curves. Furthermore, with an exploratory intent, we compared the PC-ICU score performance in the development cohort to previously published trigger subsets using the DeLong algorithm.^23,26,33

Sensitivity analyses

To confirm the robustness of the score, we conducted a series of sensitivity analyses. These included 1) multiple imputation of missing covariates in the development cohort to investigate score performance in all eligible patient cases (n = 79,796), as well as subgroup analyses to assess the score performance among 2) different ICU specialties captured using ICU types and the primary services (medical, surgical, and other including oral, ear, nose and throat, eye, gynaecology, obstetric, and orthopaedic services at admission) as previous data reported different PC utilization between surgical and medical patients,³⁴ 3) age groups (dichotomised based on the median in our cohort) to test consistency in score performance, 4) in all cases after March 11th 2020 as data indicated a shift in providing PC during the COVID-19 pandemic,³⁵ and 5) in non-cancer patients as they are less prone to receive specialist PC.³⁶ In clinical reality, a patient might have PC needs, as determined by the primary care team, and while a request is submitted, no actual specialist PC consultation may take place. Hence, we conducted a 6) sensitivity analysis and investigated score performance for patients that received a PC consult, as assured by respective notes in the electronic health record.

Statistical software

Analyses were performed in Stata (version MP 16.0, StataCorp LLC, College Station, TX, U.S.) and R Statistical Software (version 4.4.0, R Foundation for Statistical Computing, Vienna, Austria). A p-value below 0.05 was considered statistically significant. Visualization was performed using Stata and GraphPad Prism (version 9.0.2, GraphPad Software, Boston, MA, U.S.).

Role of the funding source

The funder of the study (German Research Foundation) had no role in study design, data collection, data analysis, data interpretation, or writing of the report. TT and MSS have accessed and verified the data, and were responsible for the decision to submit the manuscript.

Results

Overall, 99,582 patients were included (Fig. 1), with 60,091 in the development cohort and 1449 as well as 38,042 in the validation cohorts I and II, respectively.

Fig. 1.

Study flow diagram. The study flow depicts the total study cohort and respective numbers from each study site. ICU, intensive care unit; VC, validation cohort; ∗ = multiple criteria may apply.

Score development

Out of the 60,091 patients in the development cohort, 3294 (5.5%) received specialist PC during their ICU stay. PC involvement in this cohort increased over the study period (2011: 5.1%; 2022: 7.0%; p = 0.002). Patient characteristics by specialist PC involvement are shown in Table 1. Patients receiving PC were older, had higher comorbidity load and longer ICU stays with higher mortality. In total, 43 predictive variables were selected through adaptive-lasso-logistic regression for the comprehensive score (see Supplemental digital content 2.3), and the selection of ten predictors for the short PC-ICU (Fig. 2) score was confirmed by a second adaptive-lasso-logistic regression. A comparison of the comprehensive and PC-ICU score values (Fig. 3A) showed good agreement between the scores (mean difference 2.13 ± 1.06). Both scores demonstrated good discrimination (comprehensive score: AU-ROC, 0.85 [95% CI 0.84–0.86], PC-ICU score: AU-ROC, 0.81 [95% CI 0.81–0.82]). The predicted probability of PC involvement increased by the PC-ICU score values (Fig. 3B). The Brier score was 0.0462 for the comprehensive and 0.0468 for the PC-ICU score, indicating high prediction accuracy. The PC-ICU score point values ranged from 0 to 17 with a median of 7 (interquartile range [IQR] 5–7). The cut-off point discriminating between low and high likelihood of receiving specialist PC consultation was 7.5, which was present in 12,048 of cases (20.1%). For the cut-off, sensitivity was 0.64 and specificity was 0.82. The calibration plots for the scores are presented in Supplemental digital content 3.1, the Hosmer–Lemeshow goodness-of-fit tests were statistically significant for both the comprehensive model (χ² = 109.16) and the PC-ICU score (χ² = 87.45), which may reflect the large sample size.³⁷ The calibration slope and intercept were 1.00 (95% CI 0.97–1.03) and 3.65e-09 for the comprehensive model, and 1.00 (95% CI 0.97–1.03) and of −3.73e-08 for the PC-ICU score, respectively, indicating excellent agreement between predicted and observed probabilities. The Youden index–based cut-off (7.5) corresponded to a predicted probability of 5.9%, which demonstrated clinical utility in the decision curve analysis, provided in Supplemental digital content 4.

Table 1.

Patient characteristics and distribution of variables by specialist palliative care involvement in the development cohort.

	No specialist palliative care n = 56,797	Specialist palliative care n = 3294	Absolute standardised mean difference
Age, years	64.3 ± 16.1	69.3 ± 14.6	0.325
Sex
Male	31,103 (54.8%)	1788 (54.3%)	0.010
Female	25,694 (45.2%)	1506 (45.7%)
Body mass index, kg/m2	28.7 ± 6.9	27.0 ± 6.7	0.251
Marital status
Married	27,610 (48.6%)	1561 (47.4%)	0.087
Single	16,649 (29.3%)	893 (27.1%)
Widowed	6785 (11.9%)	535 (16.2%)
Ethnicity/race
White	39,701 (71.1%)	2247 (69.6%)	0.030
Hispanic	2597 (4.7%)	125 (3.9%)
Black	6990 (12.5%)	462 (14.3%)
Asian	1769 (3.2%)	120 (3.7%)
Religious denomination	39,719 (69.9%)	2396 (72.7%)	0.062
Federal insurance	30,454 (53.6%)	2085 (63.3%)	0.197
Adverse admission	13,661 (24.1%)	993 (30.1%)	0.137
History of
Home oxygen therapy	2629 (4.6%)	386 (11.7%)	0.263
Arterial hypertension	18,425 (32.4%)	1430 (43.4%)	0.228
Neurological disorders	12,830 (22.6%)	1219 (37.0%)	0.319
Diabetes mellitus	12,853 (22.6%)	896 (27.2%)	0.106
Renal failure	15,356 (27.0%)	1266 (38.4%)	0.246
Lymphoma	2085 (3.7%)	197 (6.0%)	0.108
Leukemia	1185 (2.1%)	97 (2.9%)	0.055
Metastatic cancer	4507 (7.9%)	1207 (36.6%)	0.735
Solid tumor	10,951 (19.3%)	1625 (49.3%)	0.667
Stroke	10,999 (19.4%)	821 (24.9%)	0.134
Dementia	5621 (9.9%)	524 (15.9%)	0.180
Parkinson's disease	2003 (3.5%)	151 (4.6%)	0.054
Duration hospital to ICU admission, days	0 (0–1)	0 (0–1)	0.303
ICU length of stay, days	2 (1–4)	5 (2–10)	0.543
Death in hospital	3872 (6.8%)	1258 (38.2%)	0.811
BIDMC: PC-ICU score parameters
Underweight	1597 (2.8%)	214 (6.5%)	0.176
Solid tumor	10,951 (19.3%)	1625 (49.3%)	0.667
Metastatic cancer	4507 (7.9%)	1207 (36.6%)	0.735
Home oxygen dependence	2629 (4.6%)	386 (11.7%)	0.261
Emergency/urgent hospital admission	44,478 (78.3%)	3207 (97.4%)	0.609
Non-surgical ICU admission	38,932 (68.5%)	3008 (91.3%)	0.593
ICU readmission	1269 (2.2%)	263 (8.0%)	0.263
SOFA ≥2	39,471 (69.5%)	2877 (87.3%)	0.444
ECMO use	45 (0.1%)	8 (0.2%)	0.041
Limiting code status	1213 (2.1%)	374 (11.4%)	0.374

Data are expressed as frequency (prevalence in %), mean ± standard deviation, or median (interquartile range [25th–75th percentile]).

ICU, intensive care unit; BIDMC, Beth Israel Deaconess Medical Center; SOFA, sequential organ failure assessment; ECMO, extracorporeal membrane oxygenation.

Fig. 2.

Variables included in the PC-ICU score. Grey dots indicate respective point values for each variable. BMI, body mass index; ICU, intensive care unit; SOFA, sequential organ failure assessment; ECMO, extracorporeal membrane oxygenation. Possible limiting code status: DNR, do not resuscitate; DNI, do not intubate; CMO, comfort measures only.

Fig. 3.

Score agreement and PC-ICU score prediction. Agreement between score values of the comprehensive and PC-ICU score (A) and the predicted probability of receiving palliative care depending on each PC-ICU score point value with the density of the score value distribution in grey bars (B).

External validation

The validation cohort I of 1449 patients included data from Jul 01, 2020 until Jun 30, 2021. 9.7% (n = 141/1449) of patients received specialist PC during the ICU course. The distribution of the PC-ICU score parameters is shown in Table 2, the AU-ROC in this cohort was 0.78 (95% CI 0.74–0.82), the Brier score 0.0721. For validation cohort II, 38,042 patients from Jan 06, 2019 until Oct 26, 2022 were included. Specialist PC was involved in 13.2% (n = 5038/38,042) of ICU cases. Score performance of the PC-ICU score showed an AU-ROC of 0.67 (95% CI 0.66–0.67), the Brier score was 0.1097. In this cohort, there was a pronounced variation in specialist PC utilization between surgical and non-surgical ICU admissions (Table 2). Fig. 4 summarises the AU-ROC for the development and validation cohorts.

Table 2.

Patient characteristics and PC-ICU score parameter distribution of variables by specialist palliative care involvement in the validation cohorts.

	Validation cohort I: University of Nebraska Medical Center			Validation cohort II: Emory University School of Medicine
	No specialist palliative care n = 1308	Specialist palliative care n = 141	ASD	No specialist palliative care n = 33,004	Specialist palliative care n = 5038	ASD
Age, years	61.7 ± 14.3	65.5 ± 13.4	0.274	61.1 ± 15.8	63.8 ± 15.6	0.171
Sex
Male	779 (59.6%)	80 (56.7%)	0.057	18,472 (56.0%)	2689 (53.4%)	0.052
Female	529 (40.4%)	61 (43.3%)		14,532 (44.0%)	2349 (46.6%)
Ethnicity/race	missing					0.137
White				15,919 (48.2%)	1886 (37.4%)
Black				14,105 (42.7%)	2712 (53.8%)
Asian				959 (2.9%)	115 (2.3%)
Other				2021 (6.1%)	325 (6.5%)
Marital status			0.116			0.035
Married/Life partner	700 (56.9%)	63 (50.4%)		14,786 (44.8%)	2056 (40.8%)
Divorced/Separated	131 (10.6%)	18 (14.4%)		2858 (8.7%)	531 (10.5%)
Single	278 (22.6%)	28 (22.4%)		8634 (26.2%)	1391 (27.6%)
Widowed	114 (9.3%)	14 (11.2%)		2263 (6.9%)	466 (9.2%)
Others/Unknown	8 (0.6%)	2 (1.6%)		4463 (13.5%)	594 (11.8%)
Religious denomination	missing			22,682 (68.7%)	3478 (69.0%)	0.007
Federal health insurance	684 (55.4%)	76 (59.4%)	0.081	11,876 (36.0%)	2198 (43.6%)	0.157
Body mass index, kg/m2	29.7 ± 6.8	30.3 ± 8.1	0.097	28.4 ± 6.8	27.5 ± 7.2	0.126
PC-ICU score parameters
Underweight	38 (2.9%)	1 (0.7%)	0.165	1354 (4.1%)	398 (7.9%)	0.160
Solid tumor	222 (17.0%)	14 (9.9%)	0.207	3481 (10.5%)	1038 (20.6%)	0.280
Metastatic cancer	64 (4.9%)	2 (1.4%)	0.199	2071 (6.3%)	854 (17.0%)	0.338
Home oxygen dependence	50 (3.8%)	8 (5.7%)	0.087	2091 (6.3%)	587 (11.7%)	0.187
Emergency/urgent hospital admission	682 (52.1%)	118 (83.7%)	0.717	11,989 (36.3%)	1974 (39.2%)	0.059
Non-surgical ICU admission	671 (51.3%)	66 (46.8%)	0.090	19,724 (59.8%)	3844 (76.3%)	0.360
ICU readmission	35 (2.7%)	6 (4.3%)	0.086	5817 (17.6%)	1327 (26.3%)	0.212
SOFA ≥2	996 (76.1%)	130 (92.2%)	0.450	21,468 (65.0%)	4050 (80.4%)	0.350
ECMO use	46 (3.5%)	31 (22.0%)	0.574	304 (0.9%)	109 (2.2%)	0.101
Limiting code status	41 (3.1%)	57 (40.4%)	1.009	246 (0.7%)	100 (2.0%)	0.107

Data are expressed as frequency (prevalence in %) and mean ± standard deviation.

ASD, absolute standardised mean difference; ICU, intensive care unit; SOFA, sequential organ failure assessment; ECMO, extracorporeal membrane oxygenation.

Fig. 4.

Score performances in the development and validation cohorts. Receiver operating characteristic curves of the comprehensive (light blue, AU-ROC: 0.85) and PC-ICU (dark blue, AU-ROC: 0.81) scores in the development cohort, and the PC-ICU score in the validation cohort I (dark grey, AU-ROC: 0.78) and II (light grey, AU-ROC: 0.67).

Comparison with other trigger factors

In exploratory analyses, we compared the PC-ICU score performance in predicting specialist PC involvement with previously published trigger factors.^23,26 The AU-ROC for the PC-ICU score was superior to the trigger subset as published by Secunda et al. (AU-ROC 0.81 vs. 0.70, p < 0.001).²³ Comparison to the trigger subset published by Hua et al. also yielded a higher AU-ROC for the PC-ICU score (0.81 vs. 0.69, p < 0.001).²⁶

Sensitivity analyses

After multiple imputation of missing covariates, resulting in a cohort of 79,796 patients, the PC-ICU score parameter selection was confirmed in adaptive-lasso-logistic regression and yielded a confirmatory AU-ROC of 0.80 (95% CI 0.79–0.81). The Youden-based cut-off value was identical with 7.5. Among all patients receiving specialist PC in the development cohort, 81.9% (n = 2697/3294) were medical, 16.6% (n = 546/3294) surgical and 1.5% (n = 51/3294) were labelled with other (e.g., orthopaedic) services. In the overall ICU population, the distribution was as follows: medical 55.6% (n = 33,422/60,091), surgical 39.6% (n = 23,789/60,091), and others 4.8% (n = 2880/60,091). When investigating the PC-ICU score performance among the different ICU specialties separately, we observed good performance across all specialties (medical: AU-ROC 0.76, surgical: 0.84, other: 0.86). The median age in the ICU population was 66 (IQR 55–76) years. The PC-ICU score performed well in subgroups based on age (AU-ROC: 0.87 for ≤65 compared to 0.77 for patients aged >65). In cases during the COVID-19 pandemic (n = 15,316; 5.7% specialist PC), the AU-ROC was 0.83 (95% CI 0.82–0.84). Among 47,076 patients without a diagnosis of solid tumor or metastatic cancer, 3.4% (n = 1610) received specialist PC. The PC-ICU score performed with an AU-ROC of 0.76 (95% CI 0.75–0.77). Score performance for patients that received a PC consult (n = 2537) was comparable, yielding an AU-ROC of 0.82 (95% CI 0.81–0.82). Additional information is provided in Supplemental digital content 5.

Discussion

The PC-ICU score was developed and externally validated in three independent cohorts of almost 100,000 patients and includes ten predictive factors on demographics, comorbidities, admission data, and the current patient status. The score predicts specialist PC consultation during intensive care early upon ICU admission with good discriminative and predictive abilities and demonstrated superior performance compared to previously published trigger subsets. Its application is to help identify patients within 24 h of ICU admissions whose needs make them good candidates for receipt of and benefit from specialist PC.

Prior studies indicate that up to 20% of patients admitted to the ICU develop PC needs during their stay.¹¹ Previously, protocols for triggered specialist PC consultations were based on clinical characteristics associated with death or resource utilization, such as advanced cancer or cardiac arrest.^24,38, 39, 40 However, these trigger factors showed poor performance to predict mortality or actual PC needs.^24,26 The published triggers did not include complex pain, symptom management, or conflict resolution, which may also benefit from PC team input. Furthermore, no prior study has developed multivariable models for predicting patients who might benefit from specialist PC. The PC-ICU score is intended as an automated, supportive tool integrated into clinical workflows to flag patients at high risk of unmet PC needs. While not a substitute for clinical assessment, it may help standardise referrals, reduce variability, and support time-efficient, team-based decision-making in the ICU.

In the PC-ICU score, each of the ten parameters is assigned up to three score points. Based on this study, a patient diagnosed with solid pulmonary cancer who was urgently admitted to the medical ICU due to an unspecified infection with a sequential organ failure assessment (SOFA) score of four would have a PC-ICU score value of eight, suggesting a high likelihood of receiving PC consultation during intensive care and thus triggering early integration of specialist PC. A prior diagnosis of metastatic cancer, an urgent/emergency hospital admission and a present limiting code status such as Do-Not-Resuscitate and Do-Not-Intubate orders (as determined prior to or during ICU admission) were strong predictors for specialist PC consultation during intensive care. Our data further indicate a strong association between home oxygen dependency and specialist PC consultation in the ICU. This is an important finding because previous trigger subsets did not include this factor. Interestingly, age and admission from a care facility, or another hospital which defined adverse admission were not selected in the score development process. The focus on cancer diagnoses aligns with previous data showing non-cancer patients receiving more intensive treatments at the end of life but are less likely to receive specialist PC, a pattern also reflected in our data.^5,36 Although the predictors were selected in a data-driven manner, the PC-ICU score remains a valuable tool for identifying PC needs in non-cancer patients, where other included variables play a key role. Our sensitivity analysis restricted to non-cancer patients confirmed good discriminatory performance of the score in this population as well.

The rate of PC in the development cohort was 5.5%. This aligns with a previously published PC prevalence in the ICU of 4.0% and the median U.S. penetration rate (initial PC consults/inpatient hospital admissions) of 5.0%.^23,41 We observed an increase in the prevalence of PC involvement throughout the study period, increasing from 5.1% in 2011 to 7.0% in 2022. In line with previous data from institutions in- and outside the U.S., we speculate that this trend reflects the rising awareness of the benefits of integrating and improving PC in the ICU.³⁹ In both validation cohorts, higher rates of PC were observed. The findings may suggest that the score performance is better when PC prevalence is lower. In addition to different utilization patterns, patient characteristics between the three cohorts differed, including age, federal insurance status, and race and ethnicity. While specialist PC utilization was seen more frequently in patients with cancer and higher among non-surgical ICU admissions in the development cohort, PC was consulted more frequently in surgical admissions and patients on extracorporeal life support and with existing limiting code status in validation cohort I. In this cohort, score evaluation yielded an AU-ROC of 0.78, allowing us to conclude successful external validation of the PC-ICU score. As PC practices are subject to substantial differences between institutions even within the U.S.,⁹ we tested the PC-ICU score in an environment (validation cohort II) with a fundamentally different outcome incidence (13.2% vs. 5.5% in the development cohort). In this cohort, PC consultation occurred less often in emergency hospital admission cases but one quarter of PC patients were readmitted to the ICU after a previous ICU stay. The PC-ICU score yielded an AU-ROC of 0.67 in this cohort, suggesting fair but significantly lower discriminative ability as in validation cohort I, likely due to the population heterogeneity and differences in specialist PC utilization. This finding highlights the impact of differences in PC practices and encourages future studies evaluating the set of variables included in the PC-ICU score and their relative importance in further external cohorts.

Specialist PC consultation served as the outcome in this study reflecting previous real-world behaviour of ICU clinicians regarding PC integration. We included patients with specialist PC requests but no conducted consultation in the outcome definition because a variety of clinical or institutional factors might hinder an actual specialist PC consultation while the need was determined by the primary care team. The ability to consult specialist PC depends on individual clinicians' attitude towards PC and knowledge about benefits and triggers, as well as the available resources. The PC-ICU score can provide evidence-based support for an individual physician's decision to involve specialist PC, and may help overcome individual reluctance by providing objective triggers for referral, fostering a more consistent and team-based approach to recognising patient needs. Importantly, clinicians should use the PC-ICU score alongside their own assessment, as it is not intended to replace clinical judgment, especially in recognising PC needs across disease groups, including non-oncological patients, as both research and clinical guidelines demonstrate and advocate for the benefits of PC integration for patients with non-malignant illness.

ICU providers responsible for patient care are often reluctant to involve PC as they fear the word “palliative” might signal patients and their family that therapeutic options have been exhausted.^42,43 Communication skills training can improve the precision of treatment information on outcome.⁴⁴ We would like to emphasise that the intention of the PC-ICU score is not to select patients where therapeutic care might be limited, but rather facilitate a multidisciplinary approach early on to improve care and experience of critically ill patients and their families. The score might facilitate informed discussions and support physicians in their decision making to allocate additional resources to patients. We think that an open discussion around integration of PC and barriers to utilization could be furthered through integration of the score.^8,16,45

Besides individual aspects, this study echoes the included institution's specialist PC referring practice. Based on these data-driven findings, qualitative investigation of how provider and institutional practices colour specialist PC utilization is warranted to combine objective and subjective measures.⁴⁶ For the PC-ICU score, investigations on how the included clinical characteristics correlate with actual specialist PC needs should be conducted next. Future prospective evaluation should focus on how score implementation might change physical, psychosocial, spiritual and structural (e.g., code status adherence, family meetings, resource utilization) outcomes.⁴⁷ In line with previous research showing that early PC integration in the ICU leads to improved patient satisfaction, decreased length of stay, more goal concordant care, and overall decreased ICU and post-ICU healthcare resource utilization,^13,14 we speculate that integrating the PC-ICU score in ICU workflows might optimise ICU care. Specifically, early PC involvement has been associated with fewer ICU readmissions, likely due to clearer goals of care, improved communication, and enhanced care coordination.^15,48 By proactively identifying patients at high need for PC, the PC-ICU score may help reduce avoidable readmissions and promote more efficient use of ICU resources.

The study aim of creating a simple and accurate tool that can be used within 24 h following ICU admission to predict specialist PC consultation during intensive care introduces inherent limitations. First, this study is limited by its retrospective design investigating previous specialist PC consultation practices with predictors available in electronic health records. Many relevant and non-physical predictors for specialist PC could not be obtained. Also, using routinely collected hospital registry data such as billing codes and administrative clinical data carries the inherent risk of unaccounted confounding by unmeasured variables and bias through missing data that were not available in electronic health records. To address these issues, we conducted a sensitivity analysis including imputation of missing data, which yielded confirmatory results. Potential temporal variations throughout the study period, particularly those related to the COVID-19 pandemic, were addressed through data exploration and sensitivity analysis. Furthermore, scoring systems inherently reflect the populations in which they were developed and tested, including cultural, religious, and socioeconomic factors, as well as the specific healthcare resources available in those settings. We used independent cohorts from three institutions across the U.S. for score development and validation, as opposed to splitting a single cohort into a development and validation population. This approach yields a reliable estimation of the PC-ICU's score performance outside the development cohort and in different geographical regions, patient populations, and healthcare settings in the U.S. Inequalities in access to PC services have been observed among marginalised groups including race, ethnicity, and age.⁴⁹ Of note, for score development we used real world data including different backgrounds of patients regarding their sex (45.3% female), marital status (e.g., 48.5% married, 29.2% single, 12.2% widowed), ethnicity and race (69.8% White, 12.4% Black, 4.5% Hispanic) and present religious denomination (70.1%). Further, the validation cohorts from different geographical regions represent heterogeneous populations. For instance, in Atlanta, GA, 44.2% of included patients were Black. We now advocate for additional prospective evaluation of the score in national and international locations, settings and patient populations (e.g., older and frail patients) to further enhance generalizability, especially in patients with varying access to healthcare services, in particular to intensive and palliative care. Another limitation is the variable availability of PC specialists in different institutions or countries, as triggers may not be applicable to systems without dedicated teams. However, by creating a short and concise version of the prediction model, the PC-ICU score helps foster a cultural shift that encourages widespread implementation of PC in the ICU, using research as a tool to guide progress. In settings with limited or no access to specialist PC, the PC-ICU score can still function as a prompt for ICU teams to recognise unmet palliative needs and initiate primary PC.

In summary, we developed and externally validated the PC-ICU score (available at: www.pc-icu.com), which includes ten parameters and predicts specialist PC consultation during intensive care early upon ICU admission with good discriminative ability. The PC-ICU score showed a better discriminative ability than other measures for PC integration, helps increase awareness for multi-disciplinary care, and may enable early integration of specialist PC. Future studies should investigate whether these clinical characteristics driving previous clinicians’ behaviour actually correlate with PC needs, and study the score in environments of different PC practices. The impact of implementing the PC-ICU score on early PC involvement and patient-centred outcomes, specifically those related to healthcare utilization and quality of life in the ICU should be explored.

Contributors

TT: conceptualization, data curation, formal analysis, funding acquisition, writing – original draft.

RH: data curation, resources, validation, writing – original draft.

EM: data curation, resources, validation, writing – original draft.

EA: conceptualization, methodology, visualization, writing – original draft.

LJW: conceptualization, formal analysis, methodology, writing – review & editing.

SRe: conceptualization, data curation, formal analysis, methodology, writing – review & editing.

SRi: formal analysis, methodology, visualization, writing – review & editing.

BSP: data curation, formal analysis, writing – review & editing.

GC: formal analysis, software, methodology, writing – review & editing.

SK: data curation, validation, resources, writing – review & editing.

JS: conceptualization, methodology, writing – review & editing.

MS: conceptualization, methodology, writing – review & editing.

RMA: methodology, resources, writing – review & editing.

KL: data curation, methodology, resources, writing – review & editing.

MN: conceptualization, resources, supervision, writing – review & editing.

MSS: conceptualization, project administration, resources, supervision, writing – original draft.

TT and MSS have accessed and verified the data, and were responsible for the decision to submit the manuscript.

Data sharing statement

Due to the sensitive nature of the data collected for this study, requests to access the dataset from qualified researchers trained in human subject research and confidentiality may be sent to Maximilian S. Schaefer at msschaef@bidmc.harvard.edu.

Declaration of interests

TT was funded by the German Research Foundation to conduct this work (Walter Benjamin Fellowship, project number: 522518834). TT received travel support to attend conferences through a DAAD (German Academic Exchange Service, Deutscher Akademischer Austauschdienst) congress stipend and a HeRa (Heine Research Academies) congress travel grant. EA is an associate editor for BMC Anesthesiology. LJW is an associate editor for BMC Anesthesiology and received funding for an investigator-initiated study from Merck & Co., which does not pertain to this manuscript. MSS received an unrestricted philanthropic grant from Jeffrey and Judith Buzen. MSS is an associate editor for BMC Anesthesiology. He received honoraria for lectures from Fisher & Paykel Healthcare and Mindray Medical International Limited. MSS received funding for investigator-initiated studies from Merck & Co., which do not pertain to this manuscript. All funders had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

RH, EM, SRe, SRi, BSP, CG, SK, JS, MS, RMA, KL and MN declare no competing interests.