Abstract
OBJECTIVES
To develop a scoring system model that predicts mortality within 30 days of admission of patients older than 80 years admitted to intensive care units (ICUs).
DESIGN
Prospective cohort study.
SETTING
A total of 306 ICUs from 24 European countries.
PARTICIPANTS
Older adults admitted to European ICUs (N = 3730; median age = 84 years [interquartile range = 81‐87 y]; 51.8% male).
MEASUREMENTS
Overall, 24 variables available during ICU admission were included as potential predictive variables. Multivariable logistic regression was used to identify independent predictors of 30‐day mortality. Model sensitivity, specificity, and accuracy were evaluated with receiver operating characteristic curves.
RESULTS
The 30‐day‐mortality was 1562 (41.9%). In multivariable analysis, these variables were selected as independent predictors of mortality: age, sex, ICU admission diagnosis, Clinical Frailty Scale, Sequential Organ Failure Score, invasive mechanical ventilation, and renal replacement therapy. The discrimination, accuracy, and calibration of the model were good: the area under the curve for a score of 10 or higher was .80, and the Brier score was .18. At a cut point of 10 or higher (75% of all patients), the model predicts 30‐day mortality in 91.1% of all patients who die.
CONCLUSION
A predictive model of cumulative events predicts 30‐day mortality in patients older than 80 years admitted to ICUs. Future studies should include other potential predictor variables including functional status, presence of advance care plans, and assessment of each patient's decision‐making capacity.
Keywords: critical care, prognosis, older adults, predict, model
More than 10% of the patients admitted to the intensive care unit (ICU) are 80 years and older.1 This proportion of “very old intensive care patients” (VIPs) is estimated to increase up to 36% in 2025.2, 3 However, once VIPs have been admitted to the ICU for an acute medical reason, their overall 30‐day mortality is high. The frailer patients in particular have a high mortality.4 Despite careful patient selection before ICU admission, more than half of these VIPs will die or will experience major functional deterioration in the 6 months following their admission.5, 6
As a result of our current uncertainty in predicting which VIPs could potentially benefit from ICU treatment, we often offer them an “ICU trial.” This means admitting VIPs to an ICU, offering them life‐sustaining treatment for a period of 2 to 3 days, and then reevaluating if they show any improvement. If patients deteriorate, limitations in life‐sustaining therapies would be required.7 For ICU physicians, this ICU trial postpones the difficult ICU admission triage decision by a few days, and by then, some patients have improved. However, some will still receive life‐sustaining therapy, and a decision to continue treatment should be discussed with the patient or his or her legal representatives.8 Inevitably, during such shared decision‐making processes, the question of chances of survival emerges. Most intensivists estimate a patient's chances of outcome on experience and on preferences. Current severity scoring systems are not tailored for VIPs,1 and proposed models for VIP are not precise enough.9, 10
We hypothesize that a cumulative prognostic score can predict 30‐day mortality and thus support physicians and relatives with the decision to continue care or start a new treatment.
METHODS
We present only a very brief discussion of methods here. A more elaborate description of our methods, adhering to all Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statements,11 is provided in Supplementary Materials S1.
In short, all patients older than 80 years who were acutely admitted to participating ICUs in 24 European countries were included in this study.4, 7 The mortality 30 days after ICU admission was the primary outcome.
Based on variables present at admission (eg, age, sex, reason for ICU admission, the abbreviated Clinical Frailty Scale)12 or treatments provided during ICU stay (eg, worst Sequential Organ Failure Assessment [SOFA], (non‐)invasive mechanical ventilation, use of vasoactive drugs, renal replacement therapy [RRT]), a multivariable logistic regression model was constructed. The discrimination, accuracy, and calibration of the model were assessed13, 14 before a simple bedside model was constructed. This simple bedside model was based on the beta of each predictor in the model as described previously.15
The total number of points assigned to each patient is called the cumulative prognostic score (CPS) and correlates with 30‐day mortality. The performance of the bedside model (the sensitivity and specificity at several cutoff points of the CPS) is assessed.
RESULTS
Participants
In total, 306 ICUs from 24 countries participated and included 5187 VIPs. Of these patients, 4252 were acutely admitted. Follow‐up at 30 days with complete data on all variables was obtained in 88% (3730/4252). The median number of patients recruited per country and per ICU was 104 and 12, respectively. Demographics of the patients included in the final analyses are presented in Table 1.
Table 1.
Demographics of Included Nonelective Patients
| No. of patients | % | |
|---|---|---|
| Total | 3730 | 100 |
| ICU mortality | 1065 | 28.6 |
| 30‐day mortality | 1562 | 41.9 |
| Male | 1932 | 51.8 |
| Age, y, median (25th percentile‐75th percentile) | 84 (81‐87) | ‐ |
| CFS, median (25th percentile‐75th percentile) | 4 (3‐6) | ‐ |
| SOFA score, median (25th percentile‐75th percentile) | 7 (4‐11) | ‐ |
| Intubation and mechanical ventilation | 1924 | 51.6 |
| Vasoactive drugs | 2155 | 57.8 |
| NIV | 984 | 26.4 |
| RRT | 405 | 10.9 |
| Reason for ICU admission | ||
| Respiratory failure | 900 | 24.1 |
| Circulatory failure | 537 | 14.4 |
| Respiratory and circulatory failure | 448 | 12 |
| Sepsis | 480 | 12.9 |
| Multi‐trauma without head injury | 55 | 1.5 |
| Multi‐trauma with head injury | 57 | 1.5 |
| Head injury | 110 | 2.9 |
| Intoxication | 13 | 0.3 |
| Nontrauma | 293 | 7.9 |
| Emergency surgery | 379 | 10.2 |
| Other | 458 | 12.3 |
Abbreviations: CSF, Clinical Frailty Scale; ICU, intensive care unit; NIV, noninvasive ventilation; RRT, renal replacement therapy; SOFA, Sequential Organ Failure Assessment.
Multivariable Logistic Regression Model
No multicollinearity was found between the variables available for model development, and there was no interaction between age and the Clinical Frailty Scale. During model development, the following variables were selected using the Lasso procedure: age, sex, reason for ICU admission categorized into 11 options (Supplementary Materials S1), vasoactive drugs, Clinical Frailty Scale, SOFA score, intubation with mechanical ventilation, and RRT. The continuous variable SOFA score was included as restricted cubic spline. The final regression model is presented in Table 2, where the SOFA score for simplification is presented as categorical data instead of splines.
Table 2.
Final Multivariable Regression Model
| Covariate | Odds ratio 30‐day mortality |
|---|---|
| Age | 1.05 (1.03‐1.07) |
| Sex, male | 1.26 (1.09‐1.47) |
| Mechanical ventilation | 2.07 (1.72‐2.50) |
| RRT | 1.70 (1.33‐2.18) |
| CFS | 1.19 (1.14‐1.25) |
| SOFA score | |
| <4 | Reference |
| ≥4 and <7 | 1.80 (1.37‐2.37) |
| ≥7 and <10 | 2.70 (2.02‐3.60) |
| ≥10 | 4.40 (3.26‐5.92) |
| Reason for ICU admission | |
| Respiratory failure | 3.58 (.73‐17.48) |
| Circulatory failure | 3.74 (.76‐18.39) |
| Respiratory and circulatory failure | 4.55 (.92‐22.43) |
| Sepsis | 3.20 (.65‐15.77) |
| Multi‐trauma without head injury | 3.62 (.66‐19.77) |
| Multi‐trauma with head injury | 5.06 (.94‐27.23) |
| Isolated head Injury | 4.30 (.84‐22.09) |
| Intoxication | Reference |
| Nontrauma CNS causes | 5.45 (1.10‐27.01) |
| Emergency surgery | 1.87 (.38‐9.27) |
| Other | 2.0 (.40‐9.88) |
Abbreviations: CFS, Clinical Frailty Scale; CNS, central nervous system; ICU, intensive care unit; RRT, renal replacement therapy; SOFA, Sequential Organ Failure Assessment.
The overall discrimination and accuracy of the final regression model that predicts mortality within 30 days after ICU admission was good; the area under the curve (AUC) was .80 (.80‐.81). The Brier score was .18 (.18‐.18). In the different diagnostic subgroups, the performance of the model was also good (Supplementary Materials S2). The calibration belt of the sepsis patients showed more uncertainty for the low‐ and high‐risk patients; for the emergency surgery patients, the model is not able to predict high mortality risks. However, the calibration belt of the total population and all other diagnostic subgroups showed no abnormalities (Supplementary Materials S3).
“Bedside” Model
The “bedside” model, based on a point system (Supplementary Materials S4), shows parameters that influence mortality 30 days after ICU admission and the weight assigned to these variables. In theory, the minimum and maximum scores a patient can obtain are 0 and 26 points, respectively. The minimum and maximum scores obtained in the study population were 1 and 26, respectively. The reason for ICU admission and the SOFA score were the two most important factors associated with 30‐day mortality. Among the reasons for ICU admission, multi‐trauma with head injury and nontrauma central nervous system causes were associated with a high 30‐day mortality.
The sensitivity and specificity for various thresholds is based on the assigned number of points in the prediction model (ie, CPS of the patients) (Figure 1). When all patients with a 30‐day CPS higher than 10 points are selected (corresponding to 76.0% of all patients), 91.8% of all patients who died during the 30 days after ICU admission are captured (sensitivity). Of these 76.0% selected patients, 50.6% will die during the 30 days after ICU admission (positive predicted value).
Figure 1.
Sensitivity and specificity of the simple bedside model for 30‐day mortality. When a patient has >10 points on the Cumulative Prognostic Score (CPS), that patient has a 50.6% positive predictive value (PPV) on 30‐day mortality and a negative predictive value (NPV) of 85.8%. The other values are as follows: at >12 points, a PPV of 55.9% and an NPV of 81.1%; at >16 points, a PPV of 67.6% and an NPV of 69.2%; and at >20 points, a PPV of 77.0% and an NPV of 60.5%.
Supplementary Materials S5 lists the characteristics of the patients with a 30‐day CPS higher than 10 points and the patients with a 30‐day mortality score higher, lower, or equal to 10 points.
DISCUSSION
This study demonstrates that a cumulative event model correlates with outcome in acutely admitted ICU patients older than 80 years. This model is based on reason for ICU admission, age, sex, frailty, SOFA score (all available at admission), and the organ support during ICU stay: invasive mechanical ventilation and RRT. Although this model discriminates between those dying and surviving 30 days after ICU admission rather accurately (AUC = .80), it remains difficult to predict which patients are going to die with 100% certainty. Even if we use a high cutoff (eg, >19 points), these patients still have a 25% chance of survival in the first 30 days after ICU admission.
Previous research identified variables associated with a poor outcome in this very old patient group: age,1, 4 sex,4, 9 mechanical ventilation,16 circulatory shock,17 acute kidney injury,18 and the presence of comorbidities.19 However, up to now, only a few studies have tried to build a model from these variables to predict outcome after ICU admission. Ball et al developed a model based on age, serum creatinine, Glasgow Coma Scale, and pH.10 The data in that study were collected during the first day of ICU admission and did not include the response to active treatment for which the ICU trial is intended. Heyland et al made a model to predict functional outcome 1 year after ICU admission.9 Although that model had a good performance (AUC = .81), the selected variables were less easily collected at the bedside because they were derived from more complex scores, for example, the Acute Physiology, Age, Chronic Health Evaluation II score and the Charlson Comorbidity Index.
It is very difficult to predict outcome in acutely ill patients older than 80 years, and many are admitted to the ICU for an ICU trial. However, such an admission should be reevaluated in a shared decision‐making conference with the patient or, when the patients lack decision‐making capacity, with the family or designated surrogate. Our model can be used to estimate, in a more objective way than just subjective clinical intuition, what the chances of 30‐day mortality will be. We believe this information can help both the intensivist and the family members to put treatment into perspective.20 Indeed, although almost all intensivists claim that they value the opinions of surrogates (eg, relatives, family, legal representatives, and caregivers),21 these family opinions on ICU admissions are, in reality, rarely sought.8, 22 One of the reasons why this is omitted is the uncertainty that intensivists feel during the prognostication of patients.22, 23, 24, 25
And yet, a poor prognosis is one of the most important reasons to implement limitations in life‐sustaining therapies.26, 27 Indeed, family members reported that what was most important to them was that the “patient should be comfortable and suffer as little as possible.” The belief that “life should be preserved at all costs” was their least important value considered in making treatment decisions. A substantial proportion of the caregivers (24%) reported that “comfort care without life support” was their most preferred treatment goal, but 14% were “unsure about their treatment preferences.”28 Caregivers and surrogates who had a shared decision‐making conference with their intensivist experienced less decisional conflicts than family members who had not talked to a physician.28 Strikingly, despite family members only prioritizing “comfort measures,” 83.7% of these patients still received life‐sustaining treatments, and approximately 20% received such treatments for more than a week.28 Our bedside model for 30‐day mortality can assist both intensivists and family members in the decision‐making process to continue or cease further treatment.
The strong feature of this study is the international inclusion of more than 3700 very old patients and the high follow‐up rate at 30 days after ICU admission. However, some limitations need to be discussed. First, we did not collect data on the timing of discussions to withhold or withdraw life‐sustaining treatment. For example, some patients had advanced directives not to instigate certain treatments that might have influenced outcome. A second limitation of the model is that it does not predict patients who will have a definite poor outcome (100% mortality). The predicted mortality at more than 19 points is 75%, but this means that 25% of those patients do survive the first 30 days. A longer follow‐up will undoubtedly show there is substantial additional mortality in these very old patients. For this reason a new study (the so‐called Very old Intensive care Patient study 2 [VIP2]) will look at mortality at day 180 after ICU admission. Third, we combined data from admission with treatment data from the subsequent treatment days. This simplifies and strengthens prognostication but prevents the model being used for triage purposes before admission. Fourth, we only looked at survival, although we know that many patients do not fully recover, and many older adult patients prioritize “quality of life” above “quantity of life.” Future research should focus on functional outcome and quality of life. And, finally, we did not include data on nutritional status, functional status (activities of daily living and instrumental activities of daily living), cognitive impairment, dementia, delirium, depression, and comorbidities (eg, active cancer). These variables will undoubtedly influence outcome. Our CPS can become more comprehensive if such variables are included.
In conclusion, this relatively simple cumulative events model can help assess the chances of 30‐day mortality in very old patients who were acutely admitted to the ICU. This model may assist both intensivists and family members during the shared decision‐making process to estimate the otherwise subjective chances of survival.
Supporting information
Supplementary Appendix S1. Supplementary Materials.
Supplementary Materials S1. Expanded methods.
Supplementary Materials S2. Performance of multivariate logistic regression model.
Supplementary Materials S3. GiViTi calibration belt for the developed prediction model.
Supplementary Materials S4. Prediction model based on a point system.
Supplementary Materials S5. Demographics for patients with ≤10 and >10 Cumulative Prognostic Score.
Supplementary Materials S6. VIP1 study contributors.
ACKNOWLEDGMENTS
This study could not be performed without the help of many contributors. These VIP1 study contributors are listed in Supplementary Materials S6.
Conflict of Interest: The authors have declared no conflicts of interest for this article.
Author Contributions: All authors contributed to obtaining ethical clearance for their countries, including patients, and corrected the final draft of the manuscript. Statistical analyses: Brinkman, Soliman, Bertolini, and Boumendil. Writing the first draft: de Lange.
Sponsor's Role: There was no sponsor for this article. The Society of Intensive Care Medicine (ESICM) endorsed the study and was awarded a scientific prize.
[Corrections added May 20, 2019, after first online publication. The author name “Bernadro Bollen Pinto” was corrected as “Bernardo Bollen Pinto”]
REFERENCES
- 1. Flaatten H, Artigas A, de Lange DW, et al. The status of ICM research and a future agenda for very elderly patients in the ICU. Intensive Care Med. 2017;43:1319‐1328. [DOI] [PubMed] [Google Scholar]
- 2. Laake JH, Dybwik K, Flaatten HK, Fonneland IL, Kvåle R, Strand K. Impact of the post–World War II generation on intensive care needs in Norway. Acta Anaesthesiol Scand. 2010;54(4):479‐484. [DOI] [PubMed] [Google Scholar]
- 3. Angus D. Admitting elderly patients to the intensive care unit—is it the right decision? JAMA. 2017;318:1443‐1444. [DOI] [PubMed] [Google Scholar]
- 4. Flaatten H, De Lange DW, Morandi A, et al. The impact of frailty on ICU and 30‐day mortality and the level of care in very elderly patients (≥ 80 years). Intensive Care Med. 2017;43:4940‐4948. [DOI] [PubMed] [Google Scholar]
- 5. Boumendil A, Angus DC, Guitonneau AL, et al. Variability of intensive care admission decisions for the very elderly. PLoS One. 2012;7:e34387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Guidet B, Leblanc G, Simon T, et al. Effect of systematic intensive care unit triage on long‐term mortality among critically ill elderly patients in France: a randomized clinical trial. JAMA. 2017;318:1450‐1459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Guidet B, Flaatten H, Boumendil A, et al. Withholding or withdrawing of life sustaining therapy in very elderly patients (≥ 80 years) admitted to the intensive care unit. Intensive Care Med. 2018;44:1027‐1038. [DOI] [PubMed] [Google Scholar]
- 8. Le Guen J, Boumendil A, Guidet B, Corvol A, Saint‐Jean O, Somme D. Are elderly patients’ opinions sought before admission to an intensive care unit? Results of the ICE‐CUB study. Age Ageing. 2016;45:303‐309. [DOI] [PubMed] [Google Scholar]
- 9. Heyland DK, Stelfox HT, Garland A, et al. Predicting performance status 1 year after critical illness in patients 80 years or older: development of a multivariable clinical prediction model. Crit Care Med. 2016;44:1718‐1726. [DOI] [PubMed] [Google Scholar]
- 10. Ball IM, Bagshaw SM, Burns KE, et al. A clinical prediction tool for hospital mortality in critically ill elderly patients. J Crit Care. 2016;35:206‐212. [DOI] [PubMed] [Google Scholar]
- 11. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594. [DOI] [PubMed] [Google Scholar]
- 12. Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005;173:489‐495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Steyerberg EW, Bleeker SE, Moll H, et al. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003;56:441‐447. [DOI] [PubMed] [Google Scholar]
- 14. Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G. Calibration belt for quality‐of‐care assessment based on dichotomous outcomes. PLoS One. 2011;6:e16110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Brandenburg R, Brinkman S, de Keizer NF, Kesecioglu J, Meulenbelt J, de Lange DW. The need for ICU admission in intoxicated patients: a prediction model. Clin Toxicol (Phila). 2017;55:4‐11. [DOI] [PubMed] [Google Scholar]
- 16. Schortgen F, Follin A, Piccari L, et al. Results of noninvasive ventilation in very old patients. Ann Intensive Care. 2012;21:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Biston P, Aldecoa C, Devriendt J, et al. Outcome of elderly patients with circulatory failure. Intensive Care Med. 2014;40:50‐56. [DOI] [PubMed] [Google Scholar]
- 18. Chronopoulos A, Rosner MH, Cruz DN, Ronco C, et al. Acute kidney injury in elderly intensive care patients: a review. Intensive Care Med. 2010;36:1454‐1464. [DOI] [PubMed] [Google Scholar]
- 19. Flaatten H, Oeyen S, de Lange DW. Predicting outcomes in very old ICU patients: time to focus on the past? Intensive Care Med. 2018;44:1344‐1345. [DOI] [PubMed] [Google Scholar]
- 20. Guidet B, de Lange DW, Flaatten H. Should this elderly patient be admitted to the ICU? Intensive Care Med. 2018;44:1926‐1928. [DOI] [PubMed] [Google Scholar]
- 21. Guidet B, De Lange DW, Christensen S, et al. Attitudes of physicians towards the care of critically ill elderly patients—a European survey. Acta Anaesthesiol Scand. 2018;62:207‐219. [DOI] [PubMed] [Google Scholar]
- 22. Heyland DK, Barwich D, Pichora D, et al. Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med. 2013;173:778‐787. [DOI] [PubMed] [Google Scholar]
- 23. Teno JM, Fisher E, Hamel MB, et al. Decision‐making and outcomes of prolonged ICU stays in seriously ill patients. J Am Geriatr Soc. 2000;48(S1):S70‐S74. [DOI] [PubMed] [Google Scholar]
- 24. Sinuff T, Adhikari NK, Cook DJ, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med. 2006;34:878‐885. [DOI] [PubMed] [Google Scholar]
- 25. Soliman IW, Cremer OL, de Lange DW, et al. The ability of intensive care unit physicians to estimate long‐term prognosis in survivors of critical illness. J Crit Care. 2018;43:148‐155. [DOI] [PubMed] [Google Scholar]
- 26. Hoel H, Skjaker SA, Haagensen R, Stavem K. Decisions to withhold or withdraw life‐sustaining treatment in a Norwegian intensive care unit. Acta Anaesthesiol Scand. 2014;58(3):329‐336. [DOI] [PubMed] [Google Scholar]
- 27. Keenan SP, Busche KD, Chen LM, McCarthy L, Inman KJ, Sibbald WJ. A retrospective review of a large cohort of patients undergoing the process of withholding or withdrawal of life support. Crit Care Med. 1997;25:1324‐1331. [DOI] [PubMed] [Google Scholar]
- 28. Heyland DK, Dodek P, Mehta S, et al. Admission of the very elderly to the intensive care unit: family members’ perspectives on clinical decision‐making from a multicenter cohort study. Palliat Med. 2015;29:324‐335. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Appendix S1. Supplementary Materials.
Supplementary Materials S1. Expanded methods.
Supplementary Materials S2. Performance of multivariate logistic regression model.
Supplementary Materials S3. GiViTi calibration belt for the developed prediction model.
Supplementary Materials S4. Prediction model based on a point system.
Supplementary Materials S5. Demographics for patients with ≤10 and >10 Cumulative Prognostic Score.
Supplementary Materials S6. VIP1 study contributors.