Abstract
How to cite this article: Siddiqui SS, Patnaik R, Kulkarni AP. General Severity of Illness Scoring Systems and COVID-19 Mortality Predictions: Is “Old Still Gold?” Indian J Crit Care Med 2022;26(4):416–418.
Keywords: COVID-19 mortality, Critically ill adults, Predictive model
Severity of illness scoring systems in intensive care unit (ICU) are used to assess the disease severity, prognostication and mortality prediction, and with some systems prediction of length of stay and better resource allocation. An ideal scoring system should have easy to record variables, should be well calibrated, have high level of discrimination, and be applicable to all critically ill patients.1 Widely used general severity of illness scoring systems for critically ill patients include various iterations of mortality prediction model (MPM), acute physiology and chronic health evaluation (APACHE), and simplified acute physiology score (SAPS). Coronavirus disease-2019 (COVID-19)pandemic spreads globally in a short time. About 5–10% of COVID-19 patients with severe pneumonia require ICU admission for ventilatory support.2 The heterogeneous nature of COVID-19 with pulmonary and extrapulmonary manifestations makes it an enigma for the medical fraternity. Furthermore, long-COVID or post-COVID syndrome is another distinct entity occurring in a minority of patients. These factors make it difficult to predict outcomes in critically ill COVID-19 patients. Any scoring system used to predict mortality in COVID-19 should incorporate parameters of all relevant organ systems.
The study by Asmarawati et al. published in this issue of the journal compared the utility of sequential organ failure assessment (SOFA), quick sequential organ failure assessment (qSOFA), acute physiology and chronic health evaluation (APACHE II), and new early warning signs (NEWS-2) scores estimate mortality of COVID-19 patients requiring intensive care unit (ICU).3 We would like to underscore a few important observations about this study. Firstly, the 4 scoring systems used in the study are of diverse genre, some of them being used primarily as triaging/screening tools in high workload areas, in-patient wards (qSOFA and NEWS-2) to define the level of care, while SOFA is used to define the various organ dysfunction/failure, while APACHE II takes other factors also into account, like age, comorbid illnesses apart from physiological disturbances (Table 1).
Table 1.
Comparison of scoring systems used in the study
| qSOFA | NEWS-2 | SOFA | APACHE II | |
|---|---|---|---|---|
| Year of publication | 2016 | 2017 | 1994 | 1985 |
| Origin (patient population) | USA | UK | Europe | USA |
| Abbreviation expansion | Quick sequential organ failure assessment | National early warning score | Sequential organ failure assessment | Acute physiology, age, and chronic health evaluation |
| Total variables | 3 | 8 | 6 | 17 |
| Range of score | 0–3 | 0–23 | 0–24 | 0–71 |
| Prime utility locations | ED/Ward/HDU | ED/Ward/HDU | ICU | ICU |
| Variables/parameters | Clinical | Clinical | Clinical and investigational | History (age, comorbidities), clinical and investigational |
| Time taken for calculation | Fast | Fast | Medium | Prolonged time |
| Serial monitoring | Yes | Yes | Yes | Not much used |
| Remarks | Identify patients with suspected infection who are at greater risk for a poor outcome outside the intensive care unit (ICU) | Early warning system for identifying acutely ill patients Used mainly in the UK |
Worst values over 24 hours are used Includes a treatment-related variable (dose of vasopressor agent) |
Worst values over 24 hours are used Not validated for children (aged <16 years), patients with an ICU stay of less than 8 hours, or those who are readmitted to the unit within the same hospital admission Several patient populations like burn and CABG patients were not included in the original study of designing the score |
ED, emergency department; HDU, high-dependency unit; CABG, coronary artery bypass grafting; ICU, intensive care unit
Notwithstanding the differences in the scoring systems, the authors found similar discrimination by all scoring systems with similar area under the receiver operating characteristics (AUROC) curves on day 0 and day 5. The ability of a prognostic index to predict an outcome (e.g. inhospital mortality) is evaluated based on its calibration and discrimination.4 Calibration refers to the relation between predicted mortality and observed mortality. The calibration of a prognostic model generally deteriorates over time due to changes in ICU admission and discharge criteria, the evolution of intensive care support, and medical advancement. In contrast, discrimination refers to the ability of a prognostic index to differentiate between patients who will and will not survive. This metric is based on the AUROC curve, with a larger area indicative of greater accuracy.
Secondly, while serial use of qSOFA, NEWS-2, and SOFA scores have been well documented, the application of daily APACHE II scores to predict patient survival rate is not used often.5 A recent study assessing the utility of dynamic APACHE II score to predict outcome of ICU patients concluded that APACHE II score on day 3 of ICU admission is an optimal predictor of outcomes in ICU patients.6 However, in the current study, APACHE II score on days 0 and 5 was used for mortality prediction; the basis of using day 5 score has not been given by the authors. Though, both the studies are different in terms of their patient population, geographic region, it remains largely speculative which is the optimal day to calculate APACHE II for the best prediction of mortality in COVID-19 patients.
Thirdly, although the study incorporated consecutive sampling, majority were critically ill COVID-19 patients (54.7%) with preponderance of acute respiratory distress syndrome (ARDS) patients (50.9 %) and consequent high mortality (43.4 %).
Lastly, all these 4 scoring systems used in this study involve parameters of multiple organs; however, none of them incorporate any parameter representing immune system. COVID-19 manifestations result from the release of various cytokines, including interleukin (IL)-1 IL-6, IL-12, IL-18, IL-33, interferon (IFN)-gamma, IFN-alpha, tumor necrosis factor (TNF)-alpha, and transforming growth factor (TGF)-beta. The elevation in the neutrophil-to-lymphocyte ratio (NLR) is related to immune disorders in patients with COVID-19. In fact, secondary hemophagocytic lymphohistiocytosis after COVID-19 infection has also been reported. Thus, the key role of the immune system cannot be overemphasized in this context. This is especially important considering the fact that most of the treatments involve immunomodulation. This includes steroids that are the sine qua non for treating severe COVID-19 infections. Not involving the immune system as part of the prediction model for COVID-19 by the authors is probably a missed opportunity.
With the continued rage of COVID-19 pandemic, several bespoke scores for COVID-19 prognostication have also been developed, like Coronavirus Clinical Characterization Consortium (4C) mortality score, COVID-GRAM Critical Illness Risk Score, and CANPT score.7–9 These COVID-specific prognostic scores are different from other prognostic scores as they also incorporate one aspect of the immune system: for example, 4C mortality score has C-reactive protein; COVID-GRAM and CANPT scores have NLR. Whether incorporation of these parameters will improve their mortality prediction requires larger validation studies and also comparison with the currently available general scoring systems.
Mortality prediction with general scoring systems in a specific disease like COVID-19 may have certain other pitfalls also. Prognostication using the general scoring systems on being applied to a new group of patient population, affected by a new disease, whose pathophysiology largely remains unknown and no definite therapy, is difficult. Additionally, the presence of different strains, likely to need varied approach of management in different parts of the world would confound the ability to predict outcomes. In other words, if a model fits adequately in the first wave of COVID-19, it may not be the appropriate mortality prediction model later as the treatment and disease both evolve.10
In the present era of electronic medical record, machine-learning-based methods to predict inhospital mortality can reduce the predicament of varied clinical phenotypes. Linden et al. have shown that this remains to be the future endeavor.11
In conclusion, the study findings demonstrate the utility of general severity of illness scoring systems for prediction of mortality in moderate-to-severe COVID-19. Finding unity in diverse manifestations of COVID-19 with general mortality prediction models seems to have a role. Thus, the idiom “old is gold” in this context still holds true, till we get a better scoring system.
Orcid
Suhail S Siddiqui https://orcid.org/0000-0001-7109-0566
Rohit Patnaik https://orcid.org/0000-0002-2321-0743
Atul P Kulkarni https://orcid.org/0000-0002-5172-7619
Footnotes
Source of support: Nil
Conflict of interest: None
References
- 1.Bouch CD, Thompson JP. Severity scoring systems in the critically ill. Contin Educ Anaesth Crit Care Pain. 2008;8(5):181–185. doi: 10.1093/bjaceaccp/mkn033. [DOI] [Google Scholar]
- 2.Guan WJ, Ni ZY, Hu Y, Liang WH, Ou CQ, He JX, et al. Clinical characteristics of coronavirus disease 2019 in China. New Engl J Med. 2020;382(18):1708–1720. doi: 10.1056/NEJMoa2002032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Asmarawati TP, Suryantoro SD, Rosyid AN, Marfiani E, Windradi C, Mahdi BA, et al. Predictive Value of Sequential Organ Failure Assessment (SOFA), Quick Sequential Organ Failure Assessment (qSOFA), Acute Physiology and Chronic Health Evaluation (APACHE II), and New Early Warning Signs (NEWS-2) Scores Estimate Mortality of COVID-19 Patients Requiring Intensive Care Unit (ICU). Indian J Crit Care Med. 2022;26(4):464–471. doi: 10.5005/jp-journals-10071-24170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Breslow MJ, Badawi O. Severity scoring in the critically ill: part 1- Interpretation and accuracy of outcome prediction scoring systems. Chest. 2012;141(1):245–252. doi: 10.1378/chest.11-0330. [DOI] [PubMed] [Google Scholar]
- 5.Rogers J, Fuller HD. Use of daily Acute Physiology and Chronic Health Evaluation (APACHE) II scores to predict individual patient survival rate. Crit Care Med. 1994;22(9):1402–1405. doi: 10.1097/00003246-199409000-00008. [DOI] [PubMed] [Google Scholar]
- 6.Tian Y, Yang YA, Zhou J, Diao X, Chen H, Cai K, et al. Dynamic APACHE II score to predict outcome among intensive care unit patients. Front Med. 2021;8:744907. doi: 10.3389/fmed.2021.744907. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Knight SR, Ho A, Pius R, Buchan I, Carson G, Drake TM, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO clinical characterisation protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339. doi: 10.1136/bmj.m3339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. 2020;180(8):1081–1089. doi: 10.1001/jamainternmed.2020.2033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Chen Y, Zhou X, Yan H, Huang H, Li S, Jiang Z, et al. CANPT score: a tool to predict severe COVID-19 on admission. Front Med. 2021:68. doi: 10.3389/fmed.2021.608107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sperrin M, McMillan B. Prediction models for covid-19 outcomes. BMJ. 2020;20:371. doi: 10.1136/bmj.m3777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Linden T, Hanses F, Domingo-Fernández D, DeLong LN, Kodamullil AT, Schneider J, et al. Machine learning based prediction of COVID-19 mortality suggests repositioning of anticancer drug for treating severe cases. Artif Intell Life Sci. 2021;17:100020. doi: 10.1101/2021.11.11.21266048. [DOI] [PMC free article] [PubMed] [Google Scholar]