Abstract
Background: Whether combining the Sequential Organ Failure Assessment (SOFA) score at admission with inflammation-based markers can improve performance of prediction and risk stratification of patients with sepsis, compared to use of the SOFA score alone, remains unknown. Methods: Data from septic patients included in the Medical Information Mart for Intensive Care database (MIMIC-IV) database were used for model development and internal validation. We developed a predictive nomogram model that included SOFA score, Charlson Comorbidity Index (CCI), red cell distribution width (RDW), neutrophil-to-lymphocyte ratio (NLR), lymphocyte-to-monocyte ratio (LMR), and mean corpuscular volume (MCV) values. The primary outcome was the performance of the risk score. Results: Data from 4704 septic patients included in the database were used for the primary cohort and to build the model. The multivariate analyses included SOFA score, CCI, RDW, NLR, LMR, and MCV values. These values were used for nomogram model construction. The nomogram model showed good calibration, and had better discrimination in terms of area under the receiver operating characteristic (AUROC) curve results than use of the SOFA score alone (0.724 (95% CI: 0.705-0.743) vs. 0.585 (95% CI: 0.562-0.609), respectively; P<0.001). It also had better classification in terms of net reclassification improvement (20.5% (95% CI: 16.2%-24.7%; P<0.001)) and integrated discrimination improvement (6.0% (95% CI: 5.1%-6.8%; P<0.001)). The validation cohort results supported these findings. Conclusion: The results suggested that this simple-to-use nomogram model provided a relatively accurate risk of death prediction in patients with sepsis.
Keywords: Sequential organ failure assessment score, red cell distribution width, neutrophil-to-lymphocyte ratio, lymphocyte-to-monocyte ratio, mean corpuscular volume, nomogram, sepsis
Introduction
Sepsis is characterized by systemic inflammation and acute organ dysfunction due to infection [1], The short-term mortality rate is relatively high (30%-50%) [2]. Identifying patients with sepsis in the intensive care unit (ICU) who are at an increased risk of death during the early phase of sepsis can assist with development and provision of timely and adequate interventions [3].
Risk prediction systems are used in the ICU. These systems include use of underlying medical comorbidities and physiologic data to estimate patient mortality and outcomes. The commonly usedSequential Organ Failure Assessment (SOFA) score [4] consists of six distinct organ dysfunction measures. Its use over time to assess illness severity and predict future organ dysfunction and mortality in patients who are critically ill has been validated [5].
SOFA score values are highly reliable and routinely available for prediction and risk stratification of patients with sepsis. However, no components of the score include the contribution of the dysregulated systemic inflammation associated with disease progression during sepsis [6]. Use of inflammation-based markers (e.g., red cell distribution width (RDW) [7], neutrophil-to-lymphocyte ratio (NLR) [8], lymphocyte-to-monocyte ratio (LMR), and mean corpuscular volume (MCV) [9]) to identify critically ill patients at risk of death from sepsis has been examined. It remains unknown whether combining the SOFA score recorded at admission with these inflammation-based markers improves performance of prediction and risk stratification of patients with sepsis, compared to the SOFA score alone.
In this study, we examined a nomogram model that combined use of the SOFA score at admission with RDW, NLR, LMR, and MCV values. We also investigated whether the new model had better performance for prediction and risk stratification in patients with sepsis than SOFA only.
Materials and methods
Study design
We performed a retrospective analysis of data from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database [10]. These high-quality, comprehensive ICU patient data are third-party anonymized and publicly available. The patients were admitted to ICUs at the Beth Israel Deaconess Medical Center (2008-2019). The Beth Israel Deaconess Medical Center and Massachusetts Institute of Technology institutional review boards (IRBs) approved use of the database (certification number 31723135). IRB approval from our institution was not required because pre-existing IRB approval applied.
Study population
Data associated with septic patients were eligible for inclusion in the analysis [1]. A sepsis diagnosis in MIMIC-IV was consistent with the third sepsis definition. Specifically, data from patients with clinically confirmed or suspected infection and a ≥2-point acute SOFA score change were screened for inclusion in the analysis [1]. The presence of infection was identified according to ICD-9 and ICD-10 codes. Only patients with a post-24 h ICU admission sepsis onset were included in the analysis. The definition of sepsis onset in the MIMIC-IV database is discussed elsewhere. The exclusion criteria were: (1) patient age <18 years; (2) an ICU stay <24 h; (3) an admission with outliers present (values greater than the mean ± three times the standard deviation). If a patient was admitted to the ICU more than once, only data associated with the first ICU stay were analyzed. Patients who were not excluded were randomly assigned to a primary cohort group or an internal validation cohort group at a 7:3 ratio, respectively.
Data extraction
A structured query programming language approach was used to extract the data. Baseline characteristics for the 24 h period immediately post-ICU admission were extracted: age, gender, admission type, Charlson Comorbidity Index (CCI), severity at admission measured by SOFA score, and Simplified Acute Physiology Score II (SAPS II), use of mechanical ventilation, renal replacement therapy (RRT), and vasopressor administration. Vital sign data (e.g., mean arterial pressure (MAP), heart rate, temperature (°C), and respiratory rate) were also extracted. Clinical pathology data for hemoglobin concentration, white blood cell, neutrophil, lymphocyte, and platelet counts, RDW, MCV, pH, lactate concentration, and PaO2/FiO2 were also extracted. Neutrophil count divided by the lymphocyte count was used to calculate NLR. Lymphocyte count divided by the monocyte count was used to calculate LMR. The value associated with the greatest severity was used if a variable was recorded more than once during the initial 24-h period. The primary outcome variable for the analysis was the performance of the risk score.
Statistical analysis
Results for continuous variables were presented as mean (standard deviation) or median (interquartile range) values and were compared using the Student’s t-test for variables with data that met the normal distribution assumption. Mann-Whitney U tests were used for the analysis of variables with data that did not meet the normal distribution assumption. Categorical variables were described using proportions. Chi-square test or Fisher’s exact test were used for between-group comparisons.
Uni- and multivariate analyses were used to examine the associations of SOFA score, CCI, NLR, LMR, MCV, and RDW with in-hospital mortality in the primary model. Model overfitting error was controlled using a backward step-down process based on the Akaike information criterion. A nomogram was constructed based on the results of a logistic regression analysis. Nomogram accuracy and validity were evaluated based on calibration, discrimination, and clinical utility. Calibration curves were used to analyze the extent of agreement between predictions of the nomogram and the clinical data. The area under the receiver operating characteristic (AUROC) curve analysis was performed to compare the predictive performance between the nomogram model and SOFA score alone (bootstrap). Bootstrapping validation was performed in the primary cohort. The AUROCs between the primary cohort and validation cohort were also compared using bootstrap. To assess the clinical usefulness of the predictive nomogram, decision curve analysis (DCA) was used to quantify the net benefits at different threshold probabilities. Net reclassification was also used to determine if the nomogram improved risk classification beyond that achieved using the SOFA score alone. In the analysis, in-hospital mortality results of 0-0.2, 0.2-0.4, and 0.4-1 were defined as low risk, moderate risk, and high risk, respectively. Net reclassification improvement (NRI) was used to assess reclassification improvement over risk categories. Integrated discrimination improvement (IDI) was used as a continuous version of the NRI that did not require risk categories that were defined a priori. Nomogram accuracy and validity were validated in the validation cohort.
A risk score for each patient was calculated based on the nomogram model, and patients in the primary cohort were classified eventually into groups by tertile of the risk score. Multivariate modeling of the association between groups and mortality was performed with logistic regression. Baseline variables that were considered clinically relevant or that showed a univariate relationship with outcome (P<0.10) were entered into a multivariate logistic regression model as covariates, and included age, gender, SOFA score, CCI, SAPS II, use of mechanical ventilation, and initial lactate level.
Statistical analyses were performed using the software application, R (version 4.0.3). P<0.05 was considered significant.
Results
Baseline characteristics
An examination of the database found that 6,720 patients met the definition of sepsis. The study flowchart is presented in Figure 1. For the 4704 patients in the primary cohort, the mean SOFA score was 3.69 (1.89), and the in-hospital mortality rate was 14.3%. The validation cohort consisted of 2016 patients who achieved a mean sofa score of 3.73 (1.90), with an in-hospital mortality rate of 14.7%. There were no intergroup differences for baseline characteristics between the development and validation cohorts. Results for baseline characteristics of the patients in the primary and validation cohorts are presented in Table 1.
Figure 1.
Study flowchart in MIMIC-IV.
Table 1.
Baseline characteristics and outcomes of the primary cohort and validation cohort
| Baseline | Primary cohort (n=4704) | Validation cohort (n=2016) |
|---|---|---|
| Age (years) | 67.0 [56.0, 78.0] | 67.0 [56.0, 78.0] |
| Male, n (%) | 2586 (55.0) | 1155 (57.3) |
| BMI | 29.09 (8.03) | 28.91 (8.15) |
| Admission Type, n (%) | ||
| Emergency | 2921 (62.1) | 1260 (62.5) |
| Elective | 165 (3.5) | 61 (3.0) |
| Urgent | 945 (20.1) | 405 (20.1) |
| Other | 673 (14.3) | 290 (14.4) |
| Charlson Comorbidity Index | 6.0 [4.0, 8.0] | 6.0 [4.0, 8.0] |
| SOFA | 3.69 (1.89) | 3.73 (1.90) |
| SAPS II | 53.0 [40.0, 72.0] | 54.0 [41.0, 73.0] |
| Intervention in the first 24 h, n (%) | ||
| Vasopressor | 1964 (41.8) | 866 (43.0) |
| Renal replacement therapy | 200 (4.3) | 86 (4.3) |
| Mechanical ventilation | 1974 (42.0) | 881 (43.7) |
| Vital signs | ||
| MAP (mmHg) | 75.0 (9.7) | 74.7 (9.7) |
| Heart rate (bpm) | 88.7 (16.1) | 87.9 (16.4) |
| Temperature (°C) | 36.9 (0.6) | 37.0 (0.6) |
| Respiratory rate (bpm) | 20.3 (4.1) | 20.3 (4.3) |
| Laboratory data | ||
| WBC (*109/L) | 11.7 [8.1, 16.4] | 11.8 [7.7, 16.3] |
| PLT (*109/L) | 168.5 [113.7, 238.0] | 164.2 [111.8, 238.0] |
| Hemoglobin (g/L) | 9.96 (1.84) | 9.99 (1.84) |
| NLR | 9.14 [5.15, 17.20] | 9.06 [4.98, 17.77] |
| LMR | 1.94 [1.08, 3.35] | 1.92 [1.05, 3.38] |
| MCV (%) | 91.47 (7.46) | 91.26 (7.53) |
| RDW (%) | 15.79 (2.49) | 15.74 (2.52) |
| pH | 7.37 (0.08) | 7.36 (0.08) |
| PaO2/FiO2 (mmHg) | 240.5 [177.1, 317.2] | 232.6 [172.3, 310.6] |
| Lactate level (mmol/L) | 2.35 (1.87) | 2.32 (1.88) |
| Outcome | ||
| In-hospital mortality, n (%) | 672 (14.3) | 296 (14.7) |
| Length of ICU stay | 3.1 [2.0, 6.0] | 3.1 [1.9, 6.0] |
| Length of hospital stay | 9.0 [6.0, 16.0] | 9.0 [6.0, 16.0] |
SOFA, Sequential Organ Failure Assessment; SAPS II, Simplified Acute Physiology Score II; MAP, Mean Arterial Pressure; WBC, White Blood Cell count; PLT, Platelet; NLR, Neutrophil-to-Lymphocyte Ratio; LMR, Lymphocytes to Monocytes Ratio; MCV, Mean Corpuscular Volume; RDW, Red cell Distribution Width.
Prognostic nomogram of modified SOFA scores for 28-day mortality
Uni-and multivariate logistic regression analyses revealed thatSOFA score, CCI, NLR, LMR, MCV, and RDW were independent risk factors for in-hospital mortality, with adjusted odds ratios (ORs) of 1.12 (95% CI: 1.07-1.16), 1.15 (95% CI: 1.12-1.18), 1.01 (95% CI: 1.01-1.02), 1.03 (95% CI: 1.00-1.05), 1.03 (95% CI: 1.02-1.04), and 1.15 (95% CI: 1.12-1.18), respectively (all P values <0.05) (Table 2). The Akaike information criterion result for the model was 3531.98. The Cox & Snell R Square and Nagelkerke R Square was 0.125 and 0.206, respectively.
Table 2.
Univariable model and full multivariable model for the relationship between risk factors and in-hospital mortality in primary cohort
| Univariable model | Full multivariable model | |||||
|---|---|---|---|---|---|---|
|
|
|
|||||
| OR | 95% CI | P value | OR | 95% CI | P value | |
| SOFA | 1.18 | 1.13-1.22 | <0.001 | 1.12 | 1.07-1.16 | <0.001 |
| CCI | 1.19 | 1.16-1.23 | <0.001 | 1.15 | 1.12-1.18 | <0.001 |
| NLR | 1.02 | 1.01-1.02 | <0.001 | 1.01 | 1.01-1.02 | <0.001 |
| LMR | 0.99 | 0.96-1.01 | 0.24 | 1.03 | 1.00-1.05 | 0.023 |
| MCV | 1.04 | 1.02-1.05 | <0.001 | 1.03 | 1.02-1.04 | <0.001 |
| RDW | 1.21 | 1.17-1.24 | <0.001 | 1.15 | 1.12-1.18 | <0.001 |
SOFA, Sequential Organ Failure Assessment; CCI: Charlson Comorbidity Index; NLR, Neutrophil-to-Lymphocyte Ratio; LMR, Lymphocytes to Monocytes Ratio; MCV, Mean Corpuscular Volume; RDW, Red cell Distribution Width.
A nomogram was constructed using the risk factors identified by multivariate analysis (Figure 2). To generate the nomogram, each independent prognostic parameter was assigned a weighted score based on a point scale. The total score was calculated as follows: (3.566 * SOFA-7.133) + (4.524 * CCI) + (0.569 * NLR) + (0.913 * LMR) + (0.852 * MCV-51.118) + (4.545 * RDW-45.455).
Figure 2.
Nomogram predicting in-hospital mortality in septic patients. SOFA, Sequential Organ Failure Assessment; CCI, Charlson Comorbidity Index; NLR, Neutrophil-to-Lymphocyte Ratio; LMR, Lymphocyte to Monocyte Ratio; MCV, Mean Corpuscular Volume; RDW, Red Cell distribution Width.
Nomogram model validation
The calibration plot (Figure 3) indicated an adequate nomogram fit for outcome prediction in both cohorts (i.e., primary and validation). ROC curve analysis for outcome discriminatory ability (Figure 4) revealed an AUC of 0.724 (95% CI: 0.705-0.743) for the nomogram, which was better than SOFA alone (0.585, 95% CI: 0.562-0.609; P<0.001). The AUC of the nomogram model was 0.745 (95% CI: 0.678-0.765) in bootstrapping validation. The result for the validation cohort was similar. The AUC of the nomogram model or SOFA did not differ significantly between the primary cohort and the validation cohort (P=0.47 for nomogram model, P=0.32 for SOFA). The DCA curve (Figure 5) indicated that a better net benefit was gained using the nomogram than the SOFA score; the threshold probabilities were 0-0.3 vs. 0-0.28, respectively, for the primary cohort and 0-0.31 vs. 0-0.17, respectively, for the validation cohort.
Figure 3.
Calibration curves for predicting the outcome of septic patients in the emergency department in the primary cohort (A), validation cohort (B).
Figure 4.
Receiver operating characteristic curve analysis and comparison of the AUCs for the nomogram and SOFA score in the primary cohort (A), validation cohort (B). AUC: Area Under the Curve; SOFA: Sequential Organ Failure Assessment.
Figure 5.
Decision curves for the primary cohort (A), and validation cohort (B) implicating the net benefit with respect to the use of the nomogram for predicting mortality of septic patients. SOFA: Sequential Organ Failure Assessment.
The results of the reclassification statistics analysis indicated an improved classification ability of the nomogram model, compared with the original SOFA score. The nomogram resulted in an additive NRI of 20.5% (95% CI: 16.2%-24.7%; P<0.001), and an IDI of 6.0% (95% CI: 5.1%-6.8%; P<0.001) in the primary cohort. It resulted in an additive NRI of 28.5% (95% CI: 21.9%-35.1%; P<0.001), and an IDI of 8.1% (95% CI: 6.6%-9.7%; P<0.001) in the validation cohort.
Classification of patients with sepsis based on risk score
The risk score was 95.1 (79.8-113.4) in the primary cohort. Patients were classified into four groups by total points (very low risk: <79.8, low risk: 79.9-95.1, medium risk: 95.2-113.4, high risk: ≥13.5). Multivariate logistic regression showed that compared to patients with very low risk, the patients with low risk, medium risk, and high risk were associated with increased in-hospital mortality, with adjusted ORs of 1.75 (95% CI: 1.07-2.90), 3.68 (95% CI: 2.25-6.17), and 5.85 (95% CI: 3.32-10.48), respectively (all P values <0.05).
Discussion
The results of this study suggested that a new SOFA score modified using CCI, RDW, NLR, LMR, and MCV through a nomogram had better performance for prediction and risk stratification for septic patients.
Because it is simple to use and includes graded assessment of organ failure, the SOFA score is the most widely used scoring system for patients with sepsis. As the definition and clinical management of sepsis have changed over time, the priority for sepsis management has become not just addressing organ failure but also the severity of organ failure and the exaggerated inflammatory response [11]. Because of these changes, there was an inevitable decline in the existing SOFA score performance and the model has been updated over time. Glasgow Coma Scale (GCS) is not available for use or is not applied consistently at some healthcare facilities, thus Vasilevskis et al. [12] proposed a modified SOFA score that uses the Richmond Agitation-Sedation Scale (RASS) instead of the GCS. They found that compared to a GCS-based SOFA score (0.799), a Mean RASS-based SOFA score can achieve an AUC of 0.814 for prediction of ICU mortality. A modified cardiovascular component of the SOFA score was developed and validated by Yadav et al. [13]. All clinically-used vasoactive agents are accounted for by this modification. A shock index is used as a mean arterial pressure substitute, and serum lactate serves as a shock state biomarker. They found that ROC curves used to predict ICU mortality, hospital mortality, and 28-day mortality are significantly higher for the modified cardiovascular component of the SOFA score than for the original cardiovascular component of the SOFA score. In this study, we constructed a new SOFA score by modifying the assessment of inflammation-based markers, including RDW, NLR, and MCV through a nomogram.The AUC of the nomogram to predict 28-day mortality of septic patients was as high as 0.704, compared to the SOFA score alone (P<0.001).
In critically ill patients, RDW can be independently associated with all-cause mortality risk [14-16]. RDW can provide additional prognostic value beyond that of severity scores, including the APACHE II and SOFA scores [17,18]. Consistent with these findings, the results of this study suggested that RDW was an important independent risk factor for 28-day mortality in septic patients admitted to the ICU. RDW can be affected by factors including anemia, ischemia, renal function, and inflammatory responses. Sepsis-associated inflammatory responses can result in inadequate erythropoiesis, altered iron metabolism, and inhibition of erythrocyte maturation. Immature red blood cells are then released into the circulation to result in an elevated RDW, which may partially represent the inflammation state of patients with sepsis [19]. On the other hand, reactive erythropoiesis participates in compensation for acute stress and the resulting tissue hypoxia. How well this process is carried out might be determined by genetic factors. The level of response may also reflect the extent of physiologic reserves available to the patient, and explain why RDW is independent of organ dysfunction severity during sepsis.
Sepsis pathophysiology includes unregulated inflammation; the inflammatory response includes neutrophils, lymphocytes, and red blood cells. Changes in NLR, LMR, and MCV indicate the presence of inflammatory response or immune status imbalances, or imbalances in both. Hence, NLR, LMR, and MCV are used over time as inflammation-based markers for patients who are critically ill, have cardiovascular disease, or have tumors [20,21]. Patients with increased NLR, LMR, or MCV values have poorer outcomes. Lu et al. found a significant relatively shorter overall survival time to be associated with nasopharyngeal carcinoma and an NLR ≥2.28. Cox regression analysis found that compared to quintile 3 (8.6 (7.1-9.9)), quintile 1 (0.2 (0.1-0.7)) and quintile 5 (31 (24.6-46.8)) NLR values are significant risk factors for prediction of 28-day mortality in critically ill patients with sepsis [8]. In this study, we found that an NLR ≥12.8 and an MCV ≥93.3 in the initial blood sample collected after ICU admission had significant associations with increased in-hospitalmortality.
The predictive performance of the SOFA score for in-hospital mortality was relatively low, compared to previous research [6,22]. This may be because of different environments to calculate SOFA scores in the MIMIC IV database. There are many missing values in MIMIC IV regarding the parameters to calculate SOFA scores, especially the bilirubin, and all missing values were transformed to 0 when calculating the SOFA score, which could affect the predictive value of the SOFA score [23].
This study was the first to construct and use a nomogram model to examine prognostic accuracy and validity of combining the SOFA score with inflammation-based markers, including CCI, RDW, NLR, LMR, and MCV in patients with sepsis in the ICU. We also proposed a new risk stratification strategy for septic patients based on risk score.
This study had some limitations. First, we performed a retrospective analysis using data from an online database. Missing and outlier data were common and may have increased the study bias. Well-designed prospective studies are required to further evaluate our nomogram. Second, we only used admission SOFA scores to construct the nomogram. An evaluation over time could be more informative and aid in treatment adjustment decisions. Third, we evaluated the prediction performance only for in-hospital mortality. Prediction of shorter- or longer-term mortality and other outcomes (e.g., length of ICU stay, hospitalization costs, and quality of life) could also be evaluated in future studies.
Conclusions
Compared to use of the SOFA score alone, a nomogram model based on SOFA score, CCI, RDW, NLR, LMR, and MCV values provided more accurate death prediction and risk stratification for patients with sepsis in the ICU.
Acknowledgements
Jiangsu Provincial Key Medical Discipline (Laboratory) (ZDXKA2016025) (Dr. Yi Yang), the Key Research and Development Plan of Jiangsu Province (BE2018743) (Dr. Yi Yang).
Disclosure of conflict of interest
None.
References
- 1.Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC. The third international consensus definitions for sepsis and septic shock (sepsis-3) JAMA. 2016;315:801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, Iwashyna TJ. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312:90–92. doi: 10.1001/jama.2014.5804. [DOI] [PubMed] [Google Scholar]
- 3.Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, Edelson DP. Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017;195:906–911. doi: 10.1164/rccm.201604-0854OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, Reinhart CK, Suter PM, Thijs LG. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure. On behalf of the working group on sepsis-related problems of the european society of intensive care medicine. Intensive Care Med. 1996;22:707–710. doi: 10.1007/BF01709751. [DOI] [PubMed] [Google Scholar]
- 5.Matics TJ, Sanchez-Pinto LN. Adaptation and validation of a pediatric sequential organ failure assessment score and evaluation of the sepsis-3 definitions in critically ill children. JAMA Pediatr. 2017;171:e172352. doi: 10.1001/jamapediatrics.2017.2352. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ferreira FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286:1754–1758. doi: 10.1001/jama.286.14.1754. [DOI] [PubMed] [Google Scholar]
- 7.Kim CH, Park JT, Kim EJ, Han JH, Han JS, Choi JY, Han SH, Yoo TH, Kim YS, Kang SW, Oh HJ. An increase in red blood cell distribution width from baseline predicts mortality in patients with severe sepsis or septic shock. Crit Care. 2013;17:R282. doi: 10.1186/cc13145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hwang SY, Shin TG, Jo IJ, Jeon K, Suh GY, Lee TR, Yoon H, Cha WC, Sim MS. Neutrophil-to-lymphocyte ratio as a prognostic marker in critically-ill septic patients. Am J Emerg Med. 2017;35:234–239. doi: 10.1016/j.ajem.2016.10.055. [DOI] [PubMed] [Google Scholar]
- 9.Dratch A, Kleine CE, Streja E, Soohoo M, Park C, Hsiung JT, Rhee CM, Obi Y, Molnar MZ, Kovesdy CP, Kalantar-Zadeh K. Mean corpuscular volume and mortality in incident hemodialysis patients. Nephron. 2019;141:188–200. doi: 10.1159/000495726. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Johnson AE, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3:160035. doi: 10.1038/sdata.2016.35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, Pilcher DV. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317:290–300. doi: 10.1001/jama.2016.20328. [DOI] [PubMed] [Google Scholar]
- 12.Vasilevskis EE, Pandharipande PP, Graves AJ, Shintani A, Tsuruta R, Ely EW, Girard TD. Validity of a modified sequential organ failure assessment score using the richmond agitation-sedation scale. Crit Care Med. 2016;44:138–146. doi: 10.1097/CCM.0000000000001375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Yadav H, Harrison AM, Hanson AC, Gajic O, Kor DJ, Cartin-Ceba R. Improving the accuracy of cardiovascular component of the sequential organ failure assessment score. Crit Care Med. 2015;43:1449–1457. doi: 10.1097/CCM.0000000000000929. [DOI] [PubMed] [Google Scholar]
- 14.Fava C, Cattazzo F, Hu ZD, Lippi G, Montagnana M. The role of red blood cell distribution width (RDW) in cardiovascular risk assessment: useful or hype? Ann Transl Med. 2019;7:581. doi: 10.21037/atm.2019.09.58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wang RR, He M, Ou XF, Xie XQ, Kang Y. The predictive value of RDW in AKI and mortality in patients with traumatic brain injury. J Clin Lab Anal. 2020;34:e23373. doi: 10.1002/jcla.23373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pedrazzani C, Tripepi M, Turri G, Fernandes E, Scotton G, Conci S, Campagnaro T, Ruzzenente A, Guglielmi A. Prognostic value of red cell distribution width (RDW) in colorectal cancer. Results from a single-center cohort on 591 patients. Sci Rep. 2020;10:1072. doi: 10.1038/s41598-020-57721-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Yang Y, Liang S, Geng J, Wang Q, Wang P, Cao Y, Li R, Gao G, Li L. Development of a nomogram to predict 30-day mortality of patients with sepsis-associated encephalopathy: a retrospective cohort study. J Intensive Care. 2020;8:45. doi: 10.1186/s40560-020-00459-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lorente L, Martín MM, Argueso M, Solé-Violán J, Perez A, Marcos Y Ramos JA, Ramos-Gómez L, López S, Franco A, González-Rivero AF, Martín M, Gonzalez V, Alcoba-Flórez J, Rodriguez MÁ, Riaño-Ruiz M, Guillermo O Campo J, González L, Cantera T, Ortiz-López R, Ojeda N, Rodríguez-Pérez A, Domínguez C, Jiménez A. Association between red blood cell distribution width and mortality of COVID-19 patients. Anaesth Crit Care Pain Med. 2021;40:100777. doi: 10.1016/j.accpm.2020.10.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Zhang L, Yu CH, Guo KP, Huang CZ, Mo LY. Prognostic role of red blood cell distribution width in patients with sepsis: a systematic review and meta-analysis. BMC Immunol. 2020;21:40. doi: 10.1186/s12865-020-00369-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Korkmaz C, Demircioglu S. The association of neutrophil/lymphocyte and platelet/lymphocyte ratios and hematological parameters with diagnosis, stages, extrapulmonary involvement, pulmonary hypertension, response to treatment, and prognosis in patients with sarcoidosis. Can Respir J. 2020;2020:1696450. doi: 10.1155/2020/1696450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kakhki RD, Dehghanei M, ArefNezhad R, Motedayyen H. The predicting role of neutrophil- lymphocyte ratio in patients with acute ischemic and hemorrhagic stroke. J Stroke Cerebrovasc Dis. 2020;29:105233. doi: 10.1016/j.jstrokecerebrovasdis.2020.105233. [DOI] [PubMed] [Google Scholar]
- 22.Ho KM, Lee KY, Williams T, Finn J, Knuiman M, Webb SA. Comparison of acute physiology and chronic health evaluation (APACHE) II score with organ failure scores to predict hospital mortality. Anaesthesia. 2007;62:466–473. doi: 10.1111/j.1365-2044.2007.04999.x. [DOI] [PubMed] [Google Scholar]
- 23.Hu W, Chen H, Wang H, Peng Q, Wang J, Huang W, Liu A, Xu J, Li Q, Pan C, Xie J, Huang Y. Identifying high-risk phenotypes and associated harms of delayed time-to-antibiotics in patients with ICU onset sepsis: a retrospective cohort study. J Crit Care. 2022;74:154221. doi: 10.1016/j.jcrc.2022.154221. [DOI] [PubMed] [Google Scholar]