Abstract
Objectives
The assessment of illness severity at admission can contribute to decreased mortality in patients with the coronavirus disease (COVID-19). This study was conducted to evaluate the effectiveness of the Sequential Organ Failure Assessment (SOFA) and Quick Sequential Organ Failure Assessment (qSOFA) scoring systems at admission for the prediction of mortality risk in COVID-19 patients.
Methods
We included 140 critically ill COVID-19 patients. Data on demographics, clinical characteristics, and laboratory findings at admission were used to calculate SOFA and qSOFA against the in-hospital outcomes (survival or death) that were ascertained from the medical records. The predictive accuracy of both scoring systems was evaluated by the receiver operating characteristic (ROC) curve analysis.
Results
The area under the ROC curve for SOFA in predicting mortality was 0.890 (95% CI: 0.826–0.955), which was higher than that of qSOFA (0.742, 95% CI 0.657–0.816). An optimal cutoff of ≥3 for SOFA had sensitivity, specificity, positive predictive value, and negative predictive value of 90.00%, 83.18%, 50.00%, and 97.80%, respectively.
Conclusions
This novel report indicates that SOFA could function as an effective adjunctive risk-stratification tool at admission for critical COVID-19 patients. The performance of qSOFA is accepted but inferior to that of SOFA.
Abbreviations: AUC, area under the curve; COVID-19, novel coronavirus disease; GCS, Glasgow Coma Scale; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operating characteristic; SOFA, Sequential Organ Failure Assessment; qSOFA, Quick Sequential Organ Failure Assessment
Keywords: Sequential organ failure assessment, Quick sequential organ failure assessment, Novel coronavirus disease, Mortality
1. Introduction
The coronavirus disease (COVID-19), caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), emerged in December 2019 and has rapidly spread worldwide [1]. Globally, the onslaught of the COVID-19 outbreak on the public health system continues despite unprecedented mobilization. The mortality of critical ill patients with COVID-19 has been reported variously as low as 11% and as high as 61% [[2], [3], [4], [5]]. Bhatraju et al. [5] reported that the ratio of patients with COVID-19 over 65 years of age had died greater than patients less than 65 years of age. In a 43 patients' study, Saurabh Aggarwal et al. reported three patients aged 70 years or older died [6].
In critically ill COVID-19 patients, early medical intervention to reduce mortality depended on early effective assessment [7]. The application of scoring systems can facilitate effective evaluation by emergency or critical care physicians to screen severe patients. At present, however, there are no specific scoring systems for the evaluation of COVID-19 patients. Another option is to adopt existing scoring systems that are used for predicting the mortality of severe patients.
Scoring systems, such as the Sequential Organ Failure Assessment (SOFA) and the more recent Quick Sequential Organ Failure Assessment (qSOFA), can help emergency or critical care physicians for prognosis and predicting mortality. The SOFA scoring system was developed in 1996 by an international group of experts [8]. SOFA describes the condition of multiple organ dysfunction through several parameters, including the oxygenation index (arterial oxygen tension [PaO2]/fraction of inspiration oxygen [FiO2]), mean arterial pressure, Glasgow Coma Scale(GCS), creatinine or urine volume, bilirubin, and platelets, for the respiratory, circulatory, neurologic, renal, hepatogenic, and coagulation systems, respectively. The function of each organ systems is scored from 0 to 4, and the individual SOFA scores are then summed to a total score that is between 0 and 24. A higher score infers aggravating conditions in the patients. Although SOFA was not initially designed to predict mortality, several studies revealed that the SOFA score could predict morbidity and mortality and this specific application has been validated in severely ill patients [[9], [10], [11], [12], [13]].Therefore, it is of interest to determine whether the SOFA score could predict mortality in the COVID-19 population.
The qSOFA score was introduced in the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) and is composed of three clinical parameters: systolic blood pressure ≤ 100 mmHg, respiratory rate ≥ 22 breaths/min, and altered mental status. The qSOFA score was created initially for the evaluation of septic patients, but several recent studies have described its effectiveness for predicting mortality in patients with different diseases [14,15].
Both SOFA and qSOFA have widespread application to screen patients with life-threatening conditions in the emergency or critical care department [16]. Several studies [[17], [18], [19]] have suggested that the SOFA and qSOFA were high-performance assessments to evaluate the risk of mortality in critically ill patients. These scoring systems can assist emergency or critical care physicians to predict mortality of such patients, especially in resource-constrained scenarios.
The COVID-19 epidemic is a scenario of shortage of medical resources. However, to our knowledge, no study has validated the performance of SOFA and qSOFA for severely/critically ill COVID-19 patients. This study was conducted with an aim to evaluate the prognostic value of SOFA and qSOFA models with regard to the in-hospital mortality of COVID-19 patients.
2. Material and methods
2.1. Study design
This study was undertaken through a retrospective analysis of electronic medical records of patients managed by an emergency medical team that was deployed to Wuhan. The team independently managed the COVID-19 ward in a local hospital that was temporarily constructed for COVID-19 patients and comprised 80 beds.
2.2. Study subjects and setting
The Diagnosis and Treatment Plan of Novel Coronavirus [20], issued by the National Health Commission of China, was adopted as a criterion for the diagnosis, classification, and treatment of COVID patients in the COVID-19 ward.
We searched for all adult patients (age > 18 years) who had been diagnosed with severe/critical COVID-19 and admitted to the ward between January 31, 2020 and March 7, 2020 through the computerized registration system of the ward. The criterion of severe/critical COVID-19 is followed the Diagnosis and Treatment Plan of Novel Coronavirus, which can be download in http://www.nhc.gov.cn (Table 1 ) [20]. We identified 140 records that were eligible for study inclusion. Then, all the cases was divided into sub-group of young (age < 65 years) and senior (age ≥ 65 years) for analysis. This study was conducted with the approval of the local institutional review board and was undertaken in accordance with the principles of the Declaration of Helsinki for research on human participants.
Table 1.
The criterion⁎ of severe/critical COVID-19.
| Classification | |
|---|---|
| Minor | The confirmed patient has mild symptoms，without radiography results of pneumonia. |
| Common | The confirmed patient has fever and respiratory symptoms，with radiography results of pneumonia. |
| Severe | the confirmed COVID-19 patients who meet any one of the follows: 1. Shortness of breath, with RR ≥ 30 times/min 2. Oxygen saturation ≤ 93% 3. Oxygenation index ≤ 300 mmHg§ 4. Chest radiographic images showed that the lesion progressed more than 50% within 24–48 h |
| Critical | the severe confirmed COVID-19 patients who meet any one of the follows 1. Respiratory failure needs mechanical ventilation 2. Shock 3. Combining other organ failure needs ICU admission |
| Inclusion and exclusion criteria | |
| Inclusion criteria | 1. Age > 18 years 2. Diagnosed with severe/critical COVID-19 3. Admitted to the COVID-19 ward between January 31, 2020 and March 7, 2020 |
| Exclusion criteria | 1. Cases with missing data 2. Pregnant women |
COVID-19: novel coronavirus disease; RR: respiratory rate; ICU: intensive care unit.
The criterion of severe/critical COVID-19 is followed the Diagnosis and Treatment Plan of Novel Coronavirus, which can be download in http://www.nhc.gov.cn.
High altitude areas need correction.
2.3. Data collection
From the medical records, we extracted data that were collected at the time of admission, including details of the demographics, clinical manifestations, and laboratory results, to calculate the SOFA and qSOFA scores and recorded the patient outcomes.
2.4. Statistical analysis
Descriptive data are reported as mean ± standard deviation or median [25%quartile, 75%quartile] for continuous variables, and percentages for categorical data, as appropriate. We divided the study sample into survivors and non-survivors and carried out intergroup comparisons of variables, including demographics, clinical features, laboratory findings, and both SOFA and qSOFA scores. Continuous data were compared by the Student t-test or the Mann–Whitney U test; categorical variables were tested by using either the chi-square test or Fisher exact test (when the expected value was <5 in one cell), as appropriate.
The receiver operating characteristic (ROC) curve analysis was applied for a discriminatory evaluation of the performance of the SOFA and qSOFA scores. The classification performance of both scoring systems to discriminate between survivors and non-survivor was evaluated by calculating the area under the curve (AUC) of the ROC and its 95% confidence intervals (95%CI). We conducted the Delong test [21] for a comparison of the AUCs of SOFA and qSOFA.
For each rapid scoring system, the score with the largest Youden Index was defined as the optimal cutoff value for predicting COVID-19 mortality. Based on the optimal cutoff value, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for both scoring systems. A p-value < 0.05 was accepted as statistically significant.
Data were entered into the IBM Statistical Product and Service Solutions (SPSS) version 20.0 for Windows and were analyzed by MedCalc® 12.7.
3. Results
3.1. Baseline analysis
We initially included 140 severely or critically ill patients who were admitted to the COVID-19 ward. However, the final analysis dataset included 127 patients, after excluding 13 patients with missing data (Fig. 1 ). In total, the mean age of the patients in this study population was 61.25 (SD ± 15.53) years, and 75 patients (70.09%)were male. The baseline characteristics and comorbidities, including diabetes, hypertension, cardiovascular disease, chronic pulmonary disease, cerebral vascular disease, and malignant tumor, of the survivors and non-survivors in this study sample are listed in Table 2 .
Fig. 1.
Disposition of the study population.
Table 2.
Comparison of the baseline characteristics of survivors and non-survivors.
| Variable | Survivors N = 107 |
Non-survivors N = 20 |
p value |
|---|---|---|---|
| Male (number) | 60 (56.07%) | 15 (75.00%) | 0.090 |
| Female (number) | 47 (43.92%) | 5 (25.00%) | |
| Age (years) | 58.62 ± 14.68 | 75.36 ± 12.14 | 0.000⁎ |
| Pre-hospital day (day) | 10 [7, 14] | 7 [4, 14] | 0.308 |
| Underlying disease | |||
| Diabetes | 10 (9.35%) | 0 (0.00%) | 0.361 |
| Hypertension | 26 (24.30%) | 5 (25.00%) | 0.999 |
| Cardiovascular disease | 5 (4.67%) | 2 (10.00%) | 0.304 |
| Chronic pulmonary disease | 5 (4.67%) | 2 (10.00%) | 0.304 |
| Cerebral vascular disease | 0 (0.00%) | 3 (15.00%) | 0.003⁎ |
| Malignant tumor | 4 (3.74%) | 1 (5.00%) | 0.582 |
| Symptom and Sign | |||
| Fever | 87 (81.31%) | 15 (75.00%) | 0.544 |
| Cough | 64 (59.81%) | 15 (75.00%) | 0.221 |
| Sore throat | 1 (0.93%) | 2 (10.00%) | 0.064 |
| Diarrhea | 19 (17.76%) | 6 (30.00%) | 0.222 |
| Fatigue | 46 (42.99%) | 12 (60.00%) | 0.226 |
| Temperature | 36.72 ± 0.63 | 36.9 ± 0.9 | 0.302 |
| PR (/min) | 88.45 ± 19.05 | 89.2 ± 13.48 | 0.867 |
| RR (/min) | 20.50 ± 3.79 | 23.50 ± 5.16 | 0.003⁎ |
| HR (/min) | 88.45 ± 19.05 | 89.2 ± 13.48 | 0.867 |
| SBP | 130.57 ± 17.87 | 141.75 ± 25.34 | 0.018⁎ |
| DBP | 79.44 ± 11.56 | 83.8 ± 13.96 | 0.137 |
| MBP | 96.48 ± 12.26 | 103.12 ± 16.03 | 0.037⁎ |
| SpO2 (%) | 95.03 ± 4.25 | 85.5 ± 12.6 | 0.003⁎ |
| GCS (score) | 15 [15, 15] | 15 [15, 15] | 0.121 |
| Laboratory characteristics | |||
| White blood cell count (109/l) | 5.60 [4.49, 6.77] | 8.97 [5.63, 12.68] | 0.009⁎ |
| Red blood cell count (1012/l) | 3.90 [3.62, 4.24] | 3.91 [3.34, 4.19] | 0.569 |
| Lymphocyte count (109/l) | 1.28 [0.81, 1.65] | 0.47 [0.36, 0.92] | 0.000⁎ |
| Platelets count (109/l) | 233.36 ± 96.22 | 139.25 ± 72.85 | 0.000⁎ |
| Bilirubin (μmol/l) | 13.02 ± 9.32 | 24.33 ± 21.14 | 0.029⁎ |
| Creatinine (μmol/l) | 65.25 ± 28.47 | 100.05 ± 55.87 | 0.013⁎ |
| PaO2 (mmHg) | 108.07 ± 32.75 | 76.65 ± 34.96 | 0.000⁎ |
| FiO2 (%) | 21 [21, 29] | 29 [29, 43.25] | 0.027⁎ |
| Chest radiographic images showed pneumonia | 104 (97.20%) | 20 (100.00%) | 0.999 |
| Ventilator applying | 8 (7.48%) | 8 (40.00%) | 0.001⁎ |
| Length of hospital stay§ | 20 [10, 33.5] | 6 [3.75, 9] | 0.000⁎ |
| SOFA (score) | 1 [0, 2] | 4 [3, 5] | 0.000⁎ |
| qSOFA (score) | 0 [0, 0] | 1 [0, 1] | 0.000⁎ |
PR: pulse rate; HR: heart rate; RR: respiratory rate; SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial pressure;SpO2: peripheral oxygen saturation; GCS: Glasgow Coma Scale; PaO2: arterial oxygen tension; FiO2: fraction of inspiration oxygen; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment.
p < 0.05.
The length of hospital stay of survivors is the duration from admission to discharge, while the length of hospital stay of non-survivors is the duration from admission to death.
From among the 140 cases that were initially identified, we excluded 13 cases because of missing data. The final analysis dataset included 127 cases.
Variables that showed significant differences between the survivors and non-survivors included age, respiratory rate, systolic blood pressure, mean arterial pressure, peripheral oxygen saturation, platelet count, bilirubin, creatinine, PaO2, and FiO2. The median [25% quartile, 75% quartile] values of the SOFA and qSOFA scores of survivors versus non-survivors were 1[0,2] versus 4[3,5] and 0[0,0] versus 1[0,1], respectively.
3.2. Overall analysis
The ROC curves of SOFA and qSOFA scoring systems for predicting in-hospital mortality are depicted in Part A of Fig. 2 , and their AUCs were 0.915 (95% CI: 0.852–0.957) and 0.742 (95% CI: 0.657–0.816), respectively; there was significant difference between the two scoring systems(p = 0.007; Table 3 ).
Fig. 2.
Classification performances of the SOFA and qSOFA scores.
Table 3.
The AUC of SOFA and qSOFA models in predicting in-hospital mortality.
| Models | AUC of ROC | 95%C.I. of AUC | ΔAUC | z statistic | p value |
|---|---|---|---|---|---|
| Overall | |||||
| SOFA | 0.890 | 0.826 to 0.955 | 0.148 | 2.612 | 0.009⁎ |
| qSOFA | 0.742 | 0.657 to 0.816 | |||
| Subgroup | |||||
| Age < 65 years | |||||
| SOFA | 0.912 | 0.813 to 0.969 | 0.209 | 1.002 | 0.316 |
| qSOFA | 0.703 | 0.574 to 0.813 | |||
| Age ≥ 65 years | |||||
| SOFA | 0.921 | 0.827 to 0.973 | 0.148 | 2.131 | 0.033⁎ |
| qSOFA | 0.773 | 0.652 to 0.867 |
AUC: area under the curve of the Receiver Operating Characteristic; 95%C.I.: 95% confidence interval; ΔAUC: difference between AUCs; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment.
p < 0.05.
(A)The ROC of SOFA and qSOFA models for overall cases; (B) The ROC of SOFA and qSOFA models in the subgroup of persons aged ≥65 years; (C) The ROC of SOFA and qSOFA models in the subgroup of persons aged <65 years. The area under the curve of the solid line (for SOFA) was greater than the area under the curve of the dotted line (for qSOFA) for overall and subgroup analysis. SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment; ROC: receiver operating characteristics curve.
Based on the largest Youden Index, an optimum cutoff value of 3 was used to predict in-hospital mortality by using the SOFA score, with a sensitivity, specificity, PPV, and NPV of 90.00%, 83.18%, 50.00%, and 97.80%, respectively. Similarly, an optimum cutoff value of 1 was used to predict in-hospital mortality by using the qSOFA score, with a sensitivity, specificity, PPV, and NPV of 70.00%, 80.37%, 40.00%, and 93.48%, respectively (Table 4 ).
Table 4.
Sensitivities, specificities, and accuracy rates of SOFA and qSOFA for predicting in-hospital mortality.
| Models | Cutoff value | Youden Index | Sen. (%) | Spe. (%) | Accuracy (%) | PPV (%) | NPV (%) |
|---|---|---|---|---|---|---|---|
| Overall | |||||||
| SOFA | 1 | 0.50 | 100.00 | 50.47 | 58.27 | 27.40 | 100.00 |
| 2 | 0.67 | 95.00 | 71.96 | 75.59 | 38.78 | 98.72 | |
| 3 | 0.73⁎ | 90.00 | 83.18 | 84.25 | 50.00 | 97.80 | |
| 4 | 0.58 | 70.00 | 87.85 | 85.04 | 51.85 | 94.00 | |
| 5 | 0.49 | 55.00 | 94.39 | 88.19 | 64.71 | 91.82 | |
| 6 | 0.23 | 25.00 | 98.13 | 86.61 | 71.43 | 87.50 | |
| 7 | 0.18 | 20.00 | 98.13 | 85.83 | 66.67 | 86.78 | |
| 8 | 0.20 | 20.00 | 100.00 | 87.40 | 100.00 | 86.99 | |
| qSOFA | 1 | 0.50⁎ | 70.00 | 80.37 | 78.74 | 40.00 | 93.48 |
| 2 | −0.03 | 0.00 | 97.20 | 81.89 | 0.00 | 83.87 | |
| 3 | 0.00 | 0.00 | 100.00 | 84.25 | N/A | 84.25 | |
| Subgroup | |||||||
| Age < 65 years | |||||||
| SOFA | 1 | 0.54 | 100.00 | 54.24 | 56.45 | 10.00 | 100.00 |
| 2 | 0.71 | 100.00 | 71.19 | 72.58 | 15.00 | 100.00 | |
| 3 | 0.81⁎ | 100.00 | 81.36 | 82.26 | 21.43 | 100.00 | |
| 4 | 0.53 | 66.67 | 86.44 | 85.48 | 20.00 | 98.08 | |
| 5 | 0.27 | 33.33 | 93.22 | 90.32 | 20.00 | 96.49 | |
| 6 | 0.32 | 33.33 | 98.31 | 95.16 | 50.00 | 96.67 | |
| 7 | 0.32 | 33.33 | 98.31 | 95.16 | 50.00 | 96.67 | |
| 8 | 0.33 | 33.33 | 100.00 | 96.77 | 100.00 | 96.72 | |
| qSOFA | 1 | 0.43⁎ | 66.67 | 76.27 | 75.81 | 12.50 | 97.83 |
| 2 | −0.03 | 0.00 | 96.61 | 91.94 | 0.00 | 95.00 | |
| 3 | 0.00 | 0.00 | 100.00 | 95.16 | N/A | 95.16 | |
| Age ≥ 65 years | |||||||
| SOFA | 1 | 0.46 | 100.00 | 45.83 | 60.00 | 39.53 | 100.00 |
| 2 | 0.67 | 94.12 | 72.92 | 78.46 | 55.17 | 97.22 | |
| 3 | 0.74⁎ | 88.24 | 85.42 | 86.15 | 68.18 | 95.35 | |
| 4 | 0.60 | 70.59 | 89.58 | 84.62 | 70.59 | 89.58 | |
| 5 | 0.55 | 58.82 | 95.83 | 86.15 | 83.33 | 86.79 | |
| 6 | 0.21 | 23.53 | 97.92 | 78.46 | 80.00 | 78.33 | |
| 7 | 0.16 | 17.65 | 97.92 | 76.92 | 75.00 | 77.05 | |
| 8 | 0.18 | 17.65 | 100.00 | 78.46 | 100.00 | 77.42 | |
| qSOFA | 1 | 0.56⁎ | 70.59 | 85.42 | 81.54 | 63.16 | 89.13 |
| 2 | −0.02 | 0.00 | 97.92 | 72.31 | 0.00 | 73.44 | |
| 3 | 0.00 | 0.00 | 100.00 | 73.85 | N/A | 73.85 |
Sen.: sensitivity; Spe.: specificity; PPV: positive predictive value; NPV: negative predictive value; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment; N/A: cannot calculate due to the denominator is zero.
Largest Youden Index.
The sensitivity and the specificity of SOFA were considerably higher than those of qSOFA. Therefore, the SOFA score can better predict mortality in critically ill COVID-19 patients than the qSOFA score.
For verify importance of SOFA as predictor in COVID-19, we conduct univariable and multivariable logistic regression analysis. In univariable logistic analysis, the age, SOFA and qSOFA were predictors in COVID-19 (p < 0.05); while in multivariable logistic regression analysis, only the age and SOFA were predictors in COVID-19 (Table 5 ).
Table 5.
Univariate and multivariate logistic analysis for inpatient death of COVID-19.
| Variables | Univariate analysis |
Multivariate analysis |
||||||
|---|---|---|---|---|---|---|---|---|
| χ2 | p value | OR | 95% C.I. | χ2 | p value | OR | 95% C.I. | |
| Age | 17.509 | 0.000⁎ | 1.140 | 1.072–1.213 | 11.061 | 0.001⁎ | 1.129 | 1.051–1.212 |
| Gender | 5.919 | 0.051 | 3.830 | 1.298–11.297 | 1.736 | 0.188 | 2.648 | 0.622–11.270 |
| SOFA | 22.647 | 0.000⁎ | 2.070 | 1.534–2.793 | 9.917 | 0.002⁎ | 1.867 | 1.266–2.753 |
| qSOFA | 11.925 | 0.001⁎ | 4.699 | 1.952–11.309 | 0.511 | 0.475 | 1.618 | 0.432–6.055 |
OR: odds ratio; 95% C.I.: 95% confidence intervals for OR; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment.
p < 0.05.
3.3. Subgroup analysis
The ROC of SOFA and qSOFA scores of subgroups in predicting in-hospital mortality is depicted in Fig. 2(B and C). The AUC of SOFA and qSOFA scores in subgroup of persons aged <65 years were 0.912 and 0.703, and the difference between the two was no statistically significant (p = 0.316). And the AUC of SOFA and qSOFA scores in the subgroup of persons aged ≥65 years were 0.921 and 0.773, and the difference between the two was found to be statistically significant (p = 0.033) (Table 3).
According to the largest Youden index, an optimum cut-off value of 3 was used to predict in-hospital mortality using the SOFA, with sensitivity, specificity, PPV, and NPV of 100.00%, 81.36%, 21.43%, and 100.00% in the subgroup of persons aged <65 years. Likewise, an optimum cut-off value of 3 was used, with sensitivity, specificity, PPV, and NPV of 88.24%, 85.42%, 68.18%, and 95.35% in the subgroup of persons aged ≥65 years (Table 4).
According to the largest Youden index, an optimum cut-off value of 1 was used to predict in-hospital mortality using the qSOFA, with sensitivity, specificity, PPV, and NPV of 66.67%, 76.27%, 12.50%, and 97.83% in the subgroup of persons aged <65 years. Likewise, an optimum cut-off value of 1 was used, with sensitivity, specificity, PPV, and NPV of 70.59%, 85.42%, 63.16%, and 89.13% in the subgroup of persons aged ≥65 years. (Table 4).
4. Discussion
Given the medical resource constraints during the COVID-19 pandemic, [22] the early evaluation of severely or critically ill COVID-19 patients is paramount to ensure early medical monitoring and interventions for these patients. Thus, an important task of the emergency or critical care physicians is to screen patients with increased mortality from among the severely or critically ill COVID-19 patients [23]. However, this task has become particularly difficult due to the shortage of medical resources after the COVID-19 outbreak [24]. A scoring system can help emergency and critical care physicians to more quickly and accurately identify patients with increased mortality. Each scoring system has their own advantages and disadvantages. The SOFA score, with higher accuracy, has been widely validated as a tool for this purpose across a range of healthcare settings and environments [25]. However, SOFA is complex and time-consuming, because it needs six parameters and four of them are from laboratory results. While the superiorities of qSOFA are simple, rapid and practical, but some studies suggests that qSOFA has a low sensitive for in-hospital mortality in hospitalized patients with suspected infection [26]. In the absence of a specific risk-scoring system for COVID-19, we studied the performance of the SOFA and qSOFA in the evaluation of severely or critically ill patients in order to identify an effective risk-scoring system for COVID-19 patients.
To our knowledge, this is the first research to evaluate the performance of the SOFA and qSOFA in predicting the mortality risk of severely ill COVID-19 patients. This retrospective study has investigated the performance of both SOFA and qSOFA scores as predictors of prognosis in severely or critically ill COVID-19 patients. Initially, we determined that a SOFA score of ≥3 and a qSOFA score of ≥1 were associated with mortality in severely ill COVID-19 patients. Furthermore, the study showed that the SOFA score is a highly sensitive marker of in-hospital mortality in COVID-19 patients and is prognostically superior to qSOFA in this setting.
In our analysis, the SOFA illustrated acceptable predictive performance for in-hospital mortality on ROC curve analysis in both overall and subgroup analysis. At the time of admission, a SOFA score ≤ 3 was highly predictive of spontaneous survival in COVID-19 patients. The high NPV in overall analysis of 97.80% for SOFA functions as a gate keeper by accurately identifying low-risk patients.
The greater predictive advantage of the SOFA is attributable to the higher number of clinical parameters and inclusion of six important variables in the scoring system, which facilitated more accurate patient stratification. On univariate analysis, we found that five (mean arterial pressure, platelets count, bilirubin, creatinine, PaO2, and FiO2) of the six variables of the SOFA differed significantly between survivors and non-survivors. Similar findings have been reported by other observational studies. Mo et al. reported [27] that severely ill patients had a lower platelet count. In a meta-analysis conducted by Lippi and colleagues, [28] low platelet counts were associated with increased risk of mortality in patients with COVID-19. Zhang et al. [29] reported that a higher bilirubin level and lower oxygenation index were observed in severely or critically ill COVID-19 patients; furthermore, higher serum creatinine levels at admission were a predictive factor of disease severity in their patients. The pathogenic mechanisms underlying changes in the abovementioned clinical indicators in COVID-19 patients need to be further studied, although these indicators can be perceived, at the very least, as predictive factors of the mortality risk in critically ill patients [30].
The AUCs of qSOFA of both overall and subgroup analysis were lower than that of SOFA; therefore, we considered the accuracy of SOFA to be higher. The prerequisite parameters for SOFA can be determined through medical assessment and laboratory examination. Nevertheless, there is a need to recognize that, as qSOFA is simple, fast, and acceptable accuracy, it can be used in the emergency room or at admission if the parameters of the SOFA cannot be acquired in time.
This novel study explored the effectiveness of the SOFA and qSOFA scoring systems for predicting mortality risk of severely or critically ill COVID-19 patients and ascertained their practical value in identifying severely or critically ill COVID-19 patients with a higher mortality risk. However, some limitations of this study need to be considered. This study is limited by the small sample size. Studies with a larger number of severely or critically ill COVID-19 patients who fulfill the inclusion criteria would enable a more effective evaluation of the SOFA and qSOFA scoring systems. This was a single-center study and may have been affected by a selection bias with regard to the study population. Multicenter studies are required to prospectively validate the clinical utility of the SOFA and qSOFA scores in severely or critically ill COVID-19 patients. Moreover, the prognostic value of the scoring systems among patient subgroups that were stratified by gender groups or underlying diseases was not analyzed owing to the limited sample size Furthermore, because of the limited availability of information; details of out-of-hospital therapy and nursing care were not captured. Finally, there are several risk-scoring tools, such as the Acute Physiology and Chronic Health Evaluation scoring systems, that find widespread application in the critical care unit and emergency departments. However, in this retrospective study based on medical records, the information that could be collected could only permit a calculation of SOFA and qSOFA scores. Future studies are needed to comparatively explore the performance of other scoring systems in the COVID-19 patient population.
5. Conclusions
The SOFA scoring systems at admission have high classification performance in predicting the mortality risk of severely or critically ill COVID-19 patients and have acceptable NPV for screening these patients. The performance in predicting such patients of qSOFA is accepted but inferior to that of SOFA.
Funding sources
None.
CRediT authorship contribution statement
Sijia Liu:Software, Methodology, Formal analysis, Writing - original draft, Writing - review & editing.Ni Yao:Investigation, Formal analysis, Writing - original draft, Writing - review & editing.Yanru Qiu:Investigation, Validation, Writing - review & editing.Chengqi He:Conceptualization, Supervision.
Declaration of competing interest
Sijia Liu, Ni Yao, YanruQiu, and Chengqi He declared there are no conflicts of interest.
Acknowledgements
We would like to thank all the physicians and nurses who worked in the COVID-19 ward for helping us collect the data. And we would like to thank Editage (www.editage.cn) for English language editing.
References
- 1.Liu W., Yue X.G., Tchounwou P.B. Response to the COVID-19 epidemic: the Chinese experience and implications for other countries. Int J Environ Res Public Health. 2020 Mar 30;17(7) doi: 10.3390/ijerph17072304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Huang C., Wang Y., Li X. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395(10223):497–506. doi: 10.1016/S0140-6736(20)30183-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Yang X., Yu Y., Xu J. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. 2020;24 doi: 10.1016/S2213-2600(20)30079-5. S2213-2600(20)30079-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Chen Nanshan, Zhou Min, Dong Xuan, Jieming Qu, Gong Fengyun, Yang Han. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395(10223):507–513. doi: 10.1016/S0140-6736(20)30211-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bhatraju P.K., Ghassemieh B.J., Nichols M. Covid-19 in critically ill patients in the Seattle region — case series. N Engl J Med. 2020 doi: 10.1056/NEJMoa2004500. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Aggarwal Saurabh, Garcia-Telles Nelson, Aggarwal Gaurav. Clinical features, laboratory characteristics, and outcomes of patients hospitalized with coronavirus disease 2019 (COVID-19): early report from the United States. Diagnosis (Berl) 2020;7(2):91–96. doi: 10.1515/dx-2020-0046. [DOI] [PubMed] [Google Scholar]
- 7.Rong X.M., Yang L., Chu H.D., Fan M. Effect of delay in diagnosis on transmission of COVID-19. Math Biosci Eng. 2020;17(3):2725–2740. doi: 10.3934/mbe.2020149. [DOI] [PubMed] [Google Scholar]
- 8.Vincent J.L., Moreno R., Takala J., Willatts S., Mendonca A., Bruining H. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–710. doi: 10.1007/BF01709751. [DOI] [PubMed] [Google Scholar]
- 9.Schoe Abraham, Bakhshi-Raiez Ferishta, de Keizer Nicolette. Mortality prediction by SOFA score in ICU patients after cardiac surgery; comparison with traditional prognostic–models. BMC Anesthesiol. 2020;20:65–69. doi: 10.1186/s12871-020-00975-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315: 801–810. [DOI] [PMC free article] [PubMed]
- 11.Ferreira F.L., Bota D.P., Bross A., Melot C., Vincent J.L. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286:1754–1758. doi: 10.1001/jama.286.14.1754. [DOI] [PubMed] [Google Scholar]
- 12.Craig D.G., Zafar S., Reid T.W.D.J., Martin K.G., Davidson J.S., Hayes P.C. The sequential organ failure assessment (SOFA) score is an effective triage marker following staggered paracetamol (acetaminophen) overdose. Aliment Pharmacol Ther. 2012;35(12):1408–1415. doi: 10.1111/j.1365-2036.2012.05102.x. [DOI] [PubMed] [Google Scholar]
- 13.Tranca S., Petrișor C., Hagau N., Ciuce C. Can APACHE II, SOFA, ISS, and RTS severity scores be used to predict septic complications in multiple trauma patients? J Crit Care Med (TarguMures) 2016;2(3):124–130. doi: 10.1515/jccm-2016-0019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Jiawen Dai, Wu Jian, Bin Gu, Jianquan You, Mingdong Ding, Fei Qian. Value of TCC and qSOFA score on the early diagnosis of severe trauma. Chin J Emerg Med. 2019;28(2):185–189. [Google Scholar]
- 15.Jiang Jianjun, Yang Jin, Mei Jing, Jin Yongmei, Lu Youjin. Head-to-head comparison of qSOFA and SIRS criteria in predicting the mortality of infected patients in the emergency department: a meta-analysis. Scand J Trauma Resusc Emerg Med. 2018;26(1):56. doi: 10.1186/s13049-018-0527-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Maitra S., Som A., Bhattacharjee S. Accuracy of quick sequential organ failure assessment (qSOFA) score and systemic inflammatory response syndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta-analysis of observational studies. Clin Microbiol Infect. 2018 Nov;24(11):1123–1129. doi: 10.1016/j.cmi.2018.03.032. [DOI] [PubMed] [Google Scholar]
- 17.de Freitas Garbero Rodrigo, Simoes Analice Alves, Martins Gabriela Alves, da Cruz Ludmilla Vale, von Zuben Vinicius Gabriel Monteiro. SOFA and qSOFA at admission to the emergency department: diagnostic sensitivity and relation with prognosis in patients with suspected infection. Turk J Emerg Med. 2019;19(3):106–110. doi: 10.1016/j.tjem.2019.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Schlapbach L.J., Straney L., Bellomo R., MacLaren G., Pilcher D. Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among children with suspected infection admitted to the intensive care unit. Intensive Care Med. 2018;44(2):179–188. doi: 10.1007/s00134-017-5021-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Probst L., Schalk E., Liebregts T., Zeremski V., Tzalavras A., von Bergwelt-Baildon M. Prognostic accuracy of SOFA, qSOFA and SIRS criteria in hematological cancer patients: a retrospective multicenter study. J Intensive Care. 2019 Aug 7;7:41. doi: 10.1186/s40560-019-0396-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.National Health Commission. The diagnosis and treatment plan of novel coronavirus (version 6). http://www.nhc.gov.cn/yzygj/s7653p/202002/8334a8326dd94d329df351d7da8aefc2/files/b218cfeb1bc54639af227f922bf6b817.pdf. Accessed Feb 19, 2020.
- 21.DeLong E.R., DeLong D.M., Clarke-Pearson D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837. [PubMed] [Google Scholar]
- 22.Ji Y., Ma Z., Peppelenbosch M.P., Pan Q. Potential association between COVID-19 mortality and health-care resource availability. Lancet Glob Health. 2020;8(4) doi: 10.1016/S2214-109X(20)30068-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Swiss Academy Of Medical Sciences COVID-19 pandemic: triage for intensive-care treatment under resource scarcity. Swiss Med Wkly. 2020;3 doi: 10.4414/smw.2020.20229. w20229. [DOI] [PubMed] [Google Scholar]
- 24.She-yu Li, Wen-zhi Huang, Liao Xue-lian. Emergency recommendation of West China hospital for the prevention and control of novel coronavirus pneumonia in hospitals. Chin J Evid Based Med. 2020;20(2):125–133. [Google Scholar]
- 25.Lambden Simon, Laterre Pierre Francois, Levy Mitchell M., Francois Bruno. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care. 2019;23(1):374. doi: 10.1186/s13054-019-2663-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Maitra S., Som A., Bhattacharjee S. Accuracy of quick sequential organ failure assessment (qSOFA) score and systemic inflammatory response syndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta-analysis of observational studies. Clin Microbiol Infect. 2018;24(11):1123–1129. doi: 10.1016/j.cmi.2018.03.032. [DOI] [PubMed] [Google Scholar]
- 27.Mo Pingzheng, Xing Yuanyuan, Yu Xiao, Deng Liping, Zhao Qiu, Wang Hongling. Clinical characteristics of refractory COVID-19 pneumonia in Wuhan, China. Clin Infect Dis. 2020;3 doi: 10.1093/cid/ciaa270. ciaa270. [DOI] [Google Scholar]
- 28.Lippi G., Plebani M., Henry B.M. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta. 2020 Mar 13;506:145–148. doi: 10.1016/j.cca.2020.03.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Zhang X., Cai H., Hu J., Lian J., Gu J., Zhang S. Epidemiological, clinical characteristics of cases of SARS-CoV-2 infection with abnormal imaging findings. Int J Infect Dis. 2020;3:S1201–9712(20). doi: 10.1016/j.ijid.2020.03.040. 30172-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Yang W., Cao Q., Qin L., Wang X., Cheng Z., Pan A. Clinical characteristics and imaging manifestations of the 2019 novel coronavirus disease (COVID-19): a multi-center study in Wenzhou city, Zhejiang, China. J Infect. 2020;80(4):388–393. doi: 10.1016/j.jinf.2020.02.016. [DOI] [PMC free article] [PubMed] [Google Scholar]