Abstract
Background
Identification of patients on admission to hospital with coronavirus infectious disease 2019 (COVID-19) pneumonia who can develop poor outcomes has not yet been comprehensively assessed.
Objective
To compare severity scores used for community-acquired pneumonia to identify high-risk patients with COVID-19 pneumonia.
Design
PSI, CURB-65, qSOFA, and MuLBSTA, a new score for viral pneumonia, were calculated on admission to hospital to identify high-risk patients for in-hospital mortality, admission to an intensive care unit (ICU), or use of mechanical ventilation. Area under receiver operating characteristics curve (AUROC), sensitivity, and specificity for each score were determined and AUROC was compared among them.
Participants
Patients with COVID-19 pneumonia included in the SEMI-COVID-19 Network.
Key results
We examined 10,238 patients with COVID-19. Mean age of patients was 66.6 years and 57.9% were males. The most common comorbidities were as follows: hypertension (49.2%), diabetes (18.8%), and chronic obstructive pulmonary disease (12.8%). Acute respiratory distress syndrome (34.7%) and acute kidney injury (13.9%) were the most common complications. In-hospital mortality was 20.9%. PSI and CURB-65 showed the highest AUROC (0.835 and 0.825, respectively). qSOFA and MuLBSTA had a lower AUROC (0.728 and 0.715, respectively). qSOFA was the most specific score (specificity 95.7%) albeit its sensitivity was only 26.2%. PSI had the highest sensitivity (84.1%) and a specificity of 72.2%.
Conclusions
PSI and CURB-65, specific severity scores for pneumonia, were better than qSOFA and MuLBSTA at predicting mortality in patients with COVID-19 pneumonia. Additionally, qSOFA, the simplest score to perform, was the most specific albeit the least sensitive.
KEY WORDS: CURB-65, PSI, qSOFA, COVID-19, community-acquired pneumonia
BACKGROUND
At the end of 2019, a novel coronavirus was identified as the cause of an outbreak of pneumonia cases in Wuhan, Hubei Province, China. This virus, now known as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), rapidly spread around the world, causing a pandemic that has affected more than 61·8 million people as of 30 November 2020. 1–3 The disease that it produces is called coronavirus disease 2019 (COVID-19) and is characterized by a wide spectrum of infectious symptoms that range from mild to severe, with acute respiratory distress syndrome (ARDS) as its primary complication.4–6 Hospitalization is required in 6 to 20% of patients. 7,8 Mortality among hospitalized patients is high, ranging from 11 to 28%.4,5 Therefore, early and simple identification upon admission to hospital of patients who may have poor outcomes would be of considerable value and may help lead to prompt treatment and an optimized use of resources.
Several prognostic scores have been developed to identify elevated risk of death in patients with community-acquired pneumonia (CAP). Two of them, the Pneumonia Severity Index (PSI) and the CURB-65 score, are well-validated scores for supporting pneumonia prognoses.9,10 The MuLBSTA score is based on six parameters (Multilobar infiltration, hypo-Lymphocytosis, Bacterial coinfection, Smoking history, hypertension, and age) routinely measured in hospitals. MuLBSTA is a new prognostic tool for patients hospitalized with viral pneumonia11 and has been suggested as a severity score for patients with COVID-19.5 The qSOFA, based on the sepsis-3 criteria,12 has proven to be a useful tool in the emergency department,13,14 non-intensive-care-unit (ICU) wards,15–17 and even at home 18,19 for establishing a prognosis in patients with documented or suspected infection with different foci. Recent works indicate that qSOFA is also a useful tool for identifying patients with poor prognoses in viral infections such as influenza.20–22 In addition, qSOFA has shown good performance in predicting prognosis in pneumonia, both in the emergency department 23–27 and in wards.28 The advantage of the qSOFA score is that the variables are clinical and thus laboratory tests are not required.12
OBJECTIVE
In this study, we aim to ascertain the applicability and prognostic prediction value of the PSI, CURB-65, MuLBSTA, and qSOFA severity scores in COVID-19 patients with pneumonia admitted to hospital.
DESIGN
Study design and participants
This work is a multicenter retrospective cohort study of patients with SARS-CoV2 infection, aged ≥18 years, and with CAP who were hospitalized between March 1 and May 28, 2020, in Spain and included in the aforementioned SEMI-COVID-19 Registry.29 The registry is an initiative of the Spanish Society of Internal Medicine (SEMI, for its initials in Spanish), and its characteristics have been described elsewhere.29 Data are collected retrospectively and include epidemiological and clinical characteristics, such as comorbidities, symptoms, physical examination findings, laboratory and diagnostic imaging test results, and clinical outcomes.
Definition of variables
COVID-19 pneumonia was defined as compatible symptoms (cough, dyspnea, need for respiratory support, fever, or crackling or rales on auscultation), pulmonary infiltrates, or consolidation detected by a chest X-ray or computerized tomography (CT) scan, and a positive result on either a polymerase chain reaction test of a nasopharyngeal sample or a serology test for COVID-19 antibodies.
We defined PSI, CURB-65, MuLBSTA, and qSOFA as originally described 9–12. PSI is a scale with 19 variables with different point values.9 A cut-off of 91 points (risk class IV–V) was considered positive for a poor prognosis 24–28. We considered CURB-65 to be predictive of poor prognosis when three of the five variables (confusion, BUN >7 mmol/L, respiratory rate ≥30 bpm, systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg, and age ≥65 years) were positive 21–24. The MuLBSTA score is a newly designed score for viral pneumonia11 that considers the following variables: multilobar infiltrates (5 points), lymphocytes ≤0·8 x10e9/ml (4 points), bacterial infection (4 points), active smoker (3 points), prior smoker (2 points), hypertension (2 points), and age ≥60 years (2 points), with a cut-off point of 12 points or more is related with poor prognosis, as defined in the original study.11 qSOFA was considered positive when two of the three variables (altered mental status, respiratory rate ≥22 brpm, and/or systolic blood pressure <100 mmHg) were met, according to the Sepsis-3 definition.12
Outcome measures
The primary outcome measure was all-cause in-hospital mortality. As secondary outcomes, we analyzed admission to intensive care unit (ICU) and use of mechanical ventilation.
Statistical analysis
After a preliminary descriptive analysis of the data (SD, means, and percentages), we compared the differences between categorical variables using the chi-square test and continuous variables using Student’s t test or analysis of variance. Sensitivity and specificity of the scores were calculated using the cut-off points of poor prognosis proposed in the validation study of the score (MuLBSTA and qSOFA) or habitual use in other studies (PSI and CURB-65), as is detailed in the definition of variables. In the AUROC, all scores were analyzed by points, as a graded risk assessment system. When data for the calculation of the score were missing, the case was excluded. The SPSS statistical package version 22.0 (SPSS, Chicago, IL) was used for the statistical study and the Epidat version 3.1 program was used for comparing the AUROCs of the different scales. P values <0.05 were considered statistically significant.
Ethical aspects
Scientific and ethical permission to conduct this study was obtained from the Provincial Research Ethics Committee of Málaga (Spain). Informed consent was obtained from patients for the purpose of publication. Personal data were processed in strict compliance with Spanish Law 14/2007, of July 3, on Biomedical Research; Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation), and Spanish Organic Law 3/2018, of December 5, on Personal Data Protection and the Guarantee of Digital Rights.
Key results
A total of 10,238 patients were included in this study. The patient inclusion flowchart can be seen in Figure 1. Epidemiological and clinical data are found in Table 1. The mean age was 66.6 ± 16.2 years and 57.9% of patients were male. A total of 5830 (56.9%) patients had an age-adjusted Charlson Comorbidity Index of 3 or more points. The most frequent comorbidities were hypertension (49.2%), diabetes (18.8%), and COPD (12.2%). ARDS and acute kidney injury were the most common complications (34.7% and 13.9%, respectively).
Figure 1.
Flowchart of patients included from the SEMI-COVID-19 Network with COVID-19 pneumonia. COVID-19: coronavirus infectious disease 2019.
Table 1.
Epidemiological and Clinical Characteristics, Complications, Prognostic Scores and Outcomes of Patients with COVID-19 Pneumonia. COVID-19: Coronavirus Infectious Disease 2019
| All patients n 10,238 |
Non-survivors n 2135 |
Survivors n 8103 |
p | |
|---|---|---|---|---|
| Demographics | ||||
| Male sex, n (%) | 5924 (57.9) | 1336 (62.6) | 4588 (56.6) | <0.001 |
| Age, mean ± SD, years | 66.6 ± 16.2 | 79.5 ± 10.6 | 63.2 ± 15.6 | <0.001 |
| ≥ 70 years, n (%) | 4765 (46.5) | 1810 (38) | 2955 (62) | <0.001 |
| Comorbidities, n (%) | ||||
| Moderate or severe dependency | 1504 (14.7) | 787 (36.9) | 717 (8.8) | <0.001 |
| Age-adjusted Charlson comorbidity index ≥3 | 5830 (56.9) | 1934 (90.6) | 3896 (48.1) | <0.001 |
| Hypertension | 5040 (49.2) | 1496 (70.1) | 3544 (43.7) | <0.001 |
| Coronary artery disease | 703 (6.9) | 333 (15.6) | 370 (4.6) | <0.001 |
| COPD† | 1250 (12.2) | 502 (23.5) | 748 (9.2) | <0.001 |
| Cerebrovascular disease | 250 (2.4) | 116 (5.4) | 134 (1.7) | <0.001 |
| Smoking history | 2424 (23.7) | 646 (30.3) | 1778 (21.9) | <0.001 |
| Active smoking | 492 (4.8) | 98 (4.6) | 394 (4.9) | <0.001 |
| Diabetes | 1926 (18.8) | 618 (28.9) | 1308 (16.1) | <0.001 |
| Moderate-severe CKD‡ | 577 (5.6) | 261 (12.2) | 316 (3.9) | <0.001 |
| Malignant tumor | 595 (5.8) | 199 (9.3) | 396 (4.9) | <0.001 |
| Complications, n (%) | ||||
| Shock | 482 (4.7) | 370 (17.3) | 112 (1.4) | <0.001 |
| Acute kidney injury | 1423 (13.9) | 781 (36.6) | 642 (7.9) | <0.001 |
| Acute pulmonary embolism | 158 (1.5) | 28 (1.3) | 130 (1.6) | 0.790 |
| Multi-organic failure | 667 (6.5) | 375 (17.6) | 532 (6.6) | <0.001 |
| ARDS§ | 3550 (34.7) | 1730 (81) | 1820 (22.5) | <0.001 |
| Prognostic scores, n (%) | ||||
| qSOFA‖ | 9887 | 2049 (20.7) | 7838 (79.3) | ·· |
| 0 | 5788 (58.6) | 552 (26.9) | 5236 (66.8) | <0.001 |
| 1 | 3219 (32.6) | 952 (46.5) | 2267 (28.9) | <0.001 |
| 2 | 779 (7.8) | 464 (22.7) | 315 (4) | <0.001 |
| 3 | 101 (1) | 81 (3.9) | 20 (0.3) | <0.001 |
| CURB-65¶ | 9887 | 2049 (20.7) | 7838 (79.3) | ·· |
| 0-1 | 5899 (59.7) | 366 (17.9) | 5533 (70.6) | <0.001 |
| 2 | 2232 (22.6) | 657 (32.1) | 1575 (20.1) | <0.001 |
| 3 | 1282 (12.9) | 679 (33.1) | 603 (7.7) | <0.001 |
| 4 | 438 (4.4) | 317 (15.5) | 121 (1.5) | <0.001 |
| 5 | 36 (0.4) | 30 (1.4) | 6 (0.1) | <0.001 |
| PSI# | 9261 | 1906 (20.6) | 7355 (79.4) | ·· |
| I–II | 3376 (36.5) | 55 (2.9) | 3321 (45.2) | <0.001 |
| III | 2168 (23.4) | 247 (12.9) | 1921 (26.1) | <0.001 |
| IV | 2566 (27.7) | 860 (45.1) | 1706 (23.2) | <0.001 |
| V | 1151 (12.4) | 744 (39.1) | 407 (5.5) | <0.001 |
| MuLBSTA** | 9505 | 1956 (20.6) | 7549 (79.4) | ·· |
| MuLBSTA ≥12 | 1298 (13.7) | 588 (30.1) | 710 (9.4) | <0.001 |
| Outcomes, n (%) | ||||
| NIV†† | 528 (5.2) | 269 (12.6) | 259 (3.2) | <0.001 |
| Mechanical ventilation | 722 (7.1) | 340 (15.9) | 382 (4.7) | <0.001 |
| Admission to ICU‡‡ | 907 (8.9) | 375 (17.6) | 532 (6.6) | <0.001 |
| Mortality | 2135 (20.9) | – | – | – |
| Length of hospital stay, mean ± SD, days | 11.2 ± 9.2 | 9.7 ± 10.3 | 11.6 ± 8.8 | <0.001 |
†COPD: chronic obstructive pulmonary disease; ‡CKD: chronic kidney disease; §ARDS: acute respiratory distress syndrome; ‖qSOFA: quick Sequential Organ Failure Assessment; ¶CURB-65: Confusion, Urea, Respiratory rate, Blood pressure, and age ≥65 years; #PSI: Pneumonia Severity Index; **MuLBSTA: Multilobar infiltration, hypo-Lymphocytosis, Bacterial coinfection, Smoking history, hyper-Tension and Age; ††NIV: non-invasive ventilation; ‡‡ICU: intensive care unit
In-hospital mortality was 20.9% (2135 cases). A total of 907 (8.9%) patients were admitted to ICU. The mean length of hospital stay was 11.2 ± 9.2 days. As expected, the patients who died were older and had more comorbidities and complications. Prognostic scores were more frequently positive in deceased patients, as can be seen in Table 1. Patients who died also had a longer mean hospital stay (9.7 vs 11.6 days, p <0.001) and were more frequently admitted to the ICU (17.6% vs 6.6%, p <0.001).
PSI was calculated in 9261 (90.5%) cases, CURB-65 and qSOFA were calculated in 9887 (96.6%) cases, and MuLBSTA was calculated in 9505 (92.8%) cases. The PSI and CURB-65 scores showed no statistical differences upon comparison (AUROC 0.835 vs 0.825, p=0.112) for mortality and were higher than the qSOFA and MuLBSTA scores (AUROC 0.728 and 0.715, respectively, p<0.001 for both compared to PSI and CURB-65). These findings can be seen in Table 2 and Figure 2. The qSOFA score showed no differences compared to the MuLBSTA score (AUROC 0.728 vs 0.715, p=0.102) and was the most specific score (95.72%), as can be seen in Table 3, but it had a lower sensitivity. PSI risk class IV–V had the highest sensitivity (84.12%) but a lower specificity than the qSOFA and MuLBSTA scores (72.25% vs 95.72% and 91.23%, p <0.001 for both scores).
Table 2.
Statistical Data of ROC Comparisons Between PSI, CURB-65, MuLBSTA, and qSOFA Scores for In-Hospital Mortality, Admission to ICU, and Use Of Mechanical Ventilation in Patients with COVID-19 Pneumonia. ROC: Receiver Operating Characteristic Curve
| AUROC# (95% CI) | vs. PSI† | vs. CURB-65‡ | vs. MuLBSTA§ | |
|---|---|---|---|---|
| Mortality | ||||
| PSI† | 0.835 (0.826–0.845) | – | – | – |
| CURB-65‡ | 0.825 (0.815–0.835) | 0.112 | – | – |
| MuLBSTA§ | 0.715 (0.703–0.727) | <0.001 | <0.001 | – |
| qSOFA‖ | 0.728 (0.715–0.741) | <0.001 | <0.001 | 0.102 |
| Admission to ICU | ||||
| PSI† | 0.539 (0.521–0.557) | – | – | – |
| CURB-65‡ | 0.562 (0.544–0.580) | 0.064 | – | – |
| MuLBSTA§ | 0.658 (0.640–0.677) | <0.001 | <0.001 | – |
| qSOFA‖ | 0.616 (0.598–0.635) | <0.001 | <0.001 | 0.001 |
| Mechanical ventilation | ||||
| PSI† | 0.560 (0.540–0.579) | – | – | – |
| CURB-65‡ | 0.572 (0.553–0.592) | 0.349 | – | – |
| MuLBSTA§ | 0.678 (0.657–0.698) | <0.001 | <0.001 | – |
| qSOFA‖ | 0.624 (0.603–0.644) | <0.001 | <0.001 | <0.001 |
†PSI: Pneumonia Severity Index; ‡CURB-65: Confusion, Urea, Respiratory rate, Blood pressure, and age ≥65 years; §MuLBSTA: Multilobar infiltration, hypo-Lymphocytosis, Bacterial coinfection, Smoking history, hyper-Tension, and Age; ‖qSOFA: quick Sequential Organ Failure Assessment; ¶COVID-19: coronavirus infectious disease 2019; #AUROC: area under the receiver operating characteristic curve
Figure 2.
Receiver operating characteristic curves for PSI, CURB65, MuLBSTA, and qSOFA scores for in-hospital mortality in COVID19 pneumonia patients. PSI: Pneumonia Severity Index; CURB-65: Confusion, Urea, Respiratory rate, Blood pressure, and age ≥65 years; MuLBSTA: Multilobar infiltration, hypo-Lymphocytosis, Bacterial coinfection, Smoking history, hyper-Tension and Age; qSOFA: quick Sequential Organ Failure Assessment; COVID-19: coronavirus infectious disease 2019.
Table 3.
Comparison of Predictive Assessments Between PSI, CURB-65, MuLBSTA, and qSOFA Scores for In-Hospital Mortality in Patients with COVID-19 Pneumonia. PSI: Pneumonia Severity Index
| Sensitivity, % (95% CI) | Specificity, % (95% CI) | PPV¶, % (95% CI) | NPV#, % (95% CI) | |
|---|---|---|---|---|
| PSI ≥IV | 84.12 (82.45–85.79) | 72.25 (71.21–73.29) | 44.28 (42.65–45.91) | 94.55 (93.94–95.15) |
| CURB-65 ≥3 | 82.13 (80.45–83.82) | 70.59 (69.57–71.60) | 42.20 (40.65–43.74) | 93.79 (93.17–94.41) |
| MuLBSTA ≥12 | 27.54 (25.62–29.45) | 91.23 (90.61–91.85) | 45.30 (42.55–48.04) | 82.69 (81.90–83.48) |
| qSOFA≥2 | 26.59 (24.66–28.53) | 95.72 (95.27–96.18) | 61.93 (58.66–65.19) | 83.30 (82.52–84.07) |
†CURB-65: Confusion, Urea, Respiratory rate, Blood pressure, and Age ≥65 years; ‡MuLBSTA: Multilobar infiltration, hypo-Lymphocytosis, Bacterial coinfection, Smoking history, hyper-Tension, and Age; §qSOFA: quick Sequential Organ Failure Assessment; ‖COVID-19: coronavirus infectious disease 2019; ¶PPV: positive predictive value; #NPV: negative predictive value
The results of the secondary outcomes admission to ICU or use of mechanical ventilation are shown in Table 2. As we can see, in both outcomes, MuLBSTA and qSOFA were superior to PSI and CURB-65, which had no difference between them. However, the prognostic accuracy of the scores for these outcomes was suboptimal.
DISCUSSION
To our knowledge, this is the largest study evaluating prognostic scores in patients with COVID-19 pneumonia. The in-hospital mortality observed in this multicenter study in Spain was high (20.9%). This could be due to the fact that the mean age of patients was 66.6 years and there was a high frequency of comorbidities. The information obtained at admission allowed us to compare severity scores for identifying patients at high risk of in-hospital death. PSI and CURB-65 had better prognostic accuracy, with an AUROC of 0.835 and 0.825, respectively.
Mortality in our results was higher than that in other cohorts as Liang et al. (3.2%)30 or Jang et al. (5.5%)31, but similar to Fan et al. (20.3%)32 or Lalueza et al. (20.7%)33. In our cohort, the mean age was 66.6 years and 56.9% patients had an age-adjusted Charlson Comorbidity Index of 3 or more points, in consonance with cohorts with similar mortality (mean age 65.9, median Charlson Comorbidity Index of 3 (1–5))33; meanwhile, the cohorts with lower mortality had lower mean age (48.9 and 56.9 years30,31) and less comorbidities.30,31
PSI was initially designed to help practitioners identify which low-risk patients with CAP could be safely treated in an outpatient setting,9 although it has subsequently been used to assess post-discharge mortality for those treated as inpatients. The capability of PSI to predict hospital mortality in our study was similar than another study in COVID-19 pneumonia (AUROC 0.85).32 This AUROC is similar or even slightly better than that of previous studies in CAP, in which PSI presented an AUROC of 0.778 to predict 30-day mortality or transfer to ICU 28 and 0.812 to predict 30-day mortality.34 Our findings confirm that PSI is quite a good score for assessing the risk of in-hospital mortality for patients admitted with COVID-19 pneumonia.
CURB-65 was developed to stratify hospitalized patients with CAP into mortality risk groups, with a primary outcome measure of 30-day mortality.10 It has shown a diagnostic performance for mortality similar to the PSI in CAP.24 The diagnostic performance of CURB-65 for mortality found in our study was comparable to that found by Su et al.35 (AUROC 0.85) in their study on 116 patients with COVID-19 in China, whose primary outcome measure was intensive respiratory or vasopressor support. These results are consistent with other COVID-19 studies with mortality as primary outcome (AUROC 0.85)32, and better than those with composed primary outcomes including mortality (AUROC 0.75 and 0.5) 30,36.
MuLBSTA is a recently described severity score for 90-day mortality in patients with viral pneumonia.11 It has been shown to perform better than CURB-65 in viral pneumonia, mainly caused by influenza virus, human rhinovirus, and respiratory syncytial virus.11 Chen et al. suggested using MuLBSTA as a severity score for COVID-19 in their description of 99 patients in Wuhan, China 5, because the patients who died had a high percentage of variables included in this score. In our results, the AUROC of MuLBSTA for mortality (0.715) showed lower sensitivity compared to the other scores as well as when compared to the original work (0.715 vs 0.773, p <0.001). Further studies are needed in order to prove the suitability of the MuLBSTA score.
In our study, qSOFA showed a lower sensitivity than the PSI and CURB-65 scores (AUROC 0.728 vs 0.835 and 0.825, p <0.001 for both scores). These results were in line with the results found by Su et al.35 in their study previously commented. The superiority of CRB-65, a score comparable to CURB-65, versus qSOFA for prediction of intensive respiratory or vasopressor support (AUROC 0.81 vs 0.70, p 0.02) was observed in this study. It was also similar to other studies previously commented, with primary outcome of mortality (AUROC 0.73)32. In this study, PSI and CURB-65 were also superior to qSOFA (AUROC 0.85 both scores vs. 0.73)32. Jang et al.31, in a study with 110 patients, showed an AUROC of 0.779 for the primary outcome of mortality at day 28, and 0.760 for “critical outcomes,” composed by transfer to ICU, septic shock, or ARDS. Our results were superior to that showed by Lalueza et al. in a retrospective study with 237 patients33, with an AUROC of 0.69 for respiratory failure or death. Other studies have also shown that the accuracy of qSOFA in patients with COVID-19 was limited,37,38 although these studies included a small number of patients. However, qSOFA has been related to mortality in COVID-19 in another study with 191 patients (OR 12 (CI 95% 5.06–28.43)).4
In our work, the PSI and CURB-65 scores showed sensitivity and specificity values similar to those found in other studies on CAP.23,28,34 However, qSOFA sensitivity was lower than the other scores and lower for CAP (53 to 70.1%).23,26 qSOFA had a higher specificity than the other scores in our study (95.72% vs 72.25%, 90.68%, and 91.23% for PSI, CURB-65, and MuLBSTA, respectively). These results for qSOFA are consistent with the findings of Su et al.,35 who observed a similar specificity for qSOFA and CURB-65 (98.9% and 96.7%, respectively). Despite its lower sensitivity, qSOFA has the benefit of not requiring laboratory test results.
In the study by Su et al.,35 the CRB score was also analyzed, which also has the benefit of not requiring laboratory results. In that study, no difference was found between CRB and qSOFA.35 These two scores had a lower accuracy than CRB-65 and CURB-65, suggesting that age is an important risk factor for mortality. Indeed, age has been shown to be an independent risk factor for mortality in patients with COVID-19.39,40
This study has some limitations. First, due to its retrospective design, some possible confounding variables were not recorded and thus we could not calculate prognostic scores in 3.4 to 9.5% of cases. Second, we focused only on hospitalized patients and as such, we cannot be certain that our findings can be extrapolated to outpatients. Third, by excluding patients still hospitalized as of May 29, 2020, the case fatality rate in our study does not reflect the true mortality rate of COVID-19. Fourth, this study was conducted in multiple centers in Spain and its results may not be applicable to other settings with different populations or healthcare systems. Fifth, the results of the secondary outcomes may be biased by the availability of ICU beds and ventilators in the first months of the pandemic.
It is important to note that although these scores could help physicians identify patients with COVID-19 pneumonia at admission to hospital who have different risk levels for death, there are other risk factors specific to COVID-19 that should be considered, such as lymphopenia or D-dimer,4 which are not analyzed in these scores.
Zhou et al 4 showed that a D-dimer of >1 μg/ml was associated with mortality (OR 18.42 (95% CI 2.64–128.55)). In a retrospective study with 4389 patients41, anticoagulation was associated with lower mortality and intubation among hospitalized COVID-19 patients. However, our study is focused on prognostic scores, and regrettably we did not analyze the effect of treatments in mortality.
Recently, more than 22 specific scores for COVID-19 have been developed42. The CALL scoring system, composed by underlying comorbidity, advanced age, higher LDH, and lower lymphocyte count, showed an AUROC of 0.91 for progression of COVID-19 43, and is simpler than the 12-parameter MuLBSTA score proposed by Guo et al.11 Liang et al. 30 developed and validate other more complex system, the GRAM-COVID, a clinical risk score to predict the development of critical illness among hospitalized patients infected with COVID-19. 30 Performance of this risk score was satisfactory with AUC-based precision in the development and validation cohorts of 0.88. But both scores30,43 are limited by a modest sample size for constructing the risk score and a relatively small sample for validation. The PRIORITY score42 has been developed by the SEMI-COVID-19 Network to predict critical illness, in a retrospective study with 10433 patients. This score that includes epidemiological and clinical variables has shown an AUROC of 0.795 in the validation cohort.
However, development of new scores always needs an external validation and other studies to prove its usefulness. Therefore, we believe that using some well-established scores such as CURB-65 and PSI, used in pneumonia, or a general and simple sepsis score as qSOFA, will be in benefit of the clinician and the patients.
CONCLUSIONS
PSI and CURB-65, two severity scores specific to pneumonia, performed better than qSOFA and MuLBSTA for predicting all-cause in-hospital death for patients with COVID-19 pneumonia. qSOFA, the simplest score to calculate, was the most specific, albeit the least sensitive. The four scores had a suboptimal performance for the secondary outcomes admission to ICU or use of mechanical ventilation.
Acknowledgements
We want to thank the contribution of the SEMI-COVID-19 Network for data retrieval.
Declarations
Conflict of interest
The authors declare that they do not have a conflict of interest.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Lu H, Stratton C, Tang Y. Outbreak of pneumonia of unknown etiology in Wuhan, China: The mystery and the miracle. J Med Virol. 2020;92(4):401–2. doi: 10.1002/jmv.25678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lin L, Lu L, Cao W, Li T. Hypothesis for potential pathogenesis of SARS-CoV-2 infection-a review of immune changes in patients with viral pneumonia. Emerg Microbes Infect. 2020;9(1):727–32. doi: 10.1080/22221751.2020.1746199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Coronavirus disease (COVID-19) Situation Report. https://covid19.who.int/: World Health Organization; 2020, consulted on 30th november 2020.
- 4.Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395(10229):1054–62. doi: 10.1016/S0140-6736(20)30566-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chen N, Zhou M, Dong X, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395(10223):507–13. doi: 10.1016/S0140-6736(20)30211-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ruan Q, Yang K, Wang W, Jiang L, Song J. Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China. Intensive Care Med. 2020;46(5):846–8. doi: 10.1007/s00134-020-05991-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rosenberg ES, Dufort EM, Blog DS, et al. COVID-19 Testing, Epidemic Features, Hospital Outcomes, and Household Prevalence, New York State-March 2020. Clin Infect Dis. 2020. [DOI] [PMC free article] [PubMed]
- 8.Wu Z, McGoogan J. Characteristics of and important lessons from the Coronavirus Disease 2019 (COVID-19) outbreak in China. Summary of a Report of 72314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323(13):1239–42. doi: 10.1001/jama.2020.2648. [DOI] [PubMed] [Google Scholar]
- 9.Fine M, Hough L, Medsger A, et al. The hospital admission decision for patients with community acquired pneumonia. Arch Intern Med. 1997;157:36–44. doi: 10.1001/archinte.1997.00440220040006. [DOI] [PubMed] [Google Scholar]
- 10.Lim W, van der Eerden M, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58:377–82. doi: 10.1136/thorax.58.5.377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Guo L, Wei D, Zhang X, et al. Clinical Features Predicting Mortality Risk in Patients With Viral Pneumonia: The MuLBSTA Score. Front Microbiol. 2019;10:2752. doi: 10.3389/fmicb.2019.02752. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) JAMA. 2016;315(8):762–74. doi: 10.1001/jama.2016.0288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Freund Y, Lemachatti N, Krastinova E, et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. JAMA. 2017;317(3):301–8. doi: 10.1001/jama.2016.20329. [DOI] [PubMed] [Google Scholar]
- 14.April MD, Aguirre J, Tannenbaum LI, et al. Sepsis Clinical Criteria in Emergency Department Patients Admitted to an Intensive Care Unit: An External Validation Study of Quick Sequential Organ Failure Assessment. J Emerg Med. 2017;52(5):622–31. doi: 10.1016/j.jemermed.2016.10.012. [DOI] [PubMed] [Google Scholar]
- 15.Ramos-Rincón J, Fernández-Gil A, Merino E, et al. The quick Sepsis-related Organ Failure Assessment (qSOFA) is a good predictor of in-hospital mortality in very elderly patients with bloodstream infections. A retrospective observational study. Sci Rep. 2019;9(1):15075. doi: 10.1038/s41598-019-51439-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.LeGuen M, Ballueer Y, McKay R, et al. Frequency and significance of qSOFA criteria during adult rapid response team reviews: A prospective cohort study. Resuscitation. 2018;122:13–8. doi: 10.1016/j.resuscitation.2017.11.036. [DOI] [PubMed] [Google Scholar]
- 17.Siddiqui S, Chua M, Kumaresh V, Choo R. A comparison of pre ICU admission SIRS, EWS and q SOFA scores for predicting mortality and length of stay in ICU. J Crit Care. 2017;41:191–3. doi: 10.1016/j.jcrc.2017.05.017. [DOI] [PubMed] [Google Scholar]
- 18.Vaittinada Ayar P, Delay M, Avondo A, et al. Prognostic value of prehospital quick sequential organ failure assessment score among patients with suspected infection. Eur J Emerg Med. 2019;26(5):329–33. doi: 10.1097/MEJ.0000000000000570. [DOI] [PubMed] [Google Scholar]
- 19.Koyama S, Yamaguchi Y, Gibo K, Nakayama I, Ueda S. Use of prehospital qSOFA in predicting in-hospital mortality in patients with suspected infection: A retrospective cohort study. PLoS One. 2019;14(5):e0216560. doi: 10.1371/journal.pone.0216560. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Yeh CC, Chen YA, Hsu CC, et al. Quick-SOFA score>/2 predicts prolonged hospital stay in geriatric patients with influenza infection. Am J Emerg Med. 2020;38(4):780–4. doi: 10.1016/j.ajem.2019.06.041. [DOI] [PubMed] [Google Scholar]
- 21.Papadimitriou-Olivgeris M, Gkikopoulos N, Wust M, et al. Predictors of mortality of influenza virus infections in a Swiss Hospital during four influenza seasons: Role of quick sequential organ failure assessment. Eur J Intern Med. 2020;74:86–91. doi: 10.1016/j.ejim.2019.12.022. [DOI] [PubMed] [Google Scholar]
- 22.Chang SH, Yeh CC, Chen YA, et al. Quick-SOFA score to predict mortality among geriatric patients with influenza in the emergency department. Medicine (Baltimore) 2019;98(23):e15966. doi: 10.1097/MD.0000000000015966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.George N, Elie-Turenne MC, Seethala RR, et al. External Validation of the qSOFA Score in Emergency Department Patients With Pneumonia. J Emerg Med. 2019;57(6):755–64. doi: 10.1016/j.jemermed.2019.08.043. [DOI] [PubMed] [Google Scholar]
- 24.Ranzani O, Prina E, Menéndez R, et al. New Sepsis Definition (Sepsis-3) and Community-Acquired Pneumonia Mortality: a validation and clinical decision-making stuy. Am J Respir Crit Care Med. 2017;196(10):1287–97. doi: 10.1164/rccm.201611-2262OC. [DOI] [PubMed] [Google Scholar]
- 25.Song H, Moon HG, Kim SH. Efficacy of quick Sequential Organ Failure Assessment with lactate concentration for predicting mortality in patients with community-acquired pneumonia in the emergency department. Clin Exp Emerg Med. 2019;6(1):1–8. doi: 10.15441/ceem.17.262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Zhang X, Liu B, Liu Y, Ma L, Zeng H. Efficacy of the quick sequential organ failure assessment for predicting clinical outcomes among community-acquired pneumonia patients presenting in the emergency department. BMC Infect Dis. 2020;20(1):316. doi: 10.1186/s12879-020-05044-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Zhou H, Lan T, Guo S. Stratified and prognostic value of admission lactate and severity scores in patients with community-acquired pneumonia in emergency department. Medicine. 2019;98(41):e17479. doi: 10.1097/MD.0000000000017479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Asai N, Watanabe H, Shiota A, et al. Efficacy and accuracy of qSOFA and SOFA scores as prognostic tools for community-acquired and healthcare-associated pneumonia. Int J Infect Dis. 2019;84:89–96. doi: 10.1016/j.ijid.2019.04.020. [DOI] [PubMed] [Google Scholar]
- 29.Casas-Rojo J, Antón-Santos JM, Millán-Núñez-Cortés J, et al. Clinical characteristics of patients hospitalized with COVID-19 in Spain: results from the SEMI-COVID-19 Registry. Rev Clin Esp. 2020;220(8):480–494. doi: 10.1016/j.rceng.2020.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Liang W, Liang H, Ou L, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med. 2020;180(8):1081–89. doi: 10.1001/jamainternmed.2020.2033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Jang JG, Hur J, Hong KS, et al. Prognostic accuracy of the SIRS, qSOFA, and NEWS for early detection of clinical deterioration in SARS-CoV-2 infected patients. J Korean Med Sci. 2020;35(25):e234. doi: 10.3346/jkms.2020.35.e234. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Fan G, Tu C, Zhou F, et al. Comparison of severity scores for COVID-19 patients with pneumonia: a retrospective study. Eur Respir J. 2020;56:2002113. doi: 10.1183/13993003.02113-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lalueza A, Lora-Tamayo J, de la Calle C, et al. Utilidad de las escalas de sepsis para predecir el fallo respiratorio y la muerte en pacientes con COVID-19 fuera de las Unidades de Cuidados Intensivos. Rev Clin Esp 2020. DOI: 10.1016/j.rce.2020.10.004 [DOI] [PMC free article] [PubMed]
- 34.Ahnert P, Creutz P, Horn K, et al. Sequential organ failure assessment score is an excellent operationalization of disease severity of adult patients with hospitalized community acquired pneumonia - results from the prospective observational PROGRESS study. Crit Care. 2019;23(1):110. doi: 10.1186/s13054-019-2316-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Su Y, Tu GW, Ju MJ, et al. Comparison of CRB-65 and quick sepsis-related organ failure assessment for predicting the need for intensive respiratory or vasopressor support in patients with COVID-19. J Inf Secur. 2020;S0163-4453(20):30281–4. doi: 10.1016/j.jinf.2020.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Haimovich AD, Ravindra NG, Stoytchev S, et al. Development and Validation of the Quick COVID-19 Severity Index: A Prognostic Tool for Early Clinical Decompensation. Ann Emerg Med. 2020;76(4):442–53. doi: 10.1016/j.annemergmed.2020.07.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Ferreira M, Blin T, Collercandy N, et al. Critically ill SARS-CoV-2-infected patients are not stratified as sepsis by the qSOFA. Ann Intensive Care. 2020;10(1):43. doi: 10.1186/s13613-020-00664-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Myrstad M, Ihle-Hansen H, Tveita AA, et al. National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality form COVID-19. A prospective cohort study. Scan J Traum, Res, Emerg Med. 2020;28:66. doi: 10.1186/s13049-020-00764-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Liu K, Chen Y, Lin R, Han K. Clinical features of COVID-19 in elderly patients: A comparison with young and middle-aged patients. J Inf Secur. 2020;80(6):e14–e8. doi: 10.1016/j.jinf.2020.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Fu L, Wang B, Yuan T, et al. Clinical characteristics of coronavirus disease 2019 (COVID-19) in China: A systematic review and meta-analysis. J Inf Secur. 2020;80(6):656–65. doi: 10.1016/j.jinf.2020.03.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Nadkarni GN, Lala A, Bagiella E, et al. Anticoagulation, bleeding, mortality, and pathology in hospitalized patients with COVID-19. J Am Coll Cardiol. 2019;76(16):1815–26. doi: 10.1016/j.jacc.2020.08.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Martínez-Lacalzada M, Viteri-Noël A, Manzano L, et al. Predicting critical illness on initial diagnosis of COVID-19: development and validation of the PRIORITY model for outpatient applicability. MedRxiv 2020. DOI: 10.1101/2020.11.27.20237966 [DOI] [PMC free article] [PubMed]
- 43.Ji D, Zhang D, Xu J, Chen Z, et al. Prediction for Progression Risk in Patients With COVID-19 Pneumonia: The CALL Score. Clin Infect Dis. 2020;71(6):1393–99. doi: 10.1093/cid/ciaa414. [DOI] [PMC free article] [PubMed] [Google Scholar]