Abstract
Background
The Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years (CURB-65) score and the Pneumonia Severity Index (PSI) are well-established clinical prediction rules for predicting mortality in patients hospitalized with community-acquired pneumonia (CAP). SARS-CoV-2 has emerged as a new etiologic agent for CAP, but the role of CURB-65 score and PSI have not been established.
Research Question
How effective are CURB-65 score and PSI at predicting in-hospital mortality resulting from SARS-CoV-2 CAP compared with non-SARS-CoV-2 CAP? Can these clinical prediction rules be optimized to predict mortality in SARS-CoV-2 CAP by addition of procalcitonin and D-dimer?
Study Design and Methods
Secondary analysis of two prospective cohorts of patients with SARS-CoV-2 CAP or non-SARS-CoV-2 CAP from eight adult hospitals in Louisville, Kentucky.
Results
The in-hospital mortality rate was 19% for patients with SARS-CoV-2 CAP and 6.5% for patients with non-SARS-CoV-2 CAP. For the PSI score, receiver operating characteristic (ROC) curve analysis resulted in an area under the ROC curve (AUC) of 0.82 (95% CI, 0.78-0.86) and 0.79 (95% CI, 0.77-0.80) for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP, respectively. For the CURB-65 score, ROC analysis resulted in an AUC of 0.79 (95% CI, 0.75-0.84) and 0.75 (95% CI, 0.73-0.77) for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP, respectively. In SARS-CoV-2 CAP, the addition of D-dimer (optimal cutoff, 1,813 μg/mL) and procalcitonin (optimal cutoff, 0.19 ng/mL) to PSI and CURB-65 score provided negligible improvement in prognostic performance.
Interpretation
PSI and CURB-65 score can predict in-hospital mortality for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP comparatively. In patients with SARS-CoV-2 CAP, the inclusion of either D-dimer or procalcitonin to PSI or CURB-65 score did not improve the prognostic performance of either score. In patients with CAP, regardless of cause, PSI and CURB-65 score remain adequate for predicting mortality in clinical practice.
Key Words: CAP, COVID-19, D-dimer, pneumonia, procalcitonin
Abbreviations: AUC, area under the receiver operating characteristic curve; bCI, boostrap confidence interval; CAP, community-acquired pneumonia; CURB-65, Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years; PSI, Pneumonia Severity Index; ROC, receiver operating characteristic
Graphical Abstract
Take-home Points.
Study Question: How effective are Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years (CURB-65) score and the Pneumonia Severity Index (PSI) at predicting in-hospital mortality resulting from SARS-CoV-2 community-acquired pneumonia (CAP) compared with non-SARS-CoV-2 CAP?
Results: The receiver operating characteristic (ROC) curve analysis showed that the areas under the ROC curve for PSI and CURB-65 score for predicting in-hospital mortality were acceptable and similar for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP.
Interpretation: PSI and CURB-65 score can predict in-hospital mortality for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP comparatively.
SARS-CoV-2 is a novel etiologic agent of community-acquired pneumonia (CAP) that has resulted in a pandemic that has strained the resources of health care systems around the world. Patients with SARS-CoV-2 are at high risk of mortality, such that a 22.9% increase in US death counts occurred from March 2020 through January 2021 compared with the same period in previous years, and COVID-19 was documented in 72.4% of those excess deaths.1 Patients with SARS-CoV-2 CAP frequently require hospitalization for noninvasive and invasive respiratory support. Triaging patients with SARS-CoV-2 CAP requires an accurate assessment of mortality risk; however, this is challenging given the novelty of the virus. Consequences of overestimating mortality risk include prolonged hospitalization and the consumption of scarce medical resources. Because of COVID-19’s high mortality rate and the tremendous amount of health care resources required to treat patients with COVID-19, it is vital that models be evaluated to assist physicians in triaging patients such that resource allocation is optimized while mortality is minimized.
Recommended by numerous international guidelines, the most widely used clinical scoring systems to predict mortality resulting from CAP are the Pneumonia Severity Index (PSI) and the Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years (CURB-65) score. Few studies have examined the performance of these clinical risk scores in SARS-CoV-2 CAP, and they have yielded inconsistent results.2 , 3 A distinctive feature of severe SARS-CoV-2 CAP is the presence of elevated D-dimer and procalcitonin, raising the possibility that the addition of these two variables would improve the ability of prognostic scores to predict mortality.4, 5, 6, 7, 8
The primary objective of this study was to evaluate the ability of PSI and CURB-65 score to predict in-hospital mortality in patients with SARS-CoV-2 CAP compared with patients with non-SARS-CoV-2 CAP. The secondary objective of this study was to assess the prognostic value of adding D-dimer and procalcitonin to PSI and CURB-65 score in SARS-CoV-2 CAP.
Study Design and Methods
Study Design and Participants
This is a secondary analysis of two population-based cohort studies of hospitalized adults in the city of Louisville, Kentucky. The first study included patients who received a diagnosis of SARS-CoV-2 CAP from March 5, 2020, through July 1, 2020, at eight adult hospitals in Louisville, Kentucky.9 We then compared the mortality of patients diagnosed with SARS-CoV-2 CAP with that of patients in the University of Louisville Pneumonia Study, a prospective, population-based cohort of patients diagnosed with non-SARS-CoV-2 CAP in Louisville, Kentucky, from June 1, 2014, through 31 May 2016.10 For the purpose of analysis, patients who were transferred to hospice were counted as in-hospital deaths. To evaluate further the prognostic performance of CURB-65 score and PSI in patients who received a diagnosis of SARS-CoV-2 CAP, D-dimer and procalcitonin values from admission were included in these scoring systems and receiver operating characteristic (ROC) curve analysis was performed. This study was approved by the University of Louisville Institutional Review Board (Identifier, 20.0257) and each hospital’s research department. The study was exempt from informed consent.
Inclusion and Exclusion Criteria
Inclusion and exclusion criteria for patients in the SARS-CoV-2 CAP and non-SARS-CoV-2 CAP cohorts have been published previously.9 , 10 For the non-SARS-CoV-2 CAP cohort, all adults (age ≥ 18 years) hospitalized with CAP at the nine acute-care hospitals in Louisville, Kentucky, were eligible to participate in the study. In the SARS-CoV-2 CAP cohort, all adults (age ≥ 18 years) hospitalized with CAP at eight of the nine acute-care hospitals in Louisville, Kentucky, were eligible. No exclusion criteria were applied. A patient was defined as having SARS-CoV-2 CAP when the following criteria were met: (1) positive reverse-transcriptase polymerase chain reaction results for SARS-CoV-2; (2) fever, cough, or shortness of breath; and (3) evidence of pulmonary infiltrates on chest radiography or chest CT scan.
Data Collection
Research associates screened patients daily for SARS-CoV-2 CAP at all participating sites. Data from hospital admission on demographics, comorbidities, and physical examination findings were collected as well as data to calculate the PSI and CURB-65 score. Vital signs and laboratory values represent the most extreme values that occurred within the first 24 h of admission. If blood gases were not obtained, we considered that Pao 2 was > 60 and the arterial pH was > 7.35 (and thus contributed no points to the PSI calculation). Radiographic pneumonia was identified by new infiltrate as reported by a board-certified radiologist on CT scan or chest radiograph within 48 h of admission. All data were stored in a Health Insurance Portability and Accountability Act of 1996-protected online platform.
Statistical Analysis
Continuous data are expressed as median values with interquartile ranges (25th-75th quartiles), and comparison between groups was assessed by the Mann-Whitney U test. Categorical variables are expressed as number (percentage) and were compared by χ2 tests. To assess the prognostic ability of PSI and CURB-65 score to predict mortality, both scores were calculated for each patient and ROC curves were generated. ROC curve analysis was performed, and values for area under the ROC curve (AUC), sensitivity, specificity, positive predictive value, and negative predictive value were calculated. Bootstrap 95% CIs were calculated using replicates of 1,000.
D-dimer and procalcitonin were dichotomized at values that maximized sensitivity and specificity for predicting mortality. The cutoff values for D-dimer and procalcitonin were found to be 1,813 μg/mL and 0.19 ng/mL, respectively. These values were added to the PSI and CURB-65 score to evaluate their impact on the predictive ability of these severity scores. To determine the amount of improvement to the PSI and CURB-65 score, the integrated discrimination improvement was calculated for the addition of D-dimer, procalcitonin, or both.11 Analysis was performed with R version 3.4.0 software (R Foundation for Statistical Computing).
Results
Baseline Characteristics
A total of 8,081 patients were included in this study, with a total of 632 (8%) patients with SARS-CoV-2 CAP. The median age of patients with SARS-CoV-2 CAP was 63 years, a total of 294 (47%) were men, 198 (31%) were Black, and 74 (12%) were Hispanic. The baseline characteristics for patients with SARS-CoV-2 CAP are summarized in Table 1 . Within the SARS-CoV-2 CAP cohort, patients who died or pursued hospice care were older and had more comorbidities overall, including a higher incidence of COPD (24% vs 13%), diabetes (48% vs 31%), heart failure (33% vs 12%), and renal disease (32% vs 16%) compared with those discharged alive. Based on initial laboratory tests, patients with SARS-CoV-2 CAP who died or pursued hospice care also showed higher levels of D-dimer (1,309 μg/mL vs 691 μg/mL) and procalcitonin (0.48 ng/mL vs 0.10 ng/mL). D-dimer and procalcitonin values were collected from 79.7% and 70.7%, respectively, of all patients in the SARS-CoV-2 CAP cohort. Baseline characteristics for patients with non-SARS-CoV-2 CAP from the University of Louisville Pneumonia Study cohort have been published previously.10 Comparison of baseline characteristics for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP is provided in e-Table 1.
Table 1.
Baseline Characteristics and Laboratory Values for Patients With SARS-CoV-2
| Variable | Mortality or Hospice Care | Discharged Alive | P Value |
|---|---|---|---|
| Total No. | 121 | 511 | ... |
| Age, y | 72 (64-81) | 60 (45-71) | < .001 |
| Male sex | 69 (57) | 225 (44) | .013 |
| Black race | 35 (29) | 163 (32) | .6 |
| Nursing home resident | 44 (36) | 70 (14) | < .001 |
| BMI, kg/m2 | .459 | ||
| < 18 | 3 (2) | 9 (2) | |
| 18-24.9 | 36 (30) | 113 (22) | |
| 25-29.9 | 28 (23) | 128 (25) | |
| 30-34.9 | 27 (22) | 111 (22) | |
| 35-39.9 | 12 (10) | 62 (12) | |
| > 40 | 15 (12) | 88 (17) | |
| Vital signs | |||
| Heart rate | 92 (81-112) | 98 (82-110) | .542 |
| SBP < 90 mm Hg or DBP < 60 mm Hg | 63 (52) | 146 (29) | < .001 |
| Respiratory rate ≥ 30 breaths/min | 41 (34) | 75 (15) | < .001 |
| Temperature < 95 °F (35 °C) | 1 (1) | 2 (0) | > .999 |
| Temperature ≥ 100.4 °F (38 °C) | 52 (43) | 214 (42) | .907 |
| Medical history | |||
| Any comorbidity | 108 (89) | 368 (72) | < .001 |
| > 2 comorbidities | 64 (53) | 122 (24) | < .001 |
| COPD | 29 (24) | 67 (13) | .004 |
| Asthma | 6 (5) | 61 (12) | .038 |
| Obstructive sleep disorder | 14 (12) | 46 (9) | .488 |
| Diabetes | 58 (48) | 156 (31) | < .001 |
| Heart failure | 40 (33) | 59 (12) | < .001 |
| Hypertension | 88 (73) | 264 (52) | < .001 |
| Stroke | 31 (26) | 55 (11) | < .001 |
| Neoplastic disease within past year | 19 (16) | 26 (5) | < .001 |
| Renal disease | 39 (32) | 80 (16) | < .001 |
| Cirrhosis | 2 (2) | 6 (1) | > .999 |
| Current smoker | 13 (11) | 45 (9) | .625 |
| Former smoker | 40 (33) | 130 (25) | .113 |
| Laboratory results on admission | |||
| Lymphocyte count, cells/mm3 | 0.80 (0.56-1.23) | 1.08 (0.73-1.47) | < .001 |
| Neutrophil count, cells/mm3 | 6.38 (3.34-11.45) | 4.40 (2.99-6.72) | < .001 |
| Neutrophil to lymphocyte ratio | 6.38 (3.49-14.50) | 4.05 (2.63-6.77) | < .001 |
| Platelets, cells/mm3 | 173 (126-226) | 198 (163-251) | < .001 |
| Ferritin, ng/mLa | 525 (213-1,181) | 377 (166-814) | .027 |
| D-dimer, μg/mLb | 1,309 (676-2,772) | 691 (372-1,296) | < .001 |
| CRP, mg/Lc | 18 (7-40) | 14 (5-34) | .052 |
| ESR, mm/hd | 51 (32-99) | 51 (32-74) | .399 |
| Procalcitonin, ng/mLe | 0.48 (0.12-1.40) | 0.10 (0.05-0.27) | < .001 |
| Initial serum troponin, ng/mLf | 0.03 (0.01-0.08) | 0.01 (0.01-0.03) | < .001 |
| Lactate dehydrogenase, units/Lg | 625.0 (319-974) | 451.5 (251.75-731.75) | .001 |
| pHh | 7.40 (7.32-7.45) | 7.44 (7.40-7.47) | < .001 |
| Pao2 to Fio2 ratio, %i | .005 | ||
| > 300 | 25 (31) | 60 (38) | |
| 200-300 | 13 (16) | 45 (28) | |
| 100-199 | 18 (22) | 33 (21) | |
| < 100 | 24 (30) | 20 (13) |
Data are presented as No. (%) or median (interquartile range), unless otherwise indicated. For all data points, < 1% are missing, except as noted. CRP = C-reactive protein; DBP = diastolic BP; ESR = erythrocyte sedimentation rate; SBP = systolic BP.
Data missing for 30 mortality or hospice care patients (25%) and 167 patients discharged alive (33%).
Data missing for 31 mortality or hospice care patients (26%) and 154 patients discharged alive (30%).
Data missing for 38 mortality or hospice care patients (31%) and 158 patients discharged alive (31%).
Data missing for 88 mortality or hospice care patients (73%) and 412 patients discharged alive (81%).
Data missing for 20 mortality or hospice care patients (17%) and 108 patients discharged alive (21%).
Data missing for 27 mortality or hospice care patients (22%) and 161 patients discharged alive (32%).
Data missing for 35 mortality or hospice care patients (34%) and 175 patients discharged alive (29%).
Data missing for 41 mortality or hospice care patients (34%) and 353 patients discharged alive (69%).
Data missing for 41 mortality or hospice care patients (34%) and 353 patients discharged alive (69%).
Predictive Value of PSI and CURB-65 Score in SARS-CoV-2 CAP and Non-SARS-CoV-2 CAP
For patients in the SARS-CoV-2 CAP and non-SARS-CoV-2 CAP cohorts, in-hospital mortality was assessed based on PSI risk class or CURB-65 score. Patients presenting with SARS-CoV-2 CAP had a higher mortality rate in every PSI risk class and every CURB-65 score compared with patients with non-SARS-CoV-2 CAP (Fig 1 , e-Tables 2, 3). The AUC for PSI was 0.82 (95% bootstrap CI [bCI], 0.78-0.86) in patients with SARS-CoV-2 CAP and 0.79 (95% bCI, 0.77-0.80) in patients with non-SARS-CoV-2 CAP. The AUC for CURB-65 score was 0.79 (95% bCI, 0.75-0.84) in patients with SARS-CoV-2 CAP and 0.75 (95% bCI, 0.73-0.77) in patients with non-SARS-CoV-2 CAP. Figure 2 shows the ROC curves comparing PSI and CURB-65 scores in these two cohorts. Table 2 shows the sensitivity, specificity, positive predictive value, and negative predictive value for these patients. An additional post hoc analysis was performed that compared PSI, CURB-65 score, COVID-GRAM, Comorbidity-Age-Lymphocyte Count-Lactate Dehydrogenase, and National Early Warning scores in the SARS-CoV-2 CAP cohort (e-Fig 1, e-Table 4).
Figure 1.
A, B, Bar graphs showing the risk of in-hospital mortality for patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP by PSI risk (A) and CURB-65 score (B). CAP = community-acquired pneumonia; CURB-65 = Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years; PSI = Pneumonia Severity Index.
Figure 2.
A, B, Receiver operating characteristic curves for PSI (A) and CURB-65 score (B) in patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP. AUC = area under the receiver operating characteristic curve; CAP = community-acquired pneumonia; CURB-65 = Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years; PSI = Pneumonia Severity Index.
Table 2.
ROC Analysis for PSI and CURB-65 Score in Patients With SARS-CoV-2 CAP and Non-SARS-CoV-2 CAP
| Variable | Patient Population | AUC (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) |
|---|---|---|---|---|---|---|
| PSI | Non-SARS-CoV-2 CAP | 0.79 (0.77-0.80) | 0.71 (0.67-0.75) | 0.78 (0.77-0.79) | 0.16 (0.15-0.18) | 0.98 (0.97-0.98) |
| SARS-CoV-2 CAP | 0.82 (0.78-0.86) | 0.83 (0.55-0.91) | 0.66 (0.61-0.92) | 0.37 (0.31-0.61) | 0.94 (0.90-0.97) | |
| CURB-65 score | Non-SARS-CoV-2 CAP | 0.75 (0.73-0.77) | 0.72 (0.68-0.77) | 0.66 (0.65-0.67) | 0.12 (0.10-0.13) | 0.97 (0.97-0.98) |
| SARS-CoV-2 CAP | 0.79 (0.75-0.84) | 0.83 (0.50-0.89) | 0.61 (0.58-0.85) | 0.33 (0.29-0.52) | 0.94 (0.89-0.96) |
The 95% CI is calculated from the 2.5 and 97.5 percentiles of the bootstrap distribution. AUC = area under the receiver operating characteristic curve; CAP = community-acquired pneumonia; CURB-65 = Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years; NPV = negative predictive value; PPV = positive predictive value; PSI = Pneumonia Severity Index;. ROC = receiver operating characteristic.
Addition of D-Dimer and Procalcitonin to PSI and CURB-65 Scores
Using the cutoff values for D-dimer and procalcitonin, the ROC curve analysis to evaluate the prognostic ability of PSI and CURB-65 score (Fig 3 , Table 3 ) showed no improvement in performance of the two scoring systems by the addition of either D-dimer or procalcitonin alone. A statistically significant improvement was found when adding both laboratory values to the PSI or CURB-65 score, with an integrated discrimination improvement of 0.03 (95% CI, 0.01-0.04) for the PSI and 0.03 (95% CI, 0.02-0.05) for the CURB-65 score.
Figure 3.
A, B, Receiver operating characteristic (ROC) curves for PSI, PSI + D-dimer, PSI + procalcitonin, and PSI + D-dimer + procalcitonin (A), and ROC curves for CURB-65 score, CURB-65 score + D-dimer, CURB-65 score + procalcitonin, and CURB-65 score + D-dimer + procalcitonin (B) in predicting in-hospital mortality and hospice care for patients with SARS-CoV-2 community-acquired pneumonia. AUC = area under the receiver operating characteristic curve; CURB-65 = Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years; PSI = Pneumonia Severity Index.
Table 3.
Comparison of Pneumonia Scoring Systems in SARS-CoV-2 CAP
| Variable | AUC (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | IDI (95% CIa) |
|---|---|---|---|---|---|---|
| PSI | 0.82 (0.78-0.86) | 0.83 (0.55-0.91) | 0.66 (0.61-0.92) | 0.37 (0.31-0.61) | 0.94 (0.90-0.97) | |
| PSI + D-dimer | 0.83 (0.79-0.87) | 0.83 (0.58-0.90) | 0.66 (0.62-0.90) | 0.37 (0.32-0.59) | 0.94 (0.90-0.97) | 0.01 (−0.002 to 0.014) |
| PSI + procalcitonin | 0.83 (0.80-0.87) | 0.69 (0.65-0.93) | 0.80 (0.59-0.84) | 0.46 (0.31-0.52) | 0.92 (0.90-0.97) | 0.02 (−0.01 to 0.04) |
| PSI + D-dimer + procalcitonin | 0.84 (0.80-0.87) | 0.70 (0.63-0.92) | 0.80 (0.59-0.88) | 0.46 (0.32-0.55) | 0.92 (0.90-0.97) | 0.03 (0.01-0.04) |
| CURB-65 score | 0.79 (0.75-0.84) | 0.83 (0.50-0.89) | 0.61 (0.58-0.85) | 0.33 (0.29-0.52) | 0.94 (0.89-0.96) | |
| CURB-65 score + D-dimer | 0.80 (0.76-0.84) | 0.85 (0.59-0.90) | 0.59 (0.56-0.85) | 0.33 (0.29-0.50) | 0.94 (0.89-0.97) | 0.01 (0.00-0.02) |
| CURB-65 score + procalcitonin | 0.81 (0.76-0.85) | 0.73 (0.59-0.85) | 0.75 (0.63-0.85) | 0.41 (0.32-0.52) | 0.92 (0.89-0.95) | 0.03 (0.01-0.04) |
| CURB-65 score + D-dimer + procalcitonin | 0.81 (0.77-0.85) | 0.74 (0.59-0.87) | 0.74 (0.59-0.88) | 0.41 (0.33-0.54) | 0.93 (0.89-0.96) | 0.03 (0.02-0.05) |
The 95% CI was calculated from the 2.5 and 97.5 percentiles of the bootstrap distribution. AUC = area under the receiver operating characteristic curve; CAP = community-acquired pneumonia; CURB-65 = Confusion, Urea > 7 mM, Respiratory Rate ≥ 30 breaths/min, BP < 90 mm Hg (Systolic) or < 60 mm Hg (Diastolic), Age ≥ 65 Years; IDI = integrated discrimination improvement; NPV = negative predictive value; PPV = positive predictive value; PSI = Pneumonia Severity Index.
Asymptotic 95% CI.
Discussion
This study showed that PSI and CURB-65 score can be used to predict in-hospital mortality in patients with SARS-CoV-2 CAP. In fact, the predictive ability in these patients compares favorably with that in patients with non-SARS-CoV-2 CAP. A remarkable difference in the positive predictive value between these two populations was found, likely attributable to the increased prevalence of poor outcomes in patients with SARS-CoV-2 CAP compared with patients with non-SARS-CoV-2 CAP. A high negative predictive value for both PSI and CURB-65 score in SARS-CoV-2 CAP and non-SARS-CoV-2 CAP (> 94%) suggests that the optimal clinical benefit of these scoring systems lies in their ability to detect patients at low risk for mortality. Although D-dimer and procalcitonin significantly increased the prognostic value of these indexes, the observed 3% improvement may not be clinically meaningful.
In resource-poor settings, a quick and accurate assessment of mortality risk allows clinicians to triage and optimize care for those at highest risk; thus, it is imperative that models be developed, optimized, and evaluated in different patient populations. Several new predictive models, recently reviewed by Wynants et al,12 have been proposed for patients with SARS-CoV-2 CAP, but the external validity of these models remains to be established, and they are at high risk of bias because they were developed from a relatively small number of patients. In our cohort of patients with SARS-CoV-2 CAP, PSI remained the best clinical prediction score compared with several novel prediction scores that were developed specifically for SARS-CoV-2 CAP (e-Fig 1).
The formation of microthrombi, endotheliitis, severe capillaritis, and pulmonary thromboemboli is highly prevalent in patients who have died of SARS-CoV-2 CAP, as seen in autopsy studies.13, 14, 15 Microthrombosis is now regarded as an important pathogenic feature of SARS-CoV-2 CAP.16 Furthermore, elevated procalcitonin has been associated with worse outcomes in these patients.17 Despite a biological rationale, the addition of D-dimer and procalcitonin did not improve substantially the predictive ability of the models in our study.
Despite the wide use of PSI and CURB-65 score to predict mortality and severity of CAP, very few studies have examined these scoring systems in SARS-CoV-2 CAP. Two studies, by Fan et al2 (n = 654) and Satici et al3 (n = 681), revealed inconsistent results regarding the superiority of PSI over CURB-65 score in patients with SARS-CoV-2 CAP. In patients in Wuhan, China, Fan et al2 found the performances of CURB-65 score and PSI to predict in-hospital mortality to be equivalent (AUC, 0.85). In a separate study in Turkey, Satici et al3 found that PSI was slightly superior to CURB-65 score (AUC, 0.91 vs 0.88, respectively) in predicting 30-day mortality. A recent multicenter, retrospective cohort study involving 10,238 patients with COVID-19 in Spain reported AUCs of 0.835 and 0.825 for PSI and CURB-65 score, respectively.18 None of the above studies directly compared PSI and CURB-65 scores in their ability to predict in-hospital mortality in patients with SARS-CoV-2 CAP and non-SARS-CoV-2 CAP.
Within the city of Louisville, Kentucky, the overall in-hospital mortality rate for adult patients with non-SARS-CoV-2 CAP is 6.5%.10 In the current SARS-CoV-2 CAP cohort of patients, the in-hospital mortality was much higher at 19%, which is similar to mortality rates in Spain and China.9 , 18 , 19 As illustrated in Figure 1, regardless of the PSI risk class or CURB-65 score, mortality was higher for patients with SARS-CoV-2 CAP than for patients with non-SARS-CoV-2 CAP. In our study, PSI was slightly superior to CURB-65 score in its ability to predict in-hospital mortality. This is consistent with prior work that showed slight improvement of PSI over CURB-65 score in cohorts of patients with SARS-CoV-2 CAP in China and Spain.3 , 18 The superiority of PSI is likely the result of the PSI model being heavily weighted by age and comorbidities, which are known risk factors for mortality in SARS-CoV-2 CAP.17 , 20 , 21
This study has limitations. Although this was a multicenter study and involved all adult hospitals in Louisville, Kentucky, it is still limited to a single city, which may impact its generalizability. Another limitation is that D-dimer and procalcitonin levels were not collected from all patients. No defined prognostic cutoff values for D-dimer and procalcitonin exist for classification in patients with SARS-CoV-2 CAP; thus, we used an ROC curve to generate cutpoints that maximized sensitivity and specificity for our outcome. Additionally, the management of SARS-CoV-2 CAP is constantly evolving, and our data represent outcomes based on treatment provided to unvaccinated patients early in the pandemic, when treatment options were less defined. The data were collected prospectively, but this is a secondary analysis without prospective validation of the models. Because of the retrospective nature of this study, 30-day mortality was unavailable for too many patients in the SARS-CoV-2 CAP cohort to be deemed useful in analysis owing to our inability to follow up patients after hospitalization. Arterial blood gases data were not obtained from all patients. However, further evaluation of PSI in SARS-CoV-2 CAP by nonlinear imputation of Pao 2 from oxygen saturation (for patients without arterial blood gases data) did not improve the predictability of in-hospital mortality or hospice care (e-Tables 5, 6).22
This study also has several strengths. It relies on one of the largest datasets of patients with CAP in the United States. The data contain clinical variables that were collected prospectively by research associates and were validated. Additionally, recent work suggests that the city of Louisville is representative of the overall US population with regard to sociodemographic, economic, and health-related statistics.23
Although still currently debated, it is thought that infection with SARS-CoV-2 may result in a subsequent cytokine storm syndrome similar to that which has been reported in other viral infections.24 , 25 The addition of cytokines to risk models in future studies has the potential to improve their predictive ability. Additionally, predictive models for ambulatory settings for SARS-CoV-2 CAP would be instrumental in early triaging of patients who would benefit from re-evaluation or early intervention to decrease mortality. Finally, more studies of implementation of the available evidence, including predictive models, are needed.
Interpretation
In conclusion, our results illustrate that PSI and CURB-65 score can predict in-hospital mortality for patients with SARS-CoV-2 CAP with similar efficacy as compared with non-SARS-CoV-2 CAP. We also showed that initial D-dimer and procalcitonin values do not substantially improve the performance of PSI and CURB-65 score; thus, the addition of these laboratory values for the initial prognostic assessment of patients who have received a diagnosis of SARS-CoV-2 CAP is not warranted. Our study results suggest that in patients with CAP, regardless of cause, PSI and CURB-65 score remain adequate for predicting mortality in clinical practice.
Acknowledgments
Author contributions: J. B. and R. C. take responsibility for the content of the manuscript, including the data and analysis. J. B., N. S., J. A. R., and R. C. contributed to study concept, data analysis and interpretation, and writing and editing of the manuscript. T. R. C. and S. F. contributed to data analysis and interpretation and writing and editing of the manuscript.
Financial/nonfinancial disclosures: None declared.
Other contributions: The authors thank the staff and research associates at the Division of Infectious Diseases, University of Louisville.
Additional information: The e-Figure and e-Tables can be found in the Supplemental Materials section of the online article.
Footnotes
FUNDING/SUPPORT: This study was supported by The Center for Excellence for Research in Infectious Diseases at the University of Louisville.
Supplementary Data
References
- 1.Woolf S.H., Chapman D.A., Sabo R.T., Zimmerman E.B. Excess deaths from COVID-19 and other causes in the US, March 1, 2020, to January 2, 2021. JAMA. 2021;325(17):1786–1789. doi: 10.1001/jama.2021.5199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Fan G., Tu C., Zhou F., et al. Comparison of severity scores for COVID-19 patients with pneumonia: a retrospective study. Eur Respir J. 2020;56(3) doi: 10.1183/13993003.02113-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Satici C., Demirkol M.A., Sargin Altunok E., et al. Performance of pneumonia severity index and CURB-65 in predicting 30-day mortality in patients with COVID-19. Int J Infect Dis. 2020;98:84–89. doi: 10.1016/j.ijid.2020.06.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Terpos E., Ntanasis-Stathopoulos I., Elalamy I., et al. Hematological findings and complications of COVID-19. Am J Hematol. 2020;95(7):834–847. doi: 10.1002/ajh.25829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chen G., Wu D., Guo W., et al. Clinical and immunological features of severe and moderate coronavirus disease 2019. J Clin Invest. 2020;130(5):2620–2629. doi: 10.1172/JCI137244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lippi G., Plebani M. Procalcitonin in patients with severe coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chim Acta. 2020;505:190–191. doi: 10.1016/j.cca.2020.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hu R., Han C., Pei S., Yin M., Chen X. Procalcitonin levels in COVID-19 patients. Int J Antimicrob Agents. 2020;56(2):106051. doi: 10.1016/j.ijantimicag.2020.106051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Yao Y., Cao J., Wang Q., et al. D-dimer as a biomarker for disease severity and mortality in COVID-19 patients: a case control study. J Intensive Care. 2020;8:49. doi: 10.1186/s40560-020-00466-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ramirez J., Bordon J., Cavallazzi R., et al. Characteristics and outcomes of adults hospitalized with SARS-COV-2 community-acquired pneumonia in Louisville, Kentucky. University of Louisville Journal of Respiratory Infections. 2020;4(1):72. [Google Scholar]
- 10.Ramirez J.A., Wiemken T.L., Peyrani P., et al. Adults hospitalized with pneumonia in the United States: incidence, epidemiology, and mortality. Clin Infect Dis. 2017;65(11):1806–1812. doi: 10.1093/cid/cix647. [DOI] [PubMed] [Google Scholar]
- 11.Pencina M.J., D’Agostino R.B., Sr., D’Agostino R.B., Jr., Vasan R.S. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–172. doi: 10.1002/sim.2929. discussion 207-112. [DOI] [PubMed] [Google Scholar]
- 12.Wynants L., Van Calster B., Collins G.S., et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328. doi: 10.1136/bmj.m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Varga Z., Flammer A.J., Steiger P., et al. Endothelial cell infection and endotheliitis in COVID-19. Lancet. 2020;395(10234):1417–1418. doi: 10.1016/S0140-6736(20)30937-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wichmann D., Sperhake J.P., Lütgehetmann M., et al. Autopsy findings and venous thromboembolism in patients with COVID-19: a prospective cohort study. Ann Intern Med. 2020;173(4):268–277. doi: 10.7326/M20-2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Ackermann M., Verleden S.E., Kuehnel M., et al. Pulmonary vascular endothelialitis, thrombosis, and angiogenesis in Covid-19. N Engl J Med. 2020;383(2):120–128. doi: 10.1056/NEJMoa2015432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.McFadyen J.D., Stevens H., Peter K. The emerging threat of (micro)thrombosis in COVID-19 and its therapeutic implications. Circ Res. 2020;127(4):571–587. doi: 10.1161/CIRCRESAHA.120.317447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Zheng Z., Peng F., Xu B., et al. Risk factors of critical and mortal COVID-19 cases: a systematic literature review and meta-analysis. J Infect. 2020;81(2):e16–e25. doi: 10.1016/j.jinf.2020.04.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Artero A., Madrazo M., Fernández-Garcés M., et al. Severity scores in COVID-19 pneumonia: a multicenter, retrospective, cohort study. J Gen Intern Med. 2021:1–8. doi: 10.1007/s11606-021-06626-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lalueza A., Lora-Tamayo J., de la Calle C., et al. The early use of sepsis scores to predict respiratory failure and mortality in non-ICU patients with COVID-19 [in Spanish] [published online ahead of print November 7, 2020] Rev Clin Esp. 2020 doi: 10.1016/j.rce.2020.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Chalmers J.D., Singanayagam A., Akram A.R., et al. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis. Thorax. 2010;65(10):878–883. doi: 10.1136/thx.2009.133280. [DOI] [PubMed] [Google Scholar]
- 21.Wu Z., McGoogan J.M. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323(13):1239–1242. doi: 10.1001/jama.2020.2648. [DOI] [PubMed] [Google Scholar]
- 22.Brown S.M., Grissom C.K., Moss M., et al. Nonlinear imputation of Pao2/Fio2 From Spo2/Fio2 among patients with acute respiratory distress syndrome. Chest. 2016;150(2):307–313. doi: 10.1016/j.chest.2016.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Furmanek S., Glick C., Chandler T., et al. The City of Louisville encapsulates the United States demographics. University of Louisville Journal of Respiratory Infections. 2020;4(2) [Google Scholar]
- 24.McElvaney O.J., McEvoy N.L., McElvaney O.F., et al. Characterization of the inflammatory response to severe COVID-19 illness. Am J Respir Crit Care Med. 2020;202(6):812–821. doi: 10.1164/rccm.202005-1583OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Ryabkova V.A., Churilov L.P., Shoenfeld Y. Influenza infection, SARS, MERS and COVID-19: cytokine storm—the common denominator and the lessons to be learned. Clin Immunol. 2021;223:108652. doi: 10.1016/j.clim.2020.108652. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.