Table 2.
Model performance in early prediction of prognosis in COVID19 patients in the internal validation cohort.
| Model | No significant treatment versus O2 therapy or more | No critical care required versus critical care* or death | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AUC | TP/TN/FP/FN | Sensitivity | Specificity | Accuracy | Precision | NPV | AUC | TP/TN/FP/FN | Sensitivity | Specificity | Accuracy | Precision | NPV | |
| OLR | ||||||||||||||
| Model1 | 0.880 (0.855–0.904) | 193/1199/236/48 | 80.1% (74.5–84.9) | 83.6% (81.5–85.4) | 83.1% (81.2–84.8) | 45% (40.2–49.8) | 96.2% (94.9–97.1) | 0.903 (0.869–0.937) | 75/1336/251/14 | 84.3% (75–91.1) | 84.2% (82.3–85.9) | 84.2% (82.4–85.9) | 23% (18.5–28) | 99% (98.3–99.4) |
| Model2A | 0.889 (0.865–0.912) | 195/1119/209/43 | 81.9% (76.4–86.6) | 84.3% (82.2–86.2) | 83.9% (82–85.7) | 48.3% (43.3–53.3) | 96.3% (95–97.3) | 0.905 (0.869–0.940) | 81/1164/313/8 | 91% (83.1–96) | 78.8% (76.6–80.9) | 79.5% (77.4–81.5) | 20.6% (16.7–24.9) | 99.3% (98.7–99.7) |
| Model2B | 0.866 (0.841–0.892) | 181/1147/261/53 | 77.4% (71.4–82.5) | 81.5% (79.3–83.5) | 80.9% (78.9–82.8) | 41% (36.3–45.7) | 95.6% (94.3–96.7) | 0.914 (0.884–0.944) | 72/1312/247/11 | 86.7% (77.5–93.2) | 84.2% (82.2–85.9) | 84.3% (82.4–86) | 22.6% (18.1–27.6) | 99.2% (98.5–99.6) |
| Model3 | 0.894 (0.871–0.917) | 192/1082/210/40 | 82.8% (77.3–87.4) | 83.7% (81.6–85.7) | 83.6% (81.6–85.4) | 47.8% (42.8–52.8) | 96.4% (95.2–97.4) | 0.922 (0.892–0.953) | 76/1199/242/7 | 91.6% (83.4–96.5) | 83.2% (81.2–85.1) | 83.7% (81.7–85.5) | 23.9% (19.3–29) | 99.4% (98.8–99.8) |
| Model4 | 0.907 (0.884–0.929) | 189/835/172/31 | 85.9% (80.6–90.2) | 82.9% (80.5–85.2) | 83.5% (81.3–85.5) | 52.4% (47.1–57.6) | 96.4% (95–97.6) | 0.927 (0.894–0.96) | 68/1046/100/13 | 84% (74.1–91.2) | 91.3% (89.5–92.8) | 90.8% (89–92.3) | 40.5% (33–48.3) | 98.8% (97.9–99.3) |
| KMA model | 0.723 (0.693–0.753) | 129/108/1308/106 | 54.9% (61.4) | 7.6% (6.3–9.148.3-) | 14.4% (12.7–16.1) | 9% (7.5–10.6) | 50.5% (43.6–57.4) | 0.728 (0.678–0.778) | 43/1395/171/42 | 50.6% (39.5–61.6) | 89.1% (87.4–90.6) | 87.1% (85.4–88.7) | 20.1% (14.9–26.1) | 97.1% (96.1–97.9) |
| MEWS | 0.598 (0.563–0.633) | 129/314/1023/98 | 56.8% (50.1–63.4) | 23.5% (21.2–25.9) | 28.3% (26.1–30.6) | 11.2% (9.4–13.2) | 76.2% (71.8–80.2) | 0.631 (0.574–0.689) | 41/1112/371/40 | 50.6% (39.3–61.9) | 75% (72.7–77.2) | 73.7% (71.5–75.9) | 10% (7.2–13.3) | 96.5% (95.3–97.5) |
The results of other machine learning algorithms can be found in Supplementary Table S5. Values in parentheses are 95% confidence intervals. OLR ordinal logistic regression, AUC area under the receiver operator characteristics curve, TP true positive, TN true negative, FP false positive, FN false negative, NPV negative predictive value, KMA Korean Medical Association, MEWS modified Early Warning Score.
*The use of a ventilator or extracorporeal membrane oxygenation machine.