Table 4.
Final performance of machine learning models in prediction of mortality from COVID-19 in the test set.
| Classifier | AUC | TP/FP/FN/TN | Sensitivity | Specificity | PPV | NPV | Balanced accuracy |
|---|---|---|---|---|---|---|---|
| Mortality vs. recovery (with undetermined cases excluded) | |||||||
| LASSO | 0.963 (0.946, 0.979) | 68/208/7/2217 | 90.7% (83.3, 97.3) | 91.4% (90.3, 92.5) | 24.6% (19.7, 30.2) | 99.7% (99.4, 99.9) | 91.1% (86.0, 94.3) |
| Linear SVM | 0.962 (0.945, 0.979) | 69/199/6/2226 | 92.0% (85.9, 98.1) | 91.8% (90.7, 92.9) | 25.7% (20.6, 31.4) | 99.7% (99.4, 99.9) | 91.9% (87.0, 95.0) |
| RBF-SVM | 0.958 (0.945, 0.971) | 32/53/43/2372 | 42.7% (31.5, 53.9) | 97.8% (91.9, 1) | 37.6% (27.4, 48.8) | 98.2% (97.6, 98.7) | 70.2% (64.2, 76.5) |
| RF | 0.958 (0.936, 0.981) | 24/30/51/2395 | 32.0% (21.6, 42.8) | 98.8% (98.4, 99.3) | 44.4% (30.9, 58.6) | 97.9% (97.3, 98.4) | 65.4% (60.0, 71.5) |
| KNN | 0.897 (0.856, 0.937) | 61/255/14/2170 | 81.3% (73.1, 90.6) | 89.5% (88.3, 90.7) | 19.3% (15.1, 24.1) | 99.4% (98.9, 99.6) | 85.4% (79.5, 90.1) |
| Mortality vs. survival within 14 days after diagnosis | |||||||
| LASSO | 0.944 (0.921, 0.967) | 44/293/9/2871 | 83.0% (72.9, 91.5) | 90.7% (89.7, 91.9) | 13.1% (9.6, 17.1) | 99.7% (99.4, 99.9) | 86.8% (80.0, 91.8) |
| Linear SVM | 0.941 (0.914, 0.967) | 45/303/8/2861 | 84.9% (75.3, 93.0) | 90.4% (89.4, 91.6) | 12.9% (9.6, 16.9) | 99.7% (99.5, 99.9) | 87.7% (80.8, 92.3) |
| RBF-SVM | 0.919 (0.883, 0.955) | 6/18/47/3146 | 11.3% (0.3, 18.0) | 99.4% (99.1, 99.7) | 25% (9.8, 46.7) | 98.5% (98.0, 98.9) | 55.4% (51.7, 61.4) |
| RF | 0.925 (0.893, 0.958) | 12/41/41/3123 | 22.6% (11.3, 32.1) | 98.7% (98.3, 99.2) | 22.6% (12.3, 36.2) | 98.7% (98.2, 99.1) | 60.7% (55.2, 67.7) |
| KNN | 0.772 (0.705, 0.839) | 32/205/21/2959 | 60.4% (47.2, 71.4) | 93.5% (92.6, 04.4) | 13.5% (9.4, 18.5) | 99.3% (98.9, 99.6) | 77.0% (69.3, 84.0) |
| Mortality vs. survival within 30 days after diagnosis | |||||||
| LASSO | 0.953 (0.937, 0.969) | 57/309/7/2791 | 89.1% (81.4, 96.2) | 90.0% (88.9, 91.2) | 15.6% (12.0, 19.7) | 99.7% (99.5, 99.9) | 89.5% (83.8, 93.3) |
| Linear SVM | 0.948 (0.928, 0.968) | 55/324/9/2776 | 85.9% (77.3, 93.8) | 89.5% (88.4, 90.7) | 14.5% (11.1, 18.5) | 99.7% (99.4, 99.9) | 87.7% (81.7, 92.0) |
| RBF-SVM | 0.915 (0.885, 0.944) | 14/34/50/3066 | 21.9% (11.7, 31.3) | 98.9% (98.5, 99.3) | 29.2% (17.0, 44.1) | 98.4% (97.9, 98.8) | 60.4% (55.5, 66.6) |
| RF | 0.946 (0.930, 0.963) | 9/21/55/3079 | 14.1% (6.2, 21.9) | 99.3% (98.9, 99.6) | 30.0% (14.7, 49.4) | 98.2% (97.7, 98.7) | 56.7% (52.8, 62.3) |
| KNN | 0.750 (0.687, 0.813) | 37/247/27/2853 | 57.8% (44.9, 68.3) | 92.0% (91.0, 93.0) | 13.0% (9.3, 17.5) | 99.1% (98.6, 99.4) | 74.9% (67.9, 81.5) |
Values in parentheses are 95% confidence intervals.
AUC area under the receiver operating characteristic curve, TP true positive, FP false positive, FN false negative, TN true negative, PPV positive predictive value, NPV negative predictive value, LASSO least absolute shrinkage and selection operator, SVM support vector machine, RBF radial basis function kernel, RF random forest, KNN k-nearest neighbors.