Table 4.
The performance of 20 combinations of machine learning methods in predicting the occurrence of rapid progression in patients with COVID-19
| Model | No. of features |
Train_ ACC |
Train_ SN |
Train_ SP |
Train_ PPV |
Train_ NPV |
Train_AUC (95% CI) |
Test_ ACC |
Test_ SN |
Test_ SP |
Test_ PPV |
Test_ NPV |
Test_AUC (95% CI) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LASSO + SVM | 11 | 69.6% | 92.1% | 63.5% | 40.6% | 96.7% | 0.854 (0.809–0.898) |
71.6% | 87.5% | 67.2% | 42.4% | 95.1% | 0.844 (0.750–0.938) |
| LASSO + LR | 11 | 75.7% | 81.0% | 74.3% | 46.0% | 93.5% | 0.860 (0.817–0.904) |
77.0% | 62.5% | 81.0% | 47.6% | 88.7% | 0.825 (0.724–0.927) |
| LASSO + DT | 11 | 98.7% | 100.0% | 98.3% | 94.0% | 100.0% | 1.000 (0.999–1.000) |
75.7% | 43.8% | 84.5% | 43.8% | 84.5% | 0.645 (0.510–0.780) |
| LASSO + RF | 11 | 97.0% | 87.3% | 99.6% | 98.2% | 96.7% | 0.998 (0.996–1.000) |
75.7% | 0 | 96.6% | 0 | 77.8% | 0.736 (0.611–0.860) |
| Relief + SVM | 18 | 65.2% | 73.0% | 63.1% | 34.9% | 89.6% | 0.777 (0.717–0.837) |
71.6% | 81.3% | 69.0% | 41.9% | 93.0% | 0.802 (0.701–0.903) |
| Relief + LR | 18 | 73.0% | 81.0% | 70.8% | 42.9% | 93.2% | 0.831 (0.779–0.883) |
77.0% | 68.8% | 79.3% | 47.8% | 90.2% | 0.842 (0.749–0.935) |
| Relief + DT | 18 | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% | 1.000 (1.000–1.000) |
73.0% | 12.5% | 89.7% | 25.0% | 78.8% | 0.511 (0.418–0.603) |
| Relief + RF | 18 | 90.9% | 60.3% | 99.1% | 95.0% | 90.2% | 0.920 (0.878–0.962) |
81.1% | 12.5% | 100.0% | 100.0% | 80.6% | 0.713 (0.577–0.850) |
| LVW + SVM | 2 | 60.1% | 74.6% | 56.2% | 31.5% | 89.1% | 0.758 (0.696–0.820) |
54.1% | 50.0% | 55.2% | 23.5% | 80.0% | 0.642 (0.492–0.792) |
| LVW + LR | 5 | 68.2% | 77.8% | 65.7% | 38.0% | 91.6% | 0.789 (0.732–0.845) |
71.6% | 75.0% | 70.7% | 41.4% | 91.1% | 0.780 (0.669–0.892) |
| LVW + DT | 24 | 99.0% | 100.0% | 98.7% | 95.5% | 100.0% | 0.999 (0.997–1.000) |
68.9% | 31.3% | 79.3% | 29.4% | 80.7% | 0.554 (0.424–0.685) |
| LVW + RF | 6 | 98.0% | 90.5% | 100.0% | 100.0% | 97.5% | 0.999 (0.997–1.000) |
77.0% | 12.5% | 94.8% | 40.0% | 79.7% | 0.671 (0.524–0.818) |
| L1-norm-SVM+SVM | 17 | 69.9% | 93.7% | 63.5% | 41.0% | 97.4% | 0.885 (0.845–0.924) |
74.3% | 87.5% | 70.7% | 45.2% | 95.4% | 0.857 (0.766–0.947) |
| L1-norm-SVM+LR | 17 | 77.7% | 84.1% | 76.0% | 48.6% | 94.7% | 0.889 (0.851–0.927) |
81.1% | 75.0% | 82.8% | 54.6% | 92.3% | 0.851 (0.753–0.949) |
| L1-norm-SVM+DT | 17 | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% | 1.000 (1.000–1.000) |
81.1% | 31.3% | 94.8% | 62.5% | 83.3% | 0.630 (0.510–0.751) |
| L1-norm-SVM+RF | 17 | 98.7% | 93.7% | 100.0% | 100.0% | 98.3% | 1.000 (0.999–1.000) |
81.1% | 31.3% | 94.8% | 62.5% | 83.3% | 0.728 (0.581–0.876) |
| RFE + SVM | 44 | 88.2% | 100.0% | 85.0% | 64.3% | 100.0% | 0.954 (0.932–0.976) |
73.0% | 62.5% | 75.9% | 41.7% | 88.0% | 0.776 (0.666–0.875) |
| RFE + LR | 44 | 83.1% | 92.1% | 80.7% | 56.3% | 97.4% | 0.930 (0.901–0.960) |
77.0% | 62.5% | 81.0% | 47.6% | 88.7% | 0.834 (0.738–0.931) |
| RFE + DT | 44 | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% | 1.000 (1.000–1.000) |
70.3% | 18.8% | 84.5% | 25.0% | 79.0% | 0.516 (0.407–0.626) |
| RFE + RF | 44 | 97.3% | 87.3% | 100.0% | 100.0% | 96.7% | 0.998 (0.996–1.000) |
78.4% | 6.3% | 98.3% | 50.0% | 79.2% | 0.658 (0.506–0.809) |
ACC, Accuracy; AUC, Area under the curve; DT, Decision tree;LASSO, Least absolute shrinkage and selection operator; LR, Logistic regression; LVW, Las vegas wrapper; NPV, Negative predictive value; PPV, Positive predictive value; RF, Random forest;SP, Specificity; SVM, Support vector machine.