Table 2.
Performance metrics (accuracy, recall, precision, F1-score, and balanced accuracy) of the ML and DL models according to sampling method. We measured recall, precision, and F1-score as weighted averages. The bold typeface stands for the highest metrics in each measurement.
| Original Data | |||||
|---|---|---|---|---|---|
| ML/DL Models | Accuracy (%) | Recall (%) | Precision (%) | F1-Score (%) | Balanced Accuracy (%) |
| SVM | 52.1 ± 1.5 | 52.1 ± 1.5 | 53.8 ± 2.5 | 50.4 ± 1.5 | 41.8 ± 1.5 |
| DecisionTree | 86.0 ± 1.0 | 86.0 ± 1.0 | 86.4 ± 1.1 | 86.0 ± 1.1 | 79.0 ± 1.9 |
| Perceptron | 39.1 ± 3.2 | 39.1 ± 3.2 | 64.3 ± 2.8 | 34.7 ± 3.7 | 32.3 ± 2.6 |
| LightGBM | 91.2 ± 0.5 | 91.2 ± 0.5 | 91.5 ± 0.5 | 91.1 ± 0.5 | 85.8 ± 1.4 |
| AutoGluon | 91.7 ± 0.3 | 91.7 ± 0.3 | 92.0 ± 0.3 | 91.7 ± 0.3 | 86.8 ± 1.3 |
| SuperTML | 89.3 ± 0.8 | 89.3 ± 0.8 | 89.8 ± 0.8 | 89.2 ± 0.9 | 83.1 ± 2.4 |
| TabNet | 89.5 ± 0.6 | 89.5 ± 0.6 | 89.8 ± 0.6 | 89.4 ± 0.6 | 84.0 ± 1.4 |
| SMOTE (Over Sampling) | |||||
| ML/DL Models | Accuracy (%) | Recall (%) | Precision (%) | F1-Score (%) | Balanced Accuracy (%) |
| SVM | 57.7 ± 1.6 | 57.7 ± 1.6 | 63.9 ± 2.1 | 59.7 ± 1.8 | 52.5 ± 3.1 |
| DecisionTree | 86.1 ± 0.7 | 86.1 ± 0.7 | 86.6 ± 0.7 | 86.1 ± 0.7 | 80.7 ± 2.5 |
| Perceptron | 38.2 ± 3.6 | 38.2 ± 3.6 | 62.7 ± 2.9 | 35.1 ± 3.6 | 32.9 ± 2.9 |
| LightGBM | 90.8 ± 0.7 | 90.8 ± 0.7 | 91.2 ± 0.6 | 90.8 ± 0.7 | 86.1 ± 1.2 |
| AutoGluon | 91.0 ± 0.2 | 91.0 ± 0.2 | 91.3 ± 0.2 | 90.9 ± 0.2 | 86.6 ± 1.2 |
| SuperTML | 90.3 ± 0.9 | 90.3 ± 0.9 | 90.6 ± 0.9 | 90.2 ± 0.9 | 84.1 ± 1.4 |
| TabNet | 89.5 ± 0.5 | 89.5 ± 0.5 | 90.0 ± 0.5 | 89.5 ± 0.5 | 84.9 ± 1.6 |
| TomekLinks (Under Sampling) | |||||
| ML/DL Models | Accuracy (%) | Recall (%) | Precision (%) | F1-Score (%) | Balanced Accuracy (%) |
| SVM | 53.1 ± 1.6 | 53.1 ± 1.6 | 55.2 ± 1.6 | 51.5 ± 1.8 | 42.4 ± 1.4 |
| DecisionTree | 84.9 ± 0.8 | 84.9 ± 0.8 | 85.5 ± 0.8 | 84.9 ± 0.8 | 78.6 ± 2.3 |
| Perceptron | 35.6 ± 5.7 | 35.6 ± 5.7 | 66.5 ± 4.3 | 32.2 ± 4.3 | 31.0 ± 3.3 |
| LightGBM | 90.0 ± 0.6 | 90.0 ± 0.6 | 90.4 ± 0.6 | 90.0 ± 0.6 | 85.0 ± 2.5 |
| AutoGluon | 90.2 ± 0.2 | 90.2 ± 0.2 | 90.7 ± 0.1 | 90.2 ± 0.2 | 85.9 ± 1.6 |
| SuperTML | 89.0 ± 0.8 | 89.0 ± 0.8 | 89.6 ± 0.8 | 88.9 ± 0.8 | 82.4 ± 1.4 |
| TabNet | 88.4 ± 0.9 | 88.4 ± 0.9 | 88.8 ± 0.8 | 88.4 ± 0.8 | 83.0 ± 1.6 |
| SMOTETomek (Combined Sampling) | |||||
| ML/DL Models | Accuracy (%) | Recall (%) | Precision (%) | F1-Score (%) | Balanced Accuracy (%) |
| SVM | 57.5 ± 1.5 | 57.5 ± 1.5 | 63.7 ± 1.5 | 59.4 ± 1.6 | 52.5 ± 2.7 |
| DecisionTree | 85.7 ± 0.9 | 85.7 ± 0.9 | 86.2 ± 1.0 | 85.8 ± 0.9 | 80.3 ± 2.4 |
| Perceptron | 39.9 ± 3.9 | 39.9 ± 3.9 | 62.5 ± 2.4 | 36.0 ± 4.6 | 34.3 ± 3.8 |
| LightGBM | 90.4 ± 0.7 | 90.4 ± 0.7 | 90.8 ± 0.6 | 90.4 ± 0.6 | 85.8 ± 1.6 |
| AutoGluon | 90.4 ± 0.2 | 90.4 ± 0.2 | 90.7 ± 0.2 | 90.4 ± 0.2 | 85.6 ± 1.4 |
| SuperTML | 89.8 ± 1.4 | 89.8 ± 0.9 | 90.4 ± 0.9 | 89.8 ± 0.9 | 83.3 ± 1.7 |
| TabNet | 89.2 ± 0.8 | 89.2 ± 0.8 | 89.6 ± 0.9 | 89.2 ± 0.8 | 85.3 ± 2.7 |