Table 3.
Accuracies as measured by % correct, AUC, F1, and correct classification obtained by five different ML classifiers (using 10-fold cross-validation and 5-fold cross-validation).
| Classifier | Accuracy (%) | AUC | F1 | Correct classification | |
|---|---|---|---|---|---|
| 10-FOLD CROSS-FOLD VALIDATION | |||||
| Naïve Bayes | 86.84 | 0.90 (d = 1.81) | 0.87 | FMS 34/38 | HC 32/38 |
| Logistic regression | 72.37 | 0.78 (d = 1.29) | 0.72 | FMS 28/38 | HC 27/38 |
| Simple logistics | 94.74 | 0.93 (d = 2.09) | 0.95 | FMS 36/38 | HC 36/38 |
| Support vector machine | 90.79 | 0.91 (d = 1.90) | 0.91 | FMS 34/38 | HC 35/38 |
| Random forest | 88.16 | 0.94 (d = 2.20) | 0.88 | FMS 35/38 | HC 32/38 |
| 5-FOLD CROSS-VALIDATION | |||||
| Naïve Bayes | 86.84 | 0.91 (d = 1.89) | 0.87 | FMS 34/38 | HC 32/38 |
| Logistic regression | 71.05 | 0.79 (d = 1.14) | 0.71 | FMS 29/38 | HC 25/38 |
| Simple logistics | 86.84 | 0.87 (d = 1.59) | 0.87 | FMS 33/38 | HC 33/38 |
| Support vector machine | 88.16 | 0.88 (d = 1.66) | 0.88 | FMS 34/38 | HC 33/38 |
| Random forest | 86.84 | 0.95 (d = 2.32) | 0.87 | FMS 34/38 | HC 32/38 |
Perfect classification of exemplars in the two categories has an AUC of 1 and a F1 of 1. AUC stands for Area Under the Curve in ROC analysis and F1. In order to compare AUC with the best known effect size measure Cohen's d, is included. Classifiers were run with default parameters of Weka and therefore without any parameter tuning.