Table 2.
The significance of the differences between the metric distributions of different machine learning algorithms.
| CART | CTREE | LDA | SVM | KNN | RF | ||
|---|---|---|---|---|---|---|---|
| Accuracy | CART | 0.008 | 0.106 | 0.054 | 0.190 | − 0.008 | |
| Accuracy | CTREE | 0.004 | 0.098 | 0.046 | 0.182 | − 0.016 | |
| Accuracy | LDA | < 0.001 | < 0.001 | − 0.052 | 0.084 | − 0.114 | |
| Accuracy | SVM | < 0.001 | < 0.001 | < 0.001 | 0.136 | − 0.062 | |
| Accuracy | KNN | < 0.001 | < 0.001 | < 0.001 | < 0.001 | − 0.198 | |
| Accuracy | RF | 0.005 | < 0.001 | < 0.001 | < 0.001 | < 0.001 | |
| Kappa | CART | 0.016 | 0.199 | 0.107 | 0.418 | − 0.014 | |
| Kappa | CTREE | 0.005 | 0.183 | 0.091 | 0.403 | − 0.030 | |
| Kappa | LDA | < 0.001 | < 0.001 | − 0.092 | 0.220 | − 0.213 | |
| Kappa | SVM | < 0.001 | < 0.001 | < 0.001 | 0.312 | − 0.121 | |
| Kappa | KNN | < 0.001 | < 0.001 | < 0.001 | < 0.001 | − 0.433 | |
| Kappa | RF | 0.009 | < 0.001 | < 0.001 | < 0.001 | < 0.001 | |
| Mean_Balanced_Accuracy | CART | 0.011 | 0.124 | 0.110 | 0.283 | − 0.003 | |
| Mean_Balanced_Accuracy | CTREE | 0.023 | 0.113 | 0.099 | 0.272 | − 0.014 | |
| Mean_Balanced_Accuracy | LDA | < 0.001 | < 0.001 | − 0.013 | 0.159 | − 0.126 | |
| Mean_Balanced_Accuracy | SVM | < 0.001 | < 0.001 | 1.000 | 0.172 | − 0.113 | |
| Mean_Balanced_Accuracy | KNN | < 0.001 | < 0.001 | < 0.001 | < 0.001 | − 0.285 | |
| Mean_Balanced_Accuracy | RF | 1.000 | < 0.001 | < 0.001 | < 0.001 | < 0.001 | |
| Mean_F1 | CART | 0.006 | 0.142 | 0.076 | 0.161 | − 0.007 | |
| Mean_F1 | CTREE | 0.031 | 0.137 | 0.074 | 0.158 | − 0.013 | |
| Mean_F1 | LDA | < 0.001 | < 0.001 | − 0.069 | 0.028 | − 0.149 | |
| Mean_F1 | SVM | < 0.001 | < 0.001 | < 0.001 | 0.090 | − 0.083 | |
| Mean_F1 | KNN | 0.001 | 0.001 | 1.000 | 0.156 | − 0.170 | |
| Mean_F1 | RF | < 0.001 | < 0.001 | < 0.001 | < 0.001 | 0.001 | |
| Mean_Precision | CART | 0.009 | 0.143 | 0.037 | 0.037 | − 0.009 | |
| Mean_Precision | CTREE | 0.287 | 0.132 | 0.025 | 0.035 | − 0.018 | |
| Mean_Precision | LDA | < 0.001 | < 0.001 | − 0.100 | − 0.093 | − 0.151 | |
| Mean_Precision | SVM | 0.256 | 1.000 | < 0.001 | 0.011 | − 0.045 | |
| Mean_Precision | KNN | 1.000 | 1.000 | 0.173 | 1.000 | − 0.047 | |
| Mean_Precision | RF | < 0.001 | 0.001 | < 0.001 | 0.058 | 1.000 | |
| Mean_Recall | CART | 0.016 | 0.190 | 0.189 | 0.419 | − 0.002 | |
| Mean_Recall | CTREE | 0.063 | 0.174 | 0.173 | 0.403 | − 0.017 | |
| Mean_Recall | LDA | < 0.001 | < 0.001 | − 0.001 | 0.229 | − 0.192 | |
| Mean_Recall | SVM | < 0.001 | < 0.001 | 1.000 | 0.230 | − 0.191 | |
| Mean_Recall | KNN | < 0.001 | < 0.001 | < 0.001 | < 0.001 | − 0.420 | |
| Mean_Recall | RF | 1.000 | 0.003 | < 0.001 | < 0.001 | < 0.001 | |
| Mean_Sensitivity | CART | 0.016 | 0.190 | 0.189 | 0.419 | − 0.002 | |
| Mean_Sensitivity | CTREE | 0.063 | 0.174 | 0.173 | 0.403 | − 0.017 | |
| Mean_Sensitivity | LDA | < 0.001 | < 0.001 | − 0.001 | 0.229 | − 0.192 | |
| Mean_Sensitivity | SVM | < 0.001 | < 0.001 | 1.000 | 0.230 | − 0.191 | |
| Mean_Sensitivity | KNN | < 0.001 | < 0.001 | < 0.001 | < 0.001 | − 0.420 | |
| Mean_Sensitivity | RF | 1.000 | 0.003 | < 0.001 | < 0.001 | < 0.001 | |
| Mean_Specificity | CART | 0.006 | 0.057 | 0.031 | 0.147 | − 0.004 | |
| Mean_Specificity | CTREE | 0.005 | 0.051 | 0.025 | 0.140 | − 0.010 | |
| Mean_Specificity | LDA | < 0.001 | < 0.001 | − 0.026 | 0.089 | − 0.061 | |
| Mean_Specificity | SVM | < 0.001 | < 0.001 | < 0.001 | 0.115 | − 0.035 | |
| Mean_Specificity | KNN | < 0.001 | < 0.001 | < 0.001 | < 0.001 | − 0.150 | |
| Mean_Specificity | RF | 0.026 | < 0.001 | < 0.001 | < 0.001 | < 0.001 |
Significant values are in bold.
p-value adjustment: Bonferroni.
Upper diagonal: estimates of the difference.
Lower diagonal: p-value for H0: difference = 0.