Table 6. Comparing the performance of the proposed Ensemble Subspace kNN model against conventional machine learning models when using the GA+STAT feature set.
Results of 10-fold cross validation over 30 runs.
Proposed ensemble subspace kNN (EkNN) model (No. of learners (NL): 30; Subspace Dimension (SD): 16) | ||||||
---|---|---|---|---|---|---|
Parameters | AUC | ORP FPR | ORP TPR | ACC | ||
NL: 30, SD:16 | Mean | 0.818 | 0.201 | 0.836 | 0.821 | |
Std. | 0.021 | 0.027 | 0.021 | 0.020 | ||
Simple kNN model (Distance: Euclidean) | ||||||
k | AUC | ORP FPR | ORP TPR | ACC | Acc. Diff. (EkNN vs. kNN) |
|
2 | Mean | 0.768 | 0.241 | 0.730 | 0.751 | +0.070 |
Std. | 0.119 | 0.160 | 0.393 | 0.128 | −0.108 | |
5 | Mean | 0.778 | 0.300 | 0.833 | 0.783 | +0.038 |
Std. | 0.107 | 0.265 | 0.103 | 0.103 | −0.083 | |
10 | Mean | 0.753 | 0.371 | 0.845 | 0.758 | +0.063 |
Std. | 0.137 | 0.350 | 0.120 | 0.131 | −0.111 | |
Support Vector Machine models | ||||||
Kernel | AUC | ORP FPR | ORP TPR | ACC | Acc. Diff. (EkNN vs. SVM) |
|
Linear | Mean | 0.782 | 0.342 | 0.860 | 0.784 | +0.037 |
Std. | 0.126 | 0.352 | 0.110 | 0.120 | −0.100 | |
Gaussian | Mean. | 0.808 | 0.353 | 0.876 | 0.799 | +0.022 |
Std. | 0.112 | 0.416 | 0.107 | 0.111 | −0.091 | |
Naive Bayes model | ||||||
Predictor distributions | AUC | ORP FPR | ORP TPR | ACC | Acc. Diff. (EkNN vs. Naïve Bayes) |
|
Normal | Mean. | 0.695 | 0.132 | 0.455 | 0.662 | +0.159 |
Std. | 0.169 | 0.163 | 0.493 | 0.181 | −0.161 |