Table 3. The performance comparison of the feature sets.
| # Features | Data | Method | Accuracy | MCC | Sensitivity | Specificity | AUC |
|---|---|---|---|---|---|---|---|
| - | - | Majority class | 0.763 ± 0.09 (0.0) |
0.0 ± 0.0 (0.0) |
0.0 ± 0.0 (0.0) |
1.0 ± 0.0 (0.0) |
0.5 ± 0.0 (0.0) |
| 11 | IFS | - | 0.644 ± 0.124 (0.0) |
0.028 ± 0.255 (0.0) |
0.269 ± 0.252 (0.0) |
0.767 ± 0.14 (0.0) |
0.518 ± 0.139 (0.0) |
| 13.5 ± 13.6 | T1 | UFS | 0.828 ± 0.083 (0.0) |
0.573 ± 0.183 (0.0) |
0.731 ± 0.217 (0.2496) |
0.873 ± 0.095 (0.0) |
0.802 ± 0.108 (0.0037) |
| 16.5 ± 13.2 | T1 | RFE | 0.823 ± 0.079 (0.0) |
0.549 ± 0.19 (0.0) |
0.702 ± 0.23 (0.0396) |
0.876 ± 0.089 (0.0001) |
0.789 ± 0.114 (0.0003) |
| 14.5 ± 13.0 | T1 | UFS+RFE | 0.814 ± 0.077 (0.0) |
0.52 ± 0.181 (0.0) |
0.656 ± 0.226 (0.0004) |
0.876 ± 0.092 (0.0001) |
0.766 ± 0.106 (0.0) |
| 18.1 ± 10.8 | T1 | RF |
0.87 ± 0.07 (0.4397) |
0.668 ± 0.152 (0.268) |
0.755 ± 0.198 (0.7221) |
0.922 ± 0.074 (0.7848) |
0.839 ± 0.092 (0.6576) |
| 15.7 ± 12.8 | T1 | Aggr. | 0.834 ± 0.08 (0.0001) |
0.577 ± 0.186 (0.0) |
0.711 ± 0.221 (0.0709) |
0.887 ± 0.09 (0.002) |
0.799 ± 0.109 (0.0021) |
| 10.8 ± 14.2 | T1+T2 | UFS | 0.839 ± 0.085 (0.0008) |
0.584 ± 0.207 (0.0001) |
0.715 ± 0.229 (0.101) |
0.889 ± 0.091 (0.0035) |
0.802 ± 0.116 (0.0053) |
| 17.5 ± 15.8 | T1+T2 | RFE | 0.807 ± 0.089 (0.0) |
0.503 ± 0.205 (0.0) |
0.637 ± 0.265 (0.0002) |
0.874 ± 0.101 (0.0001) |
0.755 ± 0.122 (0.0) |
| 12.9 ± 10.7 | T1+T2 | UFS+RFE | 0.796 ± 0.074 (0.0) |
0.465 ± 0.175 (0.0) |
0.596 ± 0.251 (0.0) |
0.873 ± 0.097 (0.0001) |
0.734 ± 0.109 (0.0) |
| 21.2 ± 14.4 | T1+T2 | RF |
0.878 ± 0.076 (1.0) |
0.693 ± 0.166 (1.0) |
0.765 ± 0.199 (1.0) |
0.925 ± 0.081 (1.0) |
0.845 ± 0.099 (1.0) |
| 15.6 ± 13.9 | T1+T2 | Aggr. | 0.83 ± 0.087 (0.0) |
0.561 ± 0.208 (0.0) |
0.678 ± 0.246 (0.0065) |
0.89 ± 0.095 (0.0056) |
0.784 ± 0.12 (0.0001) |
| 17.9 ± 17.2 | Full | UFS | 0.754 ± 0.105 (0.0) |
0.31 ± 0.23 (0.0) |
0.431 ± 0.256 (0.0) |
0.861 ± 0.129 (0.0) |
0.646 ± 0.119 (0.0) |
| 19.7 ± 16.5 | Full | RFE | 0.758 ± 0.088 (0.0) |
0.334 ± 0.213 (0.0) |
0.483 ± 0.278 (0.0) |
0.855 ± 0.114 (0.0) |
0.669 ± 0.118 (0.0) |
| 17.4 ± 15.2 | Full | UFS+RFE | 0.743 ± 0.09 (0.0) |
0.311 ± 0.21 (0.0) |
0.477 ± 0.251 (0.0) |
0.839 ± 0.123 (0.0) |
0.658 ± 0.111 (0.0) |
| 21.9 ± 15.5 | Full | RF |
0.819 ± 0.082 (0.0) |
0.53 ± 0.176 (0.0) |
0.614 ± 0.244 (0.0) |
0.902 ± 0.087 (0.0544) |
0.758 ± 0.111 (0.0) |
| 19.2 ± 16.2 | Full | Aggr. | 0.769 ± 0.096 (0.0) |
0.371 ± 0.228 (0.0) |
0.501 ± 0.267 (0.0) |
0.864 ± 0.117 (0.0) |
0.683 ± 0.123 (0.0) |
Best feature sets obtained in the FS experiments and their general performance across 100 random data splits. The columns correspond to the number of features, the data that were used to obtain the feature set, the FS method and five performance measures (mean ± std(p − value)). Welch’s t-test was used to obtain p-values for the null hypothesis that two performance measures had identical values. Each measure was tested against the same measures of the (T1+T2, RF)) model. The IFS dataset was used as a baseline for comparison. The best results for all measures in each dataset group are highlighted in bold.