Table 5.
Ensemble methods with feature selection.
| Method | RF | ETC | GB |
|---|---|---|---|
| RFE(number of features: 10) | 64.34% (3.65%) | 73.03% (3.35%) | 68.97% (3.40%) |
| RFE(number of features: 15) | 67.93% (2.64%) | 66.13% (3.19%) | 67.90% (4.47%) |
| RFE(number of features: 20) | 66.92% (3.71%) | 73.91% (2.61%) | 69.21% (2.23%) |
| RFE(number of features: 25) | 68.77% (3.34%) | 74.00% (3.28%) | 72.66% (3.36%) |
| RFE(number of features: 30) | 65.60% (3.37%) | 72.12% (3.44%) | 65.11% (2.46%) |
| RFE(number of features: 35) | 66.99% (5.12%) | 73.00% (2.22%) | 64.06% (4.56%) |
| RFE(number of features: 40) | 63.71% (4.43%) | 71.96% (3.87%) | 61.12% (3.33%) |
| Chi-Squared statistic(>0.1) | 67.51% (4.38%) | 76.30% (3.84%) | 65.06% (3.18%) |
| Chi-Squared statistic(>0.5) | 59.69% (2.82%) | 72.79% (3.23%) | 64.77% (3.77%) |
| Chi-Squared statistic(>1.0) | 63.47% (3.48%) | 70.19% (2.55%) | 63.66% (3.36%) |
| Chi-Squared statistic(>2.0) | 52.80% (3.77%) | 54.19% (2.34%) | 54.44% (3.67%) |
| P-Test value (>0.1) | 67.22% (4.33%) | 76.22% (2.25%) | 67.16% (3.47%) |
| P-Test value (>0.5) | 67.76% (3.68%) | 75.19% (4.04%) | 68.00% (2.34%) |
| P-Test value (>0.8) | 65.49% (3.36%) | 70.21% (2.43%) | 69.99% (3.37%) |
| Model based(model: RT) | 75.35% (4.18%) | 76.14% (4.04%) | 71.62%(3.84%) |
| Model based(model: ETC) | 79.23% (3.53%) | 80.00% (4.27%) | 71.40% (4.12%) |
| Model based(model: NB) | 72.21% (3.87%) | 77.13% (2.23%) | 70.12%(4.22%) |
RF: Random Forest, ETC: Extra Tree Classifier, GB: Gradient Boost Classifier, RFE: Recursive Feature Elimination, NB: Naive Bayes.