Table 1:
Summary of variable selection methods for random forest classification
| Abbreviation in Paper | Publication | R package/Implementation | Approach | Type of forest method | Summary | Parameter Settings |
|---|---|---|---|---|---|---|
| RF | Breiman 2001 [1] | randomForest | N/A | Random forest | No variable selection | Default |
| RFtuned | Breiman 2001 [1] | randomForest | N/A | Random forest | No variable selection, tuned with tuneRF() function | Default |
| Svetnik | Svetnik 2004 [16] | Uses party, code from Hapfelmeier [15] | Performance Based | Conditional Inference Forest | Uses backward elimination based on importance measures and k-fold validation | # trees=100, # folds=5, # repetitions=20 |
| Jiang | Jiang 2004 [17] | Uses party, code from Hapfelmeier [15] | Performance Based | Conditional Inference Forest | Similar to Svetnik but provides mechanism to prevent overfitting | # trees=1000 |
| varSelRF | Diaz Uriarte 2007 [6] | varSelRF | Performance Based | Random forest | Uses backward elimination, criteria to remove variables based on maintaining similar error rate to full model | Default |
| Caret | Kuhn 2008 [8] | caret | Performance Based | Random forest | Uses recursive feature elimination, criteria to remove variables based on maintaining similar error rate to full model | Default |
| Altmann | Altmann 2010 [18] | vita | Test Based | Random forest | Based on a parametric test of repeated permutations of importance measures | Default |
| Boruta | Kursa 2010 [5] | Boruta | Test Based | Random forest | Based on a permutation test using a hold out approach for importance measures | Default |
| Hapfelmeier | Hapfelmeier 2013 [15] | Uses party, code from Hapfelmeier [15] | Test Based | Conditional Inference Forest | Similar to Altmann, but uses unbiased importance measures | # permutations=100, # trees=100, alpha=0.05 |
| RRF | Deng 2013 [11] | RRF | Performance Based | Random Forest | Based on a regularized random forest, which uses forward selection to sequentially add variables until there is no further information gain | Default |
| SRC | Ishwaran 2014 [10] | randomForestSRC | Performance Based | Random Forest | Uses backward elimination based on minimal depth of predictors | Default |
| VSURF | Genuer 2015 [7] | VSURF | Performance Based | Random Forest | Stepwise selection procedure which implements backward elimination then forward selection based on importance measures and error rate | Default |
| Janitza | Janitza 2015 [12] | Vita | Test Based | Random forest | Similar to Altmann, but also uses cross validation | Default |