Table 2.
Comparison of results when basing variable selection on the FARMS algorithm or common strategies. The number of covariates included in the final model excludes the “forced in” covariates. FARMS parameters used in this case are number of adding covariates = 8, number of starting covariates = 10, and the selecting criteria for both best subset and best model was the BIC. (All runs were executed on an Intel Xeon x5680 machine with 6 CPU cores and 95 GB RAM memory under a Linux Suse 11.0 OS).
| Time (seconds)*
Mean (IQR) |
Number of vars.** | BIC | AIC | R 2 | Adj. R 2 | ||
|---|---|---|---|---|---|---|---|
| HLA | FARMS | 1.27 (1.00; 1.40) | 9 | 2235.4 | 2183.03 | 11.67% | 10.74% |
| All subsets | >1 month | — | — | — | — | — | |
| Forward selection1 | 3.84 (3.13; 3.52) | 17 | 2259.4 | 2168.86 | 14.69% | 12.98% | |
| Forward stepwise1 | 4.32 (3.51; 3.90) | 17 | 2259.4 | 2168.86 | 14.69% | 12.98% | |
| Forward selection2 | 2.01 (1.62; 1.88) | 10 | 2235.45 | 2178.27 | 12.36% | 11.33% | |
| Forward stepwise2 | 2.35 (1.89; 2.18) | 9 | 2235.44 | 2183.03 | 11.67% | 10.74% | |
| Forward selection3 | 1.99 (1.61; 1.88) | 10 | 2235.45 | 2178.27 | 12.36% | 11.33% | |
| Forward stepwise3 | 2.38 (1.92; 2.22) | 9 | 2235.44 | 2183.03 | 11.67% | 10.74% | |
| Backward stepwise3 | 13 s | 10 | 2236.52 | 2174.58 | 12.93% | 11.81% | |
|
| |||||||
| OLP | FARMS | 33.4 (19.47; 37.95) | 12 | 2224.8 | 2158.11 | 14.77% | 13.57% |
| All subsets | >1 month | — | — | — | — | — | |
| Forward selection1 | 393.2 (324.40; 336.60) | 79 | 2396.53 | 2010.56 | 38.40% | 32.22% | |
| Forward stepwise1 | 545.9 (451.70; 469.70) | 83 | 2415.46 | 2010.53 | 38.97% | 32.51% | |
| Forward selection2 | 401.7 (329.50; 343.40) | 80 | 2403.3 | 2012.56 | 38.40% | 32.13% | |
| Forward stepwise2 | 462.4 (382.50; 401.50) | 76 | 2385.18 | 2013.51 | 37.76% | 31.77% | |
| Forward selection3 | 38.09 (31.34; 33.11) | 12 | 2224.82 | 2158.11 | 14.77% | 13.57% | |
| Forward stepwise3 | 38.31 (31.34; 33.43) | 12 | 2224.82 | 2158.11 | 14.77% | 13.57% | |
| Backward stepwise3 | >12 hours | 23 | 2232.63 | 2108.63 | 21.68% | 19.45% | |
1Selection by AIC and base model with intercept-only.
2Selection by AIC and base model with forced-in variables.
3Selection by BIC and base model with forced-in variables.
*Time obtained after 100 executions for each scenario.
**Including forced-in variables.