Table 3.
False discovery rates of variable selection methods, for each scenario
Method | Pairwise correlation between all variables |
||
---|---|---|---|
0.2 | 0.5 | 0.8 | |
5 signals among 20 variables | |||
One-at-a-time* | 0.09 (0.01) | 0.58 (0.02) | 0.40 (0.02) |
Stepwise | <0.01 (<0.01) | <0.01 (<0.01) | 0.01 (<0.01) |
Two-step BVS* | <0.01 (<0.01) | <0.01 (<0.01) | <0.01 (<0.01) |
5 signals among 100 variables | |||
One-at-a-time* | 0.25 (0.02) | 0.77 (0.02) | 0.67 (0.02) |
Stepwise | 0.01 (<0.01) | 0.03 (0.01) | 0.08 (0.01) |
Two-step BVS* | <0.01 (<0.01) | 0.01 (<0.01) | 0.01 (<0.01) |
5 signals among 1000 variables | |||
One-at-a-time* | 0.53 (0.03) | 0.93 (0.01) | 0.78 (0.02) |
Stepwise | 0.07 (0.01) | 0.14 (0.01) | 0.27 (0.01) |
Two-step BVS* | 0.01 (<0.01) | 0.01 (<0.01) | 0.06 (0.01) |
Mean false discovery rate, the proportion of noise variables selected, is displayed for 200 simulations with the corresponding Monte Carlo errors in brackets.
*Selection thresholds chosen to match the sensitivity of the stepwise method in each simulation, for which a nominal P-value inclusion threshold of 0.05 was used.