Table 3.
Friedman example: where Ip = 1 for truly important variables x1 … x5 and Ip = 0 otherwise. Hence the lower the Brier score the better the model variable selection. Items in bold show the best model with respect to the Brier score.
p | BART BMA | RF | bart Machine | dbarts default | dbarts best |
---|---|---|---|---|---|
100 | 3.24 × 10−2 | 3.82 × 10−2 | 4.30 × 10−2 | 4.66 × 10−2 | 4.17 ×10−2 |
1000 | 3.26 × 10−3 | 4.00 × 10−3 | 4.62 × 10−3 | 4.87 × 10−3 | 4.49 × 10−3 |
5000 | 6.55 × 10−4 | 8.18 × 10−4 | 9.68 × 10−4 | 9.85 × 10−4 | 9.31 × 10−4 |
10000 | 3.28 × 10−4 | 4.13 × 10−4 | 4.89 × 10−4 | 4.95 × 10−4 | 4.75 × 10−4 |
15000 | 2.18 × 10−4 | 2.76 × 10−4 | 3.29 × 10−4 | 3.31 × 10−4 | 3.18 × 10−4 |
100000 | 3.28 × 10−5 | 4.21 × 10−5 | - | 4.99 × 10−5 | 4.98 × 10−5 |
500000 | 6.62 × 10−6 | 8.54 × 10−6 | - | 9.99 × 10−6 | - |