Table 2.
Results of the simulation study. PMSE is reported as the mean (standard error) over the simulated datasets for each simulation setting. The true (false) positive rate is computed by recording the proportion of the important (unimportant) variables included for each data set and averaging over all simulated data sets. A variable is deemed to be included in the Bayesian models if the posterior inclusion probability is greater than 0.5.
| (a) Prediction mean squared error (PMSE) | |||||
|---|---|---|---|---|---|
| Design | Correlation of the predictors | MARS | COSSO | BSS-ANOVA | Linkletter |
| 1 | Ind | 3.23 (0.28) | 2.33 (0.13) | 1.67 (0.08) | 3.50 (0.12) |
| 1 | CS (t=1) | 7.60 (0.83) | 6.08 (0.40) | 4.11 (0.27) | 7.39 (0.24) |
| 1 | AR (ρ=0.5) | 5.86 (0.44) | 5.37 (0.33) | 3.72 (0.18) | 6.38 (0.22) |
| 2 | Ind | 2.26 (0.08) | 1.68 (0.04) | 1.63 (0.04) | 1.40 (0.05) |
| 3 | Ind | 5.03 (0.38) | 4.79 (0.23) | 2.72 (0.09) | 4.50 (0.08) |
| (b) Inclusion percentage for variables not in the true model | |||||
|---|---|---|---|---|---|
| Design | Correlation of the predictors | MARS | COSSO | BSS-ANOVA | Linkletter |
| 1 | Ind | 0.04 | 0.06 | 0.03 | 0.03 |
| 1 | CS (t=1) | 0.04 | 0.13 | 0.03 | 0.08 |
| 1 | AR (ρ=0.5) | 0.03 | 0.12 | 0.05 | 0.06 |
| 2 | Ind | – | – | – | – |
| 3 | Ind | 0.04 | 0.13 | 0.10 | 0.11 |
| (c) Inclusion percentage for variables in the true model | |||||
|---|---|---|---|---|---|
| Design | Correlation of the predictors | MARS | COSSO | BSS-ANOVA | Linkletter |
| 1 | Ind | 0.78 | 0.91 | 0.91 | 0.81 |
| 1 | CS (t=1) | 0.74 | 0.83 | 0.79 | 0.80 |
| 1 | AR (ρ=0.5) | 0.75 | 0.82 | 0.78 | 0.80 |
| 2 | Ind | – | – | – | – |
| 3 | Ind | 0.67 | 0.77 | 0.82 | 0.89 |