Table 1. Accuracy of estimation methods compared with the exact model evidence.
−2 log(evidence) | Normalized evidence | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
MSD | MAD | MSD | MAD | |||||||||
K | ||||||||||||
1 | 0.00e + 00 | 0.00 | 28.72 | 0.00e + 00 | 0.00 | 28.72 | 7.03e-06 | −6.50e-03 | −2.76e-02 | 4.59e-05 | 6.50e-03 | 2.88e-02 |
2 | −1.54e-03 | 2.52 | 42.65 | 6.99e-03 | 2.52 | 42.65 | 6.67e-07 | 7.76e-03 | 4.93e-02 | 2.37e-04 | 1.41e-02 | 6.49e-02 |
3 | −1.95e-03 | 3.51 | 46.65 | 7.71e-03 | 3.51 | 46.65 | −6.00e-05 | 3.19e-02 | 8.96e-02 | 5.19e-04 | 3.96e-02 | 1.37e-01 |
4 | −1.96e-03 | 3.60 | 46.74 | 7.17e-03 | 3.60 | 46.74 | −8.42e-08 | 4.07e-02 | 1.37e-01 | 6.25e-04 | 5.54e-02 | 2.27e-01 |
5 | −1.61e-03 | 3.37 | 45.59 | 6.70e-03 | 3.37 | 45.59 | −9.85e-06 | 3.23e-02 | 1.37e-02 | 6.09e-04 | 5.16e-02 | 1.27e-01 |
6 | −1.39e-03 | 3.10 | 44.72 | 6.73e-03 | 3.10 | 44.72 | 1.59e-05 | 1.42e-02 | −5.46e-02 | 6.62e-04 | 4.08e-02 | 8.33e-02 |
7 | −1.47e-03 | 2.85 | 44.78 | 6.47e-03 | 2.85 | 44.78 | −7.72e-06 | −6.09e-03 | −6.99e-02 | 6.67e-04 | 3.17e-02 | 7.97e-02 |
8 | −1.18e-03 | 2.61 | 45.11 | 5.94e-03 | 2.61 | 45.11 | 2.01e-05 | −2.56e-02 | −6.74e-02 | 6.28e-04 | 3.17e-02 | 8.03e-02 |
9 | −1.21e-03 | 2.43 | 45.53 | 5.99e-03 | 2.43 | 45.53 | 4.17e-05 | −4.09e-02 | −5.11e-02 | 6.22e-04 | 4.23e-02 | 7.94e-02 |
10 | −1.44e-03 | 2.26 | 45.90 | 5.77e-03 | 2.26 | 45.90 | −2.17e-05 | −5.30e-02 | −2.08e-02 | 5.79e-04 | 5.31e-02 | 9.44e-02 |
Mean | −1.38e-03 | 2.63 | 43.64 | 5.95e-03 | 2.63 | 43.64 | −1.41e-06 | −5.23e-04 | −2.12e-04 | 5.19e-04 | 3.67e-02 | 1.00e-01 |
Shown are mean signed difference (MSD) and mean absolute difference (MAD) of various estimation methods compared with the exact value, obtained by brute force. Formulas for and are given in Equations 9, 4, and 5, respectively. Values are shown in log space (columns 2–7) and linear space after exponentiating and normalizing to sum to 1 (columns 8–13). Values of K here denote the value used in the inference step, with each row being an average over 1000 simulations (a more detailed breakdown can be found in Table S1).