Table 4.
Overall | By Primary Target | |||||
---|---|---|---|---|---|---|
Null Hyp‐ | Selected | Predictive | ||||
Performance | Estimand | othesis | Model | Performance | Other | |
Measure | (n = 64) | (n = 21) | (n = 8) | (n = 3) | (n = 4) | |
Convergence | 12/85 (14%) | 10/61 (16%) | 1/15 (7%) | 1/6 (17%) | 0/2 | 0/1 |
Bias | 63/80 (79%) | 59/64 (92%) | 1/9 (11%) | 0/2 | 2/3 | 1/2 |
Empirical SE | 31/78 (40%) | 31/62 (50%) | 0/9 | 0/2 | 0/3 | 0/2 |
Mean squared error | 26/78 (33%) | 22/62 (35%) | 2/9 (22%) | 0/2 | 1/3 | 1/2 |
Model SE | 22/77 (29%) | 21/62 (34%) | 1/9 (11%) | 0/2 | 0/3 | 0/1 |
Type I error | 31/95 (33%) | 8/62 (13%) | 18/21 (86%) | 4/6 | 0/3 | 1/3 |
Power | 28/95 (29%) | 8/63 (13%) | 14/20 (17%) | 4/6 | 0/3 | 2/3 |
Coverage | 42/79 (53%) | 39/63 (62%) | 1/9 (11%) | 0/2 | 1/3 | 1/2 |
Conf. int. length | 11/80 (14%) | 9/63 (14%) | 0/10 | 0/2 | 1/3 | 1/2 |
Note: Denominator changes across performance measures because not all are applicable in all simulation studies.