Table 1.
Pairwise Comparison of Absolute Errors Using Different Methods of Adjustment for Optimism Using Data on Down Syndrome and Cesarean Delivery, United Kingdom, 1990–2007
| Method of Adjustment by Data Set | Median Difference in Absolute Error (IQR)a |
||||
|---|---|---|---|---|---|
| Sample Splitting | Bootstrapping | 10-Fold CV | 10-Fold CV (20 Replications) | Leave-Pair-Out CV | |
| Down syndrome data set | |||||
| Bootstrapping | 0.022*** (–0.005–0.053) | ||||
| 10-Fold CV | 0.015 (–0.028–0.046) | –0.006** (–0.024–0.003) | |||
| 10-Fold CV (20 replications) | 0.025*** (–0.008–0.0520) | 0.002 (–0.006–0.007) | 0.011*** (–0.003–0.025) | ||
| Leave-pair-out CV | 0.018*** (–0.005–0.052) | 0.000 (–0.004–0.005) | 0.007** (–0.004–0.024) | –0.002 (–0.005–0.003) | |
| Leave-1-out CV | 0.014** (–0.006–0.053) | –0.003 (–0.031–0.016) | 0.010 (–0.016–0.029) | –0.006 (–0.028–0.014) | –0.010* (–0.026–0.013) |
| Cesarean delivery data set | |||||
| Bootstrapping | 0.012*** (–0.018–0.079) | ||||
| 10-Fold CV | 0.013** (–0.032–0.066) | –0.008** (–0.035–0.017) | |||
| 10-Fold CV (20 replications) | 0.015*** (–0.023–0.069) | –0.001 (–0.014––0.008) | 0.008 (–0.018–0.028) | ||
| Leave-pair-out CV | 0.015*** (–0.023–0.072) | 0.000 (–0.009–0.006) | 0.007* (–0.018–0.030) | –0.001 (–0.006–0.007) | |
| Leave-1-out CV | 0.016** (–0.031–0.060) | –0.012** (–0.037–0.023) | 0.000 (–0.026–0.029) | –0.009** (–0.029–0.016) | –0.019*** (–0.029–0.020) |
Abbreviations: CV, cross-validation; IQR, interquartile range.
* P < 0.05, **P < 0.01, ***P < 0.001.
a The absolute error associated with the method in the row is subtracted from the absolute error associated with the method in the column, and the medians and IQRs are presented for the 50 subgroups. Hence, positive values indicate greater absolute error using the method in the column, and negative values indicate lower absolute error. Statistical comparison is by the Wilcoxon signed-rank test versus the null hypothesis of no difference.