Table 5.
Pooled Standardised Bias Estimates.
| NRS Estimator | (1) Standardized Numerical Difference* | (2) Standardized Absolute Difference* | (3) Standardized Absolute Difference** | (4) Standardized Absolute Difference*** | (5) Mean-Squared Error*** | (6) Percent Difference*** | (7) Percent Bias*** | (8) Percent Bias Removed*** | Number of Distance estimates$ |
|---|---|---|---|---|---|---|---|---|---|
| Cross-section regression | 0.232 | 0.236 | 0.229 | 0.290 | 0.180 | 340.8 | 26.3 | 33.9 | 10 |
| IV regression | 0.206 | 0.206 | 0.104 | 0.206 | 0.095 | 31.8 | 83.8 | −29.6 | 3 |
| IV (appropriate instrument) | 0.007 | 0.007 | 0.007 | 0.007 | 0.000 | 1.1 | 3.0 | 95.4 | 1 |
| IV (inappropriate instrument) | 0.305 | 0.305 | 0.184 | 0.305 | 0.142 | 47.2 | 124.2 | −92.1 | 2 |
| Panel data regression (DD) | 0.137 | 0.137 | 0.137 | 0.137 | 0.019 | 21.2 | 55.9 | 55.9 | 2 |
| Matching | 0.084 | 0.280 | 0.059 | 0.246 | 0.183 | 210.3 | 13.1 | 58.3 | 177 |
| Matching on baseline outcome | 0.120 | 0.120 | 0.039 | 0.044 | 0.004 | 1.6 | 4.3 | 55.9 | 14 |
| Matching on local comparison | −0.001 | 0.200 | 0.043 | 0.088 | 0.028 | −9.0 | −8.4 | 53.1 | 66 |
| Matching with rich controls | 0.025 | 0.282 | 0.031 | 0.232 | 0.124 | 178.4 | 8.4 | 81.3 | 116 |
| Matching with parsimonious controls | 0.208 | 0.321 | 0.087 | 0.354 | 0.305 | 353.8 | 25.9 | 44.9 | 44 |
| Kernel matching | 0.022 | 0.284 | 0.139 | 0.282 | 0.149 | 255.3 | 4.3 | 34.3 | 70 |
| Local linear matching | 0.179 | 0.257 | 0.201 | 0.285 | 0.133 | 267.8 | 18.3 | 21.4 | 6 |
| Nearest neighbor matching | 0.011 | 0.273 | 0.040 | 0.125 | 0.069 | 54.9 | −2.4 | 51.5 | 58 |
| Radius matching | 0.154 | 0.256 | 0.215 | 0.280 | 0.143 | 100.8 | 11.0 | 236.6 | 6 |
| RDD | −0.015 | 0.048 | 0.025 | 0.012 | 0.000 | 7.3 | 8.0 | 94.2 | 173 |
| RDD (prospective design) | −0.037 | 0.073 | 0.052 | 0.064 | 0.008 | −580.0 | 33.4 | 383.8 | 92 |
| RDD (retrospective design) | 0.009 | 0.020 | 0.014 | 0.010 | 0.000 | 33.1 | 6.9 | 81.5 | 81 |
| RDD (ATE comparison)^ | −0.057 | 0.091 | 0.038 | 0.029 | 0.002 | 34.4 | 17.6 | 0.1 | 71 |
Note. RDD = regression discontinuity design; DD = double-differences.
Notes: * simple average used to calculate pooled estimate; ** weighted average calculated using the inverse of the variance multiplied by the inverse of the number of estimates in the study; *** weighted average calculated using the benchmark sample size multiplied by the inverse of the number of estimates in the study; ^ indicates RDD estimate compared with RCT average treatment effect (ATE comparisons also incorporated Urquieta et al., 2009 and Lamadrid-Figueroa et al., 2010); $ sample size is for calculations in (1-7), calculations in (8) use a smaller number of studies owing to more limited availability of a prima facie estimate.