Table 1.
Performance Metrics of Propensity Score Estimation Methods in 1000 Simulated Datasets of N=1000.
| Metric | Method1 | Scenario4 | ||||||
|---|---|---|---|---|---|---|---|---|
| A | B | C | D | E | F | G | ||
| ASAM2 | LGR | 0.041 | 0.042 | 0.058 | 0.056 | 0.061 | 0.068 | 0.094 |
| CART | 0.159 | 0.148 | 0.143 | 0.171 | 0.162 | 0.15 | 0.143 | |
| PRUNE | 0.175 | 0.164 | 0.148 | 0.182 | 0.173 | 0.161 | 0.151 | |
| BAG | 0.132 | 0.127 | 0.121 | 0.144 | 0.141 | 0.119 | 0.112 | |
| RFRST | 0.08 | 0.076 | 0.076 | 0.089 | 0.086 | 0.077 | 0.075 | |
| BOOST | 0.068 | 0.065 | 0.067 | 0.073 | 0.071 | 0.065 | 0.067 | |
| Absolute bias | LGR | 7.6% | 8.3% | 13.9% | 12.1% | 16.0% | 16.8% | 29.6% |
| CART | 20.5% | 15.2% | 18.2% | 20.1% | 16.9% | 22.5% | 19.0% | |
| PRUNE | 26.3% | 19.5% | 19.1% | 22.9% | 19.1% | 23.9% | 19.9% | |
| BAG | 12.3% | 9.1% | 11.0% | 11.2% | 9.0% | 10.2% | 9.4% | |
| RFRST | 7.4% | 6.2% | 8.9% | 7.6% | 7.3% | 7.5% | 9.0% | |
| BOOST | 8.1% | 6.9% | 6.8% | 7.0% | 6.4% | 6.1% | 6.2% | |
| Standard error | LGR | 0.066 | 0.066 | 0.062 | 0.075 | 0.076 | 0.075 | 0.071 |
| CART | 0.059 | 0.059 | 0.068 | 0.06 | 0.06 | 0.061 | 0.066 | |
| PRUNE | 0.057 | 0.057 | 0.066 | 0.059 | 0.059 | 0.059 | 0.065 | |
| BAG | 0.055 | 0.055 | 0.06 | 0.056 | 0.055 | 0.056 | 0.058 | |
| RFRST | 0.062 | 0.06 | 0.064 | 0.064 | 0.061 | 0.063 | 0.062 | |
| BOOST | 0.059 | 0.058 | 0.059 | 0.061 | 0.059 | 0.06 | 0.059 | |
| 95% CI3 coverage | LGR | 97.9% | 96.8% | 89.7% | 88.5% | 80.0% | 76.0% | 32.5% |
| CART | 63.3% | 78.2% | 69.5% | 63.7% | 74.7% | 58.3% | 67.2% | |
| PRUNE | 49.1% | 66.0% | 68.0% | 56.6% | 68.4% | 54.9% | 65.7% | |
| BAG | 90.9% | 96.8% | 90.4% | 91.8% | 95.7% | 93.6% | 94.7% | |
| RFRST | 98.3% | 99.6% | 95.3% | 98.2% | 98.6% | 98.4% | 95.1% | |
| BOOST | 98.6% | 99.8% | 99.5% | 99.9% | 99.9% | 100.0% | 99.8% | |
1
LGR – logistic regression, CART – classification and regression tree, PRUNE – pruned CART, BAG – bagged CART, RFRST – random forests, BOOST – boosted CART;
2
ASAM – average standardized absolute mean distance;
3
CI – confidence interval
4
A: additivity and linearity; B: mild non-linearity; C: moderate non-linearity; D: mild non-additivity; E: mild non-additivity and non-linearity; F: moderate non-additivity; G: moderate non-additivity and non-linearity.