Table 3.
Performance Metrics of Propensity Score Estimation Methods in 1000 Simulated Datasets of N=2000.
| Metric | Method1 | Scenario4 | ||||||
|---|---|---|---|---|---|---|---|---|
| A | B | C | D | E | F | G | ||
| ASAM2 | LGR | 0.029 | 0.031 | 0.052 | 0.047 | 0.05 | 0.057 | 0.09 |
| CART | 0.177 | 0.165 | 0.142 | 0.186 | 0.178 | 0.165 | 0.153 | |
| PRUNE | 0.181 | 0.171 | 0.143 | 0.189 | 0.182 | 0.168 | 0.156 | |
| BAG | 0.155 | 0.146 | 0.127 | 0.164 | 0.158 | 0.137 | 0.121 | |
| RFRST | 0.062 | 0.061 | 0.059 | 0.068 | 0.067 | 0.058 | 0.057 | |
| BOOST | 0.052 | 0.049 | 0.054 | 0.054 | 0.053 | 0.048 | 0.053 | |
| Absolute bias | LGR | 5.5% | 6.8% | 13.5% | 10.9% | 14.7% | 16.1% | 30.2% |
| CART | 27.6% | 18.8% | 16.5% | 25.1% | 18.9% | 32.2% | 21.1% | |
| PRUNE | 29.2% | 20.9% | 16.7% | 25.8% | 19.9% | 32.7% | 21.7% | |
| BAG | 19.9% | 13.0% | 11.3% | 16.7% | 11.6% | 16.3% | 9.1% | |
| RFRST | 5.0% | 4.5% | 7.5% | 5.2% | 5.2% | 5.5% | 8.0% | |
| BOOST | 5.6% | 4.7% | 4.8% | 4.7% | 4.1% | 4.1% | 4.7% | |
| Standard error | LGR | 0.047 | 0.047 | 0.043 | 0.054 | 0.053 | 0.054 | 0.051 |
| CART | 0.039 | 0.039 | 0.045 | 0.04 | 0.04 | 0.041 | 0.044 | |
| PRUNE | 0.039 | 0.039 | 0.045 | 0.04 | 0.04 | 0.041 | 0.044 | |
| BAG | 0.038 | 0.038 | 0.042 | 0.038 | 0.038 | 0.039 | 0.04 | |
| RFRST | 0.045 | 0.043 | 0.048 | 0.046 | 0.044 | 0.046 | 0.047 | |
| BOOST | 0.042 | 0.041 | 0.042 | 0.043 | 0.042 | 0.043 | 0.042 | |
| 95% CI3 coverage | LGR | 98.1% | 95.9% | 77.6% | 78.8% | 63.4% | 57.6% | 2.9% |
| CART | 18.1% | 45.0% | 55.6% | 27.3% | 47.5% | 20.4% | 41.6% | |
| PRUNE | 15.5% | 39.0% | 54.4% | 26.0% | 43.9% | 20.0% | 40.1% | |
| BAG | 31.6% | 69.8% | 76.1% | 51.9% | 78.5% | 52.9% | 85.8% | |
| RFRST | 99.2% | 99.3% | 90.8% | 98.1% | 98.3% | 98.2% | 89.5% | |
| BOOST | 99.0% | 99.9% | 99.9% | 99.8% | 99.9% | 99.9% | 99.1% | |
1
LGR – logistic regression, CART – classification and regression tree, PRUNE – pruned CART, BAG – bagged CART, RFRST – random forests, BOOST – boosted CART;
2
ASAM – average standardized absolute mean distance;
3
CI – confidence interval
4
A: additivity and linearity; B: mild non-linearity; C: moderate non-linearity; D: mild non-additivity; E: mild non-additivity and non-linearity; F: moderate non-additivity; G: moderate non-additivity and non-linearity.