Table 2.
Performance Metrics of Propensity Score Estimation Methods in 1000 Simulated Datasets of N=500.
| Metric | Method1 | Scenario4 | ||||||
|---|---|---|---|---|---|---|---|---|
| A | B | C | D | E | F | G | ||
| ASAM2 | LGR | 0.059 | 0.057 | 0.065 | 0.077 | 0.078 | 0.081 | 0.103 |
| CART | 0.143 | 0.138 | 0.152 | 0.155 | 0.151 | 0.142 | 0.149 | |
| PRUNE | 0.179 | 0.17 | 0.165 | 0.184 | 0.181 | 0.166 | 0.165 | |
| BAG | 0.129 | 0.126 | 0.125 | 0.143 | 0.141 | 0.126 | 0.121 | |
| RFRST | 0.099 | 0.095 | 0.093 | 0.108 | 0.107 | 0.101 | 0.095 | |
| BOOST | 0.089 | 0.085 | 0.084 | 0.096 | 0.094 | 0.088 | 0.086 | |
| Absolute bias | LGR | 11.2% | 11.2% | 13.8% | 15.3% | 17.7% | 18.1% | 30.3% |
| CART | 19.0% | 16.6% | 21.8% | 19.3% | 19.0% | 20.1% | 21.6% | |
| PRUNE | 27.6% | 22.6% | 24.5% | 24.8% | 22.6% | 22.9% | 23.1% | |
| BAG | 12.6% | 10.4% | 13.5% | 11.6% | 10.3% | 10.8% | 11.4% | |
| RFRST | 10.7% | 9.2% | 11.4% | 10.5% | 9.5% | 10.5% | 11.6% | |
| BOOST | 11.0% | 9.6% | 9.2% | 10.3% | 9.3% | 9.0% | 8.6% | |
| Standard error | LGR | 0.094 | 0.094 | 0.087 | 0.105 | 0.105 | 0.103 | 0.102 |
| CART | 0.095 | 0.096 | 0.105 | 0.097 | 0.098 | 0.096 | 0.104 | |
| PRUNE | 0.085 | 0.085 | 0.097 | 0.089 | 0.089 | 0.088 | 0.097 | |
| BAG | 0.08 | 0.079 | 0.085 | 0.081 | 0.079 | 0.081 | 0.082 | |
| RFRST | 0.086 | 0.083 | 0.085 | 0.088 | 0.084 | 0.087 | 0.085 | |
| BOOST | 0.084 | 0.082 | 0.081 | 0.086 | 0.083 | 0.085 | 0.083 | |
| 95% CI3 coverage | LGR | 97.0% | 97.3% | 96.2% | 91.1% | 87.5% | 86.5% | 64.3% |
| CART | 84.5% | 88.3% | 76.9% | 82.4% | 84.0% | 81.6% | 75.7% | |
| PRUNE | 65.3% | 75.3% | 71.5% | 71.2% | 75.8% | 76.6% | 73.8% | |
| BAG | 97.5% | 98.2% | 95.4% | 98.7% | 99.1% | 99.0% | 98.1% | |
| RFRST | 98.5% | 99.7% | 97.9% | 98.5% | 98.7% | 99.0% | 97.0% | |
| BOOST | 99.0% | 99.2% | 99.5% | 99.1% | 99.6% | 99.9% | 99.8% | |
1
LGR – logistic regression, CART – classification and regression tree, PRUNE – pruned CART, BAG – bagged CART, RFRST – random forests, BOOST – boosted CART;
2
ASAM – average standardized absolute mean distance;
3
CI – confidence interval
4
A: additivity and linearity; B: mild non-linearity; C: moderate non-linearity; D: mild non-additivity; E: mild non-additivity and non-linearity; F: moderate non-additivity; G: moderate non-additivity and non-linearity.