Table 3.

Performance of the Propensity Score Estimator When Using Super Learner, Neural Networks, and Boosted CART for Propensity Score Estimation^a

Method and Scenario	ASAM, %	Estimate	Absolute Bias	MSE	Empirical SEb	Estimated SEc	95% CI Coverage, %d
Matching
Boosted CART
Scenario A	17.003	−0.521	0.138	0.041	0.092	0.117	81.6
Scenario B	15.762	−0.510	0.135	0.041	0.091	0.119	84.5
Scenario C	17.598	−0.520	0.140	0.044	0.094	0.122	84.5
Scenario D	21.470	−0.523	0.157	0.060	0.113	0.149	86.2
Neural Networks
Scenario A	13.862	−0.399	0.001	0.036	0.082	0.132	94.3
Scenario B	14.915	−0.404	0.004	0.040	0.086	0.138	94.1
Scenario C	14.354	−0.404	0.004	0.038	0.082	0.137	94.5
Scenario D	20.287	−0.395	0.006	0.069	0.114	0.176	91.0
Super Learner
Scenario A	6.734	−0.381	0.019	0.002	0.061	0.101	100.0
Scenario B	4.840	−0.403	0.003	0.001	0.067	0.085	98.5
Scenario C	7.025	−0.381	0.019	0.002	0.065	0.097	99.4
Scenario D	10.781	−0.418	0.018	0.003	0.113	0.082	85.6
Inverse Probability of Treatment Weighting
Boosted CART
Scenario A	13.297	−0.338	0.066	0.011	0.043	0.079	95.8
Scenario B	12.505	−0.343	0.062	0.011	0.040	0.074	96.7
Scenario C	14.473	−0.328	0.074	0.013	0.043	0.076	93.5
Scenario D	14.262	−0.354	0.058	0.011	0.040	0.079	97.3
Neural Networks
Scenario A	39.586	−0.561	0.257	0.150	0.180	0.226	83.0
Scenario B	36.317	−0.530	0.241	0.149	0.189	0.231	86.2
Scenario C	39.695	−0.558	0.256	0.153	0.183	0.230	84.3
Scenario D	35.386	−0.475	0.256	0.169	0.193	0.249	83.3
Super Learner
Scenario A	4.660	−0.391	0.009	0.001	0.045	0.031	83.7
Scenario B	4.470	−0.392	0.008	0.001	0.045	0.033	83.8
Scenario C	4.996	−0.391	0.009	0.001	0.050	0.032	79.1
Scenario D	8.472	−0.389	0.011	0.002	0.073	0.043	77.8

Abbreviations: ASAM, average standardized absolute mean difference; CART, classification and regression trees; CI, confidence interval; MSE, mean squared error.

^aAll tabulated results represent average values from 1,000 independent replications.

^bEmpirical Monte Carlo SE across the 1,000 simulated data sets.

^cEmpirical mean of the estimated SEs.

^dEmpirical coverage of nominal 95% CIs across the 1,000 simulated data sets.