TABLE 4.
n | Estimator type | Effect | P(reject H 0) | MSE | Bias | Variance | Rel. Eff. |
---|---|---|---|---|---|---|---|
100 | Unadjusted | 0.500 | 0.054 | 0.263 | −0.014 | 0.263 | 1.000 |
100 | Adjusted | 0.500 | 0.050 | 0.234 | −0.017 | 0.234 | 0.890 |
100 | Unadjusted | 0.585 | 0.389 | 0.226 | −0.010 | 0.226 | 1.000 |
100 | Adjusted | 0.585 | 0.431 | 0.208 | −0.011 | 0.208 | 0.919 |
100 | Unadjusted | 0.609 | 0.625 | 0.205 | −0.009 | 0.205 | 1.000 |
100 | Adjusted | 0.609 | 0.670 | 0.194 | −0.006 | 0.194 | 0.944 |
200 | Unadjusted | 0.500 | 0.053 | 0.264 | 0.010 | 0.264 | 1.000 |
200 | Adjusted | 0.500 | 0.056 | 0.236 | 0.014 | 0.236 | 0.895 |
200 | Unadjusted | 0.587 | 0.720 | 0.232 | −0.006 | 0.232 | 1.000 |
200 | Adjusted | 0.587 | 0.776 | 0.205 | −0.001 | 0.206 | 0.886 |
200 | Unadjusted | 0.612 | 0.924 | 0.217 | 0.008 | 0.217 | 1.000 |
200 | Adjusted | 0.612 | 0.953 | 0.190 | 0.021 | 0.190 | 0.879 |
500 | Unadjusted | 0.500 | 0.063 | 0.271 | −0.018 | 0.271 | 1.000 |
500 | Adjusted | 0.500 | 0.044 | 0.230 | −0.020 | 0.230 | 0.848 |
500 | Unadjusted | 0.556 | 0.710 | 0.262 | −0.001 | 0.262 | 1.000 |
500 | Adjusted | 0.556 | 0.749 | 0.231 | −0.008 | 0.231 | 0.882 |
500 | Unadjusted | 0.576 | 0.935 | 0.249 | 0.000 | 0.250 | 1.000 |
500 | Adjusted | 0.576 | 0.958 | 0.224 | −0.002 | 0.224 | 0.897 |
1000 | Unadjusted | 0.500 | 0.039 | 0.255 | −0.004 | 0.255 | 1.000 |
1000 | Adjusted | 0.500 | 0.040 | 0.227 | −0.007 | 0.228 | 0.894 |
1000 | Unadjusted | 0.540 | 0.722 | 0.243 | −0.005 | 0.243 | 1.000 |
1000 | Adjusted | 0.540 | 0.745 | 0.220 | −0.013 | 0.220 | 0.906 |
1000 | Unadjusted | 0.556 | 0.956 | 0.234 | −0.003 | 0.234 | 1.000 |
1000 | Adjusted | 0.556 | 0.970 | 0.214 | −0.009 | 0.214 | 0.915 |
BCa bootstrap is used for confidence intervals and hypothesis testing. “Effect” denotes the true estimand value; “MSE” denotes mean squared error; “Rel. Eff.” denotes relative efficiency, which we approximate as the ratio of the MSE of the estimator under consideration to the MSE of the unadjusted estimator. In each block of six rows, the first two rows involve no treatment effect and the last four rows involve a benefit from treatment. MSE and variance are scaled by n; bias is scaled by .