Table 2.
Performance of the models on the IHDP, the augmented IHDP (adding noise/correlated variables), and the Jobs dataset
| Dataset | # Covariates/Samples | Model | MSE (SD) εPEHE/εATT |
Bengio’s P value |
Raw P value |
|---|---|---|---|---|---|
| IHDP | 25/747 | DPN-SA (N-20-10-) end-to-end | 2.09 (0.22) | Ref. | Ref. |
| DPN-SA (N-20-10-) greedy stack. | 2.10 (0.20) | 0.95 | 0.74 | ||
| DPN-SA (N-20-10-) greedy stack. curr. | 2.11 (0.20) | 0.89 | 0.50 | ||
| DPN-SA (N-10-) | 2.14 (0.22) | 0.75 | 0.11 | ||
| DPN-SA (N-1-) | 2.52 (0.37) | 0.05 | 2.23E-18 | ||
| DCN-PD | 2.22 (0.21) | 0.40 | 3.37E-05 | ||
| Logistic Regression | 6.02 (1.19) | 1.56E-09 | 2.88E-70 | ||
| LASSO Logistic Regression | 5.51 (1.03) | 1.56E-09 | 2.92E-70 | ||
| Augmented IHDP (noise and correlated variables) | 100/747 | DPN-SA (N-20-10-) end-to-end | 3.15 (0.50) | 0.02 | 1.14E-23 |
| DPN-SA (N-20-10-) greedy stack. | 2.72 (0.39) | 0.30 | 4.59E-07 | ||
| DPN-SA (N-20-10-) greedy stack. curr. | 2.72 (0.37) | 0.28 | 1.92E-07 | ||
| DPN-SA (N-10-) | 2.47 (0.27) | Ref. | Ref. | ||
| DPN-SA (N-1-) | 2.50 (0.27) | 0.88 | 4.33E-01 | ||
| DCN-PD | 2.77 (0.46) | 0.27 | 8.69E-08 | ||
| Logistic Regression | 6.85 (1.16) | 1.65E-11 | 1.49E-77 | ||
| LASSO Logistic Regression | 6.71 (1.11) | 1.18E-11 | 4.72E-78 | ||
| Jobs | 46 (17)/3212 | DPN-SA (N-20-10-) end-to-end | 0.09 (0.10) | 0.28 | 0.17 |
| DPN-SA (N-20-10-) greedy stack. | 0.12 (0.10) | 0.01 | 2.08E-03 | ||
| DPN-SA (N-20-10-) greedy stack. curr. | 0.11 (0.09) | 0.03 | 0.01 | ||
| DPN-SA (N-10-) | 0.12 (0.10) | 0.01 | 2.08E-03 | ||
| DPN-SA (N-1-) | 0.07 (0.10) | Ref. | Ref. | ||
| DCN-PD | 0.13 (0.11) | 4.71E-03 | 6.47E-04 | ||
| Logistic Regression | 0.11 (0.09) | 0.03 | 0.01 | ||
| LASSO Logistic Regression | 0.15 (0.12) | 6.48E-04 | 5.20E-05 |
Abbreviations: DCN-PD, deep counterfactual networks with propensity dropout; DPN-SA, deep propensity network-sparse autoencoder; IHDP, Infant Health and Development Program