Table 3.
The model performance.
| Time | Classifier | AUROC | Sensitivity | Specificity | Brier Score | PPV | NPV |
|---|---|---|---|---|---|---|---|
| WCM (development site) | |||||||
| 12wk | Logistics regression | 0.921(0.893,0.949) | 0.79 | 0.97 | 0.074 | 0.61 | 0.99 |
| Random Forest | 0.897(0.866,0.928) | 0.80 | 0.97 | 0.054 | 0.60 | 0.99 | |
| Decision Tree | 0.903(0.873,0.933) | 0.83 | 0.96 | 0.045 | 0.59 | 0.99 | |
| XGboost | 0.908(0.878,0.938) | 0.82 | 0.97 | 0.068 | 0.60 | 0.99 | |
| MLP | 0.921(0.893,0.949) | 0.63 | 0.98 | 0.028 | 0.71 | 0.98 | |
| 18wk | Logistics regression | 0.919(0.891,0.947) | 0.79 | 0.97 | 0.074 | 0.61 | 0.99 |
| Random Forest | 0.897(0.866,0.928) | 0.80 | 0.97 | 0.056 | 0.60 | 0.99 | |
| Decision Tree | 0.890(0.858,0.922) | 0.82 | 0.96 | 0.048 | 0.59 | 0.99 | |
| XGboost | 0.902(0.872,0.932) | 0.82 | 0.97 | 0.097 | 0.60 | 0.99 | |
| MLP | 0.919(0.891,0.947) | 0.63 | 0.98 | 0.028 | 0.71 | 0.98 | |
| 24wk | Logistics regression | 0.922(0.895,0.949) | 0.79 | 0.97 | 0.074 | 0.61 | 0.99 |
| Random Forest | 0.903(0.873,0.933) | 0.80 | 0.97 | 0.057 | 0.60 | 0.99 | |
| Decision Tree | 0.895(0.864,0.926) | 0.83 | 0.96 | 0.048 | 0.59 | 0.99 | |
| XGboost | 0.919(0.891,0.947) | 0.83 | 0.96 | 0.082 | 0.57 | 0.99 | |
| MLP | 0.920(0.892,0.948) | 0.63 | 0.98 | 0.028 | 0.72 | 0.98 | |
| 30wk | Logistics regression | 0.921(0.893,0.949) | 0.79 | 0.97 | 0.074 | 0.61 | 0.99 |
| Random Forest | 0.914(0.885,0.943) | 0.83 | 0.97 | 0.056 | 0.65 | 0.99 | |
| Decision Tree | 0.887(0.855,0.919) | 0.82 | 0.96 | 0.048 | 0.59 | 0.99 | |
| XGboost | 0.912(0.883,0.941) | 0.82 | 0.96 | 0.085 | 0.57 | 0.99 | |
| MLP | 0.917(0.889,0.945) | 0.64 | 0.98 | 0.028 | 0.72 | 0.98 | |
| Childbirth | Logistics regression | 0.937(0.912,0.962) | 0.83 | 0.96 | 0.082 | 0.59 | 0.99 |
| Random Forest | 0.935(0.910,0.960) | 0.84 | 0.96 | 0.067 | 0.57 | 0.99 | |
| Decision Tree | 0.911(0.882,0.940) | 0.87 | 0.96 | 0.052 | 0.55 | 0.99 | |
| XGboost | 0.935(0.910,0.960) | 0.87 | 0.94 | 0.101 | 0.46 | 0.99 | |
| MLP | 0.933(0.907,0.959) | 0.64 | 0.99 | 0.026 | 0.75 | 0.98 | |
| CDRN (validation site) | |||||||
| 12wk | Logistics regression | 0.810(0.801,0.819) | 0.70 | 0.85 | 0.150 | 0.24 | 0.98 |
| Random Forest | 0.788(0.779,0.797) | 0.71 | 0.85 | 0.144 | 0.24 | 0.98 | |
| Decision Tree | 0.790(0.781,0.799) | 0.71 | 0.85 | 0.152 | 0.24 | 0.71 | |
| XGboost | 0.789(0.780,0.798) | 0.71 | 0.85 | 0.180 | 0.24 | 0.98 | |
| MLP | 0.812(0.803,0.821) | 0.65 | 0.87 | 0.111 | 0.26 | 0.97 | |
| 18wk | Logistics regression | 0.817(0.808,0.826) | 0.70 | 0.85 | 0.151 | 0.24 | 0.98 |
| Random Forest | 0.794(0.785,0.803) | 0.72 | 0.84 | 0.145 | 0.24 | 0.98 | |
| Decision Tree | 0.794(0.785,0.803) | 0.72 | 0.84 | 0.152 | 0.24 | 0.98 | |
| XGboost | 0.793(0.784,0.802) | 0.72 | 0.85 | 0.180 | 0.25 | 0.98 | |
| MLP | 0.817(0.808,0.826) | 0.65 | 0.87 | 0.111 | 0.26 | 0.97 | |
| 24wk | Logistics regression | 0.821(0.812,0.830) | 0.71 | 0.85 | 0.152 | 0.25 | 0.98 |
| Random Forest | 0.800(0.791,0.809) | 0.73 | 0.84 | 0.146 | 0.24 | 0.98 | |
| Decision Tree | 0.799(0.790,0.808) | 0.73 | 0.84 | 0.152 | 0.24 | 0.98 | |
| XGboost | 0.798(0.789,0.807) | 0.73 | 0.85 | 0.180 | 0.25 | 0.98 | |
| MLP | 0.824(0.815,0.833) | 0.64 | 0.88 | 0.110 | 0.27 | 0.97 | |
| 30wk | Logistics regression | 0.824(0.815,0.833) | 0.72 | 0.85 | 0.153 | 0.24 | 0.98 |
| Random Forest | 0.807(0.798,0.816) | 0.74 | 0.84 | 0.148 | 0.24 | 0.98 | |
| Decision Tree | 0.802(0.793,0.811) | 0.73 | 0.84 | 0.152 | 0.24 | 0.98 | |
| XGboost | 0.801(0.792,0.810) | 0.73 | 0.84 | 0.181 | 0.25 | 0.98 | |
| MLP | 0.827(0.818,0.836) | 0.65 | 0.88 | 0.110 | 0.27 | 0.97 | |
| Childbirth | Logistics regression | 0.886(0.879,0.893) | 0.80 | 0.84 | 0.158 | 0.26 | 0.98 |
| Random Forest | 0.860(0.852,0.868) | 0.82 | 0.87 | 0.154 | 0.26 | 0.99 | |
| Decision Tree | 0.856(0.848,0.864) | 0.86 | 0.84 | 0.149 | 0.27 | 0.99 | |
| XGboost | 0.864(0.856,0.872) | 0.84 | 0.84 | 0.178 | 0.27 | 0.99 | |
| MLP | 0.887(0.880,0.894) | 0.66 | 0.88 | 0.105 | 0.28 | 0.97 | |