TABLE 5.
Cross-validated area under the receiver-operating curve and 95% confidence interval (CI) for gradient boosted (xgb) and neural network (nn) algorithms in the augmented super learner library
| Algorithm | cv-AUC (CI) | Algorithm | cv-AUC (CI) | Algorithm | cv-AUC (CI) |
|---|---|---|---|---|---|
| xgb10,2,10 | 0.844 (0.828, 0.859) | xgb20,2,25 | 0.836 (0.819, 0.853) | nn20,1 | 0.787 (0.772, 0.802) |
| xgb50,4,50 | 0.842 (0.823, 0.861) | xgb20,4,50 | 0.834 (0.816, 0.852) | nn10,1 | 0.773 (0.758, 0.788) |
| xgb50,2,25 | 0.839 (0.823, 0.856) | xgb50,4,25 | 0.832 (0.814, 0.849) | nn10,2 | 0.773 (0.758, 0.787) |
| xgb20,2,10 | 0.839 (0.823, 0.855) | xgb20,2,50 | 0.832 (0.815, 0.848) | nn20,5 | 0.733 (0.717, 0.749) |
| xgb10,4,25 | 0.838 (0.821, 0.855) | xgb50,2,10 | 0.831 (0.815, 0.847) | nn50,5 | 0.711 (0.693, 0.729) |
| xgb50,2,50 | 0.837 (0.819, 0.856) | xgb10,2,50 | 0.831 (0.813, 0.848) | nn50,1 | 0.708 (0.693, 0.724) |
| xgb10,4,50 | 0.837 (0.820, 0.854) | xgb20,4,10 | 0.819 (0.803, 0.836) | nn50,2 | 0.5 (0.490, 0.510) |
| xgb20,4,25 | 0.837 (0.819, 0.854) | xgb10,4,10 | 0.816 (0.800, 0.832) | nn10,5 | 0.5 (0.490, 0.510) |
| xgb10,2,25 | 0.836 (0.819, 0.853) | xgb50,4,10 | 0.799 (0.782, 0.815) | nn20,2 | 0.5 (0.490, 0.510) |
xgb subscript key: (a, b, c): a = # controls per case, b=depth, c = min obs per node.
nn subscript key: (a, b): a = # controls per case, b= # nodes in hidden layer.
95% confidence intervals calculated using method of LeDell et al (2015).