TABLE 4.

Cross-validated area under the receiver-operating curve and 95% confidence interval (CI) for ensemble SL and each of the 42 algorithms in the original super learner library.

Algorithm	cv-AUC (CI)	Algorithm	cv-AUC (CI)
lasso20,dev	0.858 (0.842, 0.874)	rForest10	0.795 (0.778, 0.811)
lasso20,auc	0.855 (0.839, 0.870)	rForest20	0.772 (0.756, 0.788)
lasso10,auc	0.853 (0.836, 0.868)	glm50,pre	0.758 (0.743, 0.773)
lasso10,dev	0.849 (0.833, 0.865)	svm50,pre	0.746 (0.731, 0.760)
ridge20,dev	0.839 (0.824, 0.854)	glmStep50	0.743 (0.729, 0.758)
ridge10,dev	0.839 (0.823, 0.854)	glm50,wt	0.742 (0.728, 0.757)
SL	0.836 (0.822, 0.851)
ridge10,auc	0.836 (0.821, 0.851)	glm20,pre	0.742 (0.728, 0.757)
ridge20,auc	0.831 (0.816, 0.846)	glmStep50,pre	0.737 (0.723, 0.751)
lasso10,pre,auc	0.828 (0.812, 0.845)	glm10	0.736 (0.721, 0.751)
lasso10,pre,dev	0.826 (0.810, 0.842)	glm50	0.707 (0.693, 0.722)
lasso20,pre,dev	0.821 (0.805, 0.837)	rForest50	0.679 (0.666, 0.693)
lasso20,pre,auc	0.821 (0.805, 0.837)	glm20,wt	0.651 (0.638, 0.664)
lasso20,pre,dev	0.818 (0.801, 0.834)	rForest20,pre	0.594 (0.587, 0.601)
glm10,pre	0.814 (0.796, 0.830)	rForest10,pre	0.591 (0.584, 0.598)
ridge10,pre,auc	0.812 (0.796, 0.827)	rForest50,pre	0.588 (0.581, 0.596)
ridge10,pre,dev	0.811 (0.794, 0.826)	glm10,wt	0.552 (0.540, 0.564)
ridge20,pre,auc	0.809 (0.793, 0.825)	nnet10,5h,pre	0.5 (0.490, 0.510)
glm20	0.807 (0.790, 0.823)	nnet10,10h,pre	0.5 (0.490, 0.510)
glm50,pre,wt	0.805 (0.789, 0.821)	nnet20,5h,pre	0.5 (0.490, 0.510)
glm20,pre,wt	0.802 (0.785, 0.817)	nnet20,10h,pre	0.5 (0.490, 0.510)
glm10,pre,wt	0.799 (0.782, 0.815)	svm50	0.424 (0.399, 0.449)

Subscript key: (50, 20,10): # controls per case, pre: 23 pre-selected covariates, auc: neg AUC loss.

dev: deviance loss, wt: weighted regression, 5h, 10h: # nodes in hidden layer.

95% confidence intervals calculated using method of LeDell et al (2015).