Table 3:

Multivariate Model Performance

	Logistic Regression – Backward Stepwise	Logistic Regression – Forward Stepwise	Random Forest
Development Set (Case block 1 vs. Control block 1)
AUC	0.769	0.774	0.738
Validation Set (Case block 1 vs. Control block 1)
AUC	0.842	0.849	0.829
Sensitivity, FPR 2.5%	21.1%	15.8%	36.8%
Sensitivity, FPR 5.0%	42.1%	42.1%	57.9%
Sensitivity, FPR 10.0%	52.6%	52.6%	63.2%
Validation Set Performance at Selected Thresholda
Sensitivity	47.4%	47.4%	63.2%
Specificity	93.9%	93.2 %	94.6%
Testb – All Case Block 1 vs. Case Block 2
AUC	0.753	0.749	0.954
Sensitivity	39.5%	41.8%	91.2%
Specificity	89.4%	89.4%	83.5%
Testb – Validation Set Case Block 1 vs. Control Block 2
AUC	0.849	0.852	0.840
Sensitivity	42.1%	47.3%	63.2%
Specificity	94.6%	94.5%	93.8%
Testb – Case Block 2 vs. Validation Set Control Block 1
AUC	0.524	0.536	0.584
Sensitivity	10.6%	10.6%	16.5%
Specificity	93.9%	93.2%	94.6%
Testb – Case block 2 vs. Control block 2
AUC	0.532	0.543	0.600
Sensitivity	10.6%	10.6%	16.5%
Specificity	94.6%	94.5%	93.8%

Abbreviations: AUC – area under the curve, FPR – false positive rate

^aThreshold selected for FPR on validation set approximately 5 %

^bAll sensitivity and specificities are reported at the selected threshold