Table 5.

Predictive accuracy (95%CI) of regression and ML models in INTERRSeCT data (external validation).

Predictive accuracy metric	RF	SVM	C5.0	ABM	CART	LR	LASSO
Automated variable selection
Sensitivity	0.70 (0.64–0.76)	0.65 (0.59–0.71)	0.76 (0.70–0.81)	0.68 (0.61–0.74)	0.77 (0.71–0.82)	0.83 (0.78–0.88)	0.71 (0.60–0.79)
Specificity	0.71 (0.65–0.77)	0.77 (0.71–0.82)	0.58 (0.51–0.65)	0.70 (0.60–0.79)	0.60 (0.54–0.67)	0.56 (0.49–0.62)	0.67 (0.61–0.73)
AUC	0.70 (0.66–0.75)	0.71 (0.65–0.75)	0.66 (0.63–0.72)	0.67 (0.65–0.73)	0.69 (0.64–0.73)	0.69 (0.65–0.73)	0.67 (0.60–0.73)
MCC	0.41 (0.29–0.54)	0.42 (0.29–0.53)	0.34 (0.15–0.44)	0.38 (0.16–0.43)	0.38 (0.14–0.41)	0.40 (0.29–0.52)	0.39 (0.20–0.50)
Brier score	0.32 (0.28–0.41)	0.32 (0.29–0.42)	0.32 (0.28–0.41)	0.32 (0.29–0.42)	0.33 (0.30–0.43)	0.30 (0.27–0.39)	0.30 (0.27–0.39)
Clinical expert knowledge
Sensitivity	0.67 (0.61–0.73)	0.67 (0.61–0.73)	0.67 (0.60–0.73)	0.56 (0.49–0.62)	0.66 (0.60–0.72)	0.81 (0.75–0.85)	0.71 (0.65–0.77)
Specificity	0.71 (0.65–0.76)	0.72 (0.66–0.78)	0.75 (0.69–0.80)	0.78 (0.73–0.83)	0.69 (0.62–0.75)	0.60 (0.54–0.66)	0.69 (0.63–0.75)
AUC	0.66 (0.58–0.73)	0.67 (0.65–0.74)	0.63 (0.58–0.67)	0.68 (0.63–0.72)	0.67 (0.63–0.72)	0.68 (0.65–0.72)	0.69 (0.65–0.73)
MCC	0.38 (0.22–0.55)	0.39 (0.27–0.55)	0.42 (0.28–0.51)	0.35 (0.21–0.53)	0.35 (0.21–0.54)	0.41 (0.29–0.55)	0.40 (0.29–0.53)
Brier score	0.31 (0.25–0.37)	0.26 (0.22–0.38)	0.35 (0.27–0.36)	0.25 (0.21–0.32)	0.25 (0.21–0.38)	0.27(0.25–0.36)	0.27(0.24–0.35)

95%CI, 95% Confidence Interval; AUC, Area under the receiver operating characteristic curve; RF, Random Forest; SVM, Support Vector Machine; C5.0, C5.0 Decision Tree; ABM, Adaptive Boost Machine; CART, Classification and Regression Tree; LR, Logistic Regression; LASSO, Least Absolute Shrinkage and Selection Operation; MCC, Matthews Correlation Coefficient.