Table 3.

Model performance in predicting CRBD in the training and validation cohorts

Models	Accuracy (95%CI)	AUC (95%CI)	Sensitivity	Specificity	PPV	NPV	F1	Brier
LR	0.88 (0.85, 0.90)	0.82 (0.79, 0.85)	0.68	0.96	0.88	0.88	0.77	0.12
RF	0.93 (0.91, 0.95)	0.89 (0.87, 0.91)	0.80	0.98	0.94	0.92	0.87	0.07
CatBoost	0.89 (0.87, 0.91)	0.84 (0.81, 0.87)	0.72	0.96	0.89	0.89	0.79	0.11
XGBoost	0.89 (0.87, 0.92)	0.84 (0.82, 0.87)	0.73	0.96	0.89	0.90	0.80	0.11
GBDT	0.90 (0.88, 0.93)	0.86 (0.83, 0.88)	0.75	0.97	0.90	0.90	0.82	0.10
LightGBM	0.91 (0.89, 0.93)	0.87 (0.84, 0.90)	0.78	0.96	0.90	0.91	0.83	0.09
Validation cohort
LR	0.91 (0.86, 0.95)	0.87 (0.82, 0.92)	0.76	0.98	0.96	0.89	0.85	0.09
RF	0.91 (0.87, 0.95)	0.88 (0.83, 0.93)	0.78	0.98	0.96	0.89	0.86	0.09
CatBoost	0.91 (0.87, 0.95)	0.88 (0.83, 0.93)	0.78	0.98	0.96	0.89	0.86	0.09
XGBoost	0.91 (0.86, 0.95)	0.87 (0.82, 0.92)	0.76	0.98	0.96	0.89	0.85	0.09
GBDT	0.90 (0.86, 0.95)	0.86 (0.81, 0.92)	0.75	0.98	0.96	0.88	0.84	0.10
LightGBM	0.92 (0.88, 0.96)	0.89 (0.84, 0.94)	0.80	0.98	0.96	0.90	0.87	0.08

LR Logistic Regression, RF Random Forest, CatBoost Categorical Boosting, XGBoost eXtreme Gradient Boosting, GBDT Gradient Boosting Decision Tree, LightGBM Light Gradient Boosting Machine, PPV positive predictive value, NPV negative predictive value; AUC: area under curve