Table 3.

Performances of the predictive models in the development and validation data sets.

Metric	MELRa			RNNb			ME-SVMc			ME-DTd
	Development	Validation	Development		Validation	Development		Validation	Development		Validation
Sensitivity, % (95% CI)	91.2 (89.4-92.8)	89.4 (85-92.8)	61.6 (58.3 64.9)		54.9 (48-61.7)	52.8 (49.5-56)		46.1 (39.1-53.2)	47 (44-50.1)		44.5 (38.4-50.7)
Specificity, % (95% CI)	90.3 (88.9-91.6)	92.5 (89.9-94.7)	72.9 (70.9-75)		74.4 (70.1-78.3)	82.7 (80.9-84.4)		78.2 (74.2-81.8)	80.2 (78.4-81.9)		81.3 (77.6-84.6)
Accuracy, % (95% CI)	90.6 (89.5-91.6)	91.4 (89.2-93.3)	69.3 (67.6-71.1)		68.2 (64.5-71.7)	72.7 (71-74.4)		68.6 (65-72.1)	68.8 (67.1 – 70.4)		68.7 (65.3-72)
AUCe (95% CI)	0.980 (0.977-0.984)	0.983 (0.977-0.989)	0.747 (0.727-0.766)		0.712 (0.669-0.754)	0.761 (0.754-0.766)		0.698 (0.681-0.734)	0.695 (0.677-0.714)		0.662 (0.621-0.702)
F1-score	0.869	0.878	0.573		0.543	0.564		0.467	0.509		0.493
Brier score	0.061	0.058	0.181		0.187	0.198		0.200	0.236		0.240
LR+f (95% CI)	9.4 (8.2-10.8)	11.9 (8.8-16.3)	2.3 (2.1-2.5)		2.1 (1.8-2.6)	3.1 (2.7-3.4)		2.1 (1.7-2.6)	2.4 (2.1-2.6)		2.4 (1.9-3.0)

^aMELR: mixed-effects logistic regression.

^bRNN: recurrent neural networks.

^cME-SVM: mixed-effects support vector machine.

^dME-DT: mixed-effects decision tree.

^eAUC: area under the receiver operating characteristic curve.

^fLR+: positive likelihood ratio.