Table 3.

Summary of prediction performance of different models

Model	Precision	Recall	F1 score	AUROC
Random forest	0.8565 ± 0.0014a	0.6871 ± 0.0027	0.7545 ± 0.0022	0.9112 ± 0.0014
Decision tree	0.7592 ± 0.0084	0.7281 ± 0.0059	0.7453 ± 0.0030	0.8823 ± 0.0019
Logistic regression	0.7507 ± 0.0095	0.6020 ± 0.0089	0.6722 ± 0.0035	0.7933 ± 0.0036
Dense neural network	0.8019 ± 0.0108	0.7694 ± 0.0027	0.7855 ± 0.0049	0.9224 ± 0.012
LSTM	0.8184 ± 0.0085	0.7865 ± 0.0058a	0.8023 ± 0.0020a	0.9369 ± 0.0038
Bi-LSTM	0.7779 ± 0.0013	0.7615 ± 0.0012	0.7696 ± 0.0012	0.9377 ± 0.0065
LSTM+Attention	0.8131 ± 0.0081	0.7814 ± 0.0071	0.7969 ± 0.0035	0.9491 ± 0.0023a
Bi-LSTM+Attention	0.7710 ± 0.0086	0.7804 ± 0.0109	0.7759 ± 0.0019	0.9463 ± 0.0006
BERT	0.7709 ± 0.0123	0.6709 ± 0.0056	0.7174 ± 0.0079	0.8687 ± 0.0060

AUROC: area under the receiver-operating characteristic curve; Bi-LSTM: bidirectional long short-term memory; LSTM: long short-term memory.

^aBest result for the metric.