Table 3.

Performance in detection of actionable reports with different use of order information for each method

Method	Use of order information	AUPRC	Average precision	AUROC	F1 score	Recall	Precision	Specificity
LR	(−)	0.4574	0.4224	0.9351	0.1052	0.8729	0.0559	0.8715
	(+)	0.4218	0.4580	0.8992	0.0763	0.8136	0.0400	0.8296
GBDT	(−)	0.4813	0.4816	0.8986	0.0975	0.7881	0.0520	0.8746
	(+)	0.4767	0.4771	0.9133	0.0699	0.8305	0.0365	0.8087
LSTM	(−)	0.4617	0.4620	0.9331	0.0916	0.8644	0.0484	0.8516
	(+)	0.4265	0.4272	0.9277	0.0797	0.8856	0.0417	0.8226
BERT	(−)	0.5138	0.5142	0.9516	0.1030	0.9068	0.0546	0.9271
	(+)	0.5153	0.5157	0.9497	0.1183	0.8729	0.0634	0.9140

Maximum values are shown in bold