Table 6.

Performance characteristics of methods in detection of actionable reports without and with oversampling of positive samples in the training data

Method	Use of order information	Oversampling	AUPRC	Average precision	AUROC	F1 score
LR	( −)	( −)	0.4574	0.4224	0.9351	0.1052
		( +)	0.3166	0.3167	0.8036	0.0474
	( +)	( −)	0.4218	0.4580	0.8992	0.0763
		( +)	0.4214	0.4221	0.9277	0.1089
GBDT	( −)	( −)	0.4813	0.4816	0.8986	0.0975
		( +)	0.4854	0.4858	0.9335	0.0841
	( +)	( −)	0.4767	0.4771	0.9133	0.0699
		( +)	0.4874	0.4878	0.9307	0.0920
LSTM	( −)	( −)	0.4617	0.4620	0.9331	0.0916
		( +)	0.4188	0.4194	0.9262	0.0818
	( +)	( −)	0.4265	0.4272	0.9277	0.0797
		( +)	0.4086	0.4066	0.9255	0.0795
BERT	( −)	( −)	0.5138	0.5142	0.9516	0.1030
		( +)	0.4256	0.4273	0.9464	0.1190
	( +)	( −)	0.5153	0.5157	0.9497	0.1183
		( +)	0.4549	0.4559	0.9441	0.0953

Maximum values for each method are shown in bold