. Author manuscript; available in PMC: 2020 Jul 13.

Published in final edited form as: J Biomed Inform. 2019 Feb 10;91:103123. doi: 10.1016/j.jbi.2019.103123

Table 7:

Results of experiments on the held-out test set. The best performing SVM model uses the features ngraml_pos + posgram + sent + struct + dep + rule. Best NN model is a CNN which uses POS and dependency-enhanced word embeddings as input and incorporates high-level features. Best results are underlined.

Experiment	Overall		Per Category
Experiment	Accuracy	MacroF₁	Cat	Pr.	Rec.	F₁
Rule-based method	0.831	0.604	POS	0.780	0.568	0.657
			NEG	0.389	0.187	0.252
			NEU	0.859	0.951	0.903
Xu et al. SVM model	0.847	0.633	POS	0.783	0.576	0.664
			NEG	0.548	0.227	0.321
			NEU	0.868	0.966	0.914
Best SVM model	0.850	0.656	POS	0.740	0.616	0.672
			NEG	0.600	0.280	0.382
			NEU	0.877	0.956	0.915
Best NN model	0.882	0.721	POS	0.783	0.681	0.728
			NEG	0.930	0.341	0.497
			NEU	0.895	0.982	0.937