Table 3.

Performance of end-to-end evaluation on the test set (best F1 scores are highlighted in bold)

Model	Performance
	strict			lenient
	precision	recall	F1 score	precision	recall	F1 score
LSTM-CRFs+SVMs-1a	0.8337	0.7773	0.8045	0.9112	0.8468	0.8778
RCNN-KB+SVMs-1	0.8406	0.7730	0.8054	0.9171	0.8400	0.8769
LSTM-CRFs+SVMs-4	0.8298	0.7810	0.8046	0.9089	0.8521	0.8796
RCNN-KB+SVMs-4	0.8400	0.7762	0.8069	0.9159	0.8430	0.8779
LSTM-CRFs+GB-4	0.8403	0.7881	0.8134	0.9187	0.8593	0.8880
RCNN-KB+GB-4	0.8504	0.7827	0.8151	0.9261	0.8495	0.8861

CRFs, conditional random fields; GB, gradient boosting; KB, knowledge embedding; LSTM, long-short term memory; RCNN, recurrent convolutional neural networks; SVMs, Support Vector Machines.

^aLSTM-CRFs+SVMs-1 is the final end-to-end system submitted during this challenge (ranked second).