Table 7.

Evaluation results of the end-to-end models (ie, output from the best-performing named entity recognition and relation extraction models) on the test set (lenient evaluation).

Relation type	RIWEa+rules				PWEb+rules				(PWE+CE)c+rules				(PWE+SFE)d+rules
	Precision	Recall	F-score	Precision		Recall	F-score	Precision		Recall	F-score	Precision		Recall	F-score
Strength-drug	0.919	0.914	0.917	0.952		0.943	0.947	0.948		0.950	0.949	0.948		0.964	0.956e
Dosage-drug	0.848	0.853	0.851	0.890		0.888	0.889	0.892		0.884	0.888	0.894		0.897	0.895
Duration-drug	0.837	0.615	0.709	0.842		0.662	0.741	0.889		0.617	0.729	0.860		0.678	0.759
Frequency-drug	0.878	0.874	0.876	0.931		0.919	0.925	0.949		0.902	0.925	0.934		0.947	0.940
Form-drug	0.894	0.888	0.891	0.939		0.915	0.927	0.944		0.919	0.931	0.959		0.920	0.939
Route-drug	0.885	0.866	0.875	0.924		0.895	0.909	0.919		0.904	0.911	0.920		0.908	0.914
Reason-drug	0.635	0.333	0.437	0.702		0.371	0.485	0.744		0.343	0.470	0.503		0.472	0.487
Micro	0.865	0.770	0.815	0.909		0.802	0.852	0.918		0.797	0.855	0.871		0.830	0.850
Macro	0.859	0.733	0.784	0.902		0.770	0.824	0.918		0.765	0.824	0.849		0.801	0.821

^aRIWE: bidirectional long-short term memory with conditional random fields with random word embeddings.

^bPWE: bidirectional long-short term memory with conditional random fields with pretrained word embeddings.

^cPWE+CE: bidirectional long-short term memory with conditional random fields with pretrained word embeddings and character embeddings.

^dPWE+SFE: bidirectional long-short term memory with conditional random fields with pretrained word embeddings and semantic-feature embeddings.

^eThe best results for each metric are italicized.