Table 6.

Performance of the two-tier extraction system: our rule-based system in the second tier

Min #Inst. per Relation	#Uniq. Relation		#Instances		Methods	Per-Relation					Per-Instance
	Pos	Neg		Neg		TP	FP	Precision	Recall	F-Score	TP	FP	Precision	Recall	F-Score
1	618	2,312	1,000	4,834	ours	132	7	0.950	0.214	0.349	153	10	0.939	0.153	0.263
					hybrid+	479	344	0.582	0.775	0.665	705	584	0.547	0.705	0.616
					SST-PT+	257	108	0.704	0.416	0.523	322	178	0.644	0.322	0.429
					SVM+	131	57	0.697	0.212	0.325	140	84	0.625	0.140	0.229
					NB+	235	247	0.488	0.380	0.427	261	410	0.389	0.261	0.312
					DT+	157	73	0.683	0.254	0.370	169	120	0.585	0.169	0.262
					kNN+	219	260	0.457	0.354	0.399	237	366	0.393	0.237	0.296
2	197	695	579	2,827	ours	80	3	0.964	0.406	0.571	101	5	0.953	0.174	0.294
					hybrid+	183	164	0.527	0.929	0.673	399	327	0.550	0.689	0.612
					SST-PT+	132	73	0.644	0.670	0.657	203	118	0.632	0.351	0.451
					SVM+	88	48	0.647	0.447	0.529	106	78	0.576	0.183	0.278
					NB+	107	113	0.486	0.543	0.513	140	230	0.378	0.242	0.295
					DT+	88	41	0.682	0.447	0.540	104	85	0.550	0.180	0.271
					kNN+	125	151	0.453	0.635	0.529	152	250	0.378	0.263	0.310
3	89	314	363	1,835	ours	51	2	0.962	0.573	0.718	67	3	0.957	0.185	0.310
					hybrid+	88	98	0.473	0.989	0.640	262	236	0.526	0.722	0.609
					SST-PT+	71	48	0.597	0.798	0.683	136	82	0.624	0.375	0.468
					SVM+	43	28	0.606	0.483	0.538	59	40	0.596	0.163	0.256
					NB+	59	69	0.461	0.663	0.544	82	113	0.421	0.226	0.294
					DT+	49	17	0.742	0.551	0.632	61	26	0.701	0.168	0.271
					kNN+	59	86	0.407	0.663	0.504	84	151	0.357	0.231	0.281
4	51	181	249	1,349	ours	35	2	0.946	0.686	0.795	50	3	0.943	0.201	0.331
					hybrid+	50	67	0.427	0.980	0.595	171	171	0.500	0.687	0.579
					SST-PT+	45	32	0.584	0.882	0.703	98	57	0.632	0.394	0.485
					SVM+	33	19	0.635	0.647	0.641	47	35	0.573	0.189	0.284
					NB+	41	53	0.436	0.804	0.565	64	116	0.356	0.257	0.299
					DT+	32	7	0.821	0.627	0.711	43	11	0.796	0.173	0.284
					kNN+	39	61	0.390	0.765	0.517	59	118	0.333	0.237	0.277
5	28	107	157	984	ours	23	1	0.958	0.821	0.884	37	1	0.974	0.236	0.380
					hybrid+	28	44	0.389	1	0.560	114	105	0.521	0.726	0.607
					SST-PT+	25	16	0.610	0.893	0.725	66	31	0.680	0.420	0.519
					SVM+	21	14	0.600	0.750	0.667	29	22	0.569	0.185	0.279
					NB+	23	41	0.359	0.821	0.500	35	73	0.324	0.223	0.264
					DT+	18	3	0.857	0.643	0.735	26	4	0.867	0.166	0.279
					kNN+	24	38	0.387	0.857	0.533	35	68	0.340	0.223	0.269