Table 4.

Overall performance on multilevel event extraction compared with the state-of-the-art methods with gold standard entities.

Method	Trigger recognition (%)				Event extraction (%)
	Precision	Recall	F1 score	Precision		Recall	F1 score
EventMinea	70.79	81.69	75.84	62.28		49.56	55.20
SSLa,b	72.17	82.26	76.89	55.76		59.16	57.41
CNNa,c	80.92	75.23	77.97	60.56		56.23	58.31
mdBLSTMa,d	82.79	76.56	79.55	90.24		44.50	59.61
RLe+KBsa,f	N/Ag	N/A	N/A	63.78		56.81	60.09
DeepEventMineh	N/A	N/A	N/A	69.91		55.49	61.87
HANNh,i	N/A	N/A	N/A	63.91		56.08	59.74
Our modelh	82.20	78.25	80.18	72.26		55.23	62.80j

^aPipeline model.

^bSSL: semisupervised learning.

^cCNN: convolutional neural network.

^dmdBLSTM: bidirectional long short-term memory with a multilevel attention mechanism and dependency-based word embeddings

^eRL: reinforcement learning.

^fKB: knowledge base

^gN/A: not applicable.

^hJoint model.

ⁱHANN: hierarchical artificial neural network.

^jThe best value compared with baselines.