Table 5.

Performance of deep-learning methods with different language representation models.

Model	Level-0				Level-1				Level-2				Level-3				MicroF1
	Pa	Rb	F1	P		R	F1	P		R	F1	P		R	F1
CNNc [16]	0.908	0.940	0.924	0.380		0.236	0.291	0.351		0.415	0.380	0.250		0.231	0.240	0.841
LSTMd [16]	0.896	0.936	0.916	0.294		0.288	0.257	0.324		0.262	0.289	0.714		0.192	0.303	0.832
BERTe [16]	0.942	0.894	0.917	0.323		0.502	0.393	0.468		0.489	0.478	0.574		0.152	0.240	0.834
BERT_IDPf [16]	0.929	0.938	0.934 g	0.394		0.446	0.418	0.568		0.385	0.459	0.667		0.231	0.343	0.856
RoBERTah	0.931	0.920	0.925	0.355		0.464	0.402	0.556		0.385	0.455	0.600		0.231	0.333	0.843
RoBERTa_IDP	0.933	0.920	0.926	0.371		0.489	0.422	0.578		0.400	0.473	0.636		0.269	0.333	0.847
XLNETi	0.908	0.948	0.927	0.358		0.273	0.309	0.484		0.353	0.408	0.530		0.384	0.445	0.848
XLNET_IDP	0.933	0.920	0.926	0.361		0.471	0.409	0.577		0.431	0.493	0.625		0.192	0.294	0.846

^aP: precision.

^bR: recall.

^cCNN: convolutional neural network.

^dLSTM: long short-term memory network.

^eBERT: bidirectional encoder representations from transformers.

^f_IDP: The model is further trained on the in-domain unlabeled corpus.

^gHighest F1 values are indicated in italics.

^hRoBERTa: robustly optimized bidirectional encoder representations from transformers pretraining approach.

ⁱXLNET: generalized autoregressive pretraining for language understanding.