Table 4.

BERT model configuration.

Parameter Name	Details	Parameter Name	Details
Maximum position embeddings	5000	Number of hidden layers	2
Number of attention heads	2	Hidden size	768
Training ratio	80	Testing ratio	20
Freeze_bert	False	epsilon value	0.00000001
Learning Rate	0.00005	optimizer	Adam