Table 7.

Model settings B.E.R.T. optimization

Parameter	Value
Model	GoogleBERT
Feature	Transformer Word Embeddings
Stop Words	Partially Removed
Batch size	32
Learning Rate	2e− 5
Max Sequence Length	160
Warm up	0.1
Tune Cell	0