Table 7. Hyperparameters employed for Bert, Roberta and Electra.

Task	Model	Learning rate	Batch-size	Epochs	MSL
HOTEL REVIEW	BERT base	3e − 5	10	5	512
	RoBERTa base
	ELECTRA base
MOVIE REVIEW	BERT base	3e − 5	10	5	512
	RoBERTa base
	ELECTRA base
SENTIMENT140	BERT base	3e − 5, 2e − 5, 1e − 5	10	5	512
	RoBERTa base
	ELECTRA base
CITATION SENTIMENT CORPUS(CSC)	BERT base	1e − 5	5	5	512
	RoBERTa base
	ELECTRA base		8	8
BIOINFORMATICS CITATION CORPUS (BCC)	BERT base	2e − 5	9	9	512
	RoBERTa base	1e − 5	6	6
	ELECTRA base	3e − 5	8	8