Table 2.

Hyperparameters Comparison of Model

Model	Bert-Base-Uncased	DistilBERT
max_seq_length	384	384
doc_stride	128	128
max_query_length	64	64
train_batch_size	128	256
max_answer_length	30	30
Learning Rate	1e-8	1e-8