Table 5:
Hyperparameters for T5 DAPT
| Hyper-parameter | Setting |
|---|---|
| Optimizer | AdamW |
| Epoch | 10 (with early stopping) |
| Learning rate | 1e-3, 1e-4 |
| Batch size | 256 |
| Gradient accumulation | True |
Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.