Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
Training details for RadBERT-CL Pretraining and Finetuning Stages.
| Hyperparameter | Pretraining | Finetuning |
|---|---|---|
| batch-size | 128 | 32 |
| learning-rate | 0.1 | 2e-5 |
| optimizer | SGD | Adam |
| temperature (CL) | 0.4 | - |
| n_epochs | 100 | 10 |
| beta | - | [0.9, 0.99] |
| Aug. Probability | 0.2 | - |
