Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
T5 pretrained models used in the experiments.1
| Model | Training Corpus | Initialization | Citation |
|---|---|---|---|
| T5 220M | Colossal Clean Crawled Corpus (C4) | Random | (Raffel et al., 2020) |
| T5 770M | Random | ||
| SciFive 220M | C4 + PubMed (abstracts) + PMC | T5 220M | (Phan et al., 2021) |
| SciFive 770M | T5 770M | ||
| Clinical-T5 220M | MIMIC-III + MIMIC-IV | T5 220M | (Lehman and Johnson, 2023) |
| Clinical-T5 770M | Random |
