Table 1.
Model | Parameters | Hidden Size | Intermediate Size | Attention Heads | Layers |
---|---|---|---|---|---|
antiBERTy | 25.76M | 512 | 2048 | 8 | 8 |
antiBERTa2 | 202.64M | 1024 | 4096 | 16 | 16 |
BALM-paired | 303.92M | 1024 | 4096 | 16 | 24 |
ft-ESM2 | 652.36M | 1280 | 5120 | 20 | 33 |
Model | Parameters | Hidden Size | Intermediate Size | Attention Heads | Layers |
---|---|---|---|---|---|
antiBERTy | 25.76M | 512 | 2048 | 8 | 8 |
antiBERTa2 | 202.64M | 1024 | 4096 | 16 | 16 |
BALM-paired | 303.92M | 1024 | 4096 | 16 | 24 |
ft-ESM2 | 652.36M | 1280 | 5120 | 20 | 33 |