Official websites use .gov
A
.gov website belongs to an official
government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you've safely
connected to the .gov website. Share sensitive
information only on official, secure websites.
Structure of the transformer model.
| Model Layer | Input Vector | Output Vector | |
|---|---|---|---|
| Input layer | 1360 × 1 | 1360 × 1 | |
| Token and position embedding | 1360 × 1 | 1360 × 512 | |
| Transformer block | Multihead attention (head = 4) | 1360 × 512 | 1360 × 512 |
| Feed forward network | 1360 × 512 | 1360 × 512 | |
| Gloval average pooling | 1360 × 512 | 512 × 1 | |
| Dense | 512 × 1 | 4 × 1 | |
