Fig. 2.

Transformer attention block. The diagram illustrates how the input embeddings are passed to multiple attention heads, each performing a series of operations to generate queries, keys, and values, and leading ultimately to contextualized embeddings