Fig. 4.

The illustration of the multi-head self-attention mechanism. The multi-head self-attention mechanism operates on each amino acid node. $m_i\left(q_i,,,k_i,,,v_i\right)$ represents the query vector, key vector and value vector of amino acid node $m_i$. $m_i\left(q_{i,h},,,k_{i,h},,,v_{i,j}\right)$ represents the query vector, key vector, and value vector of the hth type of amino acid node $m_i$. This symbol $\oplus$ indicates addition by element-wise