Algorithm 1. The pseudocode of the model.

Input: Time-frequency characteristics $X_1,X_2,\dots ,X_T$

Output: Emotion categories and probabilities Y, P

// Bi-GRU algorithm (forward and backward) $\overrightarrow{h_t}$←$\overrightarrow{GRU}\left(x_t,h_{t-1}\right)$

$\stackrel{\leftharpoonup }{h_t}$→$\stackrel{\leftharpoonup }{GRU}\left(x_t,{\tilde{h}}_{t-1}\right)$ $h_t=[\overrightarrow{h_t},\stackrel{\leftharpoonup }{h_t}]$ // Multi-head Attention algorithm

$Q=h_t\cdot W_q$, $K=h_t\cdot W_k$, $V=h_t\cdot W_v$ $AttentionScor{e}_i=softmax\left(\frac{Q{W}_{Q_i}^T{K}^T}{\sqrt{d_k}}\right)$ $hea{d}_i=AttentionScor{e}_i\cdot V$ $MultiHead\left(Q,K,V\right)=Concat\left(hea{d}_1,\dots ,hea{d}_h\right)W^O$

// Linear layer and output layer $Y^{\prime }=full connect\left(MultiHead\left(Q,K,V\right)\right)$ $Y=softmax(Y^{\prime })$