Table 3:
Results for multimodal emotions analysis on IEMOCAP with aligned and non-aligned multimodal sequences.
| Task | Happy | Sad | Angry | Neutral | ||||
|---|---|---|---|---|---|---|---|---|
| Metric | Acch | F1h | Acch | F1h | Acch | F1h | Acch | F1h |
| (Word Aligned) IEMOCAP Emotions | ||||||||
| EF-LSTM | 86.0 | 84.2 | 80.2 | 80.5 | 85.2 | 84.5 | 67.8 | 67.1 |
| LF-LSTM | 85.1 | 86.3 | 78.9 | 81.7 | 84.7 | 83.0 | 67.1 | 67.6 |
| RMFN (Liang et al., 2018) | 87.5 | 85.8 | 83.8 | 82.9 | 85.1 | 84.6 | 69.5 | 69.1 |
| MFM (Tsai etal.,2019) | 90.2 | 85.8 | 88.4 | 86.1 | 87.5 | 86.7 | 72.1 | 68.1 |
| RAVEN (Wang et al.,2019) | 87.3 | 85.8 | 83.4 | 83.1 | 87.3 | 86.7 | 69.7 | 69.3 |
| MCTN (Pham et al.,2019) | 84.9 | 83.1 | 80.5 | 79.6 | 79.7 | 80.4 | 62.3 | 57.0 |
| MulT (ours) | 90.7 | 88.6 | 86.7 | 86.0 | 87.4 | 87.0 | 72.4 | 70.7 |
| (Unaligned) IEMOCAP Emotions | ||||||||
| CTC (Graves et al., 2006) + EF-LSTM | 76.2 | 75.7 | 70.2 | 70.5 | 72.7 | 67.1 | 58.1 | 57.4 |
| LF-LSTM | 72.5 | 71.8 | 72.9 | 70.4 | 68.6 | 67.9 | 59.6 | 56.2 |
| CTC + RAVEN (Wang et al., 2019) | 77.0 | 76.8 | 67.6 | 65.6 | 65.0 | 64.1 | 62.0 | 59.5 |
| CTC + MCTN (Pham et al., 2019) | 80.5 | 77.5 | 72.0 | 71.7 | 64.9 | 65.6 | 49.4 | 49.3 |
| MulT (ours) | 84.8 | 81.9 | 77.7 | 74.1 | 73.9 | 70.2 | 62.5 | 59.l |