TABLE 1:
Studies reporting cross-database AU detection results. D1→D2 denotes that models are trained on domain D1 and tested with domain D2. The column titled AU specific represents whether the study reports AU specific cross-domain performance (Yes) or average performance (No). Used evaluation metrics include 2AFC, AUC, F1, Classification Rate (CR), Recall (RC), Precision (PR), Hamming Loss, Average positive recognition rate (APRR), Average false-alarm rate (AFAR). Used databases include Cohn-Kanade (CK) [14], Extended Cohn-Kanade (CK+) [15], BP4D [16], UNBC Shoulder-Pain Archive (SP) [9], MMI [17], DISFA [8], SEMAINE (SEM) [18], SAL [19], GFT [7], RU-FACS [20], GEMEP-FERA (G-FERA) [21], ISL [22]. For more comprehensive review, see [23].
| Study | Databases | Number of subjects |
Number of sequences (s) / frames (f) |
Number of AUs |
AUs | AU specific |
Metrics |
|---|---|---|---|---|---|---|---|
| [24] | MMI→CK SAL→SEM |
MMI (10) SAL (10), |
MMI (264 s), CK+ (55 s) SAL (35 s), SEM (10 s) |
15 | Avg of 15 AUs (not specified) |
No | 2AFC, F1 CR, RC, PR |
| [25] | MMI→CK | MMI (15) | MMI (264 s) CK (143 s) |
18 | 1, 2, 4, 5, 6, 7, 9 10, 11, 12, 14, 15, 17 20, 24, 25, 27, 45 |
No | F1, CR, RC, PR |
| [26] | CK→G-FERA G-FERA→CK |
CK (>100) | CK (8000 f) G-FERA (5000 f) |
8 | 1, 2, 4, 6, 7, 12, 15, 17 |
No | F1 |
| [27] | CK+, G-FERA, SP, DISFA (Train on one, test on the rest) |
CK+ (123), G-FERA (7), SP (25), DISFA (27) |
CK+ (593 s, 593*4 f), G-FERA (87 s), SP (200 s) |
14 | 1, 2, 4, 5, 6, 7, 9, 10, 12, 15, 17, 20, 25, 26 |
No | AUC, F1 |
| [28], [29] | CK+→ISL | ISL (7) | ISL (7*19 s), CK+ (327 s, 327 * 2 f) |
13 | 1, 2, 4, 5, 6, 7, 9 12, 17, 23, 24, 25, 27 |
No | Hamming L., F1 |
| [30] | CK+→SEM | CK+ (210) | CK+ (593 s, 593 f) | 10 | 1, 2, 4, 5, 6, 7, 12, 17, 25, 26 |
No | F1 |
| [31] | CK→MMI | MMI (11), CK (>100) |
MMI (54 s) | 13 | 1, 2, 4, 5, 6, 7 9, 12, 15, 17, 23, 25, 27 |
Yes | APRR, AFAR |
| [32] | BP4D→CK+, BP4D→DISFA, DISFA→CK+, DISFA→BP4D |
CK+ (123), DISFA (27), BP4D (41) |
CK+ (582 s), DISFA (4845 f) |
10 | 1, 2, 4, 5, 6, 9 12, 15, 17, 20 |
Yes | ACC, 2AFC |
| [33] | CK+→SP | CK+ (123), SP (25) |
CK+ (593 s, 593 f), SP (48,398 f) |
6 | 4, 6, 7, 9, 10, 43 |
Yes | AUC |
| [34] | CK+→G-FERA, G-FERA→CK+ |
CK+ (123), G-FERA (10) |
CK+ (593 s CK+, 593 f) | 17 | 1, 2, 4, 5, 6, 7, 9 11, 12, 15, 17, 20 23, 24, 25, 26, 27 |
Yes | 2AFC |
| [35] | DISFA→G-FERA, G-FERA→DISFA |
DISFA (27), G-FERA (7) |
DISFA (32 s, 32*4000 f), G-FERA (87 s) |
8 | 1, 2, 4, 6, 12, 17, 25, 26 |
Yes | CR (Per seq) |
| [36] | BP4D→GFT GFT→BP4D |
BP4D (41), GFT (50) |
BP4D (328 s, 146,847 f), GFT (254,451 f) |
12 | 1, 2, 4, 6, 7, 10, 12, 14, 15 17, 23, 24 |
Yes | F1 |
| [37] | RU-FACS→G-FERA, GFT→RU-FACS |
RU-FACS (34), G-FERA (7), GFT (42) |
G-FERA (87 s), Ru-FACS (29 s, 180K f), GFT (~302K f) |
8 | 1, 2, 4, 6, 12, 14, 15, 17 |
Yes | AUC, F1 |
| [38] | MMI→CK, CK→MMI |
MMI (70) | MMI (244 s), CK (153 s) |
16 | 1, 2, 4, 5, 6, 7, 9, 10, 12, 15, 20, 24, 25, 26, 27, 45 |
Yes | F1 |
| [39] | SEM, BP4D, DISFA (Train on one, test on the rest) |
BP4D (41), SEM (31), DISFA (27) |
BP4D (150K f), SEM (93K f), DISFA (130K f), |
8 | 2, 12, 17 (all) 25 (DISFA→SEM), 1, 4, 6, 15 (DISFA→BP4D) |
Yes | F1 |
| [40] | BP4D→DISFA | BP4D (41) DISFA (27) |
BP4D (328 s, 328*300 f) DISFA (27 * 2400 f) |
8 | 1, 2, 4, 6, 9, 12, 25, 26 |
Yes | AUC, F1 |
| [41] | BP4D→DISFA DISFA→BP4D |
DISFA (27) BP4D (41) |
Varies between (10 – 500 f) |
7 | 1, 2, 4, 6, 12, 15, 17 |
Yes | AUC, F1 |