Table 5. Intra-stimulus inter-observer consistency.
All values are Spearman correlations for 1 Hz modulations of the fraction of IOA (as, for example, depicted in figure 3) with respect to a particular emotion attribute across the entire duration of the movie. The specified range corresponds to the width of the 95% confidence interval of the mean correlation for all possible combinations of partitioning observers into two sub-groups (audio-visual 4 vs. 5; audio-only 1 vs. 2). Higher correlations indicate higher consistency of agreement modulations across observer sub-groups. Correlations higher than 0.5 (arbitrary threshold) are depicted in bold for visualization purposes. The all characters column indicates the agreement for a particular emotion attribute over time irrespective of the annotated character. The columns on the right show the corresponding correlations for the two main characters. n/a fields indicate an insufficient number of annotations to compute the consistency measure.
| All characters | Forrest-only | Jenny-only | ||||
|---|---|---|---|---|---|---|
| Audio-visual | Audio-only | Audio-visual | Audio-only | Audio-visual | Audio-only | |
|
Dimensional emotion attributes
Arousal Valence Direction |
0.676 ±0.006 0.851 ±0.004 0.555 ±0.011 |
0.305 ±0.078 0.564 ±0.044 0.068 ±0.050 |
0.600 ±0.008 0.784 ±0.006 0.543 ±0.012 |
0.443 ±0.078 0.440 ±0.054 -0.059 ±0.072 |
0.688 ±0.006 0.858 ±0.003 0.576 ±0.012 |
0.330 ±0.076 0.598 ±0.079 0.053 ±0.145 |
|
Emotion categories
Admiration Anger/Rage Compassion Contempt Disappoint. Fear Fears conf. Gloating Gratification Gratitude Happiness Happy-for Hate Hope Love Pride Relief Remorse Resentment Sadness Satisfaction Shame |
0.392 ±0.006 0.719 ±0.005 0.337 ±0.010 0.433 ±0.008 0.408 ±0.008 0.749 ±0.003 0.048 ±0.009 0.644 ±0.014 0.155 ±0.010 0.317 ±0.012 0.618 ±0.006 0.478 ±0.013 0.441 ±0.011 0.333 ±0.010 0.568 ±0.008 0.473 ±0.011 0.353 ±0.019 0.557 ±0.022 n/a 0.737 ±0.005 0.112 ±0.007 0.316 ±0.006 |
0.343 ±0.094 0.377 ±0.042 0.182 ±0.013 0.157 ±0.043 0.367 ±0.095 0.546 ±0.108 n/a 0.045 ±0.038 n/a n/a 0.289 ±0.107 n/a 0.097 ±0.070 0.217 ±0.060 0.396 ±0.032 0.156 ±0.054 -0.003 ±0.001 0.323 ±0.107 n/a 0.464 ±0.023 n/a 0.185 ±0.112 |
0.306 ±0.015 0.693 ±0.006 0.232 ±0.019 n/a 0.431 ±0.007 0.729 ±0.004 n/a n/a 0.242 ±0.017 0.267 ±0.024 0.561 ±0.007 0.408 ±0.028 0.478 ±0.027 0.202 ±0.008 0.472 ±0.008 0.536 ±0.009 0.505 ±0.028 n/a n/a 0.750 ±0.005 n/a 0.290 ±0.006 |
n/a 0.141 ±0.100 n/a n/a 0.456 ±0.129 0.504 ±0.072 n/a n/a n/a n/a 0.381 ±0.060 n/a n/a n/a 0.327 ±0.092 n/a n/a 0.453 ±0.364 n/a 0.522 ±0.006 n/a 0.034 ±0.036 |
n/a 0.809 ±0.007 0.459 ±0.018 0.404 ±0.020 n/a 0.633 ±0.005 n/a n/a n/a 0.165 ±0.015 0.738 ±0.005 n/a n/a 0.439 ±0.026 0.660 ±0.009 n/a n/a 0.464 ±0.006 n/a 0.695 ±0.009 n/a 0.201 ±0.015 |
0.643 ±0.197 0.626 ±0.109 0.363 ±0.042 n/a n/a 0.392 ±0.013 n/a n/a n/a n/a 0.532 ±0.108 n/a n/a 0.804 ±0.052 0.432 ±0.040 n/a n/a n/a n/a 0.228 ±0.040 n/a n/a |
|
Emotion onset cues
Audio Context Face Gesture Narrator Verbal |
0.418 ±0.008 0.384 ±0.010 0.713 ±0.006 0.619 ±0.010 n/a 0.637 ±0.008 |
0.037 ±0.051 n/a n/a n/a 0.251 ±0.047 0.368 ±0.107 |
0.354 ±0.013 0.424 ±0.011 0.687 ±0.010 0.532 ±0.011 n/a 0.565 ±0.007 |
n/a n/a n/a n/a 0.202 ±0.045 0.455 ±0.042 |
0.203 ±0.007 0.547 ±0.010 0.880 ±0.004 0.769 ±0.009 n/a 0.783 ±0.005 |
n/a n/a n/a n/a 0.578 ±0.286 0.626 ±0.168 |