Table 4.
Multi-modal data features and their uses in mental health
| Features | Example(s) | Relevant in mental disorder(s) | Study |
|---|---|---|---|
| Acoustic | |||
| Source of sound features | jitter | increase with depression severity | Cummins et al.188 |
| Filtering features by vocal and nasal tracks | first resonant peak in the spectrum | increase with bipolar severity | Weiner et al.173 |
| Spectral features of speech | Mel-frequency cepstral coefficients | a variety of disorders | Low et al.186 |
| Prosodic features of speech | pause duration | higher in schizophrenia | Low et al.186 |
| Video | |||
| Facial | smile duration, eyebrow movement, disgust expression | increased disgust expression in suicidal ideation | Galatzer-Levy et al.189 |
| Eyes | gaze angle | more non-mutual gazes in MDD | Song et al.190 |
| Gait | arm swing and stride | reduced arm swing in MDD | Shao et al.169 |
| Posture | head pitch variance, upper body movements | reduced head movement in schizophrenia | Abbas et al.191 |
| higher head movement in ASD | de Belen et al.192 | ||
| Language | |||
| Grandiosity | unrealistic sense of superiority | increased in bipolar | Castro et al.193 |
| Semantic coherence | flow of meaning | decreased in psychosis | Morgan et al.194 |
| Rumination | repetitive thought patterns | increased in MDD | Rutowski et al.195 |
| Self-focus | self-referent information | increased in stress | Kim et al.196 |