Depression severity is associated with impaired facial emotion processing in a large international sample

Abstract

Background:

Depression is associated with biases in facial emotion processing, which have an impact on the course and treatment of depression. While decades of research have established a negativity bias in processing in depression, there is still a gap in our understanding of how depression severity impacts sensitivity to detecting differences in emotional faces.

Methods:

We examined emotion sensitivity (ES), or the ability to to detect subtle differences in emotional faces, in a large, geographically and culturally diverse, web-based sample (N = 6,598, age range = 18 – 96, 56.50% female, 66% Caucasian). Participants completed ES tasks (fear, anger, or happiness) and a Beck Depression Inventory-II, to determine depression severity.

Results:

Depression severity was correlated with overall ES performance as well as ES performance for individual emotions. Higher depression scores were associated with poorer performance in detecting happiness, fear, and anger (ps < .001). Examining performance by region, Eastern countries showed significantly poorer ES performance compared to Western countries, and were significantly more depressed.

Limitations:

Our sample is non-clinical and self-selected.

Conclusions:

This study is an extension of existing research on emotional facial processing, with an approach that takes into consideration the heterogeneity of depression symptoms and corrects psychometric confounds of traditional emotion face processing paradigms. Overall, factors related to severity, task reliability, and facial stimuli should be considered in determining the potential mechanism of facial emotion processing in the onset and course of depression.

Investigating the biased processing of emotional faces is essential to our understanding of internalizing disorders, particularly given the evidence for depression-related biases in processing positive emotions. Negative information processing biases are thought to have a central causal role in the development of symptoms of depression (e.g., Roiser et al, 2012). In general, individuals with depression show preferential processing of negative material and impaired processing of positive materials (see Clark et al., 2009 for a review). However, while negativity processing biases are well established in individuals with depression, it is unclear how exactly this processing is impacted by depression status and severity. Furthermore, emotion processing researchers use different facial stimuli and task paradigms, many of which conflate discriminability/sensitivity and response bias. Sensitivity and bias are established concepts in psychophysics and signal detection theory, where the sensitivity of an observer (i.e., discrimination ability) is distinct from response bias (i.e., response criterion) (Macmillan & Creelman, 2004). In standard facial emotion recognition paradigms, multiple emotion categories are judged together, making sensitivity and bias impossible to disentangle. Thus, as we review the emotion processing literature, these limitations should be noted.

In an early study of emotion sensitivity (ES) and facial emotion discrimination, patients with depression and controls matched for sex, age and sociodemographic data showed distinct differences in detecting emotional faces (Gur et al., 1992). Patients with depression had a harder time detecting happiness and judged more non-sad faces as sad expressions compared to controls. In another study of facial emotion recognition in unipolar depression, researchers found that patients with depression showed higher accuracy and response bias for labeling sad faces, but not faces displaying other emotions (Milders et al., 2010). Of note, labeling performance for sad expressions was not associated with symptom severity or with changes in severity over time, suggesting a stable bias for sad expressions. In a more recent study of 19 individuals with depression examining emotion categorization in depression compared to controls, depressed individuals showed greater brain activation while processing genuine versus posed facial expression of sadness, with no differences in happy facial displays (Groves et al., 2018). These authors also note their small sample size and did not correct for multiple comparisons. The small sample sizes of both of these studies limits their generalizability.

Gender and age also play a role in perception of emotion, especially in depression. With regard to gender, in a study of 60 inpatients with major depressive disorder, researchers showed that the propensity to perceive negative facial expressions may underlie the unfavorable course of depression among women specifically (Bouhuys et al., 1999). Women with depression who were inclined to perceive high levels of negative emotion in facial expressions also reported high levels of social distress, and were less improved overall after six weeks. Indeed, emotion processing may be a gender-specific mechanism that is associated with the outcomes of depression (Bouhuys et al., 1999). Additional research has provided somewhat contradictory evidence to this point, with results showing an interaction between gender and depression status in emotion processing, such that women with major depression make more errors, particularly for sad and fearful stimuli, than non-depressed women and men with major depression (Wright et al., 2009). Despite some differences between emotion categories, in general, females show enhanced facial emotion recognition abilities compared to males (Hall, 1978, Herlitz & Loven, 2009; Kret & DeGelder, 2012; Thompson & Voyer, 2014; Wingenbach et al., 2018). Additionally, age plays a role in facial emotion recognition abilities, and while facial emotion recognition abilities decrease with age (e.g., Mill et al., 2009), depression severity also decreases with age (e.g., Mezuk & Kendler, 2012). In a study of facial emotion perception in adults with anxiety and mood disorders, increased age was associated with decreased accuracy overall and slower response times in recognizing sad and fearful facial expressions (Rutter et al., 2019).

There are several limitations to the existing literature including methodological problems (e.g., Macmillan & Creelman, 2004) and conflicting results (see Ko, 2018 for a review), as well as concerns with achieving adequate power and replicability (e.g., Asendorpf et al., 2013). In short, methodological problems are based in the idea that many tasks of emotion perception conflate discriminability or sensitivity and response bias (see Rutter et al.,2019). Sensitivity and bias are established concepts in signal detection theory and psychophysics, where the sensitivity of an observer (i.e., discrimination ability) is distinct from response bias (i.e., response criterion; Macmillan & Creelman, 2004). Further limitations of existing emotional facial processing tasks are based around the issue that in traditional paradigms, emotion categories are not comparably difficult, and judgments for happiness suffer from ceiling effects relative to other emotion categories. While we have previously addressed these issues in a large sample assessing emotion sensitivity (ES) and generalized anxiety symptoms (Rutter et al., 2019), we have not yet examined the connection between depression severity and ES. Here, we expand our investigation of ES and internalizing symptomatology by using a large, diverse sample assessed via the web-based laboratory TestMyBrain.org (see Germine et al., 2012), which allowed for rapid recruitment of large samples. Previous comparisons of emotion perception tests administered on the web vs. in the lab have shown that data are comparable in quality (Germine et al., 2012; Hartshorne & Germine, 2015; Meyerson & Tryon, 2003; Rutter et al., 2019).

Thus, we aimed to resolve the inconsistencies in the literature in several ways: using a more highly powered and diverse sample than previously studied, controlling for the potentially confounding effects of age, and using a method that targets emotion sensitivity (ES), a specific aspect of emotion perception. While there are many ways to study detection of emotional expressions in faces, we use this particular task to study emotion because it is a response bias-free method to obtain sensitivity scores, using two-alternative forced choice intensity judgments for individual emotions. This allowed us to experimentally isolate ES for each emotion (happiness, anger, fear). Alternative tests of sensitivity to emotion are not response bias-free, and produce results with ceiling effects for positive emotions (happiness) that can impact results, leading to misinterpretation (see McKone et al., 2012 for a review). Given the general tendency towards smaller sample sizes and methodological limitations of prior research, we expected that our experimental design and well-powered sample could discover previously unobservable associations between depression symptoms and ES. We had two primary hypotheses: (1) higher depression scores will predict reduced sensitivity to all emotions (happiness, fear, anger) and (2) this effect will be true across the lifespan. Additionally, because we had a geographically diverse dataset and there is reason to believe that emotion sensitivity is not the same across all cultures (e.g., Dailey et al., 2010), we also aimed to explore cross-cultural effects of depression severity on ES.

Method

Participants

Participants were 6,598 visitors to TestMyBrain.org who completed three ES tasks and a modified version of the BDI-II. The TestMyBrain.org platform was approved by the Harvard Committee on the Use of Human Subjects. Participants provided electronic consent to be a part of the study. After completing the task, participants were given immediate feedback about their performance on the ES task relative to other individuals who had completed the same measures.

Participants’ ages ranged from 18 to 96 years old, and the average age was 33.80 (SD = 14.32). The sample was predominantly female (female = 56.50%; male = 42.00%; genderqueer = 0.73%; missing = 0.77%). The majority of participants identified as Caucasian (66.05%) from English speaking countries. The most common educational statuses reported were as follows: high school (13.62%), some college (26.95%), college (24.54%), and graduate school (21.42%). Data were obtained from January 2018 to June 2018.

Measures

Belmont Emotion Sensitivity Test (BEST; Rutter et al., 2019)

The Karolinska Directed Emotional Faces (KDEFS) database (Lundqvist, Flykt, & Öhman, 1998) was used for face stimuli. Faces were morphed between any two of angry, happy, and fear faces, across a set of over two dozen identities, creating three morph continua per identity. Anger, fear, and happiness sensitivity were assessed across three separate subtests, each showing 56 pairs of faces, one pair at a time, with the two faces in a pair presented on screen at the same time, for 1000 milliseconds. Participants were asked to indicate, “Which face is more happy?” “Which face is more angry?” or “Which face is more fearful?” to evaluate happiness ES, anger ES, and fear ES, respectively. Trials were ordered so that difficulty increased across three blocks (easy trials = 8, medium trials = 20, and hard trials = 28) for each subtest.

Participants were excluded from analyses if they had less than 50% accuracy (chance performance) or had mean reaction times less than 200 milliseconds. For all emotion categories (i.e., happiness, fear, anger), lower ES scores are indicative of poorer performance, or lower sensitivity in recognizing a particular emotion. See Rutter et al., 2019 for more details on creation and psychometrics of the BEST.

Beck Depression Inventory-II (BDI; Beck, Steer, & Brown, 1996)

The BDI-II is a widely used 21-item self-report measure of severity of depressive symptoms that are consistent with the diagnostic criteria of major depressive disorders in the DSM. Due to concerns about risk and reporting in our online sample, the question about suicidality was omitted from the questionnaire. Thus, total scores range from 0 to 60. The BDI-II has been shown to have strong psychometric properties in outpatient samples (Steer, Ball, Ranieri, & Beck, 1997), and shows high reliability and the ability to discriminate between depressed and non-depressed subjects (Wang & Gorenstein, 2013a).

Data Analysis

Data were analyzed in R. Results are considered statistically significant at p < .05. First, we confirmed that ES and BDI-II both varied with age and gender in the expected directions, based on previous research. Second, we tested our specific hypotheses: (1) that ES would decline with increased depression severity and (2) that this effect would hold true across the lifespan. Finally, we conducted follow-up analyses to examine cultural effects. For each major analysis, we used a conservative Bonferroni adjustment (p < 0.017) to correct for multiple comparisons (3 emotion categories). For clarity, all analyses are reported with uncorrected p-values. However, results that did not pass the corrected threshold are indicated and interpretations are based on results that are significant after Bonferroni correction.

Results

The average BDI-II score in our sample was 15.57 (SD = 11.58, range 0–57), indicating a mild level of depression. ES scores for each emotion category were calculated based on total correct out of 56 face pairs. In our sample, the average happiness score was 46.99 (SD = 4.03), average fear score was 44.69 (SD = 6.30), and average anger score was 47.80 (SD = 4.91). Based on Welch’s t-tests, ES scores significantly differed by category with anger accuracy significantly better than fear (t = 31.58, p < .001; d = .55) and happiness (t = 10.34, p < .001; d = .18), and happiness significantly better than fear (t = 24.94, p < .001; d = .43). Participants were most sensitive to detecting anger in faces, relative to fear and happiness. As previous research has shown (e.g., Wingenbach et al., 2018), we replicated that finding that females show superior ES performance across most emotion categories compared to males (anger: t = 1.98, p < .05, d = .05; fear: t= 6.92, p < .001, d = .17), with the largest effect for fear, and no significant difference for happiness (t = 1.51, p = .13). In our sample, females reported higher levels of depression than males (t = 4.0, p < .001; d = .10) across the lifespan, see Supplemental Materials Figure 1.

We conducted a series of regressions to determine the relationship between depression severity and ES scores. First, we confirmed that the relationship between depression scores and age, and ES scores and age were in the expected direction. As expected, there was a significant relationship between age and depression severity, B = −.30, R² = .06, F (1, 6596) = 419.50, p < .001, with depression scores declining across the lifespan. Next, we conducted regressions to test the effect of age on ES. The relationship between age and ES was significant for all emotions (fear, B = −.29, R² = .02, F (1, 6596) = 109.60, p < .001; anger, B = −.37, R² = .02, F (1, 6596) = 110.60, p < .001; and happiness, B = −.32, R² = .01, F (1, 6596) = 54.08, p < .001). All tests survived correction for multiple comparisons; however, effect sizes were small. These results indicate that as people age, their sensitivity to emotional faces (anger, fear, happiness) generally decreases, see Supplemental Materials Figures 2–4.

In our remaining analyses we used age, age², and gender as covariates in our regressions of depression severity on ES. As hypothesized, higher depression scores were associated with reduced sensitivity to detecting all emotions measured (happiness: β = −.08, R² = .02, F (1, 6494) = 31.90, p < .001; fear: β = −.13, R² = .04, F (1, 6494) = 72.68, p < .001; anger: β = −.14, R² = .05, F (1, 6494) = 78.86, p < .001). All tests survived correction for multiple comparisons. Effect sizes were relatively small, with the largest effect for anger, and smallest for happiness. The significant effect for happiness indicates that the effect of depression severity on ES is not just for negative facial expressions. See Figure 1. After conducting these initial regression tests, we applied a logit transformation on ES scores to account for the skewed scale, with most participants scoring relatively high and similarly across the ES categories. Logit transformation allowed us to examine meaningful differences near 0% and 100% accuracy scores (see Jaeger, 2008). Results remained largely unchanged, with no differences in significance levels or overall effect sizes (happiness: β = −.07, R² = .02, F (1, 6488) = 28.59, p < .001; fear: β = −.13, R² = .04, F (1, 6464) = 81.88, p < .001; anger: β = −.13, R² = .04, F (1, 6448) = 71.53, p < .001). Effect sizes were small.

Figure 1.

The effects of depression severity on emotion sensitivity performance. Note. Depression severity is based on adapted Beck Depression Inventory (BDI-II) score.

Finally, we wanted to explore if depression severity and ES differed by region. We split our sample based on participants from 46 Western countries (USA, Canada, European nations) that are predominantly Caucasian (n = 4769) and compared this to participants from 36 Eastern countries, predominantly in Asia (n = 1099). Results indicated that participants from Western countries had significantly lower depression scores than Eastern countries (t = −9.10, p < .001; d = .31), and performed significantly better on ES tests across emotions (happiness: t = 3.24, p < .01; d = .11; anger: t = 10.38, p < .001, d = .35; fear: t = 16.77, p < .001, d = .59), with the largest effect size for fear, and smallest for happiness. We also tested if the impact of depression severity on ES differed by region by regressing BDI score*region onto each ES scores for each emotion. We found significant effects for each emotion, with the largest effect size for fear (happiness: β = −.06, R² = .01, F (3, 5864) = 10.64, p < .001; fear: β = −.08, R² = .06, F (3, 5864) = 124.00, p < .001; anger: β = −.06, R² = .03, F (3, 5864) = 50.45, p < .001). All tests survived correction for multiple comparisons.

Discussion

To our knowledge, this was the first study to examine the impact of depression severity on emotion sensitivity in a large, international, web-based sample. While there is a plethora of existing research on emotional facial processing, our approach here is different because we used a psychometrically improved measure of emotional faces that distinguishes discriminability/sensitivity and response bias, the BEST (Rutter & Germine, 2019). Additionally, we analyzed depression severity as a dimensional variable rather than using cut scores or the presence or absence of a major depressive disorder (MDD) diagnosis to dictate our analyses. This approach also acknowledges the heterogeneity of symptoms in depression, allowing us to examine the effect of symptoms even in participants who do not meet criteria for MDD. More importantly, our approach is an expansion of the current literature based on our large sample size.

Results provided support of both of our study hypotheses, after correcting for multiple comparisons. Depression scores and ES both decreased across the lifespan, providing additional support for the prior findings that depression decreases as people age (e.g., Mezuk & Kendler, 2012) and emotion detection in faces decreases across the lifespan (Ruffmann et al., 2008; Isaacowitz et al., 2007; Rutter et al., 2019), especially negative emotions. We also replicated our prior findings that females were better than males at detecting fear, with no significant differences for happiness or anger sensitivity after Bonferroni correction.

With respect to participants’ countries, we showed that there were differences in both depression scores, and ES scores when we split our sample into Eastern and Western regions. We based this comparison on the previous finding that recognition of emotional faces is less accurate when the emotions are expressed by individuals from a cultural group different from one’s own, and there are cultural differences in facial emotion recognition abilities (Dailey et al., 2010, Jack et al., 2009; Reyes et al., 2018). In our study, all participants viewed the same set of Caucasian faces from the KDEFS database, so we could not compare differences based on the race of the facial stimuli, which is a limitation. The result that Eastern countries showed significantly higher depression scores is inconsistent with prior findings that prevalence rates of depression and disease burden are higher in North America than South-East Asian regions (Ferrari et al., 2013; World Health Organization, 2017). The differences in depression severity shown in our findings may be due to response bias on the Beck Depression Inventory. However, given that the BDI-II has been adapted in many countries and languages, the information on comparability across cultures is still remarkably scarce (Wang & Gorenstein, 2013b). Some research has shown that in college students, Chinese-heritage students score higher than European-heritage students on the BDI-II (Dere et al., 2015), while other work has highlighted the specific differential item functioning across cultures (Bryne & Campbell, 1999), with additional results demonstrating that complete measurement invariance cannot be assumed across five European countries (UK, Ireland, Spain, Norway, and Finland) (Nuevo et al., 2009). There is reason to believe that both the BDI and the ES task we used may not be psychometrically equivalent cross-culturally, which is a suggested direction for future research.

Overall, these results replicate most prior findings, indicating the importance of considering the heterogeneity and range of depressive symptoms in making determinations about emotion perception and its role in depression. It could be that a higher level of depression symptoms is related to multiple cognitive and emotional impairments (e.g., Salthouse, 1996) or specific impairments in facial processing (Sullivan & Ruffman, 2004). It could also be the case that impaired facial processing is a risk factor for depression, or in predicting response to depression treatments (Shiroma et al., 2014). Our cross-sectional design did not allow for this analysis, but future research should address this question. Additionally, future work should consider the impact of age on both depression severity and facial processing abilities, given our findings that both decrease across the lifespan.

Our study had many strengths including the large and diverse sample, which makes our results potentially more generalizable compared to prior work. Additionally, our measurement of ES represents an expansion and improvement of the traditional emotion recognition paradigms. Another strength of our study is that we did not exclude patients with particularly high or low depression scores; instead, we included the full range of depression symptoms as captured by the BDI-II. Moreover, existing studies that examine the role of depression in interpreting faces have often done so by comparing participants with MDD to non-depressed controls, not accounting for variation in symptoms and severity within both groups. Rather than classifying participants based on a symptom cutoff or a previous diagnosis of MDD, we quantify symptom severity for all participants, including those who do not meet standard diagnostic criteria for MDD. This permits us to examine the role of individual differences in symptom severity both above and below the MDD diagnostic threshold, allowing us to more completely understand the relationship between depression symptoms and emotion processing across the severity spectrum.

This study had some notable limitations, including the non-clinical and self-selected nature of our sample. Although our sample was comprised of the general public by design, in order to draw more meaningful conclusions about depression severity and ES, a larger range of depression severity and a clinical diagnosis of depression should be considered. Additionally, our self-selected sample may have captured individuals who had higher or lower ES abilities than the general population. In other words, participants interested in testing their strengths and weaknesses in facial processing may have been particularly drawn to our study. Lastly, we did not include stimuli of sad faces, which may be particularly relevant for depression. The BEST has only been validated with happy, angry, and fearful face stimuli (Rutter et al., 2019), but future projects will include sad faces, once tests are created and tested.

Results of this study support the notion that sensitivity to emotional faces is related to depression severity such that more severe depression limits the ability to detect subtle differences in emotional faces. Factors related to severity, comorbidity, and task reliability should be considered in future research. Additionally the facial stimuli themselves should be evaluated for evidence of suitability across cultures and modes of assessment (laptop, tablet, phone). Ultimately, to determine the mechanism of facial emotion processing in the onset, course, and recovery from depression, more research using longitudinal designs and multiple modes of assessment is required.

Highlights:

• Higher depression scores are related to lower emotion sensitivity to faces.

• Depression severity is most related to detecting anger, after controlling for age and gender.

• Depression severity and emotion sensitivity both decline across the lifespan.