Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Apr 13.
Published in final edited form as: Biol Psychiatry. 2009 Nov 22;67(5):414–421. doi: 10.1016/j.biopsych.2009.09.027

Elevated Amygdala Activity to Sad Facial Expressions: A State Marker of Bipolar but Not Unipolar Depression

Jorge RC Almeida 1, Amelia Versace 1, Stefanie Hassel 1, David J Kupfer 1, Mary L Phillips 1
PMCID: PMC2854029  NIHMSID: NIHMS178898  PMID: 19931855

Abstract

Background

Difficulties in emotion processing and poor social function are common to bipolar disorder (BD) and major depressive disorder (MDD) depression, resulting in many BD depressed individuals being misdiagnosed with MDD. The amygdala is a key region implicated in processing emotionally salient stimuli, including emotional facial expressions. It is unclear, however, whether abnormal amygdala activity during positive and negative emotion processing represents a persistent marker of BD regardless of illness phase or a state marker of depression common or specific to BD and MDD depression.

Methods

Sixty adults were recruited: 15 depressed with BD type 1 (BDd), 15 depressed with recurrent MDD, 15 with BD in remission (BDr), diagnosed with DSM-IV and Structured Clinical Interview for DSM-IV Research Version criteria; and 15 healthy control subjects (HC). Groups were age- and gender ratio-matched; patient groups were matched for age of illness onset and illness duration; depressed groups were matched for depression severity. The BDd were taking more psychotropic medication than other patient groups. All individuals participated in three separate 3T neuroimaging event-related experiments, where they viewed mild and intense emotional and neutral faces of fear, happiness, or sadness from a standardized series.

Results

The BDd—relative to HC, BDr, and MDD—showed elevated left amygdala activity to mild and neutral facial expressions in the sad (p < .009) but not other emotion experiments that was not associated with medication. There were no other significant between-group differences in amygdala activity.

Conclusions

Abnormally elevated left amygdala activity to mild sad and neutral faces might be a depression-specific marker in BD but not MDD, suggesting different pathophysiologic processes for BD versus MDD depression.

Keywords: Amygdala, bipolar disorder, emotional processing, fMRI, major depressive disorder, mood disorders


There is increasing recognition of the need to identify pathophysiologic processes underlying key behavioral abnormalities in psychiatric disorders as a first step toward improving diagnostic accuracy and novel treatment developments for these disorders (1). This is particularly relevant to the study of bipolar disorder (BD) depression, where difficulties in emotion processing and poor social function are common to major depressive disorder (MDD) depression, resulting in many BD depressed adults being misdiagnosed with MDD, leading to suboptimal treatment and poor outcome (2). A large number of studies highlight the amygdala as a key neural region in the processing of emotionally salient stimuli (35). Neuroimaging studies have therefore begun to focus on examination of amygdala activity in BD and MDD depression (68).

There are several questions that remain unanswered, however, regarding amygdala dysfunction in BD and MDD depression. First, it is unclear whether abnormal amygdala activity to emotional facial expressions is a persistent marker of BD during both remission and depression or whether abnormal amygdala activity to these stimuli is a state marker of depression, common to or differentiating BD and MDD depression. Earlier studies suggested elevated amygdala activity to happy and fearful facial expressions in remitted, depressed, and hypomanic BD adults (9). Later studies did not, however, demonstrate abnormal amygdala activity to either negative (fearful, disgust, sad) or positive (happy) emotional facial expressions in remitted or stabilized BD adults (BDr) (1013). Similarly, recent studies reported no abnormality in amygdala activity to fearful, angry, or sad facial expressions in depressed BD adults (BDd) (14,15). In depressed adults with MDD, studies reported abnormally elevated amygdala activity to masked, covertly presented fear, happy, and neutral facial expressions (1618); increased amygdala blood flow at rest (19); and pretreatment abnormalities in capacity of amygdala activity, the response elicited by the difference between baseline and all facial trials taken together, to sad and neutral emotional expressions (20). Other studies, however, reported no increases in amygdala activity to happy, sad, or fearful facial expressions, even in a subliminal condition, in depressed adults with recurrent MDD (2124). Major depressive disorder might, therefore, be characterized by abnormally increased baseline amygdala activity, resulting in no further increase in activity to emotional stimuli versus baseline.

These discrepant findings regarding amygdala activity to facial expressions in BDr, BDd, and MDD patients likely result from the employment of different paradigms and different neuroimaging measures across the aforementioned studies. No studies directly compared amygdala activity in BDd, BDr, and MDD depressed individuals, however. Such a study would be particularly important to understanding likely pathophysiologic processes that might differentiate BD from recurrent MDD depression.

Second, it is unclear whether abnormally elevated amygdala activity occurs predominantly to negative or to positive emotional facial expressions in BD and recurrent MDD. Although previous studies reported abnormally elevated amygdala activity to both positive and negative emotional facial expressions in BDr patients (21) and in remitted, depressed, and hypomanic adults with BD (9), studies in depressed adults with recurrent MDD reported abnormally elevated amygdala activity mainly to negative emotional facial expressions (16,17,20). Some findings suggest abnormally elevated amygdala activity to positive emotional facial expressions in depressed adults with MDD, however (16).

Additionally, although studies in healthy adults reported amygdala activity to neutral faces (2528) and one study in youth with BD reported that neutral faces assessed as hostile were associated with abnormally elevated amygdala activity relative to healthy youth (29), there have been no studies in adults with mood disorders that focused on examination of amygdala activity to neutral faces. Although we previously showed increasing magnitude of striatal activity to facial expressions of increasing intensity of sadness in depressed individuals with MDD (22), few studies in BD and recurrent MDD patients examined amygdala activity to different mild and intensely emotional facial expressions.

We therefore first aimed to examine the extent to which abnormally elevated amygdala activity to emotional facial expressions was one of the following three possibilities: 1) a persistent marker of BD during remission and depression, 2) a state marker of depression in both BD and recurrent MDD, or 3) a specific marker of depression in either BD or recurrent MDD. We secondly aimed to examine whether abnormally elevated amygdala activity occurred mainly to negative or to positive emotional facial expressions in each disorder. Our third aim was to examine the extent to which abnormally elevated amygdala activity would be elicited by emotional facial expressions other than those displaying prototypical emotion: mild intense and neutral facial expressions. Extant data did not allow us to make specific hypotheses related to these aims, because no study has yet directly compared amygdala activity to facial expressions of different intensities of emotional facial expressions in BDr, BDd, and depressed individuals with recurrent MDD. We recruited BDr, BDd, and depressed MDD individuals and a group of age-and gender ratio-matched healthy control participants (HC). We employed a well-validated facial expression processing paradigm involving displays of negative (fear, sad) and positive (happy) facial expressions as important signals of socially relevant emotions of social approval (happy), external threat (fear), and internal distress (sad) of both prototypical (intense) and mild intensities of each emotion, together with neutral expressions.

Methods and Materials

Participants

Sixty right-handed, native English-speaking individuals were recruited: 30 adults with recurrent BD type 1, 15 in depressed episode (BDd), and 15 in remission (BDr); 15 individuals with recurrent MDD in depressed episode, diagnosed according to DSM-IV criteria and the Structured Clinical Interview for DSM-IV, Research Version (SCID-P); and 15 HCs. The four groups were age- and gender-matched. The three patients groups were also matched for age of illness onset and illness duration (Table 1). Most patients were medicated: two BDd patients, two recurrent MDD patients, and one BDr patient were medication-free.

Table 1.

Demographic, Clinical Variables and Behavior Performance

BDd (n= 15)
BDr (n= 15)
MDDd (n= 15)
HC (n= 15)
Group Mean SD Mean SD Mean SD Mean SD Statistics p
Age at Scan 36.56 11.88 33.28 7.83 32.74 9.87 32.69 8.00 F(3,56) = .56 .64
Gender
    Male 1 6.7% 5 33.3% 2 13.3% 3 20.0% χ2(3) = 3.9 .3
    Female 14 93.3% 10 66.7% 13 86.7% 12 80.0%
Age of Illness Onset 22.33 10.38 18.80 6.82 19.07 7.52 NA F(2,42) = .83 .44
Illness Duration 14.23 9.82 14.67 5.48 13.67 9.87 NA F(2,42) = .05 .9
Medication Load 3.87 2.29 3.20 1.90 1.80 1.26 NA F(2,42) = 4.8 .013
HRSD-25 21.53 6.40 1.47 1.13 24.47 6.15 NA F(2,42) = 88.1 <.001
Lifetime Presence of
    Alcohol/Drugs Abuse
    or Dependence
    (NO/YES)
9/4a 69.2/30.8% 10/5 66.7/33.3% 12/3 80/20% NA χ2(2) = 7.4 .7
Behavioral Responses to
    Happy Emotional
    Labeling Experiment
    (Emotional Faces)
69.3% 30.07 84.2% 16.36 84% 15.38 90.8% 6.79 F(3,56) = 3.4 .024
Behavioral Responses to
    Happy Emotional
    Labeling Experiment
    (Neutral Faces)
73.7% 33.30 79% 24.94 83.7% 19.50 89.3% 9.23 F(3,56) = 1.2 .3
Behavioral Responses to
    Sad Emotional
    Labeling Experiment
    (Emotional Faces)
54.3% 23.10 60.5% 21.61 62.2% 21.19 67.5% 19.04 F(3,56) = .97 .4
Behavioral Responses to
    Sad Emotional
    Labeling Experiment
    (Neutral Faces)
73.3% 33.58 74.3% 28.15 73.3% 24.10 82% 22.82 F(3,56) = .35 .79
Behavioral Responses to
    Fear Emotional
    Labeling Experiment
    (Emotional Faces)
61.5% 26.30 67.7% 19.21 75.5% 22.94 71.5% 21.48 F(3,56) = 1.0 .38
Behavioral Responses to
    Fear Emotional
    Labeling Experiment
    (Neutral Faces)
69% 37.85 73% 31.72 69.7% 32.15 84.3% 24.12 F(3,56) = .75 .52

Medication load post hoc — bipolar disorder patients in depressed episode (BDd) vs. bipolar disorder patients in remission (BDr): t(28) = .9; p = .4; BDd vs. major depression disorder patients in depressed episode (MDDd): t(28) = 3.1; p = .005; BDr vs. MDDd: t(28) = 2.4; p = .024; HRSD-25 post hoc—BDd vs. BDr: t(15) = 12; p < .001; BDd vs. MDDd: t(28) = 1.3; p = .2; BDr vs. MDDd: t(15) = 14.25; p < .001; happy faces accuracy post hoc—BDd vs. BDr: t(21.6) = 1.7; p < .1; BDd vs. MDDd: t(21) = 1.7; p = .1; BDd vs. healthy control participants (HC): t(15.4) = 2.7; p = .016; BDr vs. MDDd: t(28) = .03; p > .9; BDr vs. HC: t(28) = 1.5; p = .16; MDDd vs. HC: t(28) = 1.6; p = .13. Between group differences in emotional accuracy analysis with nonparametric tests were not significant—happy experiment: emotional faces (χ2 = 5.6; p = .13); neutral faces (χ2 = .98; p = .81); sad experiment: emotional faces (χ2 = 2.4; p = .5); neutral faces (χ2 = 2.4; p = .5); fear experiment: emotional faces (χ2 = 2.8; p = .4); neutral faces (χ2 = 2.5; p = .5).

HRSD-25, 25-item Hamilton Rating Scale for Depression.

a

Information for two subjects were not available.

Medication load, an index that reflects the number and dose of different medications for each individual (the greater the number and dose of the medication, the greater the medication load; Supplement 1), was computed for all patients. The three patient groups differed on medication load [F(2,42) = 4.8; p = .013]: BDd and BDr patients had more medication load than MDD depressed individuals [t(28) = 3.1; p = .005 and t(28) = 2.4; p = .024, respectively], but BDd and BDr patients did not differ on medication load [t(28) = .9; p = .4]. The three patient groups differed in depression severity: BDd and MDD depressed individuals each had higher depression severity measured on the 25-item Hamilton Rating Scale for Depression (HRSD-25) than BDr patients [t(14.865) = 12; p < .001 and t(14.936) = 14.25; p < .001, respectively], but depressed groups did not differ on depression severity [t(28) = 1.3; p = .2]. Reported degree of freedom reflects significance of the Levene’s test for equality of variances.

All patients had experienced at least two episodes of illness in the last 4 years. Nearly 30% of recurrent BD and 20% of recurrent MDD patients had lifetime alcohol/illicit substance abuse/dependence (Table 1) but had not been abusing or dependent on alcohol/illicit substances for at least 2 months before scanning.

All participants were aware of the purpose of the study and gave written informed consent before participation in the study. The University of Pittsburgh Institutional Review Board approved the study protocol.

Exclusion criteria included history of head injury (from medical records and participant report), systemic medical illness, cognitive impairment (score <24 Mini-Mental State Examination), premorbid IQ estimate <85 (North American Adult Reading Test) (30), Axis-II borderline personality disorder, current alcohol/illicit substance dependence, and general exclusion criteria for magnetic resonance imaging. For HC, current or previous alcohol and illicit substance abuse and previous alcohol/illicit substance dependence (determined by SCID-P, saliva, and urine screen) and previous personal or family history of psychiatric illness in first- and second-degree relatives were further exclusion criteria.

Paradigm

All individuals participated in three separate 6-min, well-validated event-related experiments (viewing fear, sad, and happy faces). Each experiment involved viewing 20 mild (50%) and 20 intense (100%) emotional faces and 20 neutral faces from a standardized series (31). Each facial expression was presented for 2 sec, with an interstimulus interval (ISI) of variable duration, according to a Poisson distribution (mean ISI = 4.9 sec) (31). Participants were asked to label the emotion of each face in each experiment (fear, sad, or happy) by moving either the index (emotional faces) or middle finger (neutral faces) of the right hand to ensure that attention was directed to the emotional content of the face.

Functional Neuroimaging Data Analyses

Data were preprocessed and analyzed with statistical parametric mapping software (SPM5; http://www.fil.ion.ucl.ac.uk/spm). Data for each participant were first corrected for differences in acquisition time between slices; realigned and unwarped, co-registered, normalized, and spatially smoothed.

A first-level fixed-effect model was constructed for each experiment with three emotion intensities (neutral, mild, intense) entered as separate conditions in an event-related design with fixation cross as baseline in the design matrix. Movement parameters from the realignment stage were entered as covariates of no interest to control for subject movement. Trials were modeled with the Canonical hemodynamic Response Function. The three intensities were then entered into second-level analyses with the relevant t-contrast images (intense happy [or sad, or fear] > baseline; mild happy [or sad, or fear] > baseline; and neutral > baseline).

Three separate (one for each experiment) second-level random-effects group analyses were conducted on the t-contrast images generated in the previous single-subject analyses in a 4 (group) × 3 (emotion intensity condition) repeated-measures analysis of variance to identify neural activity in our a priori amygdala region of interest, as defined in the Wake Forest Toolbox PickAtlas Talairach Daemon template (32). To control for multiple statistical testing we maintained a cluster-level false positive detection rate at p < .05 by using a voxel threshold of p < .05 with a cluster (𝓀) extent empirically determined by Monte Carlo simulations implemented in AlphaSim of 8 voxels, which accounted for spatial correlations between blood oxygen level dependent (BOLD) signal changes in neighboring voxels. Post hoc analyses with independent t tests in SPSS 15.0 for Windows (SPSS, Inc., Chicago, Illinois) were performed to examine any significant main effect of group or group × emotion intensity interaction upon amygdala activity in each significant cluster with the mean BOLD signal extracted from β images derived in the first level analysis. We used Bonferroni correction to correct for multiple comparisons as appropriate.

Exploratory Analyses of Relationships Between Amygdala Activity and Demographic, Clinical, and Task Performance Variables

We explored possible relationships between amygdala BOLD response in BDd, BDr, and depressed recurrent MDD individuals and the following variables: age, age of illness onset, illness duration, depression severity measured with the HRSD-25, medication load (Supplement 1), taking versus not taking each psychotropic medication subclass, and comorbid alcohol/illicit substance abuse. We used a statistical threshold of p < .05 in these exploratory analyses.

Results

Face Emotion Labeling Accuracy

The BDd were less accurate than HC groups (but not other groups) when labeling the emotion of faces during the happy experiment (Table 1). No between-group differences in emotion-labeling accuracy were observed during the sad and fear experiment (Table 1).

Neuroimaging Data Analyses

Between Group Differences in Amygdala Activity

Happy Experiment

There was no main effect of group or group × emotion intensity interaction on activity in right or left amygdala.

Fear Experiment

There was a significant group × emotion intensity interaction in the left amygdala [F(6,168) = 2.81; p = .012, 𝓀 = 8 voxels]. Post hoc analyses revealed a trend increase only in left amygdala activity to mild fear expressions in BDd versus recurrent MDD individuals (Table 2).

Table 2.

Post Hoc Between-Group Pairwise Comparison of Amygdala Activity in the Sad and Fear Experiments

Sad
Experiment
BDd > BDr
BDd > MDDd
BDd > HC
Dr > MDDd
BDr > HC
MDDd > HC
tc p d tc p d tc p d tc p d tc p d tc p d
Mean Left
    Amygdalaa
5.4b < .001f 1.97 3.6 .001f 1.32 2.8 .009f 1.02 −1.8 .09g .65 −2.2 .04g .81 −.5 .6 .2
Intense Left
    Amygdalab
2.0 .055c .73 1.6 .131 .57 2 .035g .81 −.5 .63 .18 .1 .90 .05 .6 .5 .2
Mild Left
    Amygdalab
3.7d .001f 1.34 2.9 .008f 1.05 1.5 .147 .55 −1.4 .17 .51 −2.2 .04g .81 −1.1 .3 .4
Neutral Left
    Amygdalab
5.0b < .001f 1.82 3.5 .002f 1.27 2 .025g .87 −2.1c .04g .78 −2.5 .02g .90 −.8 .4 .3
Fear
    Experimentb
tc p d tc p d tc p d tc p d tc p d tc P d

Intense Left
    Amygdala
−.1 .947 .02 .5 .651 .17 −.4e .658 .16 .6 .58 .21 −.4 .69 .15 −1.1 .3 .4
Mild Left
    Amygdala
1.6 .110 .60 2.7c .013g .97 2 .060g .72 1.0 .31 .37 .8 .44 .29 .1 .9 .1
Neutral Left
    Amygdala
−.5 .653 .17 < .1 .999 .00 .2 .864 .06 .5 .63 .18 .7 .50 .25 .2 .9 .1

t, independent t test; d, Cohen’s d effect size; other abbreviations as in Table 1.

a

Statistical threshold at p < .0083.

b

Statistical threshold at p < .0028.

c

Degree of freedom = 28 unless otherwise specified.

d

Degree of freedom = 21.

e

Degree of freedom = 22.

f

Survive statistical threshold.

g

Trend level.

Sad Experiment

There was a significant main effect of group in the left amygdala (Figure 1). Post hoc analyses indicated significantly greater left amygdala activity in BDd versus BDr patients and versus depressed recurrent MDD individuals over all emotional intensities and a trend increase in mean left amygdala activity that only just failed to meet the strict significance threshold in BDd versus HC (Figure 1, Table 2). Further analyses revealed that the significant increase in left amygdala activity in BDd patients versus BDr and depressed recurrent MDD individuals was evident to mild sad and neutral expressions (Figure 1, Table 2). Analysis of the female subgroup confirmed abnormally elevated left amygdala activity specific to BDd patients (Table S5 in Supplement 1).

Figure 1.

Figure 1

(A) Anatomical location of left amygdala activity where the main effect of group was statistically different during the sad experiment. Foci of significance were overlaid on sagittal, coronal, and axial brain slices spatially normalized into the Montreal Neurologic Institute coordinates system (x = −18, y = − 3, z = −18; cluster size = 11 voxels; F = 4.96; p = .003, corrected for multiple comparisons). (B) Mean left amygdala activity to faces in the sad experiment in each group; *trend toward significance; **significant comparison after controlling for multiple comparisons. (C) Activity in the left amygdala in response to intense and mild sad and neutral faces in each group. BOLD, blood oxygen level dependent; BDd, bipolar disorder patients in depressed episode; BDr, bipolar disorder patients in remission; MDD, major depression disorder patients in depressed episode; HC, healthy control

Relationships Between Demographic, Clinical, and Task Performance Variables and Amygdala Activity

We restricted these analyses to our significant findings, which were in the sad experiment. The BDd patients showed significant negative correlations between left amygdala activity to intense sad facial expressions and illness duration and medication load (Table 3). The BDr patients showed a significant positive correlation between emotion labeling accuracy and mean left amygdala activity to all facial expressions (Table 3) and specifically between mild left amygdala activity and mild sad labeling accuracy. The BDr patients taking antidepressant medication, versus those not taking antidepressant medication, showed significantly greater left amygdala activity to all facial expressions and to mild sad and neutral facial expressions but did not differ in sad emotional labeling accuracy. Recurrent MDD individuals showed significant positive correlations between left amygdala activity to mild sad facial expressions and age at scan and age of illness onset (Table 3).

Table 3.

Relation Between Left Amygdala Activity, Clinical, Demographic, and Experiment Performance Variables in BDd, BDr, and MDDd in the Sad Experiment

BDd (n = 15)
BDr (n = 15)
MDDd (n = 15)
Mean Intense Mild Neutral Mean Intense Mild Neutral Mean Intense Mild Neutral
r r r r r r r r r r r r
Age at Scan .04 −.45 .20 .26 .25 .08 .15 .30 .42 .25 .53a .22
Age of Illness Onset .05 .03 −.17 .17 .16 −.04 .20 .13 .44 .46 .5a .02
Illness Duration −.01 −.58 .43 .13 .15 .17 −.03 .27 .09 −.11 .13 .21
HRSD-25 −.04 .36 −.10 −.26 .02 .25 −.21 .13 −.16 −.40 −.07 .10
Medication Load −.15 −.62b .07 .17 .36 .44 .22 .09 .39 .40 .22 .35
Accuracy Emotional
    Face
−.13 −.28 .09 .57a .02 .63b −.31 −.08 −.44
Accuracy Neutral
    Face
.33 −.11 −.44 −.24 .03 .27
td td td td td td td td td td td td
Gender
    (Male/Female)
.30 .03 .25 .29 −1.67 −0.73f −1.75 −.45 .51 .55 −.09 .96
Mood Stabilizers
    (ON/OFF)
.43 −1.15 −.01 1.66 .55 −.34 .67 .58h
Antipsychotic
    (ON/OFF)
.11 −1.44 .39 .93 −.55 .93 −.92 −.92 .89 .04 .66 1.77
Antidepressants
    (ON/OFF)
.57 .30 .56 .32 4.07c .87g 3.35bg 2.19a −.07 .56 .03 −.94
Benzodiazepines
    (ON/OFF)
.81 −1.09 .52 1.92 1.28 .35 .75 1.67 .60 .11 .18 1.51
Comorbid Alcohol/
    Drugs (YES/NO)
.79e 1.93e −1.67e .62e 1 .01 .83 .98 −1 −.33 −1.98 −.02

r, Pearson correlation; other abbreviations as in Table 1 and Table 2.

a

p < .05.

b

p < .01.

c

p < .001.

d

Degree of freedom = 13.

e

Degree of freedom = 11.

f

Degree of freedom = 6.

g

Degree of freedom = 8.

h

Degree of freedom = 5.

Healthy control subjects did not show any significant relationship among BOLD, age, and accuracy in either the sad or fear experiment (Table S4 in Supplement 1).

Exploratory whole brain analyses revealed a group × emotion interaction in the left amygdala during the sad experiment. Post hoc analysis confirmed increased activity in BDd patients (Table S5 and Table S6 in Supplement 1).

Discussion

The main aim of this study was to examine whether abnormally elevated amygdala activity to different negative and positive emotional facial expressions represented a persistence marker of BD during both depression and remission or a state marker of depression common to both or specific to either BD or recurrent MDD depression. We also aimed to examine amygdala activity to both negative and positive emotional facial expressions and to neutral and mild as well as intense facial expressions. We showed abnormally elevated left amygdala activity that was specific to BD depressed individuals and elicited by all facial expressions in the sad experiment, in particular, by mild sad and neutral facial expressions. Elevated left amygdala activity specific to BDd patients was confirmed by whole brain exploratory analysis and when focusing in the female subgroup only (Table S5 in Supplement 1).

Our finding of a bipolar depression-specific pattern of abnormally elevated amygdala activity is interesting, given the inconsistent findings in the literature regarding amygdala activity in BDr, BDd, and depressed adults with recurrent MDD (9,10,12,14,16,17,2022,33). There are several points to highlight in these previous studies, however. First, no previous study directly compared amygdala activity to different emotional facial expressions in BDd, BDr, and depressed adults with recurrent MDD. Second, the discrepancy of these findings might relate to the use of different paradigms and different neuroimaging measures across the studies. Third, although previous studies of depressed individuals with recurrent MDD reported elevated amygdala activity mainly to negative emotional facial expressions (16,17,20), there are points to consider about these studies. In two of these studies (16,17), facial expressions were presented covertly in a backward masking paradigm; furthermore, elevated left amygdala activity was observed to happy, fearful, and neutral masked facial expressions in one of these studies (16). The backward masking procedure might have made these emotional facial expressions appear more ambiguous and potentially threatening, which might in turn have contributed to the elevated amygdala activity observed in depressed recurrent MDD relative to healthy individuals in this previous study. In another study, greater capacity of amygdala activity to sad facial expressions was demonstrated in depressed individuals with recurrent MDD (20), but the extent to which this relates to greater magnitude of amygdala activity to negative emotional expressions remains unclear. Additional studies reported relationships between pretreatment amygdala activity to emotional facial expressions and symptom improvement after treatment (34) and depression symptom severity (22) but did not show abnormally elevated amygdala activity to these facial expressions in depressed MDD adults relative to HC. Furthermore, we previously showed elevated ventral striatal but not elevated amygdala activity to sad facial expressions in depressed individuals with recurrent MDD (22). These previous findings, together with the present findings of abnormally elevated amygdala activity in BDd but not BDr patients or depressed individuals with recurrent MDD, therefore suggest that abnormally elevated amygdala activity might represent a pathophysiological process specific to BD but not recurrent MDD depression.

The use of an overt emotion labeling task in the present study might have contributed to our findings of abnormally elevated amygdala activity only in BDd patients and not in depressed recurrent MDD individuals or BDr patients. Previous reports indicate that, unlike implicit emotion processing such as gender labeling, explicit emotion labeling of facial expressions is not consistently associated with robust amygdala activity in HC, because it might be dependent on explicit appraisal and reappraisal processes supported by lateral prefrontal cortex rather than the amygdala (35). Our finding of elevated amygdala activity in BDd patients in the present study might therefore reflect aberrant amygdala–lateral prefrontal cortical functional coupling during appraisal of emotion in BDd patients but not depressed individuals with recurrent MDD or BDr that could be examined in future studies with functional connectivity analyses. It is also possible that, with larger numbers of participants, we would have been able to demonstrate significant increases in amygdala activity in depressed individuals with recurrent MDD and BDr relative to HC during the sad- or other emotion-experiments. The effect sizes in the significant post hoc pairwise between-group comparisons for the sad experiment (Cohen’s d = 1.02 – d = 1.97) are similar to the effect sizes in previous between-group comparisons of amygdala activity (e.g., Lawrence et al. [21]), suggesting that our study was powered to detect significant between group differences in amygdala activity during each emotion experiment consistent with previous findings.

Regarding our second aim, we found a negative-emotion–specific pattern of abnormally elevated amygdala activity in BDd patients to all faces in the sad experiment relative to other groups. This finding is consistent with previous cognitive theories of depression proposing a negative emotion attentional bias in depression in general and in particular to self-relevant negative emotional material (36), such as mood-congruent, sad facial expressions in the present study. Our findings suggest, however, that the negative emotional attentional bias in BD and recurrent MDD depression might be associated with different pathophsysiologic processes in the two disorders. In further support of the latter point, we recently demonstrated that recurrent MDD depression is associated with a top-down, negative connectivity between orbital medial prefrontal cortex (OMPFC) and amygdala to positive (happy) facial expressions, suggesting an “inhibition” of amygdala activity to positive emotional facial expressions in recurrent MDD depression, whereas BD depression is associated with a disconnectivity between OMPFC and amygdala to happy faces (37). We also previously demonstrated an attentional bias away from labeling facial expressions mild happy rather than a bias toward labeling facial expressions as sad in recurrent MDD depression (38). These, together with the present findings, therefore suggest that recurrent MDD depression might be characterized more by an inhibitory response to positive emotional facial expressions that might lead to an attentional bias away from these stimuli rather than by attentional bias toward negative emotional facial expressions. By contrast, our previous and present findings in BDd patients of OMPFC–amygdala disconnectivity to happy faces and elevated amygdala activity to mood-congruent sad facial expressions suggest that BD depression might be characterized by an attentional bias to negative, mood-congruent expressions rather than an attentional bias away from happy faces.

The BDd patients showed, with regard to our third aim, elevated amygdala activity to neutral and mild rather than intense sad facial expressions relative to BDr and depressed individuals with recurrent MDD. Mild emotional and neutral facial expressions are often perceived as more ambiguous and potentially threatening than intense facial expressions (38,39) and might elicit amygdala activity because of the proposed role of this structure in processing ambiguity and potential threat (40). Although BDd patients accurately labeled facial expressions in the sad and fear experiments, abnormally elevated amygdala activity to mild sad and neutral facial expressions might therefore suggest abnormal perception of potential threat in these faces in BDd patients, consistent with previous observations of abnormally elevated amygdala activity to neutral facial expressions appraised as hostile in BD youth (29). Employment of subjective ratings of threat and hostility during facial expression emotion labeling in future studies in BD could help clarify this.

The laterality of our amygdala findings in BDd patients is interesting. Previous studies report sustained response of the left rather than right amygdala during fearful facial expression processing in HC (41,42), whereas studies that manipulated displays of facial expressions to prevent subjective awareness of these stimuli, most frequently through a backward masking procedure, reported right-sided lateralization of amygdala activity (43). In our analyses, we fitted a hemodynamic response function to the presentation of each facial stimulus that models sustained rather than initial transient activity to all stimuli of a given emotion intensity in each experiment. Our finding of a between-group difference in left- rather than right-sided amygdala activity during facial emotion labeling therefore supports a more sustained and evaluative response of the left than right amygdala during emotion processing.

Although there was no significant effect of group on emotion labeling in the sad experiment, all participants demonstrated a minimum of 54% and a maximum of 82% accuracy rather than nearer to 100% accuracy on labeling emotional faces. This was likely due to the known difficulty in accurately labeling mild emotional facial expressions from the facial expression series employed in our study (44). Previous studies in MDD depressed individuals indicated facial emotional expression labeling impairments in MDD and BD depressed individuals (4547). The more difficult nature of the facial expression-labeling task we used might have masked any between-group difference on task performance, because all individuals had some difficulty with emotion face labeling.

Our exploratory analyses indicated few significant relationships between left amygdala activity to facial expressions in the sad experiment and clinical variables. The BDd patients with longer illness duration and higher medication load had lower amygdala activity to intense sad but not mild sad or neutral facial expressions. Previous findings in MDD indicate a reduction in amygdala capacity and activity after antidepressant treatment (20,34), which parallel our finding in medicated BDd patients in the present study. It is therefore possible that the combined effects of psychotropic medication and longer illness duration might have contributed to normalization of elevated left amygdala activity, at least to intense sad facial expressions, in BDd patients. The possible perception of the more ambiguous mild sad and neutral facial expressions as potentially threatening by BDd patients might have accounted for the absence of effects of illness duration and medication upon left amygdala activity to these stimuli. Greater left amygdala activity in BDr patients was associated with more accurate emotion labeling of all facial expressions, especially the more ambiguous mild sad facial expressions. The BDr patients taking antidepressant medication also showed greater left amygdala activity to the more ambiguous mild sad and neutral facial expressions although did not differ from HC on magnitude of left amygdala activity to these expressions. Together, these findings might suggest that, in BDr patients, greater left amygdala activity during emotion labeling of mild sad and neutral facial expressions—which in turn is more significant in BDr patients potentially more vulnerable to depression (i.e., BDr patients taking vs. BDr patients not taking antidepressant medication)—might represent a persistent, attentional bias to sad faces associated with more accurate sad emotion labeling that represents a vulnerability marker to depression in BDr patients. Depressed individuals with recurrent MDD showed a positive correlation between left amygdala activity and age at scan and with age of MDD onset, suggesting that older individuals with later age of illness onset had a pattern of left amygdala activity more similar to BDd patients.

There were limitations to the present study. We did not include a group of euthymic individuals with a history of recurrent MDD and therefore did not examine whether amygdala activity to emotional facial expressions differentiated remitted and depressed phases of recurrent MDD. Future studies could therefore compare depressed with recovered individuals with a history of MDD. We deliberately focused examination on the amygdala in this first study comparing BDd, BDr, and depressed individuals with recurrent MDD, but clearly other neural regions are implicated in emotion processing (6). Many patients were medicated, especially depressed patients, because of the need for medication in these severely depressed individuals. We showed normalizing rather than confounding effects of medication upon amygdala activity in BDd patients, however, suggesting that medication was unlikely to have contributed to the elevated left amygdala activity in the sad experiment in BDd patients. There were relatively small numbers of men in each group, and we did not obtain accurate information regarding the amount of current or past use of nicotine in study participants. Greater numbers of men and tobacco-using histories could be included in future studies of BDr, BDd, and MDD depressed individuals.

Our findings suggest that abnormally elevated left amygdala activity to sad faces might represent a depression-specific marker of BD but not recurrent MDD depression that is unrelated to medication. Despite the very similar clinical presentation of BD and MDD depression, different pathophsysiologic processes might be associated with these two disorders. Together with emerging data indicating different patterns of abnormal effective connectivity during happy emotion processing in BD and MDD depression (37), the present data highlight the potential future use of neuroimaging in the search for biological markers to help improve accuracy of early diagnosis of BD depression.

Supplementary Material

1

Acknowledgments

All work was carried out within the Department of Psychiatry, University of Pittsburgh. Neuroimaging data were collected at the Brain Imaging Research Center, University of Pittsburgh, and Carnegie Mellon University. We thank Dr. K.J. Jung, S. Kurdilla, and D. Vizslay for their help acquiring neuroimaging data.

Dr. Phillips reports having support from a National Alliance for Research on Schizophrenia and Depression (NARSAD) Independent Investigator Award and 5R01 MH076971-01. Drs. Hassel and Versace report having support from NARSAD. Drs. Almeida and Kupfer reported no biomedical financial interests or potential conflicts of interest.

Footnotes

Supplementary material cited in this article is available online.

References

  • 1.Charney DS, Nestler EJ. Neurobiology of Mental Illness. 3rd ed. New York: Oxford University Press; 2009. [Google Scholar]
  • 2.Hirschfeld RM, Lewis L, Vornik LA. Perceptions and impact of bipolar disorder: How far have we really come? Results of the national depressive and manic-depressive association 2000 survey of individuals with bipolar disorder. J Clin Psychiatry. 2003;64:161–174. [PubMed] [Google Scholar]
  • 3.McDonald AJ. Is there an amygdala and how far does it extend? An anatomical perspective. Ann N Y Acad Sci. 2003;985:1–21. doi: 10.1111/j.1749-6632.2003.tb07067.x. [DOI] [PubMed] [Google Scholar]
  • 4.Sah P, Faber ES, Lopez De Armentia M, Power J. The amygdaloid complex: Anatomy and physiology. Physiol Rev. 2003;83:803–834. doi: 10.1152/physrev.00002.2003. [DOI] [PubMed] [Google Scholar]
  • 5.Swanson LW. The amygdala and its place in the cerebral hemisphere. Ann N Y Acad Sci. 2003;985:174–184. doi: 10.1111/j.1749-6632.2003.tb07081.x. [DOI] [PubMed] [Google Scholar]
  • 6.Phillips ML, Ladouceur CD, Drevets WC. A neural model of voluntary and automatic emotion regulation: Implications for understanding the pathophysiology and neurodevelopment of bipolar disorder. Mol Psychiatry. 2008;13:833–857. doi: 10.1038/mp.2008.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Frangou S. Functional neuroimaging in mood disorders. Psychiatry. 2009;8:102–104. [Google Scholar]
  • 8.Siegle GJ, Thompson W, Carter CS, Steinhauer SR, Thase ME. Increased amygdala and decreased dorsolateral prefrontal BOLD responses in unipolar depression: Related and independent features. Biol Psychiatry. 2007;61:198–209. doi: 10.1016/j.biopsych.2006.05.048. [DOI] [PubMed] [Google Scholar]
  • 9.Blumberg HP, Donegan NH, Sanislow CA, Collins S, Lacadie C, Skudlarski P, et al. Preliminary evidence for medication effects on functional abnormalities in the amygdala and anterior cingulate in bipolar disorder. Psychopharmacology. 2005;183:308–313. doi: 10.1007/s00213-005-0156-7. [DOI] [PubMed] [Google Scholar]
  • 10.Jogia J, Haldane M, Cobb A, Kumari V, Frangou S. Pilot investigation of the changes in cortical activation during facial affect recognition with lamotrigine monotherapy in bipolar disorder. Br J Psychiatry. 2008;192:197–201. doi: 10.1192/bjp.bp.107.037960. [DOI] [PubMed] [Google Scholar]
  • 11.Hassel S, Almeida JRC, Kerr N, Nau S, Ladouceur CD, Fissell K, et al. Elevated striatal and decreased dorsolateral prefrontal cortical activity in response to emotional stimuli in euthymic bipolar disorder: No associations with psychotropic medication load. Bipolar Disord. 2008;10:916–927. doi: 10.1111/j.1399-5618.2008.00641.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Malhi GS, Lagopoulos J, Sachdev PS, Ivanovski B, Shnier R, Ketter T. Is a lack of disgust something to fear? A functional magnetic resonance imaging facial emotion recognition study in euthymic bipolar disorder patients. Bipolar Disord. 2007;9:345–357. doi: 10.1111/j.1399-5618.2007.00485.x. [DOI] [PubMed] [Google Scholar]
  • 13.Robinson JL, Monkul ES, Tordesillas-Gutierrez D, Franklin C, Bearden CE, Fox PT, et al. Fronto-limbic circuitry In euthymic bipolar disorder: Evidence for prefrontal hyperactivation. Psychiatry Res. 2008;164:106–113. doi: 10.1016/j.pscychresns.2007.12.004. [DOI] [PubMed] [Google Scholar]
  • 14.Altshuler L, Bookheimer S, Townsend J, Proenza MA, Sabb F, Mintz J, et al. Regional brain changes in bipolar I depression: A functional magnetic resonance imaging study. Bipolar Disord. 2008;10:708–717. doi: 10.1111/j.1399-5618.2008.00617.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Chen CH, Lennox B, Jacob R, Calder A, Lupson V, Bisbrown-Chippendale R, et al. Explicit and implicit facial affect recognition in manic and depressed states of bipolar disorder: A functional magnetic resonance imaging study. Biol Psychiatry. 2006;59:31–39. doi: 10.1016/j.biopsych.2005.06.008. [DOI] [PubMed] [Google Scholar]
  • 16.Sheline YI, Barch DM, Donnelly JM, Ollinger JM, Snyder AZ, Mintun MA. Increased amygdala response to masked emotional faces in depressed subjects resolves with antidepressant treatment: An fMRI study. Biol Psychiatry. 2001;50:651–658. doi: 10.1016/s0006-3223(01)01263-x. [DOI] [PubMed] [Google Scholar]
  • 17.Dannlowski U, Ohrmann P, Bauer J, Kugel H, Arolt V, Heindel W, et al. Amygdala reactivity to masked negative faces is associated with automatic judgmental bias in major depression: A 3 T fMRI study. J Psychiatry Neurosci. 2007;32:423–429. [PMC free article] [PubMed] [Google Scholar]
  • 18.Abler B, Erk S, Herwig U, Walter H. Anticipation of aversive stimuli activates extended amygdala in unipolar depression. J Psychiatr Res. 2007;41:511–522. doi: 10.1016/j.jpsychires.2006.07.020. [DOI] [PubMed] [Google Scholar]
  • 19.Drevets WC, Videen TO, Price JL, Preskorn SH, Carmichael ST, Raichle ME. A functional anatomical study of unipolar depression. J Neurosci. 1992;12:3628–3641. doi: 10.1523/JNEUROSCI.12-09-03628.1992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Fu CH, Williams SC, Cleare AJ, Brammer MJ, Walsh ND, Kim J, et al. Attenuation of the neural response to sad faces in major depression by antidepressant treatment: A prospective, event-related functional magnetic resonance imaging study. Arch Gen Psychiatry. 2004;61:877–889. doi: 10.1001/archpsyc.61.9.877. [DOI] [PubMed] [Google Scholar]
  • 21.Lawrence NS, Williams AM, Surguladze S, Giampietro V, Brammer MJ, Andrew C, et al. Subcortical and ventral prefrontal cortical neural responses to facial expressions distinguish patients with bipolar disorder and major depression. Biol Psychiatry. 2004;55:578–587. doi: 10.1016/j.biopsych.2003.11.017. [DOI] [PubMed] [Google Scholar]
  • 22.Surguladze S, Brammer MJ, Keedwell P, Giampietro V, Young AW, Travis MJ, et al. A differential pattern of neural response toward sad versus happy facial expressions in major depressive disorder. Biol Psychiatry. 2005;57:201. doi: 10.1016/j.biopsych.2004.10.028. [DOI] [PubMed] [Google Scholar]
  • 23.Fu CH, Williams SC, Brammer MJ, Suckling J, Kim J, Cleare AJ, et al. Neural responses to happy facial expressions in major depression following antidepressant treatment. Am J Psychiatry. 2007;164:599–607. doi: 10.1176/ajp.2007.164.4.599. [DOI] [PubMed] [Google Scholar]
  • 24.Dannlowski U, Ohrmann P, Bauer J, Deckert J, Hohoff C, Kugel H, et al. 5-HTTLPR biases amygdala activity in response to masked facial expressions in major depression. Neuropsychopharmacology. 2008;33:418–424. doi: 10.1038/sj.npp.1301411. [DOI] [PubMed] [Google Scholar]
  • 25.Whalen PJ, Rauch SL, Etcoff NL, McInerney SC, Lee MB, Jenike MA. Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J Neurosci. 1998;18:411–418. doi: 10.1523/JNEUROSCI.18-01-00411.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Somerville LH, Kim H, Johnstone T, Alexander AL, Whalen PJ. Human amygdala responses during presentation of happy and neutral faces: Correlations with state anxiety. Biol Psychiatry. 2004;55:897–903. doi: 10.1016/j.biopsych.2004.01.007. [DOI] [PubMed] [Google Scholar]
  • 27.Morris JS, Frith CD, Perrett DI, Rowland D, Young AW, Calder AJ, et al. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature. 1996;383:812–815. doi: 10.1038/383812a0. [DOI] [PubMed] [Google Scholar]
  • 28.Todorov A, Engell AD. The role of the amygdala in implicit evaluation of emotionally neutral faces. Soc Cogn Affect Neurosci. 2008;3:303–312. doi: 10.1093/scan/nsn033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Rich BA, Vinton DT, Roberson-Nay R, Hommer RE, Berghorst LH, McClure EB, et al. Limbic hyperactivation during processing of neutral facial expressions in children with bipolar disorder. Proc Natl Acad Sci USA. 2006;103:8900–8905. doi: 10.1073/pnas.0603246103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Blair JR, Spreen O. Predicting premorbid IQ: A revision of the national adult reading test. Clin Neuropsychol. 1989;3:129–136. [Google Scholar]
  • 31.Surguladze SA, Brammer MJ, Young AW, Andrew C, Travis MJ, Williams SC, et al. A preferential increase in the extrastriate response to signals of danger. Neuroimage. 2003;19:1317–1328. doi: 10.1016/s1053-8119(03)00085-5. [DOI] [PubMed] [Google Scholar]
  • 32.Maldjian JA, Laurienti PJ, Kraft RA, Burdette JH. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI datasets. Neuroimage. 2003;19:1233–1239. doi: 10.1016/s1053-8119(03)00169-1. [DOI] [PubMed] [Google Scholar]
  • 33.Malhi GS, Lagopoulos J, Ward PB, Kumari V, Mitchell PB, Parker GB, et al. Cognitive generation of affect in bipolar depression: An fMRI study. Eur J Neurosci. 2004;19:741–754. doi: 10.1111/j.0953-816x.2003.03159.x. [DOI] [PubMed] [Google Scholar]
  • 34.Canli T, Cooney RE, Goldin P, Shah M, Sivers H, Thomason ME, et al. Amygdala reactivity to emotional faces predicts improvement in major depression. Neuroreport. 2005;16:1267–1270. doi: 10.1097/01.wnr.0000174407.09515.cc. [DOI] [PubMed] [Google Scholar]
  • 35.Lange K, Williams LM, Young AW, Bullmore ET, Brammer MJ, Williams SC, et al. Task instructions modulate neural responses to fearful facial expressions. Biol Psychiatry. 2003;53:226–232. doi: 10.1016/s0006-3223(02)01455-5. [DOI] [PubMed] [Google Scholar]
  • 36.Beck AT. Depression: Causes and Treatments. Philadelphia: University of Pennsylvania Publishing; 1967. [Google Scholar]
  • 37.Almeida JR, Versace A, Mechelli A, Hassel S, Quevedo K, Kupfer DJ, Phillips ML. Abnormal amygdala-prefrontal effective connectivity to happy faces differentiates bipolar from major depression. Biol Psychiatry. 2009;66:451–459. doi: 10.1016/j.biopsych.2009.03.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Surguladze SA, Young AW, Senior C, Brebion G, Travis MJ, Phillips ML. Recognition accuracy and response bias to happy and sad facial expressions in patients with major depression. Neuropsychology. 2004;18:212–218. doi: 10.1037/0894-4105.18.2.212. [DOI] [PubMed] [Google Scholar]
  • 39.Calder AJ, Burton AM, Miller P, Young AW, Akamatsu S. A principal component analysis of facial expressions. Vis Res. 2001;41:1179–1208. doi: 10.1016/s0042-6989(01)00002-5. [DOI] [PubMed] [Google Scholar]
  • 40.Davis M, Whalen PJ. The amygdala: Vigilance and emotion. Mol Psychiatry. 2001;6:13–34. doi: 10.1038/sj.mp.4000812. [DOI] [PubMed] [Google Scholar]
  • 41.Phillips ML, Medford N, Young AW, Williams L, Williams SC, Bullmore ET, et al. Time courses of left and right amygdalar responses to fearful facial expressions. Hum Brain Mapp. 2001;12:193–202. doi: 10.1002/1097-0193(200104)12:4&#x0003c;193::AID-HBM1015&#x0003e;3.0.CO;2-A. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Wright CI, Fischer H, Whalen PJ, McInerney SC, Shin LM, Rauch SL. Differential prefrontal cortex and amygdala habituation to repeatedly presented emotional stimuli. Neuroreport. 2001;12:379–383. doi: 10.1097/00001756-200102120-00039. [DOI] [PubMed] [Google Scholar]
  • 43.Morris JS, Ohman A, Dolan RJ. Conscious and unconscious emotional learning in the human amygdala. Nature. 1998;393:467–470. doi: 10.1038/30976. [DOI] [PubMed] [Google Scholar]
  • 44.Calder AJ, Young AW, Rowland D, Perrett DI. Computer-enhanced emotion in facial expressions. Proc Biol Sci. 1997;264:919–925. doi: 10.1098/rspb.1997.0127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Rocca CC, Heuvel E, Caetano SC, Lafer B. Facial emotion recognition in bipolar disorder: A critical review. Rev Bras Psiquiatr. 2009;31:171–180. doi: 10.1590/s1516-44462009000200015. [DOI] [PubMed] [Google Scholar]
  • 46.Csukly G, Czobor P, Szily E, Takacs B, Simon L. Facial expression recognition in depressed subjects: The impact of intensity level and arousal dimension. J Nerv Ment Dis. 2009;197:98–103. doi: 10.1097/NMD.0b013e3181923f82. [DOI] [PubMed] [Google Scholar]
  • 47.Bozikas VP, Tonia T, Fokas K, Karavatos A, Kosmidis MH. Impaired emotion processing in remitted patients with bipolar disorder. J Affect Disord. 2006;91:53–56. doi: 10.1016/j.jad.2005.11.013. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES