Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Oct 4.
Published in final edited form as: Schizophr Res. 2008 Sep 13;105(1-3):49–60. doi: 10.1016/j.schres.2008.05.010

Static posed and evoked facial expressions of emotions in schizophrenia

Christian G Kohler 1,*, Elizabeth A Martin 1, Neal Stolar 1, Fred S Barrett 1, Ragini Verma 1, Colleen Brensinger 1, Warren Bilker 1, Raquel E Gur 1, Ruben C Gur 1
PMCID: PMC5048468  NIHMSID: NIHMS200296  PMID: 18789845

Abstract

Objective

Impaired facial expressions of emotions have been described as characteristic symptoms of schizophrenia. Differences regarding individual facial muscle changes associated with specific emotions in posed and evoked expressions remain unclear. This study examined static facial expressions of emotions for evidence of flattened and inappropriate affect in persons with stable schizophrenia.

Methods

12 persons with stable schizophrenia and matched healthy controls underwent a standardized procedure for posed and evoked facial expressions of five universal emotions, including happy, sad, anger, fear, and disgust expressions, at three intensity levels. Subjects completed self-ratings of their emotion experience. Certified raters coded images of facial expressions for presence of action units (AUs) according to the Facial Action Coding System. Logistic regression analyses were used to examine differences in the presence of AUs and emotion experience ratings by diagnosis, condition and intensity of expression.

Results

Patient and control groups experienced similar intensities of emotions, however, the difference between posed and evoked emotions was less pronounced in patients. Differences in expression of frequent and infrequent AUs support clinical observations of flattened and inappropriate affect in schizophrenia. Specific differences involve the Duchenne smile for happy expressions and decreased furrowed brows in all negative emotion expressions in schizophrenia.

Conclusion

While patterns of facial expressions were similar between groups, general and emotion specific differences support the concept of impaired facial expressions in schizophrenia. Expression of emotions in schizophrenia could not be explained by impaired experience. Future directions may include automated measurement, remediation of expressions and early detection of schizophrenia.

Keywords: Emotion expression, Schizophrenia, Facial Action Coding System, Affective flattening, Inappropriate affect

1. Introduction

Facial expressions are shared in humans and animals, and are central for communication both within and across species (Darwin, 1872). Abnormal expressions of emotions have been described as characteristic symptoms of schizophrenia (Andreasen, 1984a; Bleuler, 1911) and may precede the onset of illness by many years (Walker et al., 1993). Affective flattening and other negative symptoms are present at onset of illness (Gelber et al., 2004; Shtasel et al., 1992) more common in males, increase with illness duration (Shtasel et al., 1992) and appear distinct from depression (Kohler et al., 1998). In contrast to positive symptoms of schizophrenia, negative symptoms may not respond as well to antipsychotics and have been linked to impairment in psychosocial functioning (Edwards et al., 1999; Ho et al., 1998).

Whereas there are widely used and validated instruments that measure and parse aspects of cognitive dysfunction and its neurobiology in schizophrenia, clinical assessments of affective flattening and other negative symptoms have been limited to observer based rating scales. The ability to quantify emotional expression, especially in the face, has been enhanced by work aimed at measuring unique features of universal emotions. Six universal emotions are recognized across cultures in facial expressions — happiness, sadness, anger, fear, disgust and surprise (Eibl-Eibesfeldt, 1970; Ekman and Friesen, 1975; Izard, 1994). Based on facial muscle movement, Ekman and Friesen (1978) developed the Facial Action Coding System (FACS), which identifies discrete facial muscle movements, called Action Units (AUs). FACS has been simplified and adapted for clinical research. Emotion FACS (EMFACS: Friesen, 1986) identifies AUs associated with the predicted expression of the particular emotion, and the Facial Expression Coding System (FACES: Kring et al., 1993; Kring and Sloan, 2007), which rates overall dynamic facial changes, according to number of expressions, intensity and duration.

Examinations of facial expressions beyond clinical rating scales in schizophrenia have reported on imitative expressions (Putnam and Kring, 2007; Schwartz et al., 2006; Tremeau et al., 2005), deliberate or posed expressions (Berenbaum, 1992; Schwartz et al., 2006; Tremeau et al., 2005), spontaneous expressions within dyadic interactions (Aghevli et al., 2003; Mattes et al., 1995; Schneider et al., 1990; Steimer-Krause et al., 1990), expressions associated with emotional film clips ((Berenbaum and Oltmanns, 1992; Earnst et al., 1996; Kring et al., 1999, 1993) or emotional experiences of the participants (Berenbaum and Oltmanns, 1992; Gottheil et al., 1970; Kring et al., 1993; Tremeau et al., 2005). Media for capturing facial expressions have included still photographs (Gottheil et al., 1976; Schwartz et al., 2006), videotapes (Aghevli et al., 2003; Berenbaum, 1992; Berenbaum and Oltmanns, 1992; Gaebel and Wolwer, 2004; Kring et al., 1993; Putnam and Kring, 2007; Steimer-Krause et al., 1990; Tremeau et al., 2005), and electromyographic recordings (Earnst et al., 1996; Kring et al., 1999; Mattes et al., 1995). Videotaped acquisition offers the advantage of capturing duration and frequency of emotion expressions. However, analyses of such lengthy data sets have been limited to global assessment of positive and negative emotion expressions, rather than changes in specific face regions. Other measurements of emotion expressions have included recognition rates of expressions (Gottheil et al., 1970, 1976; Putnam and Kring, 2007; Schneider et al., 1990; Schwartz et al., 2006) and FACS derived measures without analysis of specific AUs (Aghevli et al., 2003; Berenbaum, 1992; Berenbaum and Oltmanns, 1992; Gaebel and Wolwer, 2004; Kring et al., 1993; Tremeau et al., 2005). In addition, automated methods have included computerized face morphometry (Mattes et al., 1995; Schneider et al., 1990) and electromyographic measurements (Earnst et al., 1996; Kring et al., 1999; Mattes et al., 1995) that can measure minute muscle activations, albeit limited to select face regions.

Most studies have supported affective flattening in general, rather than inappropriate affect. Studies that examined specific emotions reported on selective impairment in happy (Gottheil et al., 1976), sad(Putnam and Kring, 2007), angry (Gottheil et al., 1970; Schwartz et al., 2006) and disgusted (Schwartz et al., 2006) expressions. Laterality differences of emotional expressions have not been reported, although acuity of illness may be associated with differential impairment in upper versus lower face expressions (Gaebel and Wolwer, 2004; Mattes et al., 1995; Schneider et al., 1990). While affective flattening is considered characteristic of schizophrenia, comparisons with psychiatric (Berenbaum, 1992; Gaebel and Wolwer, 2004; Schneider et al., 1990; Tremeau et al., 2005) and medical (Steimer-Krause et al., 1990) control groups have raised questions regarding specificity.

Antipsychotics, particularly first-generation, are associated with extrapyramidal symptoms, but their influence on emotion expression remains unclear. Some studies indicated an adverse effect of medications on facial expression (Gaebel and Wolwer, 2004; Schneider et al., 1992). Others (Earnst et al., 1996; Putnam and Kring, 2007; Tremeau et al., 2005) examined patients both on and off antipsychotics and found no clear effect on expressivity.

Previously, we investigated AUs in high intensity evoked expressions of universal emotions expressed by actors and determined AUs, which were essential for accurate recognition and increased recognition, when present in combinations (Kohler et al., 2004). The aim of the present study was to extend previous investigations on evidence of impaired affect in schizophrenia and to examine individual muscle movements in static facial expressions of emotions in persons with stable symptoms. We expected persons with schizophrenia to produce emotion expressions which include fewer AUs that are frequently present in expressions of controls and more AU that are infrequently present in controls. Matched groups of persons with stable schizophrenia and healthy controls underwent a standardized procedure of eliciting posed and evoked facial expressions of five universal emotions. We applied FACS to examine facial changes based on the presence of AUs within each emotion and condition, i.e. posed and evoked. Stratified by emotion, we examined the sum of frequent and infrequent AUs, combinations of frequent AUs, and differences in the presence of individual AUs. We expected different results for posed compared to evoked emotional expressions, as the former are regulated by cortical systems and are under greater volitional and cognitive control (Rinn, 1991). Specifically, we anticipated that impaired emotional expression in schizophrenia is more pronounced for evoked, rather than posed expressions.

2. Methods

2.1. Subjects

There were 12 persons (mean age=31.50 ± SD=7.74, range=21–42) with the DSM-IV diagnosis of schizophrenia based on the Diagnostic Interview for Genetic Studies (Nurnberger et al., 1994) and 12 healthy controls, case matched to patients according to gender, ethnicity, age and parental education. There were 7 men and 5 women, including 6 Caucasians and 6 African-Americans in each group. No participant in this study displayed any facial hair, which may have obscured facial movements, such as beard or mustache. Participants were recruited from the Schizophrenia Center at the University of Pennsylvania, according to the following general exclusion criteria: History of substance related disorders (DSM-IV and laboratory data including toxicology) within the past 6 months; mental retardation defined as a full scale IQ <70; not proficient in English; history of any neurologic event or disease affecting brain function; medical diseases that may affect brain function or interfere with participation. In addition, healthy controls were free of Axis I, Axis II disorders (First et al., 1995), and Axis I disorders in first-degree relatives.

Patients were screened and excluded for self-reported and observed depression, using standard rating scales. All patients lived independently or with family, none had been hospitalized within 6 months prior and six patients – compared to all controls – worked at least 20 h per week or were students. Patients had assessment for positive symptoms (Andreasen, 1984b) (mean total score ± SD=5.8±10.7, range=0–32) and negative symptoms (Andreasen, 1984a) (mean total score ± SD=29±18, range=0–55), including item ratings for affective flattening (mean score ± SD=2.3 ± 1.0, range=0–4) and inappropriate affect (mean score ± SD=0.8 ± 1.1, range=0–3) at the time of testing. Symptom assessments were performed by trained raters meeting inter-rater reliability (icc>.80). All patients were treated with standard dosages of second generation-antipsychotics (n=12) and augmenting dosages of first-generation antipsychotics, specifically long-acting haloperidol (n=2), without increase in antipsychotic medication for 3 months. Antipsychotic dosages were converted to estimated olanzapine-equivalents (mean dosage ± SD=15.5 mg ± 2.3) and chlorpromazine-equivalents (mean dosage ± SD=312.5 mg ± 188). None of the participants exhibited clinical evidence of tardive dyskinesia or acute extrapyramidal symptoms. After complete description of the study to the subjects, written informed consent was obtained.

2.2. Emotion expression

Our method of acquisition is based on obtaining emotion expressions in actors (Gur et al.2002) and controls, and required extensive training of synchronizing the different aspects of image acquisition. Participants were seated in a brightly lit room and instructed to remain within direct view of the digital camera. Research personnel, including the instructor and photographer, were located beyond the camera in an unlit area. Interaction between research personnel allowed for rapid and repeated acquisition of photographs, if necessary. Expressions were obtained of 5 universal emotions that are reliably rated cross culturally and of neutral expressions. We followed the procedure previously described (Gur et al., 2002) and the order of emotions expressed remained fixed: Happiness, Anger, Fear, Disgust and Sadness. A priori decision was made to not include surprise, since surprise can be conceptualized as the abrupt onset of any other universal emotion. The sequence of emotions was based on the assumption that happiness as the only positive emotion is most easily achieved in isolation, i.e. being first, and sadness was positioned last, since its effect in the evoked condition may last longer. Each emotion was obtained in low, medium and high intensity expressions and followed by a neutral expression to allow the participant to achieve neutral emotional state between different emotion expressions.

Expressions were obtained in two conditions traditionally used in directing with instructions on avoidance of speech, since it would interfere with facial expression of emotions. During the mechanical approach (English method) for unfelt or posed expressions participants were instructed to communicate or signal the target emotion through facial expressions. For the posed expressions, photographs were obtained based on cuing of emotional intensities by the instructor. Duration of acquisition of posed expressions was between 15 and 25 min. Subsequently, participants identified biographical emotional situations, when each emotion was experienced in all intensities, and these situations were summarized as vignettes with identification of time points for the three intensities. The genuine approach for felt or evoked expressions, is based on the acting system credited to Constantin Stanislawski that requires the participant to engage in emotion and sense memory related to a particular event and thus reliving an emotional experience. Emotional vignettes were recounted to participants in a narrative manner using exact wording derived from vignettes, and participants were instructed to communicate the emotional experience through facial expression. For the evoked expressions, photographs were obtained based on a priori decisions about which points of the narrative were associated with the corresponding emotional intensities. Duration of acquisition of evoked expressions was between 15 and 25 min. Following each acquisition of posed and evoked expressions, participants rated their subjective experience of each emotion at the three intensities on an 11-point self-rating scale (0=no emotion, 10=extreme intensity). Examples of emotion expressions are shown in Fig. 1.

Fig. 1.

Fig. 1

Open in a new tab

Examples of facial emotion expressions.

2.3. Image acquisition

Images utilized in this study were captured with a Pulnix digital color camera, mounted on a custom-made aluminum frame (Gur et al., 2002). Large-screen floodlights provided soft (diffuse) illumination, important for resulting texture fidelity. Images were transferred serially from their buffer in the digital cameras through a Bitflow multiplexer onto the computer, using a software-controlled hardware interface.

2.4. Facial Action Coding System (FACS) ratings

Captured images of mild, moderate and high intensity expressions were presented via digital video projection in pseudo-random order to three certified FACS raters, who were blinded to emotion expressed and participant status. To serve as a baseline comparison, neutral images were presented next to emotional images of the same person. FACS scoring was performed independently by each rater. AUs of Lips Part and Jaw Drop, which constitute mouth opening, were collapsed, since they represent differing degrees of the same muscle movement. For purposes of this study, we were interested in the presence, rather than intensity of AUs and intensity ratings were not included in the analysis. Laterality was collapsed so that if an AU was scored for one side, it was qualified on both. Ratings were transformed to binomial data, with the presence of an AU recorded with 1, absence with 0. According to agreement of at least 2 raters, an AU was coded as present or absent. The number of occurrences for each AU was calculated for each image, grouped by emotion and condition. Based on the distribution of AUs, those occurring in ≥15% of expressions within a particular emotion and either condition in the healthy control group were considered to be frequent AUs (see Fig. 2). Conversely, AUs occurring less frequently were considered to be infrequent AUs for the particular emotion.

Fig. 2.

Fig. 2

Open in a new tab

Frequencies of action units in emotion expressions.

Lastly within each emotion, we examined correlations between clinical assessments of affective flattening and inappropriate affect with presence of frequent and infrequent AUs.

2.5. Data analysis

Conditional logistic regression analyses, conditioning on each matched patient–control pair, were used to test for differences in the presence of AUs by diagnosis, condition, and intensity of expression. Two-way interactions were included for diagnosis by condition and diagnosis by intensity. Since diagnosis was of primary interest, the interactions were dropped from the model when not statistically significant, but the main effects for condition and intensity were retained in the model in order to adjust for these effects. Intensity was of secondary interest to show the internal validity of the procedure when examining presence of frequent AUs. When cell counts were sparse, exact conditional logistic regression was used as implemented in SAS Proc Logistic, using the Exact statement. Linear regression models were used to test for differences in the emotion experience ratings by diagnosis, emotion, and condition. Generalized estimating equations (GEE) methodology, using the exchangeable correlation structure, were used to adjust the linear regression models for clustering by matched pair. Within the patient group, Spearman correlations were conducted between clinical ratings for affective flattening and inappropriate affect (SANS) and presence of frequent and infrequent AUs within each emotion. All analyses were performed using SAS version 9.1 (SAS Institute, Cary, NC).

2.6. Inter-coder reliability

Cohen's kappa coefficients for agreement on expression of individual AUs that differed between groups ranged from .46–.79 for frequent and .32–.36 for infrequent AUs.

3. Results

Participants underwent the procedure for posed and evoked expressions of emotions without signs of undue emotional stress or difficulties. The two groups rated similar subjective experience of emotions for each emotion and condition (see Fig. 3). As expected, experiences increased with target intensities (x-sq=26.11, p<.001) and were rated as greater during evoked compared to posed expressions (x-sq=16.73, df=1, p<.001). With respect to intensities of experience, there were interactions for emotion by condition and condition by diagnosis. Differences between experience of posed and evoked emotions were more pronounced in sad, anger and fearful expressions, and less pronounced in happy expressions (x-sq=16.48, df=4, p=0.002). Differences between experience of evoked and posed emotions were less pronounced in patients (x-sq=5.11, df=1, p=0.024). Patients had higher intensities of experience compared to controls for posed emotions (x-sq=6.34, df=1, p=0.012) but not for evoked emotions (x-sq=0.17, df=1, p=0.68).

Fig. 3.

Fig. 3

Open in a new tab

Subjective experience of posed and evoked emotions.

Within each emotion, results are presented starting with differences in expression of sum total of frequent and infrequent AUs, and combination of at least two frequent AUs, followed by differences in expression of individual AUs according to group and group by condition (evoked versus posed). Within each emotion, group differences in AUs broken down by condition are represented in Fig. 4.

Fig. 4.

Fig. 4

Open in a new tab

Differences in frequencies of action units in emotion expressions.

3.1. Happy expressions

Overall, frequent AUs, which included Cheek Raiser, Lid Tightener, Lip Corner Pull and Mouth Open were less common in evoked expressions in schizophrenia (OR=.41, p<0.001), but not in posed expressions (OR=.87, p=.595; test for diagnosis by condition interaction: Wald=4.1, p=.044). Infrequent AUs were more common in schizophrenia (OR=2.0, p=.013). Combinations of at least two frequent AUs were less common in schizophrenia (OR=0.32, p=0.018). Examining the presence of individual AUs, the frequent AUs of Lid Tightener (OR=.37, p=.018) in both conditions and Cheek Raise limited to evoked expressions (OR=.15, p=.005) were less common in SZP. Upper Lip Raiser, an infrequent AU in happy expressions, was more common in SZP (OR=25.8, p<.001). For happy expressions, there was an effect of intensity on presence of frequent AUs (Wald=40.78, p<.001) but no interaction with group.

3.2. Sad expressions

Overall, frequent AUs, which included Inner Brow Raiser, Brow Lower, Lid Tightener, Lip Corner Depressor, Chin Raiser and Lip Tightener were less common in SZP (OR=.58, p=.003), while infrequent AUs were more common in SZP (OR=2.46, p<.001). Examining the presence of individual AUs, Inner Brow Raise (OR=.25, p=.016), Brow Lower (OR=.17, p<.001), and Lip Corner Depressor (OR=.03, p<.001) – all frequent AUs – were less common in SZP. Lid Droop (OR=4.18, p=.013), an infrequent AU, was more common in SZP. For sad expressions, there was an effect of intensity on presence of frequent AUs (Wald=10.97, p=.004), but no interaction with group.

3.3. Anger expressions

Overall, frequent AUs, which included Brow Lower, Lid Tightener and Lip Tightener, were less common in SZP (OR=.58, p=.008), while infrequent AUs were more common in SZP (OR=1.98, p=.003). Combinations of at least two frequent AUs were less common in schizophrenia (OR=0.40, p=0.017). Examining the presence of individual AUs, Brow Lower (OR=.19, p<.001), a frequent AU, was less common in SZP and Lid Droop (OR=11.9, p=.010), an infrequent AU, was more common in SZP. For angry expressions, there was an effect of intensity on presence of frequent AUs (Wald=16.12, p=<.001), but no interaction with group.

3.4. Fear expressions

Groups did not differ in expression of frequent AUs, which included Inner and Outer Brow Raiser, Brow Lower, Upper Lid Raiser, Lid Tightener and Mouth Open, and they did not differ in expression of infrequent AUs. Combinations of at least two frequent AUs were less common in schizophrenia (OR=0.43, p=0.027). Examining the presence of individual AUs, Brow Lower (OR=.03, p<.001) and Lid Tightener (OR=.39, p=.046) – both frequent AUs – were less common in SZP. For fear expressions, there was an effect of intensity on presence of frequent AUs (Wald=20.39, p<.001), but no interaction with group.

3.5. Disgust expressions

Overall, frequent AUs, which included Inner Brow Raiser, Brow Lower, Lid Tightener, Nose Wrinkler, Upper Lip Raiser, Lip Stretcher, Lip Tightener and Mouth Open, were less common in SZP (OR=.71, p=.013), while infrequent AUs were more common in SZP (OR=1.77, p=.018). Combinations of at least two frequent AUs were less common in schizophrenia (OR=0.46, p=0.054). Examining the presence of individual AUs, Inner Brow Raise (OR=.05, p=.004), Brow Lower (OR=.04, p<.001) and Lip Tightener (OR=.08, p=.031) – all frequent AUs – were less common in posed expressions in SZP. Lower Lip Depressor (OR=12.0, p=.037), an infrequent AU, was more common in SZP. Lip Corner Depressor (OR=.08, p=.028), an infrequent AU, was absent in SZP. For disgust expressions, there was an effect of intensity on presence of frequent AUs (Wald=30.37, p<.001), but no interaction with group.

3.6. Clinical ratings

Within the schizophrenia group, ratings for affective flattening did not correlate with presence of frequent AUs for any emotion expressed. Ratings for inappropriate affect correlated with presence of infrequent AUs for happy (r=−. 25, p=.036), sad (r=−.44, p<.001) and fearful (r=−.24, p=.047) expressions.

4. Conclusions

Impaired facial expressions of emotions represent characteristic negative symptoms in schizophrenia and directly affect interpersonal engagement and social functioning. Previous clinical studies have characterized the enduring and pervasive nature of affective flattening in schizophrenia. Experimental studies have underscored these findings and expanded on this body of knowledge by examination of affective flattening in different conditions and in comparison with other patient groups.

In the present study, involving persons with stable schizophrenia and case matched healthy controls, we applied ratings of specific facial changes associated with posed and evoked expressions of five universal emotions. Overall patterns of facial expressions of emotions were similar between patient and control groups in all emotions and, for each emotion, expression at higher intensities produced increased number of AUs in patient and control groups without differential findings. Differences in expression of AUs between groups support observations of altered facial expression of emotions in schizophrenia. Expressions of emotions usually result from activation of multiple AUs and for all emotions, except sadness, patients displayed fewer expressions with combinations of at least two frequent AUs. Most experimental studies in schizophrenia have supported affective flattening, rather than inappropriate affect, and experimental ratings of affective flattening in dynamic expressions correlated with clinical scales (Berenbaum, 1992; Putnam and Kring, 2007; Tremeau et al., 2005). In our study, experimental measures of affective flattening based on the presence of frequent AUs failed to correlate with clinical ratings affective flattening. While this lack of finding was unexpected, it may relate to the research methodology of examining static, rather than dynamic, expressions and omission of intensity ratings from data analysis.

Our prediction regarding inappropriate expressions was confirmed and in each emotion infrequent AUs were more commonly expressed in the schizophrenia group. In addition, experimental measures of inappropriate affect in happy, sad and fearful expressions correlated with clinical ratings in patients. The presence of even a single potentially erroneous AU in a particular emotion expression should not be underestimated. Whereas affective flattening may result in lack of recognition of the emotion expressed and misinterpretation as neutral or no emotion, inappropriate affect will result in misinterpretation of the emotional valence.

Our prediction regarding a more selective impairment in evoked expressions was limited to happy expressions. The prediction could not be confirmed in the negative emotions tested, in particular disgust, where some frequent AUs were less common in posed expressions in the patient group.

In happy expressions, patients commonly displayed upward turned lip corners, as the most common and characteristic feature of a smile. Other elements, which constitute the Duchenne smile and are necessary for a smile to appear sincere, such as eye lids tightened and cheeks raised, were less frequent. In all negative emotion expressions, furrowed or lowered brows were clearly less common in schizophrenia. The corrugator muscle, as the muscle associated with formation of a frown or scowl, represents a major constituent in the expression of negative emotion (Larsen et al., 2003). Different from our finding, Kring et al. (1999) reported increased corrugator muscle activity, as measured by electromyography that can measure small changes beyond visual resolution, for both positive and negative emotional stimuli in schizophrenia.

Considering similar subjective emotional experience within the two groups, our findings do not clearly support the hypotheses on the unidirectional relationship between emotion experience and expression according to either Darwin or James. Perhaps, our findings are more consistent with the concept of neuromotor dysfunction put forth by Dworkin et al. (1996), however, not all muscle movements were less frequent and there were specific differences found for each emotion. Although lower than during evoked expressions, levels of emotion experience during posed emotion expressions were elevated and in line with a previous study that reported on emotion experience and autonomic activity related to coached facial expressions in actors (Levenson et al., 1990). While subjective emotional experience cannot be readily quantified, our findings are in consistent with previous reports (Berenbaum and Oltmanns, 1992; Earnst and Kring, 1999; Kring et al., 1993; Kring and Neale, 1996) and support the notion that flat affect in schizophrenia does not indicate diminished emotional experience.

Limitations of our study pertain to subject selection, methodology for acquisition of facial expressions and data analysis. The small sample size is mediated by careful matching of patient and control groups, the intensive task design and complex analysis that accounted for presence and absence of every AU. We chose controls as comparisons, since actors or coached controls would magnify the difference found amongst groups. Given the limited power of our sample size and clinical stability of patients, the major concern was that the study would fail to detect differences and this did not occur. Due to the complexity of data, previous attempts to rate dynamic facial expressions using FACS were forced to limit analysis to global assessment of emotion expressions. Therefore to determine differences in individual muscle movements, we focused on rating static facial expressions. Inherent to this methodology is the possibility that photographs may be obtained which miss the intended emotion expression, despite extensive training — on synchronizing the different aspects of image acquisition. Emotion expressions were obtained in two conditions, posed and evoked. Advantages for obtaining more spontaneous or genuine emotion expressions rather than evoked expressions were weighed – against the need to keep acquisition methods for the two series similar – and the interference of speech on emotion expressions. The sequence of expressed emotions remained fixed and we decided against a randomized sequence of emotions where happy expressions could be affected by negative emotions and sad experiences could interfere with other expressions. Lastly, comparisons between groups were based on differences in AUs that were frequently or infrequently present in healthy controls, and did not include expression intensity ratings. Intensity ratings may have produced interesting findings regarding expression differences amongst groups, but the small sample size precluded meaningful comparisons.

Applying FACS to larger groups with wider range of clinical symptomatology is needed to replicate our findings and investigate the effect of illness acuity and symptom clusters on facial expressions. Other future directions involving measurement of facial muscle movements may lie in further development of automated measurement of facial expressions and therapeutic application of measuring facial expressions. Over the past 15 years, several automated programs have been created with the aim to better quantitate facial regions (Bartlett et al., 1999; Cohn et al., 1999). Based on MRI morphometry, our group (Verma et al., 2005) developed an automated program that examines facial changes in 10 regions that relate to the anatomic areas involved in AUs, as described by FACS. Such computerized methods will be able to provide a measure of the subtle changes in facial expression, which FACS is unable to quantify and may replace visual inspection as a more reliable and sensitive measure of facial expression of emotions. While simple feedback did not enhance emotion expression (Schwartz et al., 2006), more interactive remediation programs (Frommann et al., 2003) that utilize information about individual regional differences in expression of specific emotions, based on AUs or alternate muscle movement measurement, may enable remediation of emotion expression.

Lastly, in the past decade identification of prodromal states in schizophrenia has received increased attention. Affective flattening may be present well before clinical onset of schizophrenia (Walker et al., 1993), increase during the prodromal phase (Malla et al., 2002) and represent a marker of vulnerability. In conjunction with other candidates for endophenotypes, facial expressions of emotions in persons with vulnerability to schizophrenia may inform us about risk of illness and assist in future efforts at prevention or postponement of onset of schizophrenia.

Supplementary Material

supplement

Acknowledgments

This work was supported by NIMH MH-01839, MH-60722 and the National Alliance for Research on Schizophrenia and Depression. The funding sources had no role in design of study, interpretation of data, manuscript preparation and submission.

Role of funding source: This work was supported by NIMH MH-01839, MH-60722 and the National Alliance for Research on Schizophrenia and Depression.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.schres.2008.05.010.

Footnotes

Contributors: Drs. Kohler, R.E. Gur and R.C. Gur designed the study and Dr. Kohler prepared the manuscript. Drs. Stolar and Verma, E. Martin and F. Barrett were responsible for data acquisition, and C. Brensinger and Dr. Bilker were responsible for data analysis.

Conflict of interest: There are no potential conflicts of interest to report for Drs. Kohler, Stolar, Bilker, Verma, R.C. Gur and R.E. Gur; C. Brensinger, E. Martin and F. Barrett.

References

  1. Aghevli MA, Blanchard JJ, Horan WP. The expression and experience of emotion in schizophrenia: a study of social interactions. Psychiatry Res. 2003;119(3):261–270. doi: 10.1016/s0165-1781(03)00133-1. [DOI] [PubMed] [Google Scholar]
  2. Andreasen NC. The Scale for the Assessment of Negative Symptoms (SANS) Trans. Vol. The University of Iowa; Iowa City: 1984a. [Google Scholar]
  3. Andreasen NC. The scale for the Assessment of Positive Symptoms (SAPS) Trans. Vol. The University of Iowa; Iowa City: 1984b. [Google Scholar]
  4. Bartlett MS, Hager JC, Ekman P, Sejnowski TJ. Measuring facial expressions by computer image analysis. Psychophysiology. 1999;36(2):253–263. doi: 10.1017/s0048577299971664. [DOI] [PubMed] [Google Scholar]
  5. Berenbaum H. Posed facial expressions of emotion in schizophrenia and depression. Psychol Med. 1992;22(4):929–937. doi: 10.1017/s0033291700038502. [DOI] [PubMed] [Google Scholar]
  6. Berenbaum H, Oltmanns TF. Emotional experience and expression in schizophrenia and depression. J Abnorm Psychology. 1992;101(1):37–44. doi: 10.1037//0021-843x.101.1.37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bleuler E. Dementia Praeox oder die Gruppe der Schizophrenien. (Trans) 1911;Vol doi: 10.1192/bjp.149.5.661. [DOI] [PubMed] [Google Scholar]
  8. Cohn JF, Zlochower AJ, Lien J, Kanade T. Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. Psychophysiology. 1999;36(1):35–43. doi: 10.1017/s0048577299971184. [DOI] [PubMed] [Google Scholar]
  9. Darwin C. The expression of emotions in man and animals. Trans. Vol. Philosophical Library; New York: 1872. [Google Scholar]
  10. Dworkin RH, Clark SC, Amador XF, Gorman JM. Does affective blunting in schizophrenia reflect affective deficit or neuromotor dysfunction? Schizophr Res. 1996;20(3):301–306. doi: 10.1016/0920-9964(96)00011-4. [DOI] [PubMed] [Google Scholar]
  11. Earnst KS, Kring AM. Emotional responding in deficit and non-deficit schizophrenia. Psychiatry Res. 1999;88(3):191–207. doi: 10.1016/s0165-1781(99)00083-9. [DOI] [PubMed] [Google Scholar]
  12. Earnst KS, Kring AM, Kadar MA, Salem JE, Shepard DA, Loosen PT. Facial expression in schizophrenia. Biol Psychiatry. 1996;40(6):556–558. doi: 10.1016/0006-3223(96)00171-0. [DOI] [PubMed] [Google Scholar]
  13. Edwards J, McGorry PD, Waddell FM, Harrigan SM. Enduring negative symptoms in first-episode psychosis: comparison of six methods using follow-up data. Schizophr Res. 1999;40(2):147–158. doi: 10.1016/s0920-9964(99)00043-2. [DOI] [PubMed] [Google Scholar]
  14. Eibl-Eibesfeldt I. Ethology, the biology of behavior. Trans. Vol. Rinehart & Winston; New York: 1970. [Google Scholar]
  15. Ekman P, Friesen WV. Unmasking the face. Trans. Vol. Prentice-Hall; Englewood Cliffs: 1975. [Google Scholar]
  16. Ekman P, Friesen WV. Manual of the Facial Action Coding System (FACS) Trans. Vol. Consulting Psychologists Press; Palo Alto: 1978. [Google Scholar]
  17. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV Axis I Disorders, Non-Patient Edition (SCID-NP) Trans. Vol. New York State Psychiatric Institute/Biometrics Research Department; New York: 1995. [Google Scholar]
  18. Friesen WV. Recent developments in FACS-EMFACS. Facial Measurement Newsletter. 1986;1:1–2. [Google Scholar]
  19. Frommann N, Streit M, Wolwer W. Remediation of facial affect recognition impairments in patients with schizophrenia: a new training program. Psychiatry Res. 2003;117(3):281–284. doi: 10.1016/s0165-1781(03)00039-8. [DOI] [PubMed] [Google Scholar]
  20. Gaebel W, Wolwer W. Facial expressivity in the course of schizophrenia and depression. Eur Arch Psychiatry Clin Neurosci. 2004;254(5):335–342. doi: 10.1007/s00406-004-0510-5. [DOI] [PubMed] [Google Scholar]
  21. Gelber EI, Kohler CG, Bilker WB, Gur RC, Brensinger C, Siegel SJ, Gur RE. Symptom and demographic profiles in first-episode schizophrenia. Schizophr Res. 2004;67(2–3):185–194. doi: 10.1016/S0920-9964(03)00083-5. [DOI] [PubMed] [Google Scholar]
  22. Gottheil E, Paredes A, Exline RV, Winkelmayer R. Communication of affect in schizophrenia. Arch Gen Psychiatry. 1970;22(5):439–444. doi: 10.1001/archpsyc.1970.01740290055007. [DOI] [PubMed] [Google Scholar]
  23. Gottheil E, Thornton CC, Exline RV. Appropriate and background affect in facial displays of emotion. Comparison of normal and schizophrenic males. Arch Gen Psychiatry. 1976;33(5):565–568. doi: 10.1001/archpsyc.1976.01770050033004. [DOI] [PubMed] [Google Scholar]
  24. Gur RC, Sara R, Hagendoorn M, Marom O, Hughett P, Macy L, Turner T, Bajcsy R, Posner A, Gur RE. A method for obtaining 3-dimensional facial expressions and its standardization for use in neurocognitive studies. J Neurosci Methods. 2002;115(2):137–143. doi: 10.1016/s0165-0270(02)00006-7. [DOI] [PubMed] [Google Scholar]
  25. Ho BC, Nopoulos P, Flaum M, Arndt S, Andreasen NC. Two-year outcome in first-episode schizophrenia: predictive value of symptoms for quality of life. Am J Psychiatry. 1998;155(9):1196–1201. doi: 10.1176/ajp.155.9.1196. [DOI] [PubMed] [Google Scholar]
  26. Izard CE. Innate and universal facial expressions: evidence from developmental and cross-cultural research. Psychol Bull. 1994;115(2):288–299. doi: 10.1037/0033-2909.115.2.288. [DOI] [PubMed] [Google Scholar]
  27. Kohler C, Gur RC, Swanson CL, Petty R, Gur RE. Depression in schizophrenia: I. Association with neuropsychological deficits. Biol Psychiatry. 1998;43(3):165–172. doi: 10.1016/S0006-3223(97)00033-4. [DOI] [PubMed] [Google Scholar]
  28. Kohler CG, Turner T, Stolar NM, Bilker WB, Brensinger CM, Gur RE, Gur RC. Differences in facial expressions of four universal emotions. Psychiatry Res. 2004;128(3):235–244. doi: 10.1016/j.psychres.2004.07.003. [DOI] [PubMed] [Google Scholar]
  29. Kring AM, Neale JM. Do schizophrenic patients show a disjunctive relationship among expressive, experiential, and psychophysiological components of emotion? J Abnorm Psychol. 1996;105(2):249–257. doi: 10.1037//0021-843x.105.2.249. [DOI] [PubMed] [Google Scholar]
  30. Kring AM, Sloan DM. The Facial Expression Coding System (FACES): development, validation, and utility. Psychol Assess. 2007;19(2):210–224. doi: 10.1037/1040-3590.19.2.210. [DOI] [PubMed] [Google Scholar]
  31. Kring AM, Kerr SL, Smith DA, Neale JM. Flat affect in schizophrenia does not reflect diminished subjective experience of emotion. J Abnorm Psychol. 1993;102(4):507–517. doi: 10.1037//0021-843x.102.4.507. [DOI] [PubMed] [Google Scholar]
  32. Kring AM, Kerr SL, Earnst KS. Schizophrenic patients show facial reactions to emotional facial expressions. Psychophysiology. 1999;36(2):186–192. [PubMed] [Google Scholar]
  33. Larsen JT, Norris CJ, Cacioppo JT. Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology. 2003;40(5):776–785. doi: 10.1111/1469-8986.00078. [DOI] [PubMed] [Google Scholar]
  34. Levenson RW, Ekman P, Friesen WV. Voluntary facial action generates emotion-specific autonomic nervous system activity. Psychophysiology. 1990;27:363–384. doi: 10.1111/j.1469-8986.1990.tb02330.x. [DOI] [PubMed] [Google Scholar]
  35. Malla AK, Takhar JJ, Norman RM, Manchanda R, Cortese L, Haricharan R, Verdi M, Ahmed R. Negative symptoms in first episode non-affective psychosis. Acta Psychiatr Scand. 2002;105(6):431–439. doi: 10.1034/j.1600-0447.2002.02139.x. [DOI] [PubMed] [Google Scholar]
  36. Mattes RM, Schneider F, Heimann H, Birbaumer N. Reduced emotional response of schizophrenic patients in remission during social interaction. Schizophr Res. 1995;17(3):249–255. doi: 10.1016/0920-9964(95)00014-3. [DOI] [PubMed] [Google Scholar]
  37. Nurnberger JI, Blehar MC, Kaufmann CA, et al. Diagnostic interview for genetic studies: rationale, unique features, and training. Arch Gen Psychiatry. 1994;51:949. doi: 10.1001/archpsyc.1994.03950110009002. [DOI] [PubMed] [Google Scholar]
  38. Putnam KM, Kring AM. Accuracy and intensity of posed emotional expressions in unmedicated schizophrenia patients: vocal and facial channels. Psychiatry Res. 2007;151(1–2):67–76. doi: 10.1016/j.psychres.2006.09.010. [DOI] [PubMed] [Google Scholar]
  39. Rinn WE. Neuropsychology of facial expression. In: Feldman RS, Rime B, editors. Fundamentals of Nonverbal Behavior. Cambridge University Press; Cambridge, UK: 1991. pp. 31–72. [Google Scholar]
  40. Schneider F, Heimann H, Himer W, Huss D, Mattes R, Adam B. Computer-based analysis of facial action in schizophrenic and depressed patients. Eur arch psychiatry clin neurosci. 1990;240:67–76. doi: 10.1007/BF02189974. [DOI] [PubMed] [Google Scholar]
  41. Schneider F, Ellgring H, Friedrich J, Fus I, Beyer T, Heimann H, Himer W. The effects of neuroleptics on facial action in schizophrenic patients. Pharmacopsychiatry. 1992;25(5):233–239. doi: 10.1055/s-2007-1014412. [DOI] [PubMed] [Google Scholar]
  42. Schwartz BL, Mastropaolo J, Rosse RB, Mathis G, Deutsch SI. Imitation of facial expressions in schizophrenia. Psychiatry Res. 2006;145(2–3):87–94. doi: 10.1016/j.psychres.2005.12.007. [DOI] [PubMed] [Google Scholar]
  43. Shtasel DL, Gur RE, Gallacher F, Heimberg C, Gur RC. Gender differences in the clinical expression of schizophrenia. Schizophr Res. 1992;7(3):225–231. doi: 10.1016/0920-9964(92)90016-x. [DOI] [PubMed] [Google Scholar]
  44. Steimer-Krause E, Krause R, Wagner G. Interaction regulations used by schizophrenic and psychosomatic patients: studies on facial behavior in dyadic interactions. Psychiatry. 1990;53(3):209–228. doi: 10.1080/00332747.1990.11024505. [DOI] [PubMed] [Google Scholar]
  45. Tremeau F, Malaspina D, Duval F, Correa H, Hager-Budny M, Coin-Bariou L, Macher JP, Gorman JM. Facial expressiveness in patients with schizophrenia compared to depressed patients and nonpatient comparison subjects. Am J Psychiatry. 2005;162(1):92–101. doi: 10.1176/appi.ajp.162.1.92. [DOI] [PubMed] [Google Scholar]
  46. Verma R, Davatzikos C, Loughead J, Indersmitten T, Hu R, Kohler C, Gur RE, Gur RC. Quantification of facial expressions using high-dimensional shape transformations. J Neurosci Methods. 2005;141(1):61–73. doi: 10.1016/j.jneumeth.2004.05.016. [DOI] [PubMed] [Google Scholar]
  47. Walker EF, Grimes KE, Davis DM, Smith AJ. Childhood precursors of schizophrenia: facial expressions of emotion. Am J Psychiatry. 1993;150(11):1654–1660. doi: 10.1176/ajp.150.11.1654. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS200296-supplement-supplement.doc (68.5KB, doc)

RESOURCES