Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Sep 1.
Published in final edited form as: Schizophr Res. 2010 Sep;122(1-3):151–155. doi: 10.1016/j.schres.2010.03.034

Differential roles of low and high spatial frequency content in abnormal facial emotion perception in schizophrenia

Ryan McBain 1, Daniel Norton 1, Yue Chen 1,2,*
PMCID: PMC2923279  NIHMSID: NIHMS193478  PMID: 20435444

Abstract

While schizophrenia patients are impaired at facial emotion perception, the role of basic visual processing in this deficit remains relatively unclear. We examined emotion perception when spatial frequency content of facial images was manipulated via high-pass and low-pass filtering. Unlike controls (n=29), patients (n=30) perceived images with low spatial frequencies as more fearful than those without this information, across emotional salience levels. Patients also perceived images with high spatial frequencies as happier. In controls, this effect was found only at low emotional salience. These results indicate that basic visual processing has an amplified modulatory effect on emotion perception in schizophrenia.

Keywords: Vision, Affect, Face Recognition, Schizophrenic, Fear, Happiness

1. Introduction

Facial emotion perception plays a foundational role in interpersonal interactions and is significantly compromised in schizophrenia (Heimberg, Gur, Erwin, Shtasel, & Gur, 1992; Schneider, Gur, Gur, & Shtasel, 1995). In part, this capacity depends upon the ability to process affective information, which is largely dependent upon brain regions such as the amygdala (Adolphs, 2002; Gobbini & Haxby, 2007). However, facial emotion perception also requires the capacity to analyze basic visual content, such as spatial frequency information (Deruelle, Rondan, Salle-Collemiche, Bastard-Rosset, & Da Fonseca, 2008; Holmes, Vuilleumier, & Eimer, 2003); this involves brain areas like the extrastriate visual cortex (Haxby, Hoffman, & Gobbini, 2002). These brain systems, responsible for visual and affective processing, have been independently implicated in schizophrenia (Butler & Javitt, 2005; Chen et al., 2008; Gur et al., 2002; Holt et al., 2006; Johnson, Lowery, Kohler, & Turetsky, 2005; Lencer, Nagel, Sprenger, Heide, & Binkofski, 2005; Scholten, Aleman, Montagne, & Kahn, 2005). However, it remains unclear how visual processing contributes to impaired emotion perception in this disorder.

To characterize the relationship between visual and affective processing in schizophrenia, one prudent approach would be to examine the effects of modulating signals in one domain (e.g., basic visual attributes) on performance in the other domain (e.g., emotion perception). Spatial frequency information is one basic visual attribute of facial images. While low spatial frequencies (LSF) convey global/configural information of faces, high spatial frequencies (HSF) convey localized/fine-grain information. In terms of the underlying brain mechanisms, LSF face images preferentially activate the amygdala, whereas HSF face images stimulate specific areas of the fusiform gyrus (Vuilleumier, Armony, Driver, & Dolan, 2003). These separable contributions of LSFs and HSFs have manifestations at the behavioral level. For instance, healthy individuals have been shown to perceive images with HSFs as more fearful than images without this type of information (Vuilleumier et al., 2003).

Patients exhibit decreased activation in response to LSF gratings as opposed to HSF gratings (Butler et al., 2007; Martinez et al., 2008). From this perspective, one might hypothesize a weakened modulatory effect of spatial frequency content on emotion perception. Yet, other studies have found increased amygdala activation in schizophrenia (Marwick & Hall, 2008), an area which receives LSF inputs from the visual system. From this perspective, a stronger effect of LSF modulation would be anticipated. Thus, measuring the effects of spatial frequency modulation on emotion perception can distinguish these two competing hypotheses.

The current study applied a psychophysical approach to compare patients’ and healthy controls’ tendency to attribute fear or happiness to LSF, HSF and BSF (unfiltered) images. Fear and happiness represent two emotional expressions which differ in terms of their valence, but which have both been extensively studied in schizophrenia (Kohler et al., 2003). As such, this previous research provides an empirical foundation to which our results may be compared. Patients have also shown a greater degree of impairment in fear perception than in happiness perception (Norton, McBain, Holt, Ongur, & Chen, 2009), raising the question whether an abnormal response to LSFs may underpin this deficit.

2. Methods

2.1. Subjects

Thirty schizophrenia patients and 29 healthy controls participated (Table 1). Patients were diagnosed using the Structured Clinical Interview for DSM-IV (First, Spitzer, Gibbon, & Williams, 2002) and a review of available medical records. All patients received antipsychotic drugs (mean chlorpromazine equivalent = 560 mg, standard deviation = 403 mg). Clinical status was assessed using the Positive and Negative Symptom Scale (Kay et al., 1987). Average positive, negative and general symptom scores were 14.8 (7.4), 14.1 (6.4), and 27.9 (10.4), respectively.

Table 1.

Demographic information of the sample.

Participants
Group Age Sex Education* IQ*
SZ (n=30) 41.7 (9.4) 9 females, 21 males 13.8 (1.8) 100.5 (12.0)
NC (n=29) 44.0 (14.7) 14 females, 15 males 15.8 (2.2) 113.0 (10.9)
Open in a new tab

Participants were recruited via advertisements throughout McLean Hospital as well as Greater Boston. The study protocol was approved by the IRB of McLean Hospital, and written informed consent was obtained from each participant.

Means are reported above standard deviations. IQ was measured with the WAIS–R (verbal components). Education and age are in years.

Asterisks indicate a significant group difference (p<0.05.)

2.2. Procedures

Targets were happy, neutral and fearful face images (6 × 8.5 cm) from the NimStim Face Stimulus Set (Tottenham et al., 2009) and additional face images created by morphing emotive and neutral images from this set (FantaMorphPro 1.0, 2007). Three levels of emotional salience were utilized: 0% (neutral expression), threshold level (i.e., the minimum level of salience at which subjects could detect the emotion with 80% accuracy)1, and 100% (highly emotive expression). Spatial frequency content of the images was manipulated using high-pass and lowpass filtration functions in Adobe Photoshop 5.0. The estimated cut-off value was 13.5 cycles per face width for low-pass filtering and 41.9 cycles per face width for high-pass filtering. In each trial, the images being compared had same emotion salience but differed in spatial frequency information (HSF vs. LSF, HSF vs. BSF, or LSF vs. BSF) (Fig 1). In order to reduce testing time, a single, previously-validated facial identity was utilized throughout the experiment.

Figure 1.

Figure 1

Open in a new tab

Schematic depiction of the emotion perception task. The two images presented in a given trial were of the same emotional salience (e.g., 15%), but differed in SF content (e.g., HSF image vs. LSF image). Face images were presented for 400 ms, with a 500 ms inter-stimulus interval. The estimated cut-off values were 13.5 cycles per face width for the low-pass filtering and 41.9 cycles per face width for high-pass filtering. The contrasts of the facial images vary spatially and range from middle to high levels. Each testing session was blocked based on emotion type (happiness or fear) and consisted of 72 trials.

The task was to discern which of two presented facial images looked happier or more fearful. Testing sessions were blocked according to emotion type (happiness or fear). In each testing session, 8 trials were repeated for the 3 emotion salience levels and 3 spatial frequency filtration comparisons (72 trials in total). Stimuli were counterbalanced within each condition. The performance measurement was the proportion of trials in which subjects selected images with HSFs (HSF and BSF images) or LSFs (LSF and BSF images) as happier or more fearful, as compared with those without HSFs (i.e., LSF images) or without LSFs (i.e., HSF images). This approach to analysis was assumed in order to examine the individual impacts of HSFs and LSFs on emotion recognition, incorporating a baseline condition with high ecological validity (i.e. BSF images).

3. Results

3.1. Fear Perception

A three-way ANOVA - group (NC, SZ) × SF filtration type (HSF, LSF, BSF) × emotional salience (0%, threshold level, 100%) found significant interactions between group and SF filtration type (F=7.07, p<0.001) and between SF filtration type and emotional salience (F=2.73, p=0.029).

To investigate these interactions, two-way ANOVAs (group × emotional salience) were performed to measure the effects of each SF filtration. First, images with LSFs (LSF and BSF images) were compared with images without LSFs (i.e., HSF images). There was a main effect for group (F=7.26, p<0.01), indicating that, compared to controls, patients perceived images with LSF as more fearful (see Fig 2). Next, images with HSF information (HSF and BSF images) were compared with images without HSF information (i.e., LSF images). No effects were significant (p>0.05).

Figure 2.

Figure 2

Open in a new tab

Patients’ and controls’ perception of images with and without LSFs on the fear perception task. The x-axis denotes the emotional salience of the images being compared, while the y-axis represents the percent of the time images with LSFs were perceived as more fearful than images without LSFs. The horizontal line, a reference of chance level performance (50%), indicates the point at which fear perception of images with LSFs would be identical to fear perception of HSF images. One sample t-tests were used to inspect whether the percentage of the time that individuals perceived HSF, LSF or BSF images as more fearful differed significantly from chance (50%). These analyses were done within each group across salience levels. In patients, images with LSFs were perceived as more fearful (t=3.48, p=0.002), and there was a trend towards perceiving images with HSFs as less fearful (t= −1.87, p=.072). In the control group, no significant effect on spatial frequency was found (p>0.05).

3.2. Happiness Perception

A three-way ANOVA (group × SF filtration type × emotional salience) found a significant main effect for SF filtration (F=64.20, p<0.001), as well as interaction effects between SF filtration and emotional salience (F=7.35, p<0.001) and group × SF filtration × emotional salience (F=3.76, p<0.01).

Two-way ANOVAs (group × emotional salience) assessed the effects of each SF filtration separately. Contrasting images with HSFs (i.e., HSF and BSF images) and those without HSFs (i.e., LSF images), there was a main effect for salience (F=3.88, p=0.023): At low salience, images with HSFs were perceived as happier a greater proportion of the time than at high salience. There was also an interaction between group and salience (F=3.79, p=0.025). T-tests between groups at each salience level showed that this interaction was caused by a significant group difference at high salience (t=2.03, p<0.05) which was not present at low or moderate salience (p>0.05). As shown in Figure 3, patients perceived images with HSFs as happier across all salience levels, whereas in controls this effect was only found at lower salience levels. Comparing images with LSFs (i.e., LSF and BSF images) to images without this information (i.e., HSF images), there was a main effect for salience (F=11.65, p<0.001); at high salience, images with LSFs were perceived as happier a greater proportion of the time than at low salience.

Figure 3.

Figure 3

Open in a new tab

Patients’ and controls’ perception of images with and without HSFs on the happiness perception task. One sample t-tests assessed each group’s tendency to perceive HSF, LSF or BSF images as happier, and whether this differed significantly from chance (50%). In patients, there were biases to perceive images with HSFs as happier (t=5.11, p<0.001), and images with LSFs as less happy (t= −3.20. p=0.004). Similar biases were found in controls (images with HSFs: t=3.83, p=0.001; images with LSFs: t= −3.06, p=0.005).

3.3. Relationships with IQ and Clinical Variables

Neither PANSS scores nor medication levels were associated with patients’ inclination to perceive images with LSFs as more fearful, or images with HSFs as happier. However, IQ scores were moderately correlated with patients’ tendency to perceive images with LSFs as more fearful (r=0.41, p<0.05).

4. Discussion

4.1. Fear Perception

This study found that patients were significantly more likely than controls to perceive images with LSF information as more fearful than images without this information. Given previous evidence indicating that patients over-attributed negative emotions such as fear (Kohler et al., 2003; Tsoi et al., 2008), our findings offer a novel explanation that this misattribution may pertain to abnormal regulation of LSF information. It has been theorized that LSF information rapidly projects from the visual system to the amygdala (Ohman, 2005), a part of the brain shown to be hyperactive in schizophrenia (Marwick and Hall 2008). In a future study, it would therefore be prudent to investigate how fear perception with LSF images is related to atypical amygdala activation in schizophrenia patients. Neurophysiological studies would also benefit from examining the relationship between patients’ decreased activation in response to LSF information (Butler et al., 2007; Martinez et al., 2008) and the amplified behavioral responses to LSF facial images shown in this study. One mechanism which may underpin these seemingly diverging results is the over-compensation of cortical systems in schizophrenia patients, whereby a weakened response at one processing level may actually be associated with heightened response at a different processing level (e.g., Chen et al., 2008).

While a previous study (Vuilleumier et al., 2003) found that healthy individuals rate HSF images as more fearful than LSF and BSF images, our data indicate that controls’ perception of fear was relatively unaffected by SF manipulations (see Figure 2). In contrast to the earlier study, which asked subjects to scale the fearfulness of individual high salience images, the present study employed direct comparisons between images containing different SF information, and these comparisons were done at multiple levels of emotional salience. Whether the use of different paradigms contributed to differences in the results between the two studies merits further investigation.

4.2. Happiness Perception

In general, images with HSFs were perceived as happier, while images with LSFs were perceived as less happy. An important finding here is that this modulatory effect persisted across salience levels for patients. In controls, the effect was present only at low salience levels. This distinction again points to an abnormal and excessive role of spatial frequency content in emotion perception in schizophrenia. One possibility is that, for healthy controls, high emotional salience made SF manipulations an exiguous factor; visual modulations were “drowned out” by the salience level. In contrast, if patients under-appreciated emotional salience (as shown in previous studies like Norton et al., 2009), then SF manipulations would still have had a significant effect at this salience level, as shown in our data.

4.3. Implications for Visual and Affective Research in Schizophrenia

Previous studies have demonstrated an association between patients’ underperformance on visual and emotional tasks, but have left unresolved the question of whether abnormal visual processing has a causal effect on affective processing (Schyns & Oliva, 1999), or vice versa (Bocanegra & Zeelenberg, 2009; Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004). In the present study, one basic visual attribute of face images (spatial frequency information) was manipulated while the effect of this manipulation was observed on emotion recognition. As such, the resulting changes in emotion perception performance indicate a direct impact of basic visual information on affective processing in schizophrenia. Such bottom-up effects may play a key role in other observable patient deficits in areas like general cognitive functioning (IQ).

It should also be noted that spatial frequency filtrations in this study may have introduced slight differences in the contrast and luminance between LSF, BSF and HSF images. However, contrast and luminance of all images were set at supra-threshold levels, thus not imposing a major limiting factor on affective processing. Nevertheless, in a future study, additional conditions with modulated contrast/luminance levels could be utilized to tease out any influence of these factors.

4.4. Conclusion

Based on the rich literature showing deficient visual processing in schizophrenia, our finding that visual feature manipulations have an amplified effect on emotion perception underscores an important abnormal association between affective and basic visual systems which should be further investigated. These modulatory effects also highlight the importance of implementing bottom-up approaches to remediation of neurocognitive deficits in schizophrenia, including behavioral training on emotion perception tasks with and without the presence of HSF and LSF information.

Acknowledgments

We thank Dost Ongur for overseeing clinical evaluations of patients who participated in this study.

Role of Source Funding

Funding for this study was provided in part by a grant from the National Institutes of Health (R01 MH 61824).

Footnotes

1

Prior to the emotion perception task - the main focus of this study, the threshold for each subject was obtained, using a standardized psychophysical method ((Norton et al., 2009)).

Conflict of Interest

All authors declare that they have no conflicts of interest.

Contributors

All authors were involved in the design and execution of the study, and have contributed to the production of this manuscript.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errorsmaybe discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Adolphs R. Neural systems for recognizing emotion. Curr Opin Neurobiol. 2002;12(2):169–177. doi: 10.1016/s0959-4388(02)00301-x. [DOI] [PubMed] [Google Scholar]
  2. Bocanegra BR, Zeelenberg R. Emotion Improves and Impairs Early Vision. Psychol Sci. 2009 doi: 10.1111/j.1467-9280.2009.02354.x. [DOI] [PubMed] [Google Scholar]
  3. Butler PD, Javitt DC. Early-stage visual processing deficits in schizophrenia. Curr Opin Psychiatry. 2005;18(2):151–157. doi: 10.1097/00001504-200503000-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Butler PD, Martinez A, Foxe JJ, Kim D, Zemon V, Silipo G, Mahoney J, Shpaner M, Jalbrzikowski M, Javitt DC. Subcortical visual dysfunction in schizophrenia drives secondary cortical impairments. Brain. 2007;130(Pt 2):417–430. doi: 10.1093/brain/awl233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Chen Y, Grossman ED, Bidwell LC, Yurgelun-Todd D, Gruber SA, Levy DL, Nakayama K, Holzman PS. Differential activation patterns of occipital and prefrontal cortices during motion processing: evidence from normal and schizophrenic brains. Cogn Affect Behav Neurosci. 2008;8(3):293–303. doi: 10.3758/cabn.8.3.293. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Deruelle C, Rondan C, Salle-Collemiche X, Bastard-Rosset D, Da Fonseca D. Attention to low- and high-spatial frequencies in categorizing facial identities, emotions and gender in children with autism. Brain Cogn. 2008;66(2):115–123. doi: 10.1016/j.bandc.2007.06.001. [DOI] [PubMed] [Google Scholar]
  7. First M, Spitzer R, Gibbon M, Williams J. Structured clinical interview for DSM-IV-TR axis I disorders - patient edition (SCID-I/P, 11/2002 revision) New York, NY: Biometrics Research Department; 2002. [Google Scholar]
  8. Gobbini MI, Haxby JV. Neural systems for recognition of familiar faces. Neuropsychologia. 2007;45(1):32–41. doi: 10.1016/j.neuropsychologia.2006.04.015. [DOI] [PubMed] [Google Scholar]
  9. Gur RE, McGrath C, Chan RM, Schroeder L, Turner T, Turetsky BI, Kohler C, Alsop D, Maldjian J, Ragland JD, Gur RC. An fMRI study of facial emotion processing in patients with schizophrenia. Am J Psychiatry. 2002;159(12):1992–1999. doi: 10.1176/appi.ajp.159.12.1992. [DOI] [PubMed] [Google Scholar]
  10. Heimberg C, Gur RE, Erwin RJ, Shtasel DL, Gur RC. Facial emotion discrimination: III. Behavioral findings in schizophrenia. Psychiatry Res. 1992;42:253–265. doi: 10.1016/0165-1781(92)90117-l. [DOI] [PubMed] [Google Scholar]
  11. Holmes A, Vuilleumier P, Eimer M. The processing of emotional facial expression is gated by spatial attention: evidence from event-related brain potentials. Brain Res Cogn Brain Res. 2003;16(2):174–184. doi: 10.1016/s0926-6410(02)00268-9. [DOI] [PubMed] [Google Scholar]
  12. Holt DJ, Kunkel L, Weiss AP, Goff DC, Wright CI, Shin LM, Rauch SL, Hootnick J, Heckers S. Increased medial temporal lobe activation during the passive viewing of emotional and neutral facial expressions in schizophrenia. Schizophr Res. 2006;82(2–3):153–162. doi: 10.1016/j.schres.2005.09.021. [DOI] [PubMed] [Google Scholar]
  13. Johnson SC, Lowery N, Kohler C, Turetsky BI. Global-local visual processing in schizophrenia: evidence for an early visual processing deficit. Biol Psychiatry. 2005;58(12):937–946. doi: 10.1016/j.biopsych.2005.04.053. [DOI] [PubMed] [Google Scholar]
  14. Kohler CG, Turner TH, Bilker WB, Brensinger CM, Siegel SJ, Kanes SJ, Gur RE, Gur RC. Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am J Psychiatry. 2003;160:1768–1774. doi: 10.1176/appi.ajp.160.10.1768. [DOI] [PubMed] [Google Scholar]
  15. Lencer R, Nagel M, Sprenger A, Heide W, Binkofski F. Reduced neuronal activity in the V5 complex underlies smooth-pursuit deficit in schizophrenia: evidence from an fMRI study. Neuroimage. 2005;24(4):1256–1259. doi: 10.1016/j.neuroimage.2004.11.013. [DOI] [PubMed] [Google Scholar]
  16. Martinez A, Hillyard SA, Dias EC, Hagler DJ, Jr, Butler PD, Guilfoyle DN, Jalbrzikowski M, Silipo G, Javitt DC. Magnocellular pathway impairment in schizophrenia: evidence from functional magnetic resonance imaging. J Neurosci. 2008;28(30):7492–7500. doi: 10.1523/JNEUROSCI.1852-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Marwick K, Hall J. Social cognition in schizophrenia: a review of face processing. Br Med Bull. 2008;88(1):43–58. doi: 10.1093/bmb/ldn035. [DOI] [PubMed] [Google Scholar]
  18. Norton D, McBain R, Holt DJ, Ongur D, Chen Y. Association of Impaired Facial Affect Recognition with Basic Facial and Visual Processing Deficits in Schizophrenia. Biol Psychiatry. 2009;5:5. doi: 10.1016/j.biopsych.2009.01.026. [DOI] [PubMed] [Google Scholar]
  19. Ohman A. The role of the amygdala in human fear: automatic detection of threat. Psychoneuroendocrinology. 2005;30(10):953–958. doi: 10.1016/j.psyneuen.2005.03.019. [DOI] [PubMed] [Google Scholar]
  20. Schneider F, Gur RC, Gur RE, Shtasel DL. Emotional processing in schizophrenia: neurobehavioral probes in relation to psychopathology. Schizophr Res. 1995;17(1):67–75. doi: 10.1016/0920-9964(95)00031-g. [DOI] [PubMed] [Google Scholar]
  21. Scholten MR, Aleman A, Montagne B, Kahn RS. Schizophrenia and processing of facial emotions: sex matters. Schizophr Res. 2005;78(1):61–67. doi: 10.1016/j.schres.2005.06.019. [DOI] [PubMed] [Google Scholar]
  22. Schyns PG, Oliva A. Dr. Angry and Mr. Smile: when categorization flexibly modifies the perception of faces in rapid visual presentations. Cognition. 1999;69(3):243–265. doi: 10.1016/s0010-0277(98)00069-9. [DOI] [PubMed] [Google Scholar]
  23. Tottenham N, Tanaka JW, Leon AC, McCarry T, Nurse M, Hare TA, Marcus DJ, Westerlund A, Casey BJ, Nelson C. The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Res. 2009;168(3):242–249. doi: 10.1016/j.psychres.2008.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Tsoi DT, Lee KH, Khokhar WA, Mir NU, Swalli JS, Gee KA, Pluck G, Woodruff PW. Is facial emotion recognition impairment in schizophrenia identical for different emotions? A signal detection analysis. Schizophr Res. 2008;99(1–3):263–269. doi: 10.1016/j.schres.2007.11.006. [DOI] [PubMed] [Google Scholar]
  25. Vuilleumier P, Armony JL, Driver J, Dolan RJ. Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nat Neurosci. 2003;6(6):624–631. doi: 10.1038/nn1057. [DOI] [PubMed] [Google Scholar]
  26. Vuilleumier P, Richardson MP, Armony JL, Driver J, Dolan RJ. Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nat Neurosci. 2004;7(11):1271–1278. doi: 10.1038/nn1341. [DOI] [PubMed] [Google Scholar]

RESOURCES